Skip to main content

Querying Live Stock Prices in Python Without an API

When I first got into Python, I had a clear goal from the very start: I wanted to make a stock ticker that I could choose a company, and it would show me the price in real time. Due to the beautiful simplicity of Python, I found myself getting closer to achieving this goal in less time than I thought I would need. Once I got the basics of Python down, I started searching for how to query a live stock price.

There are plenty of APIs that are happy to give you live stock data... for a handsome fee. Google Finance had a promising API, though it seems to have since been abandoned. Most of the decent free APIs I came across had a dishearteningly low daily query limit. In the end, I decided on a less respectable method: web scraping. 

Through much trial and error, I have simplified the code quite a bit. I have also sorted through dozens of websites to see who does and does not monitor/throttle excessive requesting. I have come to the conclusion that Yahoo Finance is best for scraping stock data. I have queried hundreds of times in a day before, and have never gotten denied a request (for any other reason than a timeout). 



Perhaps there is meaning here that I don't yet understand, but Yahoo's HTML tags for the live data seem very unintuitive. You'll see what I mean. Here is my code for finding a stock's live price:

def getprice(ticker):
    headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'}
    url = "https://finance.yahoo.com/quote/"+ticker
    page = requests.get(url, headers=headers)
    soup = BeautifulSoup(page.text, "html.parser")
    james = soup.find_all("span", {"class": "Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)"})
    james = re.findall(">(.*)<", str(james))
    return james[0]
Let's break this down a little. 

 headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'}

The headers aren't necessary for querying Yahoo Finance, as they won't penalize you even if they know you're querying from Python. I include them just in case. They serve to make Yahoo Finance think that my IP address is requesting their data from Mozilla Firefox running on a Mac. 

The URL is then defined simply by taking the base page URL and adding the ticker on the end of it. The ticker is passed in when querying the price. 

  url = "https://finance.yahoo.com/quote/"+ticker

I then define the variable page, which uses the requests package to create a handle to the webpage. From there, BeautifulSoup is used to parse the handle into a bs4 object that mimics webpage HTML. 

page = requests.get(url, headers=headers)
soup = BeautifulSoup(page.text, "html.parser")

I ran out of ideas for what to name my variables, so the list that is created from using find_all to search for tags is called James. As I mentioned, Yahoo Finance's tags seem very arbitrary. In this case, the tag for the live price is "Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)". If anyone can break this down, I'd be quite grateful. For now, I'm just glad it works. 

james = soup.find_all("span", {"class": "Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)"})

From there, I use the re package to find all the text from the stock price tag, and snip the part of that text that lies between a greater-than (>) and a less-than (<), as the numeric value lies there. 

Since re.findall() always returns a list, even if there is only one result, I return james[0]. 

james = re.findall(">(.*)<", str(james))
return james[0]

This is the most important part of my code. In actual use, the last line is within an if statement that ensures that james[0] actually has anything, and if it doesn't, it has a couple of troubleshooting loops to try before giving up. Here's a quick video of the code in action:

 

The way to go about this for finding a stock's percent change is almost exactly the same, except the Yahoo Finance HTML class for percent change is "Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($negativeColor)". 


Comments

Popular posts from this blog

Tesla Continues Parabolic Rise; How Evergreen Will Play Options

Tesla has had a twelve month run that, if one can assume that a conservatively managed fund will average 6% YoY, equates to 190 years worth of gains. For much of this time, Evergreen Financial Research has pushed the case that shares of Tesla were extremely undervalued, and that the company was gripped by the lapels and held underwater by most mainstream financial news outlets. For the first time ever, Evergreen Financial Research believes that Tesla has, from a valuation standpoint, grown into its well-deserved shell. Tesla remains an interestingly valued firm; while Tesla seems to be overvalued by a fundamental perspective (EPS 1094), there is hardly a shred of a guarantee that the market will not continue to value Tesla based on the assumption that the firm succeeds in pioneering the transition to renewable energy, and eats every ICE automobile in the process. Tesla shattered many bears' expectations in Q2 of this year by posting a profit, and those who follow the stock know tha

FinTwit Roundup: 11 August 2020

Early Monday morning, the Russian Federation (more specifically, Vladimir Putin, President of Russia) announced that a new COVID-19 vaccine is not only effective, but that Putin's own daughter has already been vaccinated with it. In one of the greatest multi-generational mic-drops of all time, Russia decided to name the vaccine Sputnik V . Unsurprisingly, the news was met with great skepticism - partly because vaccine headlines have become a daily occurrence, and partly because a Russian nanobot vaccine wouldn't be the weirdest conspiracy theory that turned out to be true. Gold, silver, and other precious metals crashed today to levels not seen since ten days ago. The markets are in pure shock.    As one Twitter user points out, this is the largest negative one-day percent drop that Silver has seen since the bankruptcy of Lehman Brothers.  The -15% drop was surprising, but not as much given the macro environment. Markets have been quite lackluster for a few days now, leading so

GameStop Stock is Proving that the Rules Have Changed

Shares of GameStop Corporation have been on fire recently, and it's caught the eye of investors old and young. The majority of the gains have been attributed to the infamous WallStreetBets community on Reddit. GameStop was publicly ridiculed not more than a few months ago for not having enough long shares to be lent out for short selling. Short interest has been through the roof for a while now, and as of Friday's close, the borrowing rate for GameStop shares was $23.50, about 35% of the share price.  Some call it The Robinhood Effect, others simply call it a cramped bandwagon. Either way, this GameStop foolery has shown the power of retail investors. Generally, market movers and institutions have been the go-to scapegoat for funny business in the stock market. This, however, is a clear indication of how much influence small retail investors can have.  And who can blame GameStop short sellers? GameStop has been clinging to brick-and-mortar stores with a death grip that has drag