Bye Yahoo, and thanks for all the fish

Just a quick post in the light of a very recent event. Users of financial functions of R, MatLab, Python, or Zorro got a bad surprise in the last days. Scripts and programs based on historical price data suddenly didn’t work anymore. And our favorite free historical price data provider, Yahoo, now responds on any access to their API in this way:

No, they won’t be right back. Their engineers went fishing and, from what I hear, won’t be working on the issue anytime soon. The Yahoo Finance API is dead. Without prior announcement, Yahoo has abandoned their only remaining service that was clearly ahead of the competition.

Many scripts that I’ve posted on this blog use Yahoo for downloading price data. So here’s a quick workaround. Before running the Yahoo based script, run this one:

void main()
{
	string MyAsset;
	while(MyAsset = loop("SPY","TNT","XIV","GLD","AAPL","YHOO")) // ... enter your assets here 
	{
		string URL = strf("https://finance.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",MyAsset);
		string Content = http_transfer(URL,0);
		if(!Content) continue;
		file_write("History\\history.csv",Content,0);
		dataNew(1,0,7);
		if(!dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","History\\history.csv"))
			continue;
		printf(" %s",MyAsset);
		dataSave(1,strf("History\\%s.t6",MyAsset));
	}
}

This script downloads the data with the Google API. When you afterwards start the script with the Yahoo download, it will detect that the data is already there, and not access the Yahoo API.

Why haven’t I used the Google API already in the first place? Their data quality is not as good (many gaps and outliers), and it’s not dividend adjusted. So the results will be a bit different than results with Yahoo data. And then there’s also this ominous message:

Like the Yahoo message, this one is also wrong, but in a more positive sense. The Google Finance API is still available and was so for years, but nobody knows how long it will remain. For the moment, the above script will do, but it’s only a temporary solution. Free stock data can alternatively be downloaded from Quandl with a similar script, but for ETFs they want 50 bucks per month (I hope they didn’t bribe Yahoo and Google into abandoning their APIs!).

I’ve added the above script to the 2017 repository, in order to make all the Yahoo accessing scripts work again, although with slightly different results. You’ll need the current Zorro version 1.58 or above. And if you know a free and reliable API for ETF EOD history, please post a comment!


Addendum (June 2017): Readers mentioned AlphaVantage (www.alphavantage.co) and Stooq (www.stooq.com) as free alternatives to Yahoo. And indeed, they provide also dividend and split adjusted data. From what I’ve seen, their data has better quality than Google’s. AlphaVantage also includes the unadjusted prices, which are sometimes required for special cases. Only problem that I’ve found is that some price histories don’t go further back than 5 years. Downloading price data from AlphaVantage and Stooq was implemented in the latest Zorro version 1.60.

62 thoughts on “Bye Yahoo, and thanks for all the fish”

  1. Thanks for the link! I didn’t know this one, but indeed they also support ETFs, at least some that I tried. I’ll have to check in which way the data is adjusted.

  2. QuantQuote Free Data– QuantQuote offers free daily resolution data for the S&P500 at this web page under the Free Data tab. The data accounts for symbol changes, splits, and dividends, and is largely free of the errors found in the Yahoo data. Note, only 500 symbols are available unlike Yahoo which provides all listed symbols.

    Source: http://quant.caltech.edu/historical-stock-data.html

  3. The cookie field is not for preventing robots, but for disabling all API access by software. You can indeed step around the cookie method with some tricks, even automated, but that’s not really a professional solution. And Yahoo could easily counter. Next would be a captcha. If they don’t want us to use their API, they can effectively prevent it.

    – Alphavantage makes a good impression at first glance, but the problem – aside from the JSON format, not the best choice for financial data – is that they do not adjust for splits and dividends. However they told me they’re considering this.

  4. How can I get only historical weekly data. What parameter has to be added ?

    strf(“https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv”,MyAsset);

  5. I don’t know if the Google API has a parameter for weekly data, but you can trivially convert it from daily.

  6. I posed the qiestion to alphavantage about adjustment of their prices and they quickly reponded the following:

    We have just beta-launched the adjusted close price feature for our daily prices API. The API call, while not yet updated on our documentation page, should look something like:

    http://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=

    Best regards,

    Team Alpha Vantage

    So Alphavantage does offer and will offer dividend adjusted data.

  7. Hello, Can you contact me at support at professionaltradingsystems.com

  8. I have shifted my scripts from pandas datareader to Intrinio data web API. There are download limits for the free data, but I don’t exceed them for my needs. Data quality is good too.

  9. Also Intrinio’s EOD stock data is split and dividend adjusted. I think it’s mostly from the same vendor as Quandl’s EOD data but it also has ETF prices. Notably, it does not have mutual fund price history like Yahoo.

  10. I tried to find alternatives to the API and found several interesting services:

    EOD Histotrical Data (https://eodhistoricaldata.com/) – cheapest and with the best coverage. They have similar to Yahoo! Finance API and instructions how to migrate from Yahoo to EOD.

    Quandl and Intrinio are good, but a little bit expensive for personal usage.
    Alphavantage has very poor data (for examples, without adjusted closes).

  11. One question; How can Alpha Vantage give access for free? Indeed the are not as big as Google or Yahoo. What’s the business model? How can they sustain service costs??

  12. This is indeed an important question, since the long-term availability of their data depends on it. I hope they have a solid business model by advertising or selling premium data once they’ve built up a large user base. Maybe Denis knows more?

    – The current Zorro beta version now supports AlphaVantage as a data source.

  13. While Alpha Vantage has great data and a well-documented API, it’s pretty sketchy that they don’t describe their business model or have an “About Us” page that gives a company address and information about the actual human beings who work there.

    I’d love to use it, but it’s hard to trust it at the moment.

  14. I checked AlphaVantage’s service, its looking usefull. I am in the same worry as turtlein, After 2 months of building the system with Alphavantage, will we meet with unbearable costs?

    Dear AlphaVantage,
    What is your profit model? Do you have a gain model? It’s a very nice service, but we want it to be permanent 🙂

    Also, how can i get yours supported instrument list?

  15. I’m afraid it’s not so easy. That “crumb” in your URL is a code connected to a cookie on your PC. No cookie, no data.

  16. Problem is that it does not help to get around it. You could connect to the Yahoo main page for generating the cookie, then analyze the cookie for getting the download link. But to what avail? Yahoo will change the cookie code next month or add a captcha. Besides, the data that they still provide for manual access is not comparable to their prior service – it is not anymore dividend adjusted and full of gaps.

  17. Just stopped by since I wanted to confirm with people that Google Finance API stopped working starting today. I’ve been trying to access it all day – no luck. I’ve reprogrammed my code using alphavantage. Best substitute for real-time quotes out there I believe. Although, they could use some enabling multiple stock quote searches, everything else does a decent job for me. Opening up and closing HTTP connection delays my output a bit, but beggars can’t be choosers.

  18. Wonder what alphavantage will do now that Google api is dead? Highly unlikely for it to remain free for any amount of time. I too have concern about it being a reliable long term solution since no clearly stated business model. No doubt all those Google api calls now being replaced with alphavantage so perhaps we will soon find out.

  19. Google Finance API appears to be dead- IF you’re looking for full day (86,400 sec) intervals. You can still pull accurate and current 5 minute, 60 minute etc. data from a link like this: https://www.google.com/finance/getprices?i=300&p=8d&f=d,c,h,l,o,v&df=cpct&q=PULM

    i is interval, but change it to 86,400 (1 day) and it stops providing anything after 9/5/17.

    This is probably related to them pulling historic data. On a Google finance page for a given stock, there used to be a link to a table of historic data you can download. That link is gone. You can still access it via direct URL like https://www.google.com/finance/historical?cid=22144&startdate=Sep+8%2C+2016&enddate=Sep+7%2C+2017&num=30&ei=aQWxWYigLIKsmAHemqPIDw but again, no data after 9/5/17.

  20. Jason,

    I am aware of the same issue. There are some other alternative services online such as alphavantage or quandl. I feel frustrated about being unable to subscribe multiple stocks at the same time and just to get the market snapshot. Here is an alternative service I got, which provides market snapshots: http://hlt-capital.com/irresponsiblefeed/.

  21. I was able to download Google Historical Data for Australia just a few days ago but now all 36 countries I download have stopped. Has Google made an announcement?

  22. No, but Google seems to have stopped the D1 data service. So, the remaining free alternatives are STOOQ or Alphavantage. The C code for downloading data from Alphavantage with Zorro:

    void main()
    {
    	string MyAsset;
    	while(MyAsset = loop("SPY","TNT","XIV","GLD","AAPL","YHOO")) // ... enter your assets here 
    	{
    		StartDate = 2010;
    		EndDate = 2017;
    		assetLoad(MyAsset,FROM_AV);
    	}
    }

    For loading from STOOQ you must put a “.US” at the end of the asset name.

  23. As others have mentioned above, there is a new provider of historical data since last August called Alpha Vantage.
    My little contribution is that I have managed to create an Excel Addin called Deriscope that retrieves data from Alpha Vantage into Excel.
    It is completely free. You can download it from https://www.deriscope.com/freedownload.php
    and watch a video tutorial at https://www.youtube.com/watch?v=alWILMgeY1s
    Please note that the whole Alpha Vantage api that is consisted of 50+ technical indicators is supported.
    I have also exploited their intraday series, which are updated every minute, to get live stock prices into Excel with only one minute delay! Compare that to the 20 minute delay of Yahoo Finance!
    A tutorial with regard to live last traded prices is https://www.youtube.com/watch?v=1GoLvT2lG8I

  24. download.finance.yahoo.com died yesterday. On a Yahoo forum an administrator commenting on the 999 error message initially stated it was a technical issue and would be fixed. 2 hours ago he updated his message to say that the use of downloads.finance.yahoo.com (which has been going on for 2 decades without a yahoo whimper) violates the ToS and would not be restored.

    RIP. On to other things. I’m writing a python routine to get data from other sources. If anyone wants what I’ve got, feel free to let me know.

  25. Hey Walt, I too have run into the issue of yahoo finance not working but I’m not finding very many others on the web discussing it. I’d like to see what you come up with.

  26. Had a script for Yahoo prices, and it failed today…very depressing…thanks for all the notes here…my script was php … Hosting service was a faster link and has lots of disk space…but would look at python…

  27. Jack,

    Here’s what I’m doing. I’ve written a python3 program that collects and displays alphavantage.co data using .csv formats. Alphavantage produces much more data than I need and is slower than yahoo was. Today, I’m going to re-write the data collection module to try the json approach. I have run into one problem tracking mutual funds. It is at least 1 day behind. I don’t know how often they update their funds data, so I’m looking at a few other options. I’ll try to get a web site or possibly set up a github soon.

  28. More Info. I’ve read yahoo’s entire ToS and cannot see how using the download feature violates any of them. Their decision may be due more to risk management paranoia than anything. Their ToS clause 22 contains some express disclaimers about the accuracy of data, which I hope we were all willing to accept. I’ve been involved in the internet from the late 1970s (milnet/darpa). The internet went from the domain of a few scientists to a useful tool for everybody, to .com ads that snoop on us to send us more ads, while useful tools like yahoo are fading. I think that the current FCC position on net neutrality (they don’t like it and want to sell bandwidth to the highest bidder) will accelerate this trend. Everyone should go to the FCC web site and oppose the current proposal ending net neutrality before thanksgiving.

  29. So Ive been using the Yahoo API in an excel spreadsheet with VBA macros that fetch certain stock data… and it worked until yesterday 11/2. Anybody know of a workaround for this? Caught me completely by surprise

  30. I too am dismayed at the loss of Yahoo’s API. I’m not interested in tons of historical data, I just want to update my portfolio spreadsheet (personal use only) with the current stock or fund price. I’d appreciate any useful comments regarding a simple excel fix.

    My “current price” cell contained:
    =NUMBERVALUE(WEBSERVICE(“http://download.finance.yahoo.com/d/quotes.csv?s=”&G3&”&f=l1”)) {G3 is the cell containing the stock ticker.}

    Thanks All!

  31. Alfavantage.com does not support bulk download the way Yahoo did, i.e. stock=A,AA,AAP,AAPL,… in the url to get last quotes for let say 100 stocks. Downloading each stock in separate url is too slow. Besides, stock history is not necessary, I only need last quotes.

    Actually I’m duplicating the above comment, if someone finds the Yahoo substitute, I would appreciate.

  32. When writing your next doctoral thesis on how NOT to run a tech company please place asterisks next to Yahoo Inc. and include pictures for future generations. Discontinuation of Yahoo finance is the final straw of my use of the service (service?). I don’t need any more liberal idiotic news sites, junk mail breeders or an incomprehensible search engine…..using an old yahoo addy to fill in form….it will self destruct in 3-2-1…….

  33. Don’t blame Yahoo!. This was Verizon’s decision. It took all of 5 months from the time they bought Yahoo! to ditch something good with no notice. Typical Verizon.

  34. If you’re looking for an alternative now that both Yahoo and Google have shut down their Financial APIs, then you should check out Intrinio (www.intrinio.com). They currently offer the least expensive Real-Time and REST API for US stock prices.

    You can access it via WebSocket, or Web API. No other firm currently offers this data via WebSocket which is especially helpful as you only need a few lines of code to implement. Additionally, Intrinio’s customer service chat support is live practically 24/7.

    Good luck!

  35. If anyone is looking for instruments from exchanges in US only, including forex, I remember that barchart.com offered free API for limited request, 1000 requests per instrument per day when I checked with them about 18 months ago. You can get delayed prices of multiple instruments in a single request. They should also offer real time and unlimited request on their paid API service.
    Please check with them directly to see if they still offer the free API.

    https://www.barchart.com/ondemand/api

  36. Update:

    I’ve successfully used alphavantage.co with python retrievals. Right now it’s crude and primitive, but the key is a pretty nice set of routines written by RomelTorres, which is available for python through pip. The code is available on github (RomelTorres/alpha_vantage). I’ll work on getting something put together for importing into LibreOffice calc using this as a base, but I’ve only been using python for a couple of weeks, so I’m a babe in the woods and LibreOffice’s python macro stuff is what it is…

  37. I have been using Deriscope for free in Excel (download site at https://www.deriscope.com ) and so far works great. What I like in particular is its asynchronous fetch of stock prices from Alpha Vantage and the filtering it applies behind the scenes so that any network failures do not corrupt the displayed feeds. Stock prices are fast enough for my needs (less than one minute time delay), but fx rates suck! I hope the Alpha Vantage guys manage to fix that quickly.

Leave a Reply

Your email address will not be published. Required fields are marked *