Bye Yahoo, and thanks for all the fish

Just a quick post in the light of a very recent event. Users of financial functions of R, MatLab, Python, or Zorro got a bad surprise in the last days. Scripts and programs based on historical price data suddenly didn’t work anymore. And our favorite free historical price data provider, Yahoo, now responds on any access to their API in this way:

No, they won’t be right back. Their engineers went fishing and, from what I hear, won’t start working on the issue anytime soon. The Yahoo Finance API is dead. Without prior announcement, Yahoo has abandoned their only remaining service that was clearly ahead of the competition.

Many scripts that I’ve posted on this blog use Yahoo for downloading price data. So here’s a quick workaround. Before running the Yahoo based script, run this one:

void main()
{
	string MyAsset;
	while(MyAsset = loop("SPY","TNT","XIV","GLD","AAPL","YHOO")) // ... enter your assets here 
	{
		string URL = strf("https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",MyAsset);
		string Content = http_transfer(URL,0);
		if(!Content) continue;
		file_write("History\\history.csv",Content,0);
		dataNew(1,0,7);
		if(!dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","History\\history.csv"))
			continue;
		printf(" %s",MyAsset);
		dataSave(1,strf("History\\%s.t6",MyAsset));
	}
}

This script downloads the data with the Google API. When you afterwards start the script with the Yahoo download, it will detect that the data is already there, and not access the Yahoo API.

Why haven’t I used the Google API already in the first place? Their data quality is not as good (many gaps and outliers), and it’s not dividend adjusted. So the results will be a bit different than results with Yahoo data. And then there’s also this ominous message:

Like the Yahoo message, this one is also wrong, but in a more positive sense. The Google Finance API is still available and was so for years, but nobody knows how long it will remain. For the moment, the above script will do, but it’s only a temporary solution. Free stock data can alternatively be downloaded from Quandl with a similar script, but for ETFs they want 50 bucks per month (I hope they didn’t bribe Yahoo and Google into abandoning their APIs!).

I’ve added the above script to the 2017 repository, in order to make all the Yahoo accessing scripts work again, although with slightly different results. You’ll need the current Zorro version 1.58 or above. And if you know a free and reliable API for ETF EOD history, please post a comment!


Addendum (June 2017): Readers mentioned AlphaVantage (www.alphavantage.co) and Stooq (www.stooq.com) as free alternatives to Yahoo. And indeed, they provide also dividend and split adjusted data. From what I’ve seen, their data has better quality than Google’s. AlphaVantage also includes the unadjusted prices, which are sometimes required for special cases. Only problem that I’ve found is that some price histories don’t go further back than 5 years. Downloading price data from AlphaVantage and Stooq was implemented in the latest Zorro version 1.60.

40 thoughts on “Bye Yahoo, and thanks for all the fish”

  1. Thanks for the link! I didn’t know this one, but indeed they also support ETFs, at least some that I tried. I’ll have to check in which way the data is adjusted.

  2. QuantQuote Free Data– QuantQuote offers free daily resolution data for the S&P500 at this web page under the Free Data tab. The data accounts for symbol changes, splits, and dividends, and is largely free of the errors found in the Yahoo data. Note, only 500 symbols are available unlike Yahoo which provides all listed symbols.

    Source: http://quant.caltech.edu/historical-stock-data.html

  3. The cookie field is not for preventing robots, but for disabling all API access by software. You can indeed step around the cookie method with some tricks, even automated, but that’s not really a professional solution. And Yahoo could easily counter. Next would be a captcha. If they don’t want us to use their API, they can effectively prevent it.

    – Alphavantage makes a good impression at first glance, but the problem – aside from the JSON format, not the best choice for financial data – is that they do not adjust for splits and dividends. However they told me they’re considering this.

  4. How can I get only historical weekly data. What parameter has to be added ?

    strf(“https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv”,MyAsset);

  5. I don’t know if the Google API has a parameter for weekly data, but you can trivially convert it from daily.

  6. I posed the qiestion to alphavantage about adjustment of their prices and they quickly reponded the following:

    We have just beta-launched the adjusted close price feature for our daily prices API. The API call, while not yet updated on our documentation page, should look something like:

    http://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&apikey=

    Best regards,

    Team Alpha Vantage

    So Alphavantage does offer and will offer dividend adjusted data.

  7. Hello, Can you contact me at support at professionaltradingsystems.com

  8. I have shifted my scripts from pandas datareader to Intrinio data web API. There are download limits for the free data, but I don’t exceed them for my needs. Data quality is good too.

  9. Also Intrinio’s EOD stock data is split and dividend adjusted. I think it’s mostly from the same vendor as Quandl’s EOD data but it also has ETF prices. Notably, it does not have mutual fund price history like Yahoo.

  10. I tried to find alternatives to the API and found several interesting services:

    EOD Histotrical Data (https://eodhistoricaldata.com/) – cheapest and with the best coverage. They have similar to Yahoo! Finance API and instructions how to migrate from Yahoo to EOD.

    Quandl and Intrinio are good, but a little bit expensive for personal usage.
    Alphavantage has very poor data (for examples, without adjusted closes).

  11. One question; How can Alpha Vantage give access for free? Indeed the are not as big as Google or Yahoo. What’s the business model? How can they sustain service costs??

  12. This is indeed an important question, since the long-term availability of their data depends on it. I hope they have a solid business model by advertising or selling premium data once they’ve built up a large user base. Maybe Denis knows more?

    – The current Zorro beta version now supports AlphaVantage as a data source.

  13. While Alpha Vantage has great data and a well-documented API, it’s pretty sketchy that they don’t describe their business model or have an “About Us” page that gives a company address and information about the actual human beings who work there.

    I’d love to use it, but it’s hard to trust it at the moment.

  14. I checked AlphaVantage’s service, its looking usefull. I am in the same worry as turtlein, After 2 months of building the system with Alphavantage, will we meet with unbearable costs?

    Dear AlphaVantage,
    What is your profit model? Do you have a gain model? It’s a very nice service, but we want it to be permanent 🙂

    Also, how can i get yours supported instrument list?

  15. I’m afraid it’s not so easy. That “crumb” in your URL is a code connected to a cookie on your PC. No cookie, no data.

  16. Problem is that it does not help to get around it. You could connect to the Yahoo main page for generating the cookie, then analyze the cookie for getting the download link. But to what avail? Yahoo will change the cookie code next month or add a captcha. Besides, the data that they still provide for manual access is not comparable to their prior service – it is not anymore dividend adjusted and full of gaps.

  17. Just stopped by since I wanted to confirm with people that Google Finance API stopped working starting today. I’ve been trying to access it all day – no luck. I’ve reprogrammed my code using alphavantage. Best substitute for real-time quotes out there I believe. Although, they could use some enabling multiple stock quote searches, everything else does a decent job for me. Opening up and closing HTTP connection delays my output a bit, but beggars can’t be choosers.

  18. Wonder what alphavantage will do now that Google api is dead? Highly unlikely for it to remain free for any amount of time. I too have concern about it being a reliable long term solution since no clearly stated business model. No doubt all those Google api calls now being replaced with alphavantage so perhaps we will soon find out.

  19. Google Finance API appears to be dead- IF you’re looking for full day (86,400 sec) intervals. You can still pull accurate and current 5 minute, 60 minute etc. data from a link like this: https://www.google.com/finance/getprices?i=300&p=8d&f=d,c,h,l,o,v&df=cpct&q=PULM

    i is interval, but change it to 86,400 (1 day) and it stops providing anything after 9/5/17.

    This is probably related to them pulling historic data. On a Google finance page for a given stock, there used to be a link to a table of historic data you can download. That link is gone. You can still access it via direct URL like https://www.google.com/finance/historical?cid=22144&startdate=Sep+8%2C+2016&enddate=Sep+7%2C+2017&num=30&ei=aQWxWYigLIKsmAHemqPIDw but again, no data after 9/5/17.

  20. Jason,

    I am aware of the same issue. There are some other alternative services online such as alphavantage or quandl. I feel frustrated about being unable to subscribe multiple stocks at the same time and just to get the market snapshot. Here is an alternative service I got, which provides market snapshots: http://hlt-capital.com/irresponsiblefeed/.

  21. I was able to download Google Historical Data for Australia just a few days ago but now all 36 countries I download have stopped. Has Google made an announcement?

  22. No, but Google seems to have stopped the D1 data service. So, the remaining free alternatives are STOOQ or Alphavantage. The C code for downloading data from Alphavantage with Zorro:

    void main()
    {
    	string MyAsset;
    	while(MyAsset = loop("SPY","TNT","XIV","GLD","AAPL","YHOO")) // ... enter your assets here 
    	{
    		StartDate = 2010;
    		EndDate = 2017;
    		assetLoad(MyAsset,FROM_AV);
    	}
    }

    For loading from STOOQ you must put a “.US” at the end of the asset name.

Leave a Reply

Your email address will not be published. Required fields are marked *