Bye Yahoo, and thanks for all the fish

Just a quick post in the light of a very recent event. Users of financial functions of R, MatLab, Python, or Zorro got a bad surprise in the last days. Scripts and programs based on historical price data suddenly didn’t work anymore. And our favorite free historical price data provider, Yahoo, now responds on any access to their API in this way:

No, they won’t be right back. Their engineers went fishing and, from what I hear, won’t start working anytime soon. The Yahoo Finance API is dead. Without prior announcement, Yahoo has abandoned their only remaining service that was clearly ahead of the competition.

Many scripts that I’ve posted on this blog use Yahoo for downloading price data. So here’s a quick workaround. Before running the Yahoo based script, run this one:

void main()
{
	string MyAsset;
	while(MyAsset = loop("SPY","TNT","XIV","GLD","AAPL","YHOO")) // ... enter your assets here 
	{
		string URL = strf("https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",MyAsset);
		string Content = http_transfer(URL,0);
		if(!Content) continue;
		file_write("History\\history.csv",Content,0);
		dataNew(1,0,7);
		if(!dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","History\\history.csv"))
			continue;
		printf(" %s",MyAsset);
		dataSave(1,strf("History\\%s.t6",MyAsset));
	}
}

This script downloads the data with the Google API. When you afterwards start the script with the Yahoo download, it will detect that the data is already there, and not access the Yahoo API.

Why haven’t I used the Google API already in the first place? Their data quality is not as good (many gaps and outliers), and it’s not dividend adjusted. So the results will be a bit different than results with Yahoo data. And then there’s also this ominous message:

Like the Yahoo message, this one is also wrong, but in a more positive sense. The Google Finance API is still available and was so for years, but nobody knows how long it will remain. For the moment, the above script will do, but it’s only a temporary solution. Free stock data can alternatively be downloaded from Quandl with a similar script, but for ETFs they want 50 bucks per month (I hope they didn’t bribe Yahoo and Google into abandoning their APIs!).

I’ve added the above script to the 2017 repository, in order to make all the Yahoo accessing scripts work again, although with slightly different results. You’ll need the current Zorro version 1.58 or above. And if you know a free and reliable API for ETF EOD history, please post a comment!

 

14 thoughts on “Bye Yahoo, and thanks for all the fish”

  1. Thanks for the link! I didn’t know this one, but indeed they also support ETFs, at least some that I tried. I’ll have to check in which way the data is adjusted.

  2. QuantQuote Free Data– QuantQuote offers free daily resolution data for the S&P500 at this web page under the Free Data tab. The data accounts for symbol changes, splits, and dividends, and is largely free of the errors found in the Yahoo data. Note, only 500 symbols are available unlike Yahoo which provides all listed symbols.

    Source: http://quant.caltech.edu/historical-stock-data.html

  3. The cookie field is not for preventing robots, but for disabling all API access by software. You can indeed step around the cookie method with some tricks, even automated, but that’s not really a professional solution. And Yahoo could easily counter. Next would be a captcha. If they don’t want us to use their API, they can effectively prevent it.

    – Alphavantage makes a good impression at first glance, but the problem – aside from the JSON format, not the best choice for financial data – is that they do not adjust for splits and dividends.

  4. How can I get only historical weekly data. What parameter has to be added ?

    strf(“https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv”,MyAsset);

  5. I don’t know if the Google API has a parameter for weekly data, but you can trivially convert it from daily.

Leave a Reply

Your email address will not be published. Required fields are marked *