Alpha Vantage stockinfo only collects 4 dfs properly formatted, not 6

Question

I can get 4 tickers of stockinfo from Alpha Vantage before the rest of the DataFrames are not getting the stockinfo I ask for. So my resulting concatenated df gets interpreted as Nonetype (because the 4 first dfs are formatted differently than the last 2). This is not my problem. The fact that I only get 4 of my requests is... If I can fix that - the resulting concatenated df will be intact.

My code

import pandas as pd
import datetime
import requests
from alpha_vantage.timeseries import TimeSeries
import time

tickers = []

def alvan_csv(stocklist):
    api_key = 'demo'   # For use with Alpha Vantage stock-info retrieval.
    for ticker in stocklist:
        #data=requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=%s&apikey={}'.format(api_key) %(ticker))
        df = pd.read_csv('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&datatype=csv&symbol=%s&apikey={}'.format(api_key) %(ticker))#, index_col = 0)  &outputsize=full
        df['ticker'] = ticker
        tickers.append(df)
        # concatenate all the dfs
        df = pd.concat(tickers)
        print('\ndata before json parsing for === %s ===\n%s' %(ticker,df))
        df['adj_close'] = df['adjusted_close']
        del df['adjusted_close']
        df['date'] = df['timestamp']
        del df['timestamp']
        df = df[['date','ticker','adj_close','volume','dividend_amount','split_coefficient','open','high','low']] #
        df=df.sort_values(['ticker','date'], inplace=True)
        time.sleep(20.3)
    print('\ndata after col reshaping for === %s ===\n%s' %(ticker,df))
    return df


if __name__ == '__main__':
    stocklist = ['vws.co','nflx','mmm','abt','msft','aapl']

    df = alvan_csv(stocklist)

NB. Please note that to use the Alpha Vantage API, you need a free API-Key which you may optain here: https://www.alphavantage.co/support/#api-key Replace the demo API Key with your API Key to make this code work.

Any ideas as to get this to work?

Have you tried retrieving only the erranous tickers and seeing that they are on the correct format? — VegardKT, Aug 06 '18 at 07:55

score 0 · Accepted Answer · answered Aug 26 '18 at 06:55

Apparently Alpha Vantage has a pretty low fair usage allowance, where they measure no of queries pr. minute. So in effekt only the first 4 stocks are allowed at full speed. The rest of the stocks need to pause before downloading for not violating their fair-usage policy.

I have now introduced a pause between my stock-queries. At the moment I get approx 55% of my stocks, if I pause for 10 sec. between calls, and 100% if I pause for 15 seconds.

I will be testing exactly how low the pause can be set to allow for 100% of stocks to come through.

I must say compared to the super high-speed train we had at finance.yahoo.com, this strikes me as steam-train. Really really slow downloads. To get my 500 worth of tickers it takes me 2½ hours. But I guess beggars can't be choosers. This is a free service and I will manage with this.

Alpha Vantage stockinfo only collects 4 dfs properly formatted, not 6

1 Answers1