-1

I am working on an option quotes downloader from finance yahoo using Python 3.9 with asyncio, aiohttp:

  • there is a need to develop a chained download sequence because the 1st request for each option ticker gets all the expiration dates and the 2nd request gets the option quotes per ticker and expiration date;
  • when downloading only 2 tickers everything is working but when downloading 300-400 tickers then the data are not correct;
  • I assume not all requests are processed.

Could you advise how to find this bottleneck? Here is the code:

def get_tasks(session, tickers):
    tasks = []
    for ticker in tickers:
        print(ticker)
        url_options1 = f"https://query2.finance.yahoo.com/v7/finance/options/{ticker}?"
        tasks.append(asyncio.create_task(session.get(url_options1, headers={'User-Agent': ua}, ssl=True)))
    return tasks


def get_tasks2(session, ticker, expiries):
    tasks2 = []
    for expiry in expiries:
        url_options2 = f"https://query2.finance.yahoo.com/v7/finance/options/{ticker}?&date={expiry}"
        tasks2.append(asyncio.create_task(session.get(url_options2, headers={'User-Agent': ua}, ssl=True)))
    return tasks2

async def get_optiondata(tickers):

    results1 = []

    async with aiohttp.ClientSession() as session:
        tasks = get_tasks(session, tickers)
        responses = await asyncio.gather(*tasks)
        for response in responses:
            results1.append(await response.json())

    return results1


async def get_optiondata2(ticker, expiries):

    results2 = []

    async with aiohttp.ClientSession() as session:
        tasks2 = get_tasks2(session, ticker, expiries)
        responses = await asyncio.gather(*tasks2)
        for response in responses:
            results2.append(await response.json())
    return results2

datas = asyncio.run(get_optiondata(usTickers))

results = []
tickers = []
expiries = []

for i in range(len(datas)):
    try:
        symbol = datas[i]['optionChain']['result'][0]['underlyingSymbol']
        tickers.append(symbol)
        if datas[i]['optionChain']['result'][0]['expirationDates']:
            expiries.append(datas[i]['optionChain']['result'][0]['expirationDates'])

    except IndexError as e:
        print(f"{i} {e}")

for i in range(len(tickers)):

    datas2 = asyncio.run(get_optiondata2(tickers[i], expiries[i]), debug=True)
wovano
  • 4,543
  • 5
  • 22
  • 49
user2882782
  • 19
  • 2
  • What does " the data are not correct" mean? What exactly is happening? – dirn Oct 16 '21 at 03:50
  • Hi "data are not correct" - means that the request response does not contain all expiration date. e.g. ticker "A" - 7 expiration dates but only 2 are returned by the response – user2882782 Oct 16 '21 at 17:25
  • Are you not getting the expiration dates back from the API or are you dropping them somewhere in your code? – dirn Oct 16 '21 at 21:59

1 Answers1

0

You can modify the query parameters to that endpoint to retrieve all expiration dates for a ticker:

url = 'https://query2.finance.yahoo.com/v7/finance/options/{ticker}?getAllData=true'
putty
  • 744
  • 1
  • 6
  • 14