0

I have to make lots of post requests with varying parameters to an API. I currently for the parameter-dictionary using a loop and then run the request for each single one, which is very time consuming.

I stumbled upon the aiohttp asynchronous requests library, but it is not clear to me how to pass a variable as parameter, that will change for every request. The examples I found, all pass a constant set of headers or parameters for varying urls.

Example:

ohsome_url = 'https://api.ohsome.org/v0.9-ignite/elements/geometry/'
data = {'bpolys': ctybnds,
        'keys': key,
        'values': val}

ohsomeResponse = requests.post(ohsome_url, data=data).json()

The variables ctybnds, key, val are all extracted currently within the loop.

Thanks in advance for any insights!

Judith Levy
  • 1
  • 2

1 Answers1

0

You need to implement something like this:

import aiohttp
import asyncio


async def request(url, data):
    async with cs.get(url, json=data) as resp:
        return resp.json()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    cs = aiohttp.client.ClientSession()

    requests = [
        request(url, data)
        for url, data in []  # your varying data here
    ]

    result = loop.run_until_complete(
        asyncio.gather(*requests)
    )

    print(result)
Yurii Kramarenko
  • 1,025
  • 6
  • 16
  • thanks! I implemented it now by handing a list containing my varying parameter data over to the request function. However, I get the following errors: Unclosed client session client_session: Unclosed connector connections: ['[(, 108671.959045217)]'] connector: :20: RuntimeWarning: coroutine 'ClientResponse.json' was never awaited RuntimeWarning: Enable tracemalloc to get the object allocation traceback – Judith Levy Mar 14 '19 at 10:04
  • Also, it is not quite clear to me, if it makes a difference to use get or post request? With the requests library I always used post, so should it be cs.post(url, data=data)? Thanks so much for clarification! – Judith Levy Mar 14 '19 at 10:04
  • You can find everything you need here http://docs.aiohttp.org/en/stable/client_reference.html (Yes you can use `cs.post`) – Yurii Kramarenko Mar 14 '19 at 10:32
  • thanks for the link, problem is, that I am new to this and I don't really get it, there are seemingly tons of options. Can you maybe evolve on what your lines of code after result = do? sorry for being a pain ... – Judith Levy Mar 14 '19 at 13:05