httpx - send multiple requests from dynamic list

Question

I would like to send multiple request at once to the API. But I do not know how to trigger the requests because part of the URL is loaded from dynamic list which can have different length every time.

Right now I am looping through the list and sending the request one by one but that is kinda slow.

for url_param in parameters:
    r = httpx.get(f"http://my-api/{url_param}")

Is there any way to trigger all the requests at once with httpx and asyncio?

use async requests https://www.python-httpx.org/async/#making-async-requests — Anentropic, Aug 08 '22 at 09:00
I know I have to use async requests, but how I can do it with the dynamic list? — pipikej, Aug 08 '22 at 09:03
with `asyncio.gather` as shown here https://stackoverflow.com/a/61537571/202168 — Anentropic, Aug 08 '22 at 11:01

score 3 · Accepted Answer · edited Nov 24 '22 at 11:20

3

Load all the request in an array and send them together in an async client session

import asyncio
import httpx

async def do_tasks():
    async with httpx.AsyncClient() as client:
        tasks = [client.get(f"http://my-api/{url_param}") for url_param in parameters]
        result = await asyncio.gather(*tasks)

edited Nov 24 '22 at 11:20

Ben

305
5
12

answered Aug 10 '22 at 08:08

Abinav R

365
2
16

httpx - send multiple requests from dynamic list

1 Answers1