coroutine was never awaited, async multithreading

Question

async def existance(s, name):
    async with s.head(f"https://example.com/{name}") as r1:
        if r1.status == 404:
            print('wow i worked')

async def process(names):
    with ThreadPoolExecutor(max_workers=3) as executor:
        async with aiohttp.ClientSession() as s:
            loop = asyncio.get_event_loop()
            tasks = []
            for name in names:
                if len(name) >= 5 and len(name) < 16 and name.isalnum():
                    task = loop.run_in_executor(
                        executor,
                        existance,
                        *(s, name)
                        )
                    tasks.append(task)
            return await asyncio.gather(*tasks)

while True:
    start_time = time.time()
    loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(process(names))
    loop.run_until_complete(future)

I'm using the code above to try and split my tasks created across multiple threads while checking them all asynchronously.

I'm getting this error:

RuntimeWarning: coroutine 'existance' was never awaited
  future = asyncio.ensure_future(process(names))

I'm still somewhat of a python beginner and I can't really figure out what I should change here to get the result I want. Any help is appreciated and I'm sorry if this is a duplicate question.

[`run_in_executor`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor) doesn't take coroutines, it takes normal functions. You're trying to mix multithreading and asyncio in a way that won't work. You should either use the thread pool to do blocking IO, or use asyncio and coroutines. — Carcigenicate, Oct 21 '20 at 14:22

score 2 · Answer 1 · answered Oct 22 '20 at 09:57

Since existance is async, you don't need threads at all, so you can implement process like this:

async def process(names):
    async with aiohttp.ClientSession() as s:
        tasks = []
        for name in names:
            if len(name) >= 5 and len(name) < 16 and name.isalnum():
                tasks.append(existance(s, name))
        return await asyncio.gather(*tasks)

If existance is calling some synchronous slow function which might block the event loop (such as BeautifulSoup parsing), you can fix that by running just that function through a thread pool. Introduce run_in_executor locally like this:

# instead of:
# result = slow_calculation(arg1, arg2)   # sync/slow code
# use this:
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, slow_calculation, arg1, arg2)

But don't attempt to call async functions through run_in_executor, that won't work.

coroutine was never awaited, async multithreading

1 Answers1