2

I wrote this simple code to fetch some urls using asyncio and aiohttp. 

import asyncio
import aiohttp


async def fetch(url, session):
    async with session.get(url) as response:
        return await response.read()


async def fetch_all(urls):
    tasks = []
    async with aiohttp.ClientSession() as session:
        for url in urls:
            task = asyncio.create_task(fetch(url, session))
            tasks.append(task)

        return await asyncio.gather(*tasks)


if __name__ == '__main__':
    res = asyncio.run(fetch_all(['http://www.youtube.com', 'http://www.google.com']))
    print(res)

I'm using Python 3.8 and PyCharm and it works fine on MacOS. If I try to run this on Windows it still works but, after printing the result, it gives me 

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x036ED2B0>
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python38-32\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Program Files (x86)\Python38-32\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files (x86)\Python38-32\lib\asyncio\base_events.py", line 719, in call_soon
    self._check_closed()
  File "C:\Program Files (x86)\Python38-32\lib\asyncio\base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

However, if I replace asyncio.run() with

loop = asyncio.get_event_loop()
res = loop.run_until_complete(fetch_all(['http://www.youtube.com', 'http://www.google.com']))
print(res)

it works without problems. Is it ok to use this option? Why is this happening?

namerand
  • 43
  • 3
  • The code returns exactly what it has to return, so I don't think this is the problem. In the documentation is said that `asyncio.run` returns the result of the coroutine https://docs.python.org/3/library/asyncio-task.html#running-an-asyncio-program – namerand May 05 '20 at 11:25
  • I didn't say it is issue. And btw I did a mistake. Indeed my comment is wrong. Sorry for that. I'll remove it. – Raphael Medaer May 05 '20 at 11:29
  • This looks like a bug in asyncio (or possibly in aiohttp) on Windows. Could you try to narrow it down to use just one host, and remove `gather`, and so on? Also, can you reproduce it without `aiohttp`, e.g. if you use `asyncio.open_connection('www.google.com', 80)` and write the request directly into the TCP stream? – user4815162342 May 05 '20 at 13:37
  • Thanks for the answers, finally I found this https://github.com/aio-libs/aiohttp/issues/4324 and it seems to be an aiohttp bug, but the default event loop for Unix doesn't consider it a Runtime Error, while the event loop used in Windows yes. I'm wondering if simply ignore this will be a good idea or not – namerand May 05 '20 at 14:23

0 Answers0