Python - Unable to use multiple event loops with aiohttp

Question

I am trying to make a code similar to this one work.


class FooFoo:
    def __init__(self):
        loop = asyncio.get_event_loop()
        self.value = loop.run_until_complete(self.async_work())

    async def async_work(self):
        return 10


def build_server():
    app = web.Application()
    app.router.add_route('GET', '/', hello)
    web.run_app(app, 'localhost', '12000')

async def hello(request):
    foo = await FooFoo()
    return web.json_response{'ip': request.remote, 'value': foo.value}

Doing a curl http://127.0.0.1:/ yields this error:

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib64/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/usr/local/lib64/python3.6/site-packages/aiohttp/web_urldispatcher.py", line 155, in handler_wrapper
    result = old_handler(request)
  File "test.py", line 36, in hello
    foo = asyncio.get_event_loop().run_until_complete(FooFoo())
  File "test.py", line 24, in __init__
    self.value = loop.run_until_complete(self.async_work())
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 471, in run_until_complete
    self.run_forever()
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 425, in run_forever
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running

This happens because the server is running on the event loop and FooFoo is wants to rerun the loop.

Solutions I've tried :

Changing the hello function into a synchronous one with : foo = asyncio.get_event_loop().run_until_complete(FooFoo()), gives the same error.
Multiprocessing: Running the server instance on another thread/process using libraries : pool, threading, multiprocessing, aioprocessing, this gives a different error:

RuntimeError: Cannot run the event loop while another loop is running

This SO question gives the same error as multiprocesssing.
This SO question as well.

I need to be able to run multiple loops, or if you have a better solution I'd take it.

If it helps I am using python 3.6.8

Is there any reason why you feel the need to start an event loop in a constructor, blocking it, then trying to await the payload of the event loop again? Are you looking for a factory function perhaps? — MisterMiyagi, Sep 10 '19 at 04:19

constt · Answer 1 · 2019-09-10T04:02:50.837

If you change the logic slightly to something like this:

import asyncio

from aiohttp import web


class FooFoo(object):

    async def async_work(self):
        await asyncio.sleep(1)
        self.value = 10


async def hello(request):
    foo = FooFoo()
    await foo.async_work()

    return web.json_response({'ip': request.remote, 'value': foo.value})


def build_server():
    app = web.Application()
    app.router.add_route('GET', '/', hello)
    web.run_app(app, host='localhost', port=8080)

you'll get it working.

UPDATE Given the details from the OP comments:

the FooFoo class is a client that generates a token via asynchronous requests to a server. The user isn't supposed to instantiate FooFoo, then generate a token later.

I'd suggest using a builder pattern which is used to simplify creation of complex objects. I think this pattern fits to this case just fine. In order to apply the pattern, we add a new FooFooBuilder class:

async def request_token():
    await asyncio.sleep(1)

    return 42


class FooFooBuilder(object):
    @staticmethod
    async def build():
        token = await request_token()

        # Do token validation and error handling. 

        return FooFoo(token)

The FooFoo class will take the token as its argument during instantiation:

class FooFoo(object):
    def __init__(self, token):
        self.value = token

So that the hello request handler will change to:

async def hello(request):
    foo = await FooFooBuilder.build()

    return web.json_response({'ip': request.remote, 'value': foo.value})

With this approach, there is no need to use multiple event loops to get things done.

What I gave was a minimal example, I need some stuff to be done inside the `__init__` function, so I cannot remove it and add another instruction to execute said work. Everything has to be done during instantiation, if I could make the `__init__` `async` then it might fix this specific issue, although it will create many more and anyway it's illegal to use asynchronous magic functions. — Reda Drissi, Sep 05 '19 at 09:28
Can you tell me why is this a strict requirement to run async code during instantiation? In the case, you have to wait for some resources to be able to build `FooFoo` instance, then why not to use a factory pattern and collect all the resources asynchronously first, and then, once everything is ready, instantiate the `FooFoo` class? — constt, Sep 05 '19 at 09:45
the `FooFoo` class is a client that generates a token via asynchronous requests to a server. The user isn't supposed to instantiate `FooFoo`, then generate a token later. The environment needs to stay controlled, so some things are done at instantiation time. — Reda Drissi, Sep 05 '19 at 11:14
OK, then why not to create a `FooFooBuilder` class which will request a token and then pass it to `FooFoo` constructor and return its instance? Something like `await foo = FooFooBuilder.build()` where inside this static method a token is requested asynchronously. What I'm trying to say is that you might be complicating things a bit and hitting the wall because of it :) — constt, Sep 05 '19 at 14:30

Python - Unable to use multiple event loops with aiohttp

1 Answers1