Asyncio event loop per python process (aioprocessing, multiple event loops)

Question

I have two processes; a main process and a subprocess. The main process is running an asyncio event loop, and starts the subprocess. I want to start another asyncio event loop in the subprocess. I'm using the aioprocessing module to launch the subprocess.

The subprocess function is:

def subprocess_code():
     loop = asyncio.get_event_loop()
     @asyncio.coroutine
     def f():
        for i in range(10):
            print(i)
            yield from asyncio.sleep(1)
     loop.run_until_complete(f())

But I get an error:

    loop.run_until_complete(f())
  File "/usr/lib/python3.4/asyncio/base_events.py", line 271, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 239, in run_forever
    raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.

Is it possible to start a new, or restart the existing, asyncio event loop in the subprocess? If so, how?

process = aioprocessing.AioProcess(target=target) process.start() — solarw, Apr 17 '15 at 17:08
I think `aioprocessing` was written by our very own @dano. You could add it to the title to get his attention. — tdelaney, Apr 17 '15 at 17:20
@tdelany I found the question anyway :). I filed a bug about this back when I was working on `aioprocessing` initially: http://bugs.python.org/issue22087 — dano, Apr 17 '15 at 19:45

score 22 · Accepted Answer · answered Apr 17 '15 at 17:19

22

Sorry for disturb! I found a solution!

policy = asyncio.get_event_loop_policy()
policy.set_event_loop(policy.new_event_loop())
loop = asyncio.get_event_loop()

put this code to start new asycnio event loop inside of subprocess started from process with asyncio event loop

answered Apr 17 '15 at 17:19

solarw

441
3
10

8

No need to be sorry! [Answers to your own questions](http://stackoverflow.com/help/self-answer) are more than welcome, as they may help future visitors! – Andrea Corbellini Apr 17 '15 at 17:20
That's interesting. `aioprocessing` goes through `multiprocessing` which means that the child has the parent memory space (at least in linux). So, there is a non-active event loop that needs restarting. – tdelaney Apr 17 '15 at 17:22
I found that asyncio checks loops at threads, so new loop in new thread should be started without questions. Suppose some kind of atfork handler should remove inactive event loop automatically, to help starting new loop without tricks with policy – solarw Apr 17 '15 at 17:25
1

@solarw There's a patch in the bug I linked to above that works around the issue, but it looks like there's another patch in yet another similar bug that implements `atfork` type handlers to address it. See here: http://bugs.python.org/issue21998. There hasn't been activity on it in a few months, though. For now, what you're doing is the right way to work around the issue. – dano Apr 17 '15 at 19:55
note this broke in 3.5.3, patch is in the works:https://github.com/python/asyncio/pull/452 – amohr Mar 01 '17 at 20:07

Asyncio event loop per python process (aioprocessing, multiple event loops)

1 Answers1

Linked