How to handle exceptions from any task in an event loop?

Question

I would like to stop the program, i.e. the event loop, if any task anywhere raises an unhandled exception. The normal behaviour is that you get a Task exception was never retrieved error message and the individual task stops, but the loop continues to run all other tasks.

For example:

import asyncio

async def foo():
    while True:
        await asyncio.sleep(1)
        print("hi")

async def bug():
    await asyncio.sleep(2)
    raise TypeError

loop = asyncio.get_event_loop()
loop.create_task(foo())
loop.create_task(bug())
loop.run_forever()

Output:

"hi"
"hi"
Task exception was never retrieved
future: <Task finished coro=<bug() done, defined at <...>:9> exception=TypeError()>
Traceback (most recent call last):
  File "<ipython-input-4-bd8651340a75>", line 11, in bug
    raise TypeError
TypeError
"hi"
"hi"
"hi"
...

My project contains many dozens of coroutines spread over many files which add each other to the loop with loop.create_task(foo()) so because they don't await each other, you cannot simply handle one or two entry point coroutines in the main file and have errors bubble up.

score 2 · Accepted Answer · answered Dec 09 '18 at 09:14

2

I think I've found what I'm looking for with loop.set_exception_handler(handler)

answered Dec 09 '18 at 09:14

jsstuball

4,104
7
33
63

1

That's it, but be careful, though, the exception handler is [sometimes called](https://stackoverflow.com/a/52021212/1600898) for exceptions that are actually caught. If you control the tasks being spawned, the alternative is to catch any exceptions thrown by the task (except perhaps `CancelledErrror`) and propagate them to a global future, as shown [here](https://stackoverflow.com/a/53691009/1600898). – user4815162342 Dec 09 '18 at 09:40
Thanks @user4815162342. Both alternative methods are available to me, namely the `set_exception_handler` or the await global future solution. I like the former option as I'm single-threaded and it seems conceptually cleaner but should I be concerned by unreliability and other library bugs similar to those in the link? – jsstuball Dec 09 '18 at 12:33
1

Personally I'd prefer to catch the exceptions raised by the tasks I create and propagate them myself. Judging by the SSL issue, it seems that there is code out there that uses `set_exception_handler` as a substitute for logging. This is really a perfect use case for the new `TaskGroup` concept. – user4815162342 Dec 09 '18 at 21:02

How to handle exceptions from any task in an event loop?

1 Answers1