How to safely handle an exception inside a context manager

Question

I think I've read that exceptions inside a with do not allow __exit__ to be call correctly. If I am wrong on this note, pardon my ignorance.

So I have some pseudo code here, my goal is to use a lock context that upon __enter__ logs a start datetime and returns a lock id, and upon __exit__ records an end datetime and releases the lock:

def main():
    raise Exception

with cron.lock() as lockid:
    print('Got lock: %i' % lockid)
    main()

How can I still raise errors in addition to existing the context safely?

Note: I intentionally raise the base exception in this pseudo-code as I want to exit safely upon any exception, not just expected exceptions.

Note: Alternative/standard concurrency prevention methods are irrelevant, I want to apply this knowledge to any general context management. I do not know if different contexts have different quirks.

PS. Is the finally block relevant?

Where are you getting this information? It is my understanding that `with` contexts are *designed for* dealing with cleanup if any unhandled exceptions occur. — Joel Cornett, Jan 26 '15 at 20:16
@JoelCornett I think I got the information from SO pages dealing specifically with nested contexts. — ThorSummoner, Jan 26 '15 at 20:42

score 65 · Accepted Answer · edited Jun 20 '20 at 09:12

The __exit__ method is called as normal if the context manager is broken by an exception. In fact, the parameters passed to __exit__ all have to do with handling this case! From the docs:

object.__exit__(self, exc_type, exc_value, traceback)

Exit the runtime context related to this object. The parameters describe the exception that caused the context to be exited. If the context was exited without an exception, all three arguments will be None.

If an exception is supplied, and the method wishes to suppress the exception (i.e., prevent it from being propagated), it should return a true value. Otherwise, the exception will be processed normally upon exit from this method.

Note that __exit__() methods should not reraise the passed-in exception; this is the caller’s responsibility.

So you can see that the __exit__ method will be executed and then, by default, any exception will be re-raised after exiting the context manager. You can test this yourself by creating a simple context manager and breaking it with an exception:

DummyContextManager(object):
    def __enter__(self):
        print('Entering...')
    def __exit__(self, exc_type, exc_value, traceback):
        print('Exiting...')  
        # If we returned True here, any exception would be suppressed!

with DummyContextManager() as foo:
    raise Exception()

When you run this code, you should see everything you want (might be out of order since print tends to end up in the middle of tracebacks):

Entering...
Exiting...
Traceback (most recent call last):
  File "C:\foo.py", line 8, in <module>
    raise Exception()
Exception

Note: for generators with `@contextlib.contextmanager` one [has to](https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager) use `try ... finally` statement to make sure resources are properly freed because exceptions raised inside `with` block are captured by `contextmanager` and re-raised *inside* generator body. I.e. code after the `yield` statement will not be called if exception is raised inside such a `with` block. — Ben Usman, Aug 04 '18 at 03:42

score 18 · Answer 2 · answered Sep 30 '19 at 10:55

The best practice when using @contextlib.contextmanager was not quite clear to me from the above answer. I followed the link in the comment from @BenUsman.

If you are writing a context manager you must wrap the yield in try-finally block:

from contextlib import contextmanager

@contextmanager
def managed_resource(*args, **kwds):
    # Code to acquire resource, e.g.:
    resource = acquire_resource(*args, **kwds)
    try:
        yield resource
    finally:
        # Code to release resource, e.g.:
        release_resource(resource)

>>> with managed_resource(timeout=3600) as resource:
...     # Resource is released at the end of this block,
...     # even if code in the block raises an exception

How to safely handle an exception inside a context manager

2 Answers2

Linked