A Python contextmanager gotcha

Dear lazy web...

I've had this code sitting around as a wtf.py for a while. I've been meaning to understand what's going on and write a blog post about it for a while, but I'm lacking the time. Now that I have a few minutes, I actually sat down to look at it and I think I figured it out:

from contextlib import contextmanager


@contextmanager
def bad():
    print('in the context manager')
    try:
        print("yielding value")
        yield 'value'
    finally:
        return print('cleaning up')


@contextmanager
def good():
    print('in the context manager')
    try:
        print("yielding value")
        yield 'value'
    finally:
        print('cleaning up')


with bad() as v:
    print('got v = %s' % v)
    raise Exception('exception not raised!')  # SILENCED!

print("this code is reached")
with good() as v:
    print('got v = %s' % v)
    raise Exception('expection normally raised')

print("NOT REACHED (expected)")

For those, like me, who need a walkthrough, here's what the above does:

1. define a bad context manager (the things you use with with statements) with contextlib.contextmanager) which:

   1. prints a debug statement
   2. return a value
   3. then returns and prints a debug statement

2. define a good context manager in much the same way, except it doesn't return, it just prints statement

3. use the bad context manager to show how it bypasses an exception

4. use the good context manager to show how it correctly raises the exception

The output of this code (in Debian 11 bullseye, Python 3.9.2) is:

in the context manager
yielding value
got v = value
cleaning up
this code is reached
in the context manager
yielding value
got v = value
cleaning up
Traceback (most recent call last):
  File "/home/anarcat/wikis/anarc.at/wtf.py", line 31, in <module>
    raise Exception('expection normally raised')
Exception: expection normally raised

What is surprising to me, with this code, is not only does the exception not get raised, but also the return statement doesn't seem to actually execute, or at least not in the parent scope: if it would, this code is reached wouldn't be printed and the rest of the code wouldn't run either.

So what's going on here? Now I know that I should be careful with return in my context manager, but why? And why is it silencing the exception?

The reason why it's being silenced is this little chunk in the with documentation:

If the suite was exited due to an exception, and the return value from the exit() method was false, the exception is reraised. If the return value was true, the exception is suppressed, and execution continues with the statement following the with statement.

This feels a little too magic. If you write a context manager with __exit__(), you're kind of forced to lookup again what that API is. But the contextmanager decorator hides that away and it's easy to make that mistake...

Credits to the Python tips book for teaching me about that trick in the first place.