try-finally in Python 3 generator

Question

I have met a snippet of Python 3 code:

def gen():
    try:
        while True:
            yield 1
    finally:
        print("stop")

print(next(gen()))

After I run it, I thought at first that the output should be:

But actually the result is:

stop
1

How can this happen? What happened under the hood?

If I run for i in gen(): print(i), there will be an infinite loop which is what I expected. What is the difference between for and next here?

The relevant [line from the documentation](https://docs.python.org/3/reference/expressions.html#yield-expressions): "Yield expressions are allowed anywhere in a try construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, allowing any pending finally clauses to execute." — Patrick Haugh, May 09 '19 at 15:59

Vince W. · Accepted Answer · 2019-05-09T16:13:19.927

13

The finally clause is being executed on garbage collection of the generator object.

Consider the following two scenarios:

def gen():
    try:
        while True:
            yield 1
    finally:
        print("stop")

g1 = gen(); print('first time')
print(next(g1))
g2 = gen(); print('second time')  # no stop will be printed because we haven't hit the finally clause yet

def gen():
    try:
        while True:
            yield 1
    finally:
        print("stop")

g = gen(); print('first time')
print(next(g))
g = gen(); print('second time')   # stop will be printed when the first object g was assigned to is garbage collected

edited May 09 '19 at 16:13

answered May 09 '19 at 15:57

Vince W.

3,561
3
31
59

I think your point is right. But your example code shows right outcome only in interactive interpreter. – ruanhao May 09 '19 at 16:35
not sure what you are doing, but I just put both blocks of text into a file and ran them. Actually, I would have expected to see two `stop`s printed for both cases, as I would have expected two objects to get garbage collected, but I only see one in both cases. Feels like another question in the works... – Vince W. May 09 '19 at 16:41
again, I'm not sure I see what you are doing, but now I understand my confusion about my comment. The try block is never entered until the next method is called. So while both g1 and g2 are being garbage collected, only g1 has entered the try block, so upon exit, the finally statement will be called, whereas g2 has not yet entered the try block – Vince W. May 09 '19 at 16:48

score 7 · Answer 2 · answered May 09 '19 at 15:59

The loop terminates when the generator is closed, which happens automatically if you don't save a reference to it. Once that happens, the try statement guarantees that the finally block is executed before the generator object is garbage collected. Compare:

>>> next(gen())
stop
1

with

>>> x = gen()
>>> next(x)
1

score 0 · Answer 3 · answered Feb 03 '23 at 06:15

I'm answering an old question for future readers.

The finally clause in a generator will be executed either when the interpreter reaches it while executing code inside the generator, or when the generator is finalized(deleted) if the interpreter didn't reached it while executing code inside the generator. You can read the reference.(See the paragraph starting with "Yield expressions are allowed anywhere".)

The following example demonstrates this.

def gen(name):
    try:
        yield None
        print('after yield', name)
    finally:
        print('after finally', name)

g1 = gen('g1')
next(g1)
next(g1, None)
print('before del g1')
del g1

g2 = gen('g2')
next(g2)
print('before del g2')
del g2

This will output the following.

after yield g1
after finally g1
before del g1
before del g2
after finally g2

For the second question on the for loop, I'll give an example.

for e in gen():
    print(e)

The above code is equivalent to the following code.

iterator = gen()
while True:
    try:
        e = next(iterator)
    except StopIteration:
        break
    print(e)
del iterator

try-finally in Python 3 generator

3 Answers3

Linked

Related