3

Today I was debugging a strange issue. The program is complex, but I have simplified the part in question to just few lines reproducing the strange behaviour.

In the example I test a random generator three times in a row. If all three tests return True, the test is completed. If not, the test must be repeated from the beginning.

Function func1 works OK. Function func2 with any() should be equivalent to func1, but it isn't. It does not work, it produces an error. The func3 is broken as well, this one is an infinite busy loop.

Where is the problem? It is legal to use yield from in other ways than value = yield from ... ? I did not found anything in the docs (so far):

When yield from is used, it treats the supplied expression as a subiterator. All values produced by that subiterator are passed directly to the caller of the current generator’s methods.

# Python 3.3 or newer
import random

def yield_random():
    if random.choice((True, False)):
        yield "OK"
        return True
    return False

def func1():
    # only this function works fine
    ok3 = False
    while not ok3:
        for i in range(3):
            ok1 = yield from yield_random()
            if not ok1:
                print("-- not ok")
                break
        else:
            print("All 3 ok !")
            ok3 = True

def func2():
    # does not work
    ok3 = False
    while not ok3:
        ok3 = all((yield from yield_random()) for i in range(3))
    print("All 3 ok !")

def func3():
    # does not work
    while any(not (yield from yield_random()) for i in range(3)):
        print("-- not ok")
    print("All 3 ok !")

for x in func1():
    print("got:", x)
Morgan Thrapp
  • 9,748
  • 3
  • 46
  • 67
VPfB
  • 14,927
  • 6
  • 41
  • 75
  • 1
    What are you trying to achieve? This code doesn't make much sense. Why are you using generator to generate single value? – Andrey Nov 16 '15 at 15:53
  • `yield "OK"; return True` that is strange. – sobolevn Nov 16 '15 at 15:55
  • As I wrote, this is a simplification derived from much more complex code. It simply does not makes sense in this form. In reality the generator is a coroutine and the yielded values are asynchronous operations to be performed by a special library. – VPfB Nov 16 '15 at 16:17

1 Answers1

2

func1 is a function returning a generator while func2 and func3 are regular functions:

>>> type(func1())
<class 'generator'>
>>> type(func2())
All 3 ok !
<class 'NoneType'>

That's because:

ok3 = all((yield from yield_random()) for i in range(3))

is actually equivalent to:

def _gen():
    for i in range(3):
        r = yield from yield_random()
        yield r
ok3 = all(_gen())

This code doesn't yield anything since the yield from statement is encapsulated in a generator. You can even run it outside of a function, in your console for instance:

>>> all((yield from yield_random()) for i in range(3))
False

Though it probably doesn't do what you expect:

>>> list((yield from yield_random()) for i in range(3))
['OK', True, False, False]
Vincent
  • 12,919
  • 1
  • 42
  • 64
  • all should not be used with side effects anyway – Andrey Nov 16 '15 at 16:20
  • @Vincent: that's a fine explanation of what is actually happenning. Do you think it is in accordance with docs? _any function containing a yield keyword is a generator function_. – VPfB Nov 16 '15 at 18:06
  • @VPfB I agree this is a bit tricky, but you wouldn't expect a function that defines an inner generator to be a generator itself. And it gets even trickier if you use `yield` inside a list comprehension, see [this thread](http://stackoverflow.com/questions/32139885/yield-in-list-comprehensions-and-generator-expressions). – Vincent Nov 16 '15 at 18:39
  • @Vincent: Interesting reading, I mean that thread including the links in it. So the main point is that nested scopes are silently created in a function. You said that using yield inside comprehensions (and generator expressions) is "tricky". I woud rather use the word "gotcha". Anyway, your helpful answer is accepted, many thanks. – VPfB Nov 16 '15 at 20:55