2

I am confused by interaction between Python-2 iterators and exceptions.

Specifically, given the following code:

def gen_int():
    if not hasattr(gen_int,"x"):
        gen_int.x = 0
    while True:
        gen_int.x += 1
        yield gen_int.x

def gen_odd():
    for x in gen_int():
        if x % 2:
            yield x
        else:
            raise ValueError("gen_odd",x)

(please assume the above are out of my control!), I write

def gen_all():
    it = gen_odd()
    while True:
        try:
            yield it.next()
        except ValueError as exn:
            print exn
            func_name, x = exn
            assert func_name == "gen_odd"
            yield x

in the hope of recovering the full stream generated by gen_int.

However, the iteration stops after the first exception:

def gen_test(top):
    for x in gen_all():
        print x
        if x > top:
            break

Here are 3 invocations:

>>> gen_test(20)
1
('gen_odd', 2)
2
>>> gen_test(20)
3
('gen_odd', 4)
4
>>> gen_test(20)
5
('gen_odd', 6)
6

The question is: How do I modify gen_all so that gen_test will print all ints below top?

PS. apparently, the exception in gen_odd works as return - it marks the iterator as exhausted. is this really the case? is there a workaround?

Moses Koledoye
  • 77,341
  • 8
  • 133
  • 139
sds
  • 58,617
  • 29
  • 161
  • 278
  • 2
    you can use pass or continue in your try / except – midori Oct 26 '16 at 16:13
  • @tinySandy: could you please be more specific? – sds Oct 26 '16 at 16:16
  • Possible duplicate of [python catch exception and continue try block](http://stackoverflow.com/questions/19522990/python-catch-exception-and-continue-try-block) – midori Oct 26 '16 at 16:18
  • @tinySandy: no, this is not a dupe of that question. – sds Oct 26 '16 at 16:20
  • The `it.next` call is raising an Exception afterwards after it handled the exception (on the next iteration). If you add prints around that `it.next` call the print before will be executed three times. Not sure what exception occurs though. – xZise Oct 26 '16 at 16:22
  • 1
    Why do you `raise` a `ValueError` in the first place in `gen_odd`? – Chris_Rands Oct 26 '16 at 16:23
  • 1
    @Chris_Rands: as I said, `gen_odd` is _out of my control_ – sds Oct 26 '16 at 16:25
  • 1
    `gen_int` is a strange generator. It saves global state that each new instance uses. The other 99.99% of generators in the wild don't work this way. – tdelaney Oct 26 '16 at 16:42

1 Answers1

4

Reassign gen_odd() in the except block:

def gen_all():
    it = gen_odd()
    while True:
        try:
            yield it.next()
        except ValueError as exn:
            print exn
            func_name, x = exn
            assert func_name == "gen_odd"
            yield x
            it = gen_odd() # here

The generator function gen_odd stops once that ValueError exception is raised. You have to recall the function to create another gen function object after the previous one stops. gen_odd picks up from where it stopped since the yielded values from gen_int are bound to the function object; states are saved, else this would not have worked.

>>> gen_test(5)
1
('gen_odd', 2)
2
3
('gen_odd', 4)
4
5
('gen_odd', 6)
6
Moses Koledoye
  • 77,341
  • 8
  • 133
  • 139
  • 1
    This is a great answer but it only works because `gen_int` works off of a singleton `gen_int.x` which means it is saving state outside of the generator. It would not work for the most generators that raise exceptions. – tdelaney Oct 26 '16 at 16:40
  • @tdelaney Yes, I figured the yielded values are bound to the function object, else it would have been very different and probably more difficult – Moses Koledoye Oct 26 '16 at 16:42