Keeping try block small when catching exceptions in generator

Question

How can I keep the try block as small as possible when I have to catch an exception which can occur in a generator?

A typical situation looks like this:

for i in g():
  process(i)

If g() can raise an exception I need to catch, the first approach is this:

try:
  for i in g():
    process(i)
except SomeException as e:
  pass  # handle exception ...

But this will also catch the SomeException if it occurs in process(i) (this is what I do not want).

Is there a standard approach to handle this situation? Some kind of pattern?

What I am looking for would be something like this:

try:

  for i in g():

except SomeException as e:
  pass  # handle exception ...

    process(i)

(But this is syntactic nonsense of course.)

ecatmur · Accepted Answer · 2012-11-23T13:55:59.263

3

You could convert exceptions occurring in the inner block:

class InnerException(Exception):
  pass

try:
  for i in g():
    try:
      process(i)
    except Exception as ex:
      raise InnerException(ex)
except InnerException as ex:
  raise ex.args[0]
except SomeException as e:
  pass  # handle exception ...

Another option is to write a local generator that wraps g:

def safe_g():
  try:
    for i in g():
      yield i
  except SomeException as e:
    pass  # handle exception ...
for i in safe_g():
  process(i)

edited Nov 23 '12 at 13:55

answered Nov 23 '12 at 13:48

ecatmur

152,476
27
293
366

The other way 'round ;-) Nice idea! – Alfe Nov 23 '12 at 13:50
But is there no way to really separate the two blocks, i. e. the call of the generator and the processing of its result? – Alfe Nov 23 '12 at 13:51

score 2 · Answer 2 · answered Nov 23 '12 at 14:02

The straight-forward approach for this seems to be unwrap the for construct (which makes it impossible to catch exceptions just in the generator, due to its syntax) into its components.

gen = g()
while True:
  try:
    i = gen.next()
  except StopIteration:
    break
  process(i)

Now we can just add our expected exception to the try block:

gen = g()
while True:
  try:
    i = gen.next()
  except StopIteration:
    break
  except SomeException as e:
    pass  # handle exception ...
    break
  process(i)

Does that (besides it being ugly as hell) have disadvantages? And more: Is there a nicer solution?

(I won't accept my own answer because it being ugly, but maybe others like and upvote it.)

score 1 · Answer 3 · answered Nov 23 '12 at 13:42

1

In your generator raise a different kind of exception, that you will be able to distinguish.

class GeneratorError(Exception):
    pass

def g():
    try:
        yield <smth>
    except:
        raise GeneratorError

try:
  for i in g():
    process(i)
except GeneratorError:
    pass  # handle generator error
except SomeException as e:
  pass  # handle exception .

answered Nov 23 '12 at 13:42

warvariuc

57,116
41
173
227

It's not *my* generator but something from a lib. Do you propose to wrap it then? – Alfe Nov 23 '12 at 13:44
Just an idea... Let's what others will suggest – warvariuc Nov 23 '12 at 13:44
@Alfe you could [wrap](http://en.wikipedia.org/wiki/Adapter_pattern) the generator in your own object. – John Dvorak Nov 23 '12 at 13:45
Yes, thank you, whatever else will come :) But this does not keep the try block very small. And consider you have a large block of code instead of the simple `process(i)` which in turn also has this pattern in it; you'd need a specific `GeneratorError` for each of the nested ones. I hoped there would be something more general. – Alfe Nov 23 '12 at 13:48
@Jan: yes, but that still does not keep the try block small. I know I'm pedantic but that problem nags me for some time now and I'm looking for a solution I really like (or find out that there is none anybody can think of). – Alfe Nov 23 '12 at 13:49

score 0 · Answer 4 · answered Nov 23 '12 at 13:52

0

I don't know, if it works. You could evaluate g() into a list. I can't test it, because I don't have an iterator that throws an exception at hand.

try:
    glist = list(g())
except SomeException as e:
    pass  # handle exception ...
for i in glist:
    process(i)

answered Nov 23 '12 at 13:52

Holger

2,125
2
19
30

1

correct me if that's impossible but what if the generator is infinite in length? – John Dvorak Nov 23 '12 at 13:53
1

Right, @Jan, my first thought as well. In my usecase it is not infinite but it takes several hours and produces *massive* output. I cannot collect all that first. – Alfe Nov 23 '12 at 13:55
@Alfe several hours certainly _look_ like infinity ;-) – John Dvorak Nov 23 '12 at 13:57
And sorry @Holger, no, it does not work at all; the `list` constructor does not return anything if there is an exception thrown; there just is that exception thrown :-} – Alfe Nov 23 '12 at 14:05

Keeping try block small when catching exceptions in generator

4 Answers4