Is StopIteration raised in the mapping function of python 3 map() handled incorrectly?

Question

Consider the following example:

def fn(x):
    if x > 2:
        raise StopIteration
    return x
results = list(map(fn, range(5)))
print(results)

When I run this with python 2, I get what I expected:

Traceback (most recent call last):
  File "example.py", line 5, in <module>
    results = list(map(fn, range(5)))
  File "example.py", line 3, in fn
    raise StopIteration
StopIteration

However, if I run it with python 3, the program does not end with the StopIteration exception. It prints the following result:

[0, 1, 2]

The map function in python 3 (specifically python 3.5.1) seems to catch and handle the StopIteration exception as though the provided iterable has thrown it. Is this a bug?

`map` doesn't return a list in Py3 (it returns an iterator). — thebjorn, May 21 '16 at 11:56

score 1 · Answer 1 · answered May 21 '16 at 15:22

As @thebjorn said in comment, the error is not raised from map() but from list(), because map() in Python 3 is lazy (that's the technical term): it does not call the function itself.

Instead, it returns an iterator, which is then consumed by list().

Let's take this example:

>>> def fn(x):
...     if x > 2:
...         raise Exception()
...     return x
... 
>>> results = map(fn, range(5))
>>> results = list(results)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in fn
Exception

As you can see, the exception is raised when calling list(), because list() reads the iterator, which causes your custom function to be called.

score 1 · Answer 2 · answered May 21 '16 at 20:03

I believe it's quite clear why this happens: when fn throws, python misinterprets this as end-of-iteration that should be thrown by iterator, when there are no more elements to iterate over (I believe you know that this exception plays this special role in iteration protocol).

I guess that Python2's map first pulls the value out of the iterator in the try/catch block and only then processes it by fn (no try/catch here). Python3 (lazy) map probably simply does nothing about try/catching the errors i.e. it implicitly rethrows the exception no matter if it is produced by iterator, or fn.

Is this a bug in Python3? First note that fixing this would probably cost quite a lot of performance of map as every pull from iterator would have to be try-catched. This said, the harmless fix would probably be to explicitly disallow usage of StopIteration for other purposes than desired.

It's possible, that such note is even there in some PEP :)

Is StopIteration raised in the mapping function of python 3 map() handled incorrectly?

2 Answers2