How to type hint a generator in Python 3?

Question

According to PEP-484, we should be able to type hinting a generator function as follows:

from typing import Generator

def generate() -> Generator[int, None, None]:
    for i in range(10):
        yield i

for i in generate():
    print(i)

However, the list comprehension gives the following error in PyCharm.

Expected 'collections.Iterable', got 'Generator[int, None, None]' instead less... (⌘F1)

Any idea why PyCharm is considering this as error?

---

A few clarification after reading some answers. I am using PyCharm Community Edition 2016.3.2 (the latest version) and have imported the typing.Generator (updated in the code). The above code runs just fine, but PyCharm considers this an error:

So, I'm wondering if this is actually an error or an unsupported feature in PyCharm.

Answer 1

You need to import the typing module. As per docs:

The return type of generator functions can be annotated by the generic type Generator[yield_type, send_type, return_type] provided by typing.py module

Try this way instead:

from typing import Generator


def generate() -> Generator[int, None, None]:
    for i in range(10):
        yield i

The above will have the desired result:

l = [i for i in generate()]

Output:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

---

As pointed out in the comments, you might not use the last version of PyCharm. Try switching to version 2016.3.2 and you might be fine. Unfortunately this is a well-known bug, as per @AshwiniChaudhary comment.

More, the reported issue (for the last version of PyCharm) was submitted on December, last year. They probably fixed it and pushed the modifications into the same version.

Answer 2

This isn't a direct answer to the question, but I think it is a better solution.

I'm using the typing specification below, using Iterator[int] instead of Generator. The validation is OK. I think it is a lot clearer. It better describes the code intention and is recommended by Python docs.

from typing import Iterator

def generate() -> Iterator[int]:
    for i in range(10):
        yield i

It would also allow future refactorings if you change your Generator for a list or other iterable.

I'm using Visual Studio Code with PyLance for typing validation. PyCharm mypy should have the same behavior.

If you are using Python 3.10 or superior, change the import command above to:

from collections.abc import Iterator

Answer 3

The Iterator, Generator and Iterable are slightly different in details and carry different type of information, so understanding the difference might help choosing correct one for your code:

• Iterator is simply an object that implements the iterator protocol.

• Generator is a more specific as it not only indicates that the function returns an Iterator, but also that it is implemented using a generator (i.e., a function using yield). Generator functions return iterators.

• Finally Iterable, which is even more general than Iterator as Iterable can be any object that can be iterated over which also includes lists or tuples (and also iterators).

So how to type hint mainly depends on what your code is really doing. I'd use Iterable to indicate that a function returns an object that can be iterated over, without specifying the exact type of iterator. I'd use Iterator for functions that return an iterator, without specifying what type of iterator exactly it is (so it can be a list but can also be Generator). And finally I'd use Generator to explicitly convey that a function returns a generator and possibly specify the types of values that can be sent to or returned from the generator.