2

Python 3 has introduced generator-like objects to be returned upon calling range() and zip(). The object returned acts like a generator and can be iterated through once but doesn't 'print' well, much like the enumerate() return argument.

I was perplexed to see, however, that they are distinct object types and do not belong to types.GeneratorType, or at least this is what the types module shows. A function that would run e.g. expecting a generator would not detect them. What is their inheritance? Do they belong to a main "generator" structure, so that they e.g. could be identified along with other generators?

import types

a = [1,2,3]
b = [4,5,6]

# create some generator-type objects
obj_zip = zip(a,b)
obj_enu = enumerate(a)
obj_r = range(10)

print(type(obj_zip))
print(type(obj_enu))
print(type(obj_r))

# checking against types.GeneratorType returns False
print(isinstance(obj_zip,types.GeneratorType))
print(isinstance(obj_enu,types.GeneratorType))
print(isinstance(obj_r,types.GeneratorType))

# checking against their own distinct object types returns True
print(isinstance(obj_zip,zip))
martineau
  • 119,623
  • 25
  • 170
  • 301
dbouz
  • 779
  • 9
  • 14
  • 4
    `range` objects aren't a generator, they're an immutable sequence type; you can iterate them repeatedly, so they definitely shouldn't be of `GeneratorType`. – ShadowRanger Oct 24 '18 at 13:39

3 Answers3

5

Per the GeneratorType docs:

types.GeneratorType

The type of generator-iterator objects, created by generator functions.

Generator functions are a specific thing in the language; it means functions that use yield or yield from (or generator expressions, which are just a shorthand for inline generator functions). It's a subset of the set of iterators (all things that you can call next() on to get a new value), which is in turn a subset of iterables (all things that you can call iter() on to get an iterator; iterators themselves are iterables, where iter(iterator) behaves as the identity function).

Basically, if you're testing for "can I loop over this?", test isinstance(obj, collections.abc.Iterable). If you're checking "is this an exhaustible iterator?" (that is, will I exhaust it by looping over it?), test either isinstance(obj, collections.abc.Iterator) or for the duck-typing based approach, test iter(obj) is obj (the invariants on iterators require that iter(iterator) yield the original iterator object unchanged).

Note that range is not a generator or iterator. Per the docs:

Rather than being a function, range is actually an immutable sequence type, as documented in Ranges and Sequence Types — list, tuple, range.

Being an immutable sequence type means it is an iterable, but that's it. The fact that it is usually used as if it were an iterator is irrelevant; if it were an iterator, the second loop here would never execute:

r = range(3)
for i in r:
    print("First", i)
for i in r:
    print("Second", i)

but it works just fine, because each (implicit) call to iter(r) returns a new iterator based on the same underlying iterable.

Community
  • 1
  • 1
ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
  • Perfect, this is what I wanted, a clarification between Iterable, Iterator and Generator. So `enumerate()` and `zip()` create **iterators** (i.e. that can be exhausted) but not **generators**. – dbouz Oct 24 '18 at 13:58
0

The documentation says that enumerate is functionally equivalent to a generator. Actually it is implemented in C and returns an iterator, not a generator as described in Does enumerate() produce a generator object. Generators and iterables are almost the same. This is explained in detail in Difference between Python's Generators and Iterators.

I'm assuming you're trying to solve a real problem, like finding out if you can iterate over something. To solve that you can test if something is an instance of collections.Iterable.

a = enumerate([1,2,3])
isinstance(a, collections.Iterable)
>>> True
tback
  • 11,138
  • 7
  • 47
  • 71
  • "iterable" as a category is a superset of "generator" (all generators are iterable, but not all iterables are generators), so that's not all that useful. – ShadowRanger Oct 24 '18 at 13:42
  • Well I am trying to create a function that is running over iterables twice. So, in case the iterable is a generator, it tries to clone a generator (using `itertools.tee`), but I'm having a problem checking against generators if the object passed belongs to these generator-type objects that have a special type. – dbouz Oct 24 '18 at 13:50
0

not meant to be a full answer (ShadowRanger answer already explains everything) but just to state that types.GeneratorType is really a very limited type as shown in types.py source:

def _g():
    yield 1
GeneratorType = type(_g())

it only scopes the type of generator functions. Other "generator-like" objects don't use yield so they're not a match.

Jean-François Fabre
  • 137,073
  • 23
  • 153
  • 219