Using return (list) vs yield

Question

I've created two enumeration methods, one which returns a list and the other which returns a yield/generator:

def enum_list(sequence, start=0):
    lst = []
    num = start
    for sequence_item in sequence:
        lst.append((num, sequence_item))
        num += 1
    return lst


def enum_generator(sequence, start=0):
    num = start
    for sequence_item in sequence:
        yield (num, sequence_item)
        num += 1

A few questions on this:

(1) Is changing a list to a generator as simple as doing:

# build via list
l = list()
for item in items:
    l.append(item)

# build via iterator
# l = list() (1) <== delete this line
for item in items:
    yield item # (2) change l.append(...) to yield ...

(2) Is "lazy evaluation" the only reason to use a generator, or are there other reasons as well?

Also see https://stackoverflow.com/questions/2776829/difference-between-pythons-generators-and-iterators — Gnudiff, Oct 21 '19 at 22:39
It's hard to understand what you are asking here.What do you mean by "changing an array (do you mean list?) to a generator?" — juanpa.arrivillaga, Oct 21 '19 at 22:41
Again, I'm not really sure what you are asking. Generators are a language construct that allow you to easily write iterators, in a way that is often easier to understand / more expressive than using the full iterator protocol. Lists and generators are two different things. Lists are containers, generators are iterators. — juanpa.arrivillaga, Oct 21 '19 at 22:50

Florian Bernard · Accepted Answer · 2019-10-21T23:34:03.337

1

(1) generator are simply created as adding yield to your iteration.

(2) Yes, for lazy evaluation. But generators are also used to create stack and queue as they can be only iterate once. This property is also exploited in context manager, by yielding the context.

edited Oct 21 '19 at 23:34

answered Oct 21 '19 at 22:38

Florian Bernard

2,561
1
9
22

1

@juanpa.arrivillaga True thank you, I did the same mistake again, confounding iterator and generator. – Florian Bernard Oct 21 '19 at 22:49
1

@juanpa.arrivillaga Not too sure why you say that. A generator class is a generator too, and with `__iter__` and `__next__` defined it does not need `yield` to be a generator. For example, https://stackoverflow.com/questions/42983569/how-to-write-a-generator-class – blhsing Oct 21 '19 at 22:54
3

@blhsing There is no such thing as "a generator class". A class can implement the *iterator protocol*, in which case, it can be a *iterator*. All generators are iterators, not all iterators are generators. If **it doesn't use yield then it is not a generator** (unless it is a generator expression). So try it yourself, with the example iterator in that linked answer, and `import types; print(isinstance(Fib(), types.GeneratorType))` – juanpa.arrivillaga Oct 21 '19 at 23:22
1

you cannot create "a generator from a class by using `__iter__` and `__next__`. – juanpa.arrivillaga Oct 21 '19 at 23:24
1

It's still not really correct. Normally, you wouldn't use `yield`, `__iter__` **and** `__next__` to create an iterator, rather, you would only use `__iter__` and `__next__` to create an iterator (**without yield** unless you want your iterator to return other iterators - generators, see the linked question in these comments). you *often* see an iterable class using a generator function to define `__iter__` because it doesn't require an additional, fulll *iterator* class definition, but that would be an *iterable*, not an *iterator* – juanpa.arrivillaga Oct 21 '19 at 23:29
2

@juanpa.arrivillaga Just looked up the official glossary for generator. You're right indeed that the precise definition of a generator is a function that yields. I guess people simply mix up iterable classes with generators so much that they are practically used interchangeably, as evident by the many usages of the phrase "generator class" on SO without any correction until now. So thanks for pointing it out and clearing it up. :-) – blhsing Oct 21 '19 at 23:59
2

@blhsing and, additionally, it is important to understand the distinction between an *iterable object* and an *iterator object*. another thing that is frequently misunderstood / glossed over. I swear I'm not just being a pedant, it is actually important. At the very least, it will make you look good in an interview :) – juanpa.arrivillaga Oct 22 '19 at 00:00
1

@juanpa.arrivillaga Yes, that's why I specifically said an iterable class, i.e. an object that defines `__iter__`, as opposed to an iterator class, i.e. an object that defines `__next__`, although they can be the same class. – blhsing Oct 22 '19 at 00:04

Gnudiff · Answer 2 · 2019-10-22T06:02:34.023

0

An additional difference in your case is that since list is created before use and generator is evaluated at each next call, the generator function can check the context and come to different result for each yield, depending on external conditions, which vary with time.

Consider pseudocode:

def alloted_time():
    while True:
         if len(global_queue)>10:
            yield 5
         else:
            yield 10

If queue is large, allot 5 mins for next person, else 10.

edited Oct 22 '19 at 06:02

answered Oct 21 '19 at 22:49

Gnudiff

4,297
1
24
25

1

To make it a *useful* generator, you'd probably want a `while True:` around the whole thing. Otherwise, with only a single `yield` no matter which code path you exercise, this is basically just a function that's inconvenient to use. – ShadowRanger Oct 22 '19 at 01:22
@ShadowRanger of course! I felt I missed something, thanks. – Gnudiff Oct 22 '19 at 06:02

Using return (list) vs yield

2 Answers2

Linked