Generators and Lists in Python

Question

Be kind, I'm still learning python (but getting better). I've looked at the other posts regarding generators, and haven't found an answer to my specific question. Sorry, if I missed it.

So I am writing a method that acts as a generator. I can make it work, but not the way I want it to. I'm trying to understand generators.

If I write the following:

def genfunc(self):
    """
    self.some_lst is defined in __init__ as a list of tuples. e.g [(1,2),(2,3)]
    """
    yield (x for x in self.some_lst)

I get

Line 73: TypeError: '<invalid type>' does not support indexing

however, if I write it as:

def genfunc()
    """
    self.some_lst is defined in __init__ as a list of tuples. e.g [(1,2),(2,3)]
    """
    for x in self.some_lst:
        yield x

Everything works fine.

Two questions: 1. What am I fundamentally missing about generators? and 2. Is there a way to write this in one line as I tried (but failed) to do?

I know there are some SOers just waiting to help this newb out. Thanks in advance.

unutbu · Answer 1 · 2014-07-07T18:58:00.240

3

In Python2, you need to write

def genfunc():
    for x in self.some_lst:
        yield x

but in Python3 you could write

def genfunc():
    yield from self.some_lst

See PEP380 -- Syntax for Delegating to a Subgenerator.

yield (x for x in self.some_lst)

does not work since (x for x in self.some_lst) is an object -- a generator expression. So the yield expression merely yields that one object, not the items inside that generator expression.

edited Jul 07 '14 at 18:58

answered Jul 07 '14 at 18:50

unutbu

842,883
184
1,785
1,677

Thanks, my confusion comes from the fact that "print max(x for x in range(10)) works. I guess it is the yield, that makes the difference. – PerryDaPlatypus Jul 07 '14 at 18:52
@user3641109: that is a generator expression too; `max()` loops over it. – Martijn Pieters Jul 07 '14 at 18:53
@user3641109: `max(x for x in range(10))` is equivalent to `max( (x for x in range(10)) )`. So `max` is being applied to a generator expression. Some operators, such a `max` and `sum` [allow the parentheses around the generator expression to be omitted](http://stackoverflow.com/questions/4799459/why-can-you-omit-the-surrounding-parentheses-for-generators-in-python-when-passi). – unutbu Jul 07 '14 at 19:00

score 3 · Accepted Answer · answered Jul 07 '14 at 18:50

You are mixing generator expressions with your generator.

yield yields whatever follows, and you have put a generator expression there. Just like a generator function, this produces a generator object:

>>> (x for x in some_lst)
<generator object <genexpr> at 0x100544aa0>

This is what you yielded.

Because you essentially yielded another generator, you couldn't index it, as it was not yielding the 2-value tuples you were expecting.

Since the generator expression itself produces a generator, you could just return that generator expression directly, without using yield:

def genfunc(self):
    """
    self.some_lst is defined in __init__ as a list of tuples. e.g [(1,2),(2,3)]
    """
    return (x for x in self.some_lst)

Generators and Lists in Python

2 Answers2