Turning any single-argument function into a generator function?

Question

In David Beazley's talk on generators, he shows how to create a generator function from any single-argument function thus:

def generate(func):
    def gen_func(s):
        for item in s:
            yield func(item)
    return gen_func

and illustrates it with math.sqrt:

gen_sqrt = generate(math.sqrt)
for x in gen_sqrt(xrange(100)):
    print x

So why does:

gen_sum = generate(sum)
for x in gen_sum([1,2,3,4,5]):
    print x

produce:

TypeError                                 Traceback (most recent call last)
<ipython-input-73-ef521f2bbfc8> in <module>()
      1 gen_sum = generate(sum)
----> 2 for x in gen_sum(1):
      3     print x

<ipython-input-50-3c0ba12c2429> in gen_func(s)
      1 def generate(func):
      2     def gen_func(s): # closure
----> 3         for item in s:
      4             yield func(item)
      5     return gen_func

TypeError: 'int' object is not iterable

Is it more accurate to say that the function being a single-argument function is a necessary but insufficient condition for this approach to work? And that the other necessary condition is that the single argument must be a single item (and not a sequence)?

a sum takes two arguments ... the function has to be one that takes only one argument ... what do you expect the above to actually do? `math.sqrt(25)` vs `sum(25)` I think it becomes clear what the issue is .. — Joran Beasley, Jan 22 '16 at 21:40
The exception you show the traceback of seems to be coming from a different call to `gen_sum` than the one you show above (which will also raise a `TypeError` with the same message, but at a different part of the code and for a different reason). — Blckknght, Jan 22 '16 at 21:45
It must be a single-argument function **that accepts the kind of values *you're passing!*** `gen_sum([(1, 2), (3, 4, 5)])` would be fine, for example. — jonrsharpe, Jan 22 '16 at 21:45
@JoranBeasley `sum` takes a single argument. In my case above, that single argument is a list. — Pyderman, Jan 22 '16 at 22:14
@Pyderman no, because you're *iterating over that list* to give the single arguments. — jonrsharpe, Jan 23 '16 at 15:26

score 3 · Answer 1 · answered Jan 22 '16 at 21:43

You're passing a list whose elements are the wrong type. Of course it's not going to work, for the same reason that gen_sqrt(['a', 's', 'd', 'f']) wouldn't have worked.

You need to pass gen_sum a list of things it makes sense to call sum on, such as other lists:

for x in gen_sum([[1, 2], [3, 4]]):
    print x

Output:

3
7

score 2 · Answer 2 · answered Jan 22 '16 at 21:42

You are correct, both are necessary requirements:

def generate(func):
    def gen_func(s):
        for item in s:          # <--- requires s to be interable
            yield func(item)    # <--- requires func to accept a single argument
    return gen_func

So in

gen_sum = generate(func)
for x in gen_sum(s):
    print x

func must accept a single argument and s must be iterable.

score 1 · Answer 3 · answered Jan 22 '16 at 21:45

generate is the generator version of map.

map(lambda x: x*2, range(5))   [0, 2, 4, 6, 8]

It takes the input range and applies the function the each of the element in the range.

generate does the same, but it doesn't return a list. It yields the result of each transformation.

Now take a look at your example. What would be the first result? sum(1).

But sum expects a list, not an integer, hence the error message.

Turning any single-argument function into a generator function?

3 Answers3