3

In a Python program I'm writing, I've built up a linked list using a dictionary which maps each node to its successor (with the last node mapped to None).

(Actually, the dictionary holds what Wikipedia tells me is called a spaghetti stack, which is a tree where each node is linked to its parent but not its children. This means there are many partially-overlapping paths from the leaf nodes to the root node. I only care about one of those paths, starting from a specific leaf node. This doesn't really matter for the question, other than to rule out any solution that involves iterating over all the elements in the dictionary.)

I need to pass this list to another function as an iterable. I know I can do this using a generator function (see code below), but it seems like there ought to be a built-in function to make the iterator I need in one line (or a perhaps a generator expression). I've done a bit of searching through the documentation, but there's nothing in the itertools or functools modules seems applicable, and I'm not sure where else to look.

Here's the generator function I have now. The outer function can be eliminated (inlined) but the inner generator seems to be the only simple way to make an iterable for the data:

def makeListGenerator(nextDict, start):
    def gen(node):
        while node:
            yield node
            node = nextDict[node]
    return gen(start)

It seems like there should be a pattern for this sort of generator, but I'm not sure what it would be called. Here's a generic version:

def makeGenericGenerator(nextFunc, continueFunc, start):
    def gen(value):
         while continueFunc(value):
             yield value
             value = nextFunc(value)
    return gen(start)

I could implement the specific version using this one using the call:

makeGenericGenerator(lambda v: nextDict[v], bool, start)

Does something like that already exist in the Python standard library?

senderle
  • 145,869
  • 36
  • 209
  • 233
Blckknght
  • 100,903
  • 11
  • 120
  • 169
  • 3
    I don't really see why this would be generalised more than the generator. It's not really a common operation. Generator expressions exist for making generators from iterables, but in your case, that's not useful. The generator is pretty much it. – Gareth Latty Jun 02 '12 at 22:05
  • it is possible (if result may be a list), but not recommended e.g., http://stackoverflow.com/a/7742030/4279 – jfs Jun 03 '12 at 12:02

2 Answers2

7

The essential problem you face is that every time another value is taken from your iterable, your iterable has to remember that value, so that it knows how to generate the next value. In other words, your iterable needs to maintain its own state.

That means that, whether or not there's a good answer to your question, using a generator is probably the right solution -- because that's exactly what generators were made for! The whole point of generators is that they save their state between calls to their next method, and that's exactly what you need.

Generator expressions, on the other hand, are better for stateless transformations. A lot of people try to shoehorn state into them, and it's generally kind of ugly. I actually spent a little while trying to do it for your case, and couldn't get a generator expression to work. I finally found something that sort of works. This uses the little-known callable_iterator version of iter:

>>> d = {1:2, 2:3, 3:4, 4:5, 5:None}
>>> list(iter(lambda st=[1]: st.__setitem__(0, d[st[0]]) or st[0], None))
[2, 3, 4, 5]

I present this not as an answer, but as a demonstration of why this doesn't work very well.

senderle
  • 145,869
  • 36
  • 209
  • 233
  • Agreed with senderle. There isn't anything in the standard library which fits that exactly. Although you can solve the problem in other ways, with or without generators, they'll likely all be quite a bit more complicated or hard to read than what you already have. – the paul Jun 02 '12 at 22:36
  • Oh my. I was looking at `iter` trying to figure out what callable to put in it, too. Didn't think of this black magic trick, though. It's cool and ugly at the same time, nice stuff :) – Lev Levitsky Jun 02 '12 at 22:41
  • Thanks for the explanation, and the crazy `iter` expression! Clearly the generator I have is what I actually want in the end. – Blckknght Jun 02 '12 at 23:11
2
nodes = {
    'A':'B',
    'B':'C',
    'C':'D',
    'D':None
}

print (lambda f: lambda g: f(f,g)) (lambda f,x: [x] + f(f, nodes[x]) if x else [])('A')
# ['A', 'B', 'C', 'D']
georg
  • 211,518
  • 52
  • 313
  • 390
  • I'm not even sure I want to [understand](http://www.youtube.com/watch?v=oENQ2jlHpfo) this... if I can. – Lev Levitsky Jun 02 '12 at 23:47
  • 2
    @LevLevitsky, it uses a [fixed point combinator](http://en.wikipedia.org/wiki/Fixed-point_combinator#Example_in_Python) to produce an anonymous recursive function. It's equivalent to `def iterate_d(x): return [x] + iterate_d(nodes[x]) if x else []`. I'd do this instead, since Python doesn't require curried functions: `(lambda f, g: f(f,g))(lambda f, x: [x] + f(f, nodes[x]) if x else [], 'A')`. But then, I wouldn't do it at all, because fortunately, Python isn't restricted to [lambda calculus](http://en.wikipedia.org/wiki/Lambda_calculus)! But the named recursive version is nice; i'd +1 it. – senderle Jun 03 '12 at 14:49
  • Thanks @senderle, I'll try to wrap my head around this slowly :) – Lev Levitsky Jun 03 '12 at 15:41