Is it possible to "interrupt" a recursive function and continue it later?

Question

I have a function some_result = treesearch(node) (a variant of Monte-Carlo Tree Search) that recursively searches a large tree. It decides the order in which to traverse the tree via next_node = expensive_heuristic(node) and then propagates the result at the leaf back up the tree. I have to perform many such searches and expensive_heuristic(...) can be efficiently computed in batches, e.g., via SIMD.

So my idea is to create a list with all the searches/root nodes, batch compute expensive_heuristic to select the "direction" of traversal, and repeat this until I find a leaf and can propagate the result back up the tree.

Of course, I can re-write my search using a queue instead of a recursion (keeping the history around), but I am curious if it is possible to keep the recursive style. Can I "interrupt" a recursion and put it into a list/queue to continue it at a later stage in python?

If you've already computed the heuristic for all nodes why do you need to interrupt the recursion? — kubatucka, Jul 03 '20 at 09:15
@ShivamJha yes, I like that idea; I am not 100% how this will look like since I have to feed the result of `expensive_heuristic` back into the respective generator. It might be possible though. — FirefoxMetzger, Jul 03 '20 at 09:18
@kubatucka I am not computing the heuristic for all nodes; my tree has more than 5e57 nodes; hence the Monte-Carlo simulation part. — FirefoxMetzger, Jul 03 '20 at 09:20
You might consider studying how the standard library `os.walk` works. It, too, handles a tree structure (nested directories in a filesystem), with a recursive algorithm, using a generator that allows the client code to influence future steps. — Karl Knechtel, Jul 03 '20 at 09:35
@KarlKnechtel pretty sure it does that by just `yield`ing the same `list` of dirs that it will continue to use internally. More generally, though, with a generator you can `.send` values in to them, if you want to use them a coroutines, which will probably offer more flexibility. I've never really taken advantage of that feature though, and would probably just use a queue instead of recursion here to begin with — juanpa.arrivillaga, Jul 03 '20 at 09:55

FirefoxMetzger · Answer 1 · 2020-07-05T14:28:10.997

It is possible with yield, yield from, and send; thanks juanpa.arrivillaga for the suggestion in the comments.

The code below searches a random binary tree for up to 10 nodes and returns the largest value in it. It interrupts the generation/search whenever it has to compute a heuristic, which is used to direct the search.

import random


class Node(object):
    def __init__(self, heuristic_value):
        self.children = {"left": None, "right": None}
        self.child_values = heuristic_value
        self.value = random.randint(0, 100)
        self.terminal = random.random() > 0.9

    def add_leaf(self):
        if self.terminal:
            return self.value

        direction = self.select()
        if self.children[direction]:
            value = yield from self.children[direction].add_leaf()
        else:
            value = yield from self.expand(direction)
        value = self.backup(direction, value)

        return value

    def select(self):
        if self.child_values["left"] > self.child_values["right"]:
            return "left"
        else:
            return "right"

    def expand(self, direction):
        heuristic_value = yield
        child = Node(heuristic_value)
        self.children[direction] = child
        return child.value

    def backup(self, direction, value):
        self.child_values[direction] = value
        if value > self.value:
            return value
        else:
            return self.value


def heuristic():
    return {"left": random.randint(0, 100), "right": random.randint(0, 100)}


tree = Node(heuristic())
for _ in range(10):
    gen = tree.add_leaf()
    value = None
    while True:
        try:
            gen.send(value)
            value = heuristic()
        except StopIteration as value:
            max_val = value
            break

print(f"Largest Value: {max_val}")

Is it possible to "interrupt" a recursive function and continue it later?

1 Answers1