F# continuation recursion bug

Question

I'm having an issue with a recursive function that runs into a stack overflow on larger data sets so I've attempted to rewrite the function to use continuous recursion but to say I'm new to this would be an understatement. In the below example the first function, processList, gives the desired results on a small data set. The second function, processListCont, seems to work however I know there must be a bug since when I run the same small data set through it I get different results. Would processListCont be the correct way to express the processList function or am I missing something?

open System

type Something(id) =
    member val id = id with get, set
    member val children : list<Something> = [] with get, set
    member val processed : bool = false with get, set

let rec processList (item:Something, itemList:list<Something>) =
    for child in item.children do
        let parent = itemList |> Seq.find (fun (i:Something) -> i.id = child.id)

        if parent.processed = false then
            parent.processed <- true
            processList(parent, itemList)

let processListCont (item:Something, itemList:list<Something>) =
    let rec _processListCont (item:Something, itemList:list<Something>, f) =
        for child in item.children do
            let parent = itemList |> Seq.find (fun (i:Something) -> i.id = child.id)

            if parent.processed = false then
                parent.processed <- true
                f(parent, itemList)

    _processListCont(item, itemList, (fun (item:Something, itemList:list<Something>) -> ()))

[<EntryPoint>]
let main argv =  
    // generate some data
    let count = 10000
    let idList = List.init count (fun index -> index)
    let items = [for (id) in idList -> Something id]

    let rnd = System.Random()

    for i in items do
        i.children <- List.init 100 (fun _ -> Something (rnd.Next(0, count - 1)))

    // process the list
    for i in items do
        processList(i, items)

    Console.WriteLine("Processing completed successfully")
    Console.ReadKey()
    |> ignore
    0

There are some problems that make it difficult to apply CPS to this particular design: 1) CPS is much easier to understand when each tree node has a fixed number of children, but your nodes have a variable number. 2) CPS is easier to understand in a pure functional context, but your code is imperative: processing a node causes a side-effect in the node's parent. If you're willing to consider a different design, the slides labelled "Another Example" in [this presentation](https://www.cs.princeton.edu/~dpw/courses/cos326-12/lec/11-more-cps.pdf) might be helpful to you. — Brian Berns, Aug 13 '21 at 16:52

score 2 · Answer 1 · answered Aug 13 '21 at 20:12

The main issue is that you are calling the continuation f in the body of the for loop, but your non-tail-recursive version makes a recurisve call here.

This is tricky because you want to make a recursive call and the continuation should be "run the rest of the for loop". To express this, you'll need to use pattern matching instead of for loop.

I did not have a small example to test this, but I think something like this should do the trick:

let rec processListCont (item:Something, itemList:list<Something>) cont =
    let rec loop (children:list<Something>) cont =
        match children with 
        | child::tail ->
            let parent = itemList |> Seq.find (fun (i:Something) -> i.id = child.id)

            if parent.processed = false then
                parent.processed <- true
                processListCont (parent, itemList) (fun () -> loop tail cont)
        | [] -> cont ()
    loop item.children cont

I must've not distilled the problem down correctly as I keep getting a stack overflow when I try this. I'm not getting anywhere so I guess I will retreat to my original C# code that uses a while loop to avoid the stack overflow but gives a less accurate result. — Scott Wilson, Aug 17 '21 at 18:20

Koenig Lear · Answer 2 · 2021-08-13T20:55:56.227

Your code is unidiomatic in F# nonetheless consider the following example.

Suppose you want to add a list of numbers. You could write a function like this:

let rec add (l:int list) :int = 
    match l with
    | [] -> 0
    | x::xs -> x + (add xs)

but this would overflow the stack very quickly. Instead you could use cps to allow the code to become tail recursive:

type 
  cont = int -> int


let rec add2 (l:int list) (k:cont):int = 
    match l with
    | [] -> k 0
    | x::xs -> add2 xs (fun a -> k (a + x))

which you can use like this:

printfn "%i" (add2 [1..10000] id)

In a similar fashion you could rewrite your function like this:

type cont2 = Something list->unit

let rec p (item:Something, itemList:list<Something>) (k:cont2) =
    match item.children with
    | [] -> k []
    | child::xs -> 
        let parent = itemList |> Seq.find (fun (i:Something) -> i.id = child.id)

        if parent.processed = false then
            parent.processed <- true
            p (parent, itemList) (fun _ ->k xs)
        else
           k xs

let p2 (item:Something,itemList:Something list) = p (item,itemList) ignore

and you can call it like this:

for i in items do
    p2(i, items)

F# continuation recursion bug

2 Answers2