F# continuations goes on StackOverflowException

Question

Hi guys I'm implementing an F# function that takes two lists of type : (int*float) list. These two lists have different lentgths. The int element of the couple is an increasing code. What I wanted to do is create a new list that will contain a couple (int*float) for each two elements of the two lists that have the same code. It's important to note that codes in lists are in increasing order. These lists are probably a little long, like 2-3000 elements., so I tried to implement this function using continuation passing style in order to avoid StackOverflowExceptions. but sadly i failed.

This is the function, i hope you will give me any hints!

let identifiedDifference list1 list2 =
    let rec produceResult (l1, l2) k =
        match l1,l2 with
            | [],[] 
            | _,[]
            | [],_ -> k []
            | (code,rate:float)::xs, (code2,rate2)::ys -> 
                if code = code2 
                    then
                        produceResult (xs, ys) (fun c -> (code,Math.Abs(rate-rate2))::(k c))
                    elif code > code2
                        then produceResult (l1, ys) k
                        else produceResult (xs, l2) k
    produceResult (list1, list2) id

I've done something wrong?

desco · Answer 1 · 2011-08-19T16:55:22.883

The problem lies in this line

produceResult (xs, ys) (fun c -> (code,Math.Abs(rate-rate2))::(k c))

Here you invoke continuation but this call is not tail because you still need to cons (code,Math.Abs(rate-rate2)) to the result of (k c)

I guess you can build result list from the inside out and just reverse final result:

let identifiedDifference list1 list2 =
    let rec produceResult (l1, l2) k =
        match l1,l2 with
            | [],[] 
            | _,[]
            | [],_ -> k []
            | (code,rate:float)::xs, (code2,rate2)::ys -> 
                if code = code2 
                    then
                        produceResult (xs, ys) (fun c -> k((code,Math.Abs(rate-rate2))::c))
                    elif code > code2
                        then produceResult (l1, ys) k
                        else produceResult (xs, l2) k
    produceResult (list1, list2) List.rev

EDIT: after second look I think CPS is not needed here and using accumulator should do the trick:

let identifiedDifference list1 list2 = 
    let rec run l1 l2 acc = 
        match l1, l2 with
        | [], _ | _, [] -> List.rev acc
        | (code1, rate1 : float)::xs, (code2, rate2)::ys ->
            if code1 = code2 then
                run xs ys ((code1, abs (rate1 - rate2))::acc)
            elif code1 > code2 then
                run l1 ys acc
            else
                run xs l2 acc
    run list1 list2 []

Using the accumulator will output the results in reverse order, which his current implementation does also (which I assume isn't intentional). — Daniel, Aug 19 '11 at 17:52

score 2 · Accepted Answer · answered Aug 19 '11 at 16:54

2

(fun c -> (code,Math.Abs(rate-rate2))::(k c))

should be

(fun c ->  k ((code,Math.Abs(rate-rate2))::c))

to make it tail-recursive:

let identifiedDifference list1 list2 =
    let rec produceResult (l1, l2) k =
        match l1,l2 with
            | [],[] 
            | _,[]
            | [],_ -> k []
            | (code,rate:float)::xs, (code2,rate2)::ys -> 
                if code = code2 then produceResult (xs, ys) (fun c ->  k ((code,Math.Abs(rate-rate2))::c))
                elif code > code2 then produceResult (l1, ys) k
                else produceResult (xs, l2) k
    produceResult (list1, list2) id

This will also fix your results being returned in reverse order.

answered Aug 19 '11 at 16:54

Daniel

47,404
11
101
179

Actually with this modifications i'm getting the StackOverflowException as before. It's a little strange because I implemented it in the same way for slightly different task (a function that taken 2 lists of ordered integers produces the list of the correspondances) and it goes on tail recursion without issues – Fr.Usai Aug 19 '11 at 17:00
I just tested with two lists of 100 million each and didn't get a stack overflow. I increased the lists to 1 billion and ran out of memory, but still no stack overflow. – Daniel Aug 19 '11 at 17:02
I really don'y know Daniel, I've done copy&paste, and I'm still getting stack overflow. – Fr.Usai Aug 19 '11 at 17:10
1

You're probably running in debug mode, which disables tail calls. – Daniel Aug 19 '11 at 17:12
yes I was. Now is working! Thanks! I really didn't know that it disables tail calls! It is possible to enable tail calls in debug mode? – Fr.Usai Aug 19 '11 at 18:43
1

Yes. Project properties > Build > check Generate tail calls. – Daniel Aug 19 '11 at 18:48

score 0 · Answer 3 · answered Aug 20 '11 at 13:23

For an alternative answer, take a look at this: http://fssnip.net/75

The function takes a couple of sequences and returns pairs which match according to some matching function. I haven't volume-tested it.

The function is actually used in the larger snippet here: http://fssnip.net/76

F# continuations goes on StackOverflowException

3 Answers3