F# MailboxProcessor memory leak in try/catch block

Question

Updated after obvious error pointed out by John Palmer in the comments.

The following code results in OutOfMemoryException:

let agent = MailboxProcessor<string>.Start(fun agent ->

    let maxLength = 1000

    let rec loop (state: string list) i = async {
        let! msg = agent.Receive()

        try        
            printfn "received message: %s, iteration: %i, length: %i" msg i state.Length
            let newState = state |> Seq.truncate maxLength |> Seq.toList
            return! loop (msg::newState) (i+1)
        with
        | ex -> 
            printfn "%A" ex
            return! loop state (i+1)
    }

    loop [] 0
)

let greeting = "hello"

while true do
    agent.Post greeting
    System.Threading.Thread.Sleep(1) // avoid piling up greetings before they are output

The error is gone if I don't use try/catch block.

Increasing the sleep time only postpones the error.

Update 2: I guess the issue here is that the function stops being tail recursive as the recursive call is no longer the last one to execute. Would be nice for somebody with more F# experience to desugar it as I'm sure this is a common memory-leak situation in F# agents as the code is very simple and generic.

score 7 · Accepted Answer · edited May 23 '17 at 12:05

Solution:

It turned out to be a part of a bigger problem: the function can't be tail-recursive if the recursive call is made within try/catch block as it has to be able to unroll the stack if the exception is thrown and thus has to save call stack information.

More details here:

Tail recursion and exceptions in F#

Properly rewritten code (separate try/catch and return):

let agent = MailboxProcessor<string>.Start(fun agent ->

    let maxLength = 1000

    let rec loop (state: string list) i = async {
        let! msg = agent.Receive()

        let newState = 
            try        
                printfn "received message: %s, iteration: %i, length: %i" msg i state.Length
                let truncatedState = state |> Seq.truncate maxLength |> Seq.toList
                msg::truncatedState
            with
            | ex -> 
                printfn "%A" ex
                state

        return! loop newState (i+1)
    }

    loop [] 0
)

score 2 · Answer 2 · answered Jun 03 '14 at 06:36

2

I suspect the issue is actually here:

while true do
    agent.Post "hello"

All the "hello"s that you post have to be stored in memory somewhere and will be pushed much faster than the output can happen with printf

answered Jun 03 '14 at 06:36

John Palmer

25,356
3
48
67

Brilliant. Darn, just when I thought I've tracked down the cause of my original problem. Looks like it will require more investigation. – Grozz Jun 03 '14 at 06:40
I've updated the question to better reflect my problem and avoid that error. – Grozz Jun 03 '14 at 06:48

score 2 · Answer 3 · answered Jun 03 '14 at 13:30

2

See my old post here http://vaskir.blogspot.ru/2013/02/recursion-and-trywithfinally-blocks.html

random chars in order to satisfy this site rules *

answered Jun 03 '14 at 13:30

Vasily Kirichenko

671
5
13

2

This answer would be better if you summarized the contents of your blog post rather than simply putting a link in the answer. A link-only answer has the drawback of being less useful (harder to scan the answers to find good information) and being vulnerable to link rot. – N_A Jun 03 '14 at 15:03

score 0 · Answer 4 · answered May 17 '18 at 08:56

Basically anything that is done after the return (like a try/with/finally/dispose) will prevent tail calls.

See https://blogs.msdn.microsoft.com/fsharpteam/2011/07/08/tail-calls-in-f/

There is also work underway to have the compiler warn about lack of tail recursion: https://github.com/fsharp/fslang-design/issues/82

F# MailboxProcessor memory leak in try/catch block

4 Answers4