Mono Continuations - Memory keeps increasing after store()

Question

Here's Mono Continuations' continuation_store (...). From looking at the code below, it appears as though store() follows these two branches:

1. cont->saved_stack && num_bytes <= cont->stack_alloc_size
   • use the memory directly
2. else
   • gc free the used memory, and create some new memory.

However, the weird thing is if I repeatedly use continuation_store(), the memory usage increases until at a later step a huge and laggy GC operation is done. Can anyone explain why this happens?

Thanks

static int
continuation_store (MonoContinuation *cont, int state, MonoException **e)
{
    MonoLMF *lmf = mono_get_lmf ();
    gsize num_bytes;

    if (!cont->domain)
        *e =  mono_get_exception_argument ("cont", "Continuation not initialized");
    if (cont->domain != mono_domain_get () || cont->thread_id != GetCurrentThreadId ())
        *e = mono_get_exception_argument ("cont", "Continuation from another thread or domain");

    cont->lmf = lmf;
    cont->return_ip = __builtin_return_address (0);
    cont->return_sp = __builtin_frame_address (0);

    num_bytes = (char*)cont->top_sp - (char*)cont->return_sp;

    /*g_print ("store: %d bytes, sp: %p, ip: %p, lmf: %p\n", num_bytes, cont->return_sp, cont->return_ip, lmf);*/

    if (cont->saved_stack && num_bytes <= cont->stack_alloc_size) 
    {
        /* clear to avoid GC retention */
        if (num_bytes < cont->stack_used_size)
            memset ((char*)cont->saved_stack + num_bytes, 0, cont->stack_used_size - num_bytes);
    } 
    else 
    {
        tasklets_lock ();
        internal_init ();
        if (cont->saved_stack) {
            mono_g_hash_table_remove (keepalive_stacks, cont->saved_stack);
            mono_gc_free_fixed (cont->saved_stack);
        }
        cont->stack_used_size = num_bytes;
        cont->stack_alloc_size = num_bytes * 1.1;
        cont->saved_stack = mono_gc_alloc_fixed (cont->stack_alloc_size, NULL);
        mono_g_hash_table_insert (keepalive_stacks, cont->saved_stack, cont->saved_stack);
        tasklets_unlock ();
    }
    memcpy (cont->saved_stack, cont->return_sp, num_bytes);

    return state;
}

Answer 1

Note the call to mono_gc_free_fixed does nothing in the default boehm collector: https://github.com/mono/mono/blob/master/mono/metadata/boehm-gc.c#L528

The else branch just removes the memory from the keepalive_stacks hash table. This hash table keeps a reference to the allocated memory so it is not garbage collected. Once the pointer to the allocated memory is removed from this hash table, it will then be reclaimed during the next garbage collection. This collection is triggered at a later time once reaching a certain amount of memory allocation.