Strategy for recovering from NULL == malloc() due to memory exhaustion

Question

Reading Martin Sustrick's blog on challenges attendant with preventing "undefined behavior" in C++, vs C, in particular the problem attendant with malloc() failing due to memory exhaustion, I was reminded of the many, many times I have been frustrated to know what to do in such cases.

With virtual systems such conditions are rare, but on embedded platforms, or where performance degradation attendant with hitting the virtual system equates to failure, as is Martin's case with ZeroMQ, I resolved to find a workable solution, and did.

I wanted to ask the readers of StackOverflow if they've tried this approach, and what their experience with it was.

The solution is to allocate a chunk of spare memory off the heap with a call to malloc() at the start of the program, and then use that pool of spare memory to stave off memory exhaustion when and if it occurs. The idea is to prevent capitulation in favor of an orderly retreat (I was reading the accounts of Kesselring's defense of Italy last night) where error messages and IP sockets and such will work long enough to (hopefully) at least tell the user what happened.

#define SPARE_MEM_SIZE (1<<20)  // reserve a megabyte
static void *gSpareMem;

// ------------------------------------------------------------------------------------------------
void *tenacious_malloc(int requested_allocation_size)   {
    static int remaining_spare_size = 0;    // SPARE_MEM_SIZE;
    char err_msg[512];
    void *rtn = NULL;

    // attempt to re-establish the full size of spare memory, if it needs it
    if (SPARE_MEM_SIZE != remaining_spare_size) {
        if(NULL != (gSpareMem = realloc(gSpareMem, SPARE_MEM_SIZE))) {
            remaining_spare_size = SPARE_MEM_SIZE;
            // "touch" the memory so O/S will allocate physical memory
            meset(gSpareMem, 0, SPARE_MEM_SIZE);
            printf("\nSize of spare memory pool restored successfully in %s:%s at line %i :)\n",
                            __FILE__, __FUNCTION__, __LINE__);
        }   else   {
            printf("\nUnable to restore size of spare memory buffer.\n");
        }
    }
    // attempt a plain, old vanilla malloc() and test for failure
    if(NULL != (rtn = malloc(requested_allocation_size))) {
        return rtn;
    }   else  {
        sprintf(err_msg, "\nInitial call to malloc() failed in %s:%s at line %i",
                                                __FILE__, __FUNCTION__, __LINE__);
        if(remaining_spare_size < requested_allocation_size)    {
            // not enough spare storage to satisfy the request, so no point in trying
            printf("%s\nRequested allocaton larger than remaining pool. :(\n\t --- ABORTING --- \n", err_msg);
            return NULL;
        }   else   {
            // take the needed storage from spare memory
            printf("%s\nRetrying memory allocation....\n", err_msg);
            remaining_spare_size -= requested_allocation_size;
            if(NULL != (gSpareMem = realloc(gSpareMem, remaining_spare_size))) {
                // return malloc(requested_allocation_size);
                if(NULL != (rtn = malloc(requested_allocation_size))) {
                    printf("Allocation from spare pool succeeded in %s:%s at line %i :)\n",
                                            __FILE__, __FUNCTION__, __LINE__);
                    return rtn;
                }   else  {
                    remaining_spare_size += requested_allocation_size;
                    sprintf(err_msg, "\nRetry of malloc() after realloc() of spare memory pool "
                        "failed in %s:%s at line %i  :(\n", __FILE__, __FUNCTION__, __LINE__);
                    return NULL;
                }
            }   else   {
                printf("\nRetry failed.\nUnable to allocate requested memory from spare pool. :(\n");
                return NULL;
            }
        }
    }   
}
// ------------------------------------------------------------------------------------------------
int _tmain(int argc, _TCHAR* argv[])    {
    int     *IntVec = NULL;
    double  *DblVec = NULL;
    char    *pString = NULL;
    char    String[] = "Every good boy does fine!";

    IntVec = (int *) tenacious_malloc(100 * sizeof(int));
    DblVec = (double *) tenacious_malloc(100 * sizeof(double));
    pString = (char *)tenacious_malloc(100 * sizeof(String));

    strcpy(pString, String);
    printf("\n%s", pString);


    printf("\nHit Enter to end program.");
    getchar();
    return 0;
}

Answer 1

The best strategy is to aim for code that works without allocations. In particular, for a correct, robust program, all failure paths must be failure-case-free, which means you can't use allocation in failure paths.

My preference, whenever possible, is to avoid any allocations once an operation has started, instead determining the storage needed and allocating it all prior to the start of the operation. This can greatly simplify program logic and makes testing much easier (since there's a single point of possible failure you have to test). Of course it can also be more expensive in other ways; for example, you might have to make two passes over input data to determine how much storage you will need and then process it using the storage.

In regards to your solution of pre-allocating some emergency storage to use once malloc fails, there are basically two versions of this:

1. Simply calling free on the emergency storage then hoping malloc works again afterwards.
2. Going through your own wrapper layer for everything where the wrapper layer can directly use the emergency storage without ever freeing it.

The first approach has the advantage that even standard library and third-party library code can utilize the emergency space, but it has the disadvantage that the freed storage could be stolen by other processes, or threads in your own process, racing for it. If you're sure the memory exhaustion will come from exhausting virtual address space (or process resource limits) rather than system resources, and your process is single-threaded, you don't have to worry about the race, and you can fairly safely assume this approach will work. However, in general, the second approach is much safer, because you have an absolute guarantee that you can obtain the desired amount of emergency storage.

I don't really like either of these approaches, but they may be the best you can do.

Answer 2

On a modern 64 bit computer, you can malloc significantly more memory than you have RAM. In practice, malloc doesn't fail. What happens in practice is that your application starts thrashing, and once you have say 4GB of RAM and your allocations exceed that, your performance will drop to zero because you are swapping like mad. Your performance goes down so much that you never get to the point where malloc can't return memory.

Answer 3

"On a modern 64 bit computer, you can allocate, e.g. by calling malloc(), significantly more memory than your physically installed RAM."

Note that RAM does not equal virtual memory, which brings us to the next point.

Whether one can obtain more virtual memory than physically available RAM is operating system dependent: GNU/Linux operating systems usually come misconfigured out of the box to over provision memory, which is an incorrect behavior in any scenario, and is particularly detrimental in mission critical deployments.

All traditional UNIX systems will not allow for requesting more virtual memory than is available, and will return ENOMEM, which is the correct behavior, because it means that no hacks like an "out of memory killer" are necessary, which in turn means that running processes will continue to function, provided they already have all the memory they require.