How to handle malloc failing and returning NULL?

Question

I'm a bit confused on how to check if a memory allocation failed in order to prevent any undefined behaviours caused by a dereferenced NULL pointer. I know that malloc (and similiar functions) can fail and return NULL, and that for this reason the address returned should always be checked before proceeding with the rest of the program. What I don't get is what's the best way to handle these kind of cases. In other words: what is a program supposed to do when a malloc call returns NULL?

I was working on this implementation of a doubly linked list when this doubt raised.

struct ListNode {

    struct ListNode* previous;
    struct ListNode* next;
    void* object;
};

struct ListNode* newListNode(void* object) {

    struct ListNode* self = malloc(sizeof(*self));

    if(self != NULL) {

        self->next = NULL;
        self->previous = NULL;
        self->object = object;
    }

    return self;
}

The initialization of a node happens only if its pointer was correctly allocated. If this didn't happen, this constructor function returns NULL.

I've also written a function that creates a new node (calling the newListNode function) starting from an already existing node and then returns it.

struct ListNode* createNextNode(struct ListNode* self, void* object) {

    struct ListNode* newNext = newListNode(object);

    if(newNext != NULL) {

        newNext->previous = self;

        struct ListNode* oldNext = self->next;

        self->next = newNext;

        if(oldNext != NULL) {

            newNext->next = oldNext;
            oldNext->previous = self->next;
        }
    }

    return newNext;
}

If newListNode returns NULL, createNextNode as well returns NULL and the node passed to the function doesn't get touched.

Then the ListNode struct is used to implement the actual linked list.

struct LinkedList {

    struct ListNode* first;
    struct ListNode* last;
    unsigned int length;
};

_Bool addToLinkedList(struct LinkedList* self, void* object) {

    struct ListNode* newNode;

    if(self->length == 0) {

        newNode = newListNode(object);
        self->first = newNode;
    }
    else {

        newNode = createNextNode(self->last, object);
    }

    if(newNode != NULL) {

        self->last = newNode;
        self->length++;
    }

    return newNode != NULL;
}

if the creation of a new node fails, the addToLinkedList function returns 0 and the linked list itself is left untouched.

Finally, let's consider this last function which adds all the elements of a linked list to another linked list.

void addAllToLinkedList(struct LinkedList* self, const struct LinkedList* other) {

    struct ListNode* node = other->first;

    while(node != NULL) {

        addToLinkedList(self, node->object);
        node = node->next;
    }
}

How should I handle the possibility that addToLinkedList might return 0? For what I've gathered, malloc fails when its no longer possible to allocate memory, so I assume that subsequent calls after an allocation failure would fail as well, am I right? So, if 0 is returned, should the loop immediately stop since it won't be possible to add any new elements to the list anyway? Also, is it correct to stack all of these checks one over another the way I did it? Isn't it redundant? Would it be wrong to just immediately terminate the program as soon as malloc fails? I read that it would be problematic for multi-threaded programs and also that in some istances a program might be able to continue to run without any further allocation of memory, so it would be wrong to treat this as a fatal error in any possible case. Is this right?

Sorry for the really long post and thank you for your help!

There's not really a general answer - it depends on the larger context of the program. Is there other useful work your program, as a whole, can do, without allocating this memory? If yes, do it. If not, exit. — Nate Eldredge, Mar 18 '20 at 15:04
It also depends somewhat on the platform. If `malloc` returns `NULL` on a platform such as Windows or Linux, then most likely something has gone seriously wrong in your program and it is almost always a fatal error. — Jabberwocky, Mar 18 '20 at 15:08
@Jabberwocky It could also just be that there isn't enough available memory for the allocation. Sometimes there are artificial restrictions on memory too. — Govind Parmar, Mar 18 '20 at 15:09
Once `malloc` has returned `NULL` do subsequent calls also return `NULL`?. Well this depends. The answer is most of the time "yes". It's very hard to give a general answer. — Jabberwocky, Mar 18 '20 at 15:10
@GovindParmar yes, of course, there are many possible scenarios. — Jabberwocky, Mar 18 '20 at 15:10
Actually you should consider available memory a bit like the gas in your car. You never want to run out of it. You always need some reserves, and when you run out of it you're most of the time out of luck — Jabberwocky, Mar 18 '20 at 15:16
Does this answer your question? [What if malloc fails?](https://stackoverflow.com/questions/11788803/what-if-malloc-fails) — ggorlen, Mar 18 '20 at 15:19

score 6 · Answer 1 · answered Mar 18 '20 at 15:09

6

It depends on the broader circumstances. For some programs, simply aborting is the right thing to do.

For some applications, the right thing to do is to shrink caches and try the malloc again. For some multithreaded programs, just waiting (to give other threads a chance to free memory) and retrying will work.

For applications that need to be highly reliable, you need an application level solution. One solution that I've used and battle tested is this:

Have an emergency pool of memory allocated at startup.
If malloc fails, free some of the emergency pool.
For calls that can't sanely handle a NULL response, sleep and retry.
Have a service thread that tries to refill the emergency pool.
Have code that uses caching respond to a non-full emergency pool by reducing memory consumption.
If you have the ability to shed load, for example, by shifting load to other instances, do so if the emergency pool isn't full.
For discretionary actions that require allocating a lot of memory, check the level of the emergency pool and don't do the action if it's not full or close to it.
If the emergency pool gets empty, abort.

answered Mar 18 '20 at 15:09

David Schwartz

179,497
17
214
278

1

Regarding (3), make sure that there is a limit to how many retries and sleeps are done before the code gives up, and have a policy about what happens when you do run out of retries. Consider how long the delay is — what's acceptable? A second might be OK in a word processor; it probably isn't in the launch controls for a SpaceX Falcon 9 rocket. – Jonathan Leffler Mar 18 '20 at 15:12
1

Very nice answer, albeit rather for advanced programmers. – Jabberwocky Mar 18 '20 at 15:13
1

@JonathanLeffler It's really hard to sanely handle memory allocation failures at literally every point in your code where memory is allocated. For the majority of cases, you can only retry forever or abort the process. – David Schwartz Mar 18 '20 at 15:14
3

"Retry forever" worries me — YMMV. – Jonathan Leffler Mar 18 '20 at 15:14
1

@JonathanLeffler That code will keep draining the emergency pool (per 2). Eventually, either the other mechanisms will allow the allocation to succeed or 8 will kick in. It's just not possible to identify every single place memory is allocated and sanely handle failure in that code. – David Schwartz Mar 18 '20 at 15:16
Dang. You sure know how to write non-error-prone code. Maybe make a static memory pool? – S.S. Anne Mar 18 '20 at 20:28
This might honestly be a bit too advanced for me, but it's still really, really interesting. It seems like a brilliant and elegant approach, I'd like to try to replicate something like this just for the sake of it (as of now I don't work on anything that might require a solution this much complex). Thank you for your reply! PS: How much memory would be sensible to reserve to the emergency pool? – Gian Mar 18 '20 at 22:17
@Gian Looking at my (10 years old now) code, it defaulted to 1,024 blocks each 64KB in size. Both parameters were tunable either by overriding defaults set by the application code or tuned by the admin in a configuration file. The default low water mark to trigger severe load shedding was 1/2 and the default high water mark to stop it was 3/4. The default rate to check on the status and possibly try to re-acquire lost blocks was 120ms. – David Schwartz Mar 18 '20 at 22:42

chux - Reinstate Monica · Accepted Answer · 2020-03-18T20:20:33.473

2

How to handle malloc failing and returning NULL?

Consider if the code is a set of helper functions/library or application.

The decision to terminate is best handled by higher level code.

Example: Aside from exit(), abort() and friends, the Standard C library does not exit.

Likewise returning error codes/values is a reasonable solution for OP's low-level function sets too. Even for addAllToLinkedList(), I'd consider propagating the error in the return code. (Non-zero is some error.)

// void addAllToLinkedList(struct LinkedList* self, const struct LinkedList* other) {
int addAllToLinkedList(struct LinkedList* self, const struct LinkedList* other) {
  ...
  if (addToLinkedList(self, node->object) == NULL) {
    // Do some house-keepeing (undo prior allocations)
    return -1;
  }

For the higher level application, follow your design. For now, it may be a simple enough to exit with a failure message.

if (addAllToLinkedList(self, ptrs)) {
  fprintf(stderr, "Linked List failure in %s %u\n", __func__, __LINE__);
  exit(EXIT_FAILURE);
}

Example of not exiting:

Consider a routine that read a file into a data structure with many uses of LinkedList and the file was somehow corrupted leading to excessive memory allocations. Code may want to simply free everything for that file (but just for that file), and simply report to the user "invalid file/out-of-memory" - and continue running.

if (addAllToLinkedList(self, ptrs)) {
  free_file_to_struct_resouces(handle);
  return oops;
}
...
return success;

Take away

Low level routines indicate an error somehow. Higher level routines can exit code if desired.

edited Mar 18 '20 at 20:20

answered Mar 18 '20 at 15:31

chux - Reinstate Monica

143,097
13
135
256

So I should just code functions that use malloc & co so that it’s always possible to check if something went wrong when these were called? Doing so should allow whoever is going to use those function in a program of his/her own to decide, on a case to case basis, whether to keep propagating the error out or to immediately kill the execution. – Gian Mar 18 '20 at 20:47
Specifically, if one of these function is used to build another helper/library function, the programmer should keep signaling the error in some way, while in a code which manages the actual execution of a program it **might** be sensible to just terminate on failure (or to just handle it in some way and to display an error message, the same way you did with that last example). Did I get it right? – Gian Mar 18 '20 at 20:47
1

Basically, this is a more versatile approach. I mean, malloc itself could have been made so that it would always terminate the program on failure (and I think there's even an alternative function that does so), but the fact that it doesn't gives options to the user to treat these occurences in the most appropriate way depending on the context. – Gian Mar 18 '20 at 20:47
1

@Gian Agree on all 3. Think of code in layers. The closer to the bottom, the less likely an `exit()` is desirable for making portable code. If I employed a tool set/library that did all sorts of great things, but would `exit()` on rare occasions, I could see setting aside an otherwise nice package. Yes, handling errors in C is a chore. – chux - Reinstate Monica Mar 18 '20 at 20:53
This actually makes a lot of sense keeping in mind what I've recently studied (I'm still in university) about exceptions (which get thrown at a lower level and get handled at a higher level). It actually makes me appreciate those a whole lot more. Now I actually get why they are so convenient: basically they automatize this process of propagating the error upward. – Gian Mar 18 '20 at 22:13
1

@Gian Error handle is a deep subject with many POVs. Sometimes error handling is nil, get the program shipped ASAP. Other times it is life critical. For me, error handling is a pragmatic yet big factor in code design - how to detect/handle efficiently can steer algorithm selection. This is the engineering of programming: not excessive attention to errors, but enough. – chux - Reinstate Monica Mar 18 '20 at 22:41

How to handle malloc failing and returning NULL?

2 Answers2

Linked