4

I'm about to teach an introductory computer science course in C and I'd like to demonstrate to students why they should check whether malloc() returned a NULL. My plan was to use ulimit to restrict the amount of available memory such that I could exercise different code paths with different limits. Our prescribed environment is CentOS 6.5.

My first attempts to make this happened failed and the shell showed "Killed". This led to me discovering the Linux OOM killer. I have since tried to figure out the magic set of incantations that will cause the results I'm looking for. Apparently I need to mess with:

  • /etc/sysctl.conf
  • ulimit -m
  • ulimit -v
  • vm.overcommit_memory (which apparently should be set to 2, according to an Oracle article)

This far either I get "Killed" or a segmentation fault, neither of which is the expected outcome. The fact that I'm getting "Killed" with vm_overcommit_memory=2 means that I definitely don't understand what's going on.

If anyone can find a way to artificially and reliably create a constrained execution environment on CentOS so that students learn how to handle OOM (and other?) kinds of errors, many course instructors will thank you.

Jason Foster
  • 81
  • 1
  • 2
  • 3
    `void *mymalloc(size_t n) { return rand() % 3 ? malloc(n) : NULL; }` then `#define malloc mymalloc` –  Dec 22 '13 at 01:18
  • 1
    Have you tried a `hard` limit in `/etc/security/limits.conf`? – Elliott Frisch Dec 22 '13 at 01:25
  • I really like the "random failure" approach, but I can see some students saying "Sure, you can simulate a failure but what happens in reality?" I'm planning to do some demos on an Arduino (where malloc() had better fail), but would like to be able to demo in a Linux shell. – Jason Foster Dec 22 '13 at 01:35
  • 1
    @JasonFoster (Just FYI, in reality, `malloc()` will hardly ever fail on Linux. Almost all modern OSes overcommit memory, meaning that they make `malloc()` return a non-NULL pointer *even if there isn't enough memory,* and they only actually allocate the underlying memory block if that pointer is ever dereferenced. And if *at that time* the allocation fails -- you will get a segfault or similar.) –  Dec 22 '13 at 01:37
  • I hadn't found `limits.conf` but some quick searching seems to imply that it's effectively a `ulimit` which doesn't seem to be working. The ideal solution is something that I can change in real time without logging in as a different user. – Jason Foster Dec 22 '13 at 01:38
  • @H2CO3 I'd found that message in my searches, but I figured that there has to be a way to make it happen. There's enough online chatter about the OOM Killer and how it should be disabled on servers that I was figuring that my objective would be achievable. I'm stuck on Linux, so I'm hoping that (to use some of the language in other posted) it can be made to act rationally. – Jason Foster Dec 22 '13 at 01:43
  • @JasonFoster Honestly, I wish it would, too! From the programmer's point of view, the inability for checking an error condition (i. e. `malloc()` not returning `NULL` on failure) is... just wrong. –  Dec 22 '13 at 01:45
  • @H2CO3: which other modern O/S other than Linux does over-commitment of memory? AFAIK, it is the only one that does so — certainly as the routine default. _[...time passeth...]_ Hmmm...it appears Mac OS X also allows over-commitment. Too much influence of Linux, I fear. – Jonathan Leffler Dec 22 '13 at 04:00
  • @JonathanLeffler Yes... and this time, the result is not better... –  Dec 22 '13 at 06:46

1 Answers1

1

It is possible to [effectively] turn off overcommitting from kernel >= 2.5.30.

Following Linux Kernel Memory :

// save your work here and note your current overcommit_ratio value

# echo 2 > overcommit_memory
# echo 1 > overcommit_ratio

this sets the VM_OVERCOMMIT_MEMORY to 2 indicating not to overcommit past the overcommit_ratio, which is set to 1 (ie no overcommitting)

Null malloc demo

#include <stdio.h>
#include <stdlib.h>

int main(int argc,char *argv[])
{
  void *page = 0; int index;
  void *pages[256];
  index = 0;
  while(1)
  {
    page = malloc(1073741824); //1GB
    if(!page)break;
    pages[index] = page;
    ++index;
    if(index >= 256)break;
  }
  if(index >= 256)
  {
    printf("allocated 256 pages\n");
  }
  else
  {
    printf("memory failed at %d\n",index);
  }
  while(index > 0)
  {
    --index;
    free(pages[index]);
  }
  return 0;
}

Output

$ cat /proc/sys/vm/overcommit_memory 
0
$ cat /proc/sys/vm/overcommit_ratio 
50
$ ./code/stackoverflow/test-memory 
allocated 256 pages
$ su
# echo 2 > /proc/sys/vm/overcommit_memory 
# echo 1 > /proc/sys/vm/overcommit_ratio 
# exit
exit
$ cat /proc/sys/vm/overcommit_memory 
2
$ cat /proc/sys/vm/overcommit_ratio 
1
$ ./code/stackoverflow/test-memory 
memory failed at 0

remember to restore your overcommit_memory to 0 and overcommit_ratio as noted

amdixon
  • 3,814
  • 8
  • 25
  • 34
  • Your solution has got me most of the way to a working setup (thanks!) but I've got two clarification questions. After applying the vm setting I tried using `ulimit -b` and `ulimit -m` to no effect. But `ulimit -v` did what I wanted (namely "toggling" between the working and failing states). I don't know how to explain that to the students (or to myself for that matter). The other question is how I would calculate (or identify) how much memory there is available for a process to `malloc()`? Thanks for the great answer and for moving things forward! – Jason Foster Dec 22 '13 at 18:09
  • `ulimit -m` will only limit the `RAM` memory and doesn't limit your use of _swap space_. `ulimit -v` limits virtual memory (includes _swap space_) available for the process. Can try adjusting [resizing your swap space](http://www.oradba.ch/2012/05/resize-swap-space-on-linux/) to confirm that notion.. In terms of how much memory is available for a process to alloc you could take a look at output of `ps v` and sum across `%MEM` usage – amdixon Dec 23 '13 at 08:23