5

I see totally different behavior when running a piece of program that tries to exceed RSS on different machines. The code is something like:

...
  char** s = (char**)malloc(10000*sizeof(char*));

  for (i = 0; i < 10000; i++){
    s[i] = (char*)malloc(1000*1000*sizeof(char));
    if (s[i] == NULL) {
      printf("cannot allocate memory for s[%d]",i);
      exit(1);
    }
  }

  int j = 0;
  while(1){
    for (i = 0; i < 10000; i++){
      for (j = 0; j < 1000*1000; j++) {
        s[i][j] = 1;
      }
      if ((i % 100) == 0) printf("i = %d\n", i);
    }
  }
  for (i = 0; i < 10000; i++)
    free(s[i]);
  free(s);
...

The above code tries to allocate around 10GB of memory using malloc. The first two machines I tried this code on run on linux kernel 2.6 and the last one runs linux kernel 2.4. Here are the behaviors I see on these machines:

Machine1: the memory is allocated using memory overcommit, but when assigning values to the memory locations in the while loop, it only allocates as much as RSS allows. Thus OOM Killer kills the process when i=3800 is printed which is around 4GB of memory this machine has.

Machine2: memory is allocated using memory overcommit and the while loop goes on forever, allocating pages from virtual memory. The process goes a bit slower after i = 3800 is printed which is normal.

machine3: this machines has only 2GB of memory. The memory cannot even be allocated. Seems like the over commit is not set or kernel 2.4 does not support allocating virtual machine pages using malloc! Thus in the first for loop it exits while allocating memory for i = 2138

My desired action is the one happening in machine2. Does anyone know which (kernel?) options has to be set to allow OS to allocate virtual memory pages using malloc and start paging while the required memory exceeds RSS?

Thanks

Pirooz
  • 1,268
  • 1
  • 13
  • 24
  • 1
    Where is `if(s != NULL)` of the first allocation? – Luca Matteis Feb 01 '10 at 20:42
  • Personal preference: Instead of `char **s = (char **)malloc(10000 * sizeof(char *));` and `s[i] = (char *)malloc(10000 * sizeof(char));` I would do `char **s = malloc(10000 * sizeof *s);` and `s[i] = malloc(10000 * sizeof **s);`. That way, if `s` changes from a `char **` to a `wchar_t **` in the future, all your `malloc` calls will adapt as needed. In general, casting `malloc()`'s return value isn't necessary in C, and is considered by some (like myself) to be a bad idea as it hinders maintainability. – Chris Lutz Feb 01 '10 at 20:44
  • Thanks for all the great comments. I have tried a simpler version where I only allocate a one dimensional array with big chunks of memory (Say around 1-2GBs) and I see the same behavior. Machines 1&2 are 64 bits and Machine3 is 32 bits. – Pirooz Feb 02 '10 at 03:14

1 Answers1

5

You won't be able to allocate 100GB on a 32-bit machine and address it using regular pointers, which is what your code seems to use. The fact that machine 1 terminates the process when it hits approx 4GB and machine 2 doesn't strongly suggests that machine 2 is running a 64-bit OS.

Timo Geusch
  • 24,095
  • 5
  • 52
  • 70
  • I made a mistake in my original message which I edited and fixed. If you look at the code it is requesting for only 10GBs of memory and not 100GBs. Anyhow, both machines 1&2 are running 64-bit OS's. These two are supposed to be identical since their hardware is exactly the same and configured identically. – Pirooz Feb 02 '10 at 03:17