Malloc & calloc: different memory size allocated

Question

So, I have this piece of code:

#include <stdio.h>
#include <stdlib.h>

int main()
{
    char *p;
    long n = 1;

    while(1) {
        p = malloc(n * sizeof(char));
        //p = calloc(n, sizeof(char));

        if(p) {
            printf("[%ld] Memory allocation successful! Address: %p\n", n , p);
            n++;
         } else {
            printf("No more memory! Sorry...");
            break;
        }
    }

    free(p);
    getch();
    return 0;
}

And I run in on Windows. Interesting thing:

if we use malloc, the program allocates about 430 MB of memory and then stops (photo here => https://i.stack.imgur.com/ZYg75.png)
if we use calloc, the program allocates about 2 GB of memory and then stops (photo here => https://i.stack.imgur.com/vqGQX.png)
(strange test): if we use both of them in the same time, it uses maximum of (~400MB + ~2GB) / 2 => ~1.2GB

However, if I run the same code on Linux, the allocation goes on and on (after 600k allocations and many GB used it still continues until eventually it is killed) and approximately the same amount of memory is used.

So my question is: shouldn't they allocate the same amount of memory? I thought the only difference was that calloc initialize the memory with zero (malloc returns uninitialized memory). And why it only happens on Windows? It's strange and interesting in the same time.

Hope you can help me with an explanation for this. Thanks!

Edit:

Code::Blocks 13.12 with GNU GCC Compiler
Windows 10 (x64)
Linux Mint 17.2 "Rafaela" - Cinnamon (64-bit) (for Linux testing)

There's a difference between *virtual* memory allocated and *physical* memory allocated... and also between how the two operating systems handle, and display, such memory allocations. — DevSolar, Dec 01 '15 at 14:58
We were studying memory allocation at the university at the programming course and I just wanted to see how everything works. I wrote this program just to see how fast my memory is filling up (I have 8GB Ram but in Windows maxium of 2GB is used with this program - another thing I noticed). — Adrian Pop, Dec 01 '15 at 15:02
You should be more precise about the environment: which compiler? 32 or 64 bits? which version of Windows and Linux... — chqrlie, Dec 01 '15 at 15:04
Try adding `memset(p,0, n)` right alfter `malloc`. Then you should get the same result as with the `calloc` version. `calloc` clears the allocated memory, but `malloc` doesn't. — Jabberwocky, Dec 01 '15 at 15:32
Michael, you're right, it's working like this. Thanks you too for "enlightening" me. Have a nice day! — Adrian Pop, Dec 01 '15 at 15:53

chqrlie · Accepted Answer · 2015-12-01T15:24:49.403

2

Looking at the program output, you actually allocate the same number of blocks, 65188 for malloc, 65189 for calloc. Ignoring overhead, that's slightly less than 2GB of memory.

My guess is that you compile in 32 bit mode (pointers are dumped as 32 bits), which limits the amount of memory available to a single user process to less than 2GB. The difference on the process map display comes from how your program uses the memory it allocates.

The malloc version does not touch the allocated pages: more than 3 quarters of them are not actually mapped, hence only 430MB.

The calloc version shows 2GB of mapped memory: chances are your C library function calloc clears the allocated memory, even for pages obtained from the OS. This is not optimal, but only visible if you do not touch the allocated memory, a special case anyway. Yet it would be faster to not clear pages obtained from the OS as they are specified to be zero filled anyway.

In Linux, you may be compiling to 64 bits, getting access to much more than 2GB of virtual process space. Since you do not touch the memory, it is not mapped, and the same seems to happen in the calloc case as well. The C runtime is different (64 bit glibc on Linux, 32 bit Microsoft Library on Windows). You should use top or ps in a different terminal in Linux to check how much memory is actually mapped to your process in both cases.

edited Dec 01 '15 at 15:24

answered Dec 01 '15 at 15:16

chqrlie

131,814
10
121
189

`calloc` definitly zero fills the allocated memory whereas `malloc` doesnt. – Jabberwocky Dec 01 '15 at 15:29
@MichaelWalz On Linux, the `calloc` doesn't force allocation of the physical memory - as long as it is not touched, it stays virtual. Just like `malloc`. Did test precisely that about few weeks ago. – Dummy00001 Dec 01 '15 at 15:32
@Dummy00001 strange, many docs for `calloc` including [this one](http://linux.die.net/man/3/calloc) state that `calloc` zero fills the allocated memory. – Jabberwocky Dec 01 '15 at 15:35
Thanks for the answer! I was compiling in 32 bit mode so I'll try to compile the code in 64 bit mode and see what's happening. Nice explanation for the rest :) Have a nice day! – Adrian Pop Dec 01 '15 at 15:40
@MichaelWalz: `calloc` does not need to clear memory it just mapped from the OS if `mmap` is specified as returning zero filled pages, or more precisely if the pages will be filled with 0 when they eventuelly get mapped. – chqrlie Dec 01 '15 at 15:43
@MichaelWalz: the C Standard says: 7.22.3.3 *The calloc function allocates space for an array of nmemb objects, each of whose size is size. The space is initialized to all bits zero.* If `calloc` uses a system call that returns a block of memory guaranteed to be 0 filled when mapped, it does not need to waste time filling it explicitly with `memset`. – chqrlie Dec 01 '15 at 15:49
@chqrlie that sounds reasonable. Thank you. – Jabberwocky Dec 01 '15 at 15:54
2

@MichaelWalz, yes, as @chqrlie has explained. C standard is portable and has no concept of the virtual memory. How OS chooses to "fill" or "initialize" the memory, is irrelevant to the standard. Years ago, Linux/glibc used always normal `memset()` on the `calloc`ed memory (which forced physical memory allolcation) - but today they have optimized it to skip the `memset()`. – Dummy00001 Dec 02 '15 at 09:05

Malloc & calloc: different memory size allocated

1 Answers1