Realloc() returning NULL when memory is available

Question

I have a c++ program running on a Windows 7 machine with 12GB memory. The compiler and linker is Visual Studio 2013 Express.

the program uses a library OGDF. I compiled the library source codes into a static library with Release X64 configuration and referenced the library in my project.

when I run the problem (Debug x64 configuration) , a exception are thrown from the code in the OGDF library indicating that no enough memory available;

    E *p = static_cast<E *>( realloc(m_pStart, sNew*sizeof(E)) );
    if(p == 0) OGDF_THROW(InsufficientMemoryException);

I paused the program and opened the debug window and checked the value of sNew = 9M and sizeof(E) = 8, so it was allocating 72M memory and failed.

At the debug time I opened windows task manager, it shows that my program's memory usage (working set size and commited size) is less than 2MB

So I am very confused why the REALLOC will fail since there are enough memory available (>4GB)? even if there are lots of fragments in my heap, the committed size is less than 2MB so there should be enough memory.

For test propose I inserted the following code before my call to the library function:

void* ddd = malloc(1200000000);
char* b = (char*)ddd;
char ttt = 3;
int g = 0;
for (g = 0; g < 1200000000;++g)
{
    *(b + g) = ttt;
}
ddd=realloc(ddd, 2400000000);
b = (char*)ddd;
for (g = 0; g < 2400000000; ++g)
{
    *(b + g) = ttt;
}

the code above runs OK (Debug x64 configuration) and the task manager shows my memory usage (working set and committed size) is about 2.3GB before I called free() . So in my program I can allocate more than 2GB memory on my heap, why does the 72MB alloc in the library code fail ?

EDIT:

I found the problem.

The debugger showed me WRONG local variable data when I am using release-configuration compiled library file. The actual cause is the library is calling realloc(ptr,0).

Answer 1

Your question is very similar to "Why won't you write me a check for $500? You have $2,000 in the bank."

Operating systems get requests for memory before that memory is used. An operating system can't grant a memory allocation request unless it has enough backing store for all the requests it has already permitted, whether or not they are currently using any RAM.

For example, if you malloc 1GB but haven't accessed the allocated virtual address space yet, much less than 1GB of RAM will be used by that allocation. But until that allocation is released, the system must reserve 1GB of backing store (RAM or swap) just in case your program starts using that allocated space.

If the system has too many such allocations, even if it has plenty of free RAM, it will refuse new allocations. Windows won't overcommit because that risks having to forcibly terminate applications should those mappings later require more backing store than the OS has.

A good way to avoid these misunderstandings in the future is to avoid using the word "memory" by itself. If you mean RAM, say "RAM" or "physical memory". If you mean backing store, say "backing store". If you mean address space, say "virtual memory".

You say things like, "I have lots of free memory, so I should be able to allocate memory." That sounds like a mystery. But if you had more accurately said, "I have lots of free RAM, why can't I allocate more virtual memory (or backed memory)?", you would be halfway to the answer.