-3

I cannot allocate memory for my 1D array in c++. I tried using malloc and new operator. merge has value 0 when I use malloc. In case of new operator I obtain exception std::bad_alloc at memory location .... data_length has value of 131596814. I have got 8GB ram and plenty of it is free. I am using Visual Studio 2012 and Debug/Release mode doesn't affect it.

unsigned int* merge = (unsigned int*) malloc(2 * data_length * sizeof(unsigned int));

unsigned int* merge = new unsigned int[data_length * 2];

Am I doing something wrong here?

benderto
  • 896
  • 11
  • 39
  • 1
    Compiling in 32bit mode? – Mat May 01 '15 at 16:22
  • Remember that when you allocate memory, the allocation will be one big continuous chunk, so if there's no continuous almost 5 gigs of memory (4 times 131596814) available the allocation will fail. – Some programmer dude May 01 '15 at 16:22
  • Remember, if your machine has 8GB of memory, it will be **shared by other executables & tasks**. Your program may not have that amount available. You may have to do things the old fashioned way and process the data in chunks / pieces, depending on how much memory is allocated to your program. – Thomas Matthews May 01 '15 at 16:24
  • 1
    Joachim Pileborg: isn't it like 0.5GB? Thomas Matthews: yeah I know, but 500Mb isn't that much. But yes at the end I don't see any other way around. – benderto May 01 '15 at 16:26
  • Mat: yes, 32bit mode – benderto May 01 '15 at 16:32
  • 3
    Looks like 1 GB to me. And that can easily fail on 32 bit if you have other large allocations. Remember that 32 bit has a 2GB address space by default. And this 2GB address space is fragmented. – drescherjm May 01 '15 at 16:32
  • Try compiling in 64-bit mode, or use an OS memory allocator. – Thomas Matthews May 01 '15 at 16:34
  • @benderto Yeah sorry, counted wrong. But still, it has to be 500 MB of continuous memory, and the OS might not be able to map that for your process. – Some programmer dude May 01 '15 at 16:45
  • Although compiling for x64 is the easiest solution, see here for a possible work around for 32 bit: http://stackoverflow.com/a/25864285/487892 – drescherjm May 01 '15 at 16:55

1 Answers1

3

Arrays in C/C++ (and many other languages) must be allocated as one continuous block of memory.

You are trying to allocate 1GB block - this is nearly impossible if complied for x86 (32bit), especially if it is not very first allocation. The restriction comes from default 2GB address space on Windows (which can be extended up to almost 4GB if your host system is x64 and your process is marked as "large address aware") and address space fragmentation caused by DLL loaded at semi-random locations in address space as well as previous memory allocations by the program.

Fixes:

  • redesign program to not require huge arrays - allocation is only first problem there. Indexing (not every language supports long for indexes), growing existing array requires full copy.
  • Consider using libraries that provide "chunked" arrays that store data as list of smaller chunks while exposing regular array/iterator based interface
  • if you must handle large amount of data switching to x64 (64bit) is basically requirement as address space restriction practically disappears and 1GB blocks of address space are easy to find.
Alexei Levenkov
  • 98,904
  • 14
  • 127
  • 179