Huge std::vector does not release all memory on destruction

Question

When using a very large vector of vectors we've found that part of the memory is not released.

#include <iostream>
#include <vector>
#include <unistd.h>

void foo()
{
    std::vector<std::vector<unsigned int> > voxelToPixel;
    unsigned int numElem = 1<<27;
    voxelToPixel.resize( numElem );

    for (unsigned int idx=0; idx < numElem; idx++)
        voxelToPixel.at(idx).push_back(idx);

}

int main()
{
    foo();
    std::cout << "End" << std::endl;
    sleep(30);
    return 0;
}

That leaves around 4GB of memory hanging until the process ends.

If we change the for line to

for (unsigned int idx=0; idx < numElem; idx++)
    voxelToPixel.at(0).push_back(idx);

the memory is released.

Using gcc-4.8 on a linux machine. We've used htop to track the memory usage on a computer with 100 GB of RAM. You will need around 8 GB of RAM to run the code. Can you reproduce the problem? Any ideas on why that is happening?

EDIT: We've seen that that does not happen in a Mac (with either gcc or clang). Also, in linux, the memory is freed if we call foo two times (but happens again the third time).

@stefan smaller how? in number of elements? I've seen this happening on 2^25, but for smaller it's difficult to tell. — quimnuss, Sep 24 '14 at 16:02

Mike Seymour · Accepted Answer · 2014-09-24T15:37:05.550

29

Small allocations (up to 128kb by default, I think) are managed by an in-process heap, and are not returned to the OS when they're deallocated; they're returned to the heap for reuse within the process. Larger allocations come directly from the OS (by calling mmap), and are returned to the OS when deallocated.

In your first example, each vector only needs to allocate enough space for a single int. You have a hundred million small allocations, none of which will be returned to the OS.

In the second example, as the vector grows, it will make many allocations of various sizes. Some are smaller than the mmap threshold, these will remain in the process memory; but, since you only do this to one vector, that won't be a huge amount. If you were to use resize or reserve to allocate all the memory for each vector before populating it, then you should find that all the memory is returned to the OS.

edited Sep 24 '14 at 15:37

answered Sep 24 '14 at 15:25

Mike Seymour

249,747
28
448
644

Each of the `1<<27` inner vectors only contains a single `int`, so `resize`/`reserve` wouldn't help in this case. There's a `resize` for the outer vector already. – Mark Ransom Sep 24 '14 at 15:32
@MarkRansom: Whoops, you're right, I misread the code. – Mike Seymour Sep 24 '14 at 15:32
@MikeSeymour That sounds about right... do you know if there's a way to force the process to release the unused heap memory? What will happen if the OS is running out of memory? – quimnuss Sep 24 '14 at 16:31
1

@quimnuss: You could use a custom allocator to allocate all the memory in a huge block, dole it out to the vectors, and release it all after use; [Boost.Pool](http://www.boost.org/doc/libs/1_56_0/libs/pool/doc/html/index.html) might be useful. You could try `malloc_trim` - but that can only release memory up to the first "used" part of the heap. Once the heap is fragmented, with blocks of used memory scattered throughout it, there's not much you can do about it. If the OS runs out of memory, then it unleashes the dreaded oom-killer, and life becomes truly scary. – Mike Seymour Sep 24 '14 at 16:41
2

@quimnuss There's no need to force the process to release unused heap memory. The OS can already take the physical memory away, and virtual memory is cheap. – David Schwartz Sep 24 '14 at 19:12
2

In Linux, there is not careful allocation of memory. You write into the area of virtual memory that's allowed for the heap and Linux finds some RAM to back it. You can inform Linux that you've finished using an entire area of memory by setting the size of your data segment with the `brk` syscall. The C++ allocator will handle this for you. If you don't free the memory, but also don't use it for a while, Linux will page it out to disk and let other processes or the disk cache use the RAM instead. – Stuart Caie Sep 24 '14 at 19:19
As the two previous comments imply, stop worrying about it, there is no problem, the C library and OS are working as designed. – Jonathan Wakely Sep 25 '14 at 10:36

Huge std::vector does not release all memory on destruction

1 Answers1