strange memory behave of resize and reserve about vector

Question

I know reserve(n) only allocate n space but size is zero.

while resize(n) allocate n space and size is n.

but when I observe the memory change by window task manager in the test , I'm confused by the result.

I have a code already take 100MB, then I test the code below step by step:

vector<Eigen::Vector4f> vec;
vec.resize(5000000); //memory still take 100MB???
vec.push_back(Eigen::Vector4f::Zero()); //memory take 181MB???

I'm curious why resize didn't change the memory until there is a push_back

vector<Eigen::Vector4f> vec;
vec.reserve(5000000); //memory still take 100MB???
for(int i = 0 ; i < 5000000; i++){
    vec.push_back(Eigen::Vector4f::Zero()); // memory increase one by one from 100MB to 181MB
}

reserve also didn't allocate memory ? why?

Probably commited vs uncommited OS memory pages. However this is outside the control of your C++ program and depends on detailed knowledge of how the OS reserves and/or allocates memory. — Richard Critten, Apr 18 '19 at 10:13
In your first case there are 5000001 `Eigen::Vector4f` objects owned by `vec`, and it may well be the case that you've just moved 5000000 of them — Caleth, Apr 18 '19 at 11:35

score 8 · Answer 1 · answered Apr 18 '19 at 10:28

It's not caused by the usage of reserve() or resize(). It is caused by your host operating system doing lazy allocation.

The first step is that operator new() (used by vectors default allocator) requests a lot of memory from the OS. The OS indicates to your program that it has done the allocation (e.g. by returning a valid handle for memory to operator new(), which in turn returns a valid pointer to your program) but does NOT actually allocate the memory.

Subsequently, when your program does something that relies on the memory being allocated, the OS catches the inevitable signal from the hardware memory management unit (e.g. as a page fault) and then allocates the page. That is happening in your usage of vectors push_back().

Some operating systems deliberately do this, or can be configured to, to avoid over-allocating memory. The hope of such an approach is that a large amount of memory may be requested but never actually used by a program. The reason is that large allocations can affect system-wide performance (e.g. slow down the operating system, cause other programs to be swapped out, etc).

One consequence is that the allocation may - as far as your program is concerned - succeed but a later usage of that allocated memory may fail (e.g. a call of push_back() after usage of reserve() can fail).

strange memory behave of resize and reserve about vector

1 Answers1