5

I have a program with three pools of structs. For each of them I use a list a of used structs and another one for the unused structs. During the execution the program consumes structs, and returns them back to the pool on demand. Also, there is a garbage collector to clean the "zombie" structs and return them to the pool.

At the beginning of the execution, the virtual memory, as expected, shows around 10GB* of memory allocated, and as the program uses the pool, the RSS memory increases.

Although the used nodes are back in the pool, marked as unused nodes, the RSS memory do not decreases. I expect this, because the OS doesn't know about what I'm doing with the memory, is not able to notice if I'm doing a real use of them or managing a pool.

What I would like to do is to force the unused memory to go back to virtual memory whenever I want, for example, when the RSS memory increases above X GB.

Is there any way to mark, given the memory pointer, a memory area to put it in virtual memory? I know this is the Operating System responsability but maybe there is a way to force it.

Maybe I shouldn't care about this, what do you think?

Thanks in advance.

  • Note 1: This program is used in High Performance Computing, that's why it's using this amount of memory.

I provide a picture of the pool usage vs the memory usage, for a few files. As you can see, the sudden drops in the pool usage are due to the garbage collector, what I would like to see, is this drop reflected in the memory usage.

Struct Pools Usage & Memory Usage

Carlos Vega
  • 1,341
  • 2
  • 13
  • 35
  • 3
    You might want to state your platform/OS, as this is highly system dependent. – Some programmer dude Jul 21 '14 at 09:34
  • I did the test in a Linux 2.6.32 (70GB of RAM) and Mac OS X Mavericks (16GB of RAM) – Carlos Vega Jul 21 '14 at 09:37
  • 2
    I don't see why would you want to force the system to mark your structures as "free" since you can re-use it in your program at leisure. – n0p Jul 21 '14 at 09:39
  • That's one of my concerns. Should I care? But, anyway, is there a way to force it? Suppose another process is using the machine, and there isn't enough memory for both process. How to tell which process the operating system would prioritise? Is there any way to say, "Hey, if you need memory I'm not using this, take it"? – Carlos Vega Jul 21 '14 at 09:45
  • 1
    Not trusting your C runtime library's implementation of malloc/free is almost never not a mistake. Of course you can get what you want, but you'll have to completely bypass the heap and allocate virtual memory yourself. Keep in mind that you can only release it when no allocations remain in the VM segment you allocated, that tends to be harder than it looks and only works if your program releases chunks of structs in roughly the same order it allocated them. And keep in mind that VM is just numbers to the processor, there is little point in releasing a number quickly. – Hans Passant Jul 21 '14 at 09:46
  • Aslo, more details. I do not own the test machine. It's a SUSE Linux Enterprise Server 11 with, again, 70GB of RAM. Not owning the machine limits my capacity to perform tests. – Carlos Vega Jul 21 '14 at 09:47
  • At the risk of starting a fire storm (and that is *not* the intent), `zombie` has a terrible connotation (not just of the Hollywood ilk). Would it be at-all feasible to implement this in such a way that that simply does not happen in the first place? (apologies in advance if that is a sour subject). It would seem that would alleviate you of much of this concern. (nice graphs btw). – WhozCraig Jul 21 '14 at 09:47
  • The zombie thing is not relevant. At some point, some structs become useless, like dead connections, and got to be removed (returned to the pool). – Carlos Vega Jul 21 '14 at 09:51
  • For now, what I got is that "I shouldn't care too much about this" and "In case I would like to do it, is not easy". I knew that conditions of the virtual memory. I tried to use aligned memory but do it takes long time to alloc the memory and I don't improve my execution times. – Carlos Vega Jul 21 '14 at 09:55

2 Answers2

3

You can do this as long as you are allocating your memory via mmap and not via malloc. You want to use the madvise function with the POSIX_MADV_DONTNEED argument.

Just remember to run madvise with POSIX_MADV_WILLNEED before using them again to ensure there is actually memory behind them.

This does not actually guarantee the pages will be swapped out but gives the kernel a strong hint to do so when it has time.

Vality
  • 6,577
  • 3
  • 27
  • 48
  • Ok, but, there is no way to do it without mmap. It has some portability problems. – Carlos Vega Sep 07 '15 at 11:18
  • 1
    @carlosvega Yes, that is correct, it is highly unportable, however I can tell you there is no way to do this as part of the C standard and many platforms fundamentally cannot do this sort of thing. This method will however work on a fair number of unix likes so is likely as good as you can get. – Vality Sep 08 '15 at 03:13
0

Git 2.19 (Q3 2018) offers an example of memory pool of struct, using mmap, not malloc.

For a large tree, the index needs to hold many cache entries allocated on heap.
These cache entries are now allocated out of a dedicated memory pool to amortize malloc(3) overhead.

See commit 8616a2d, commit 8e72d67, commit 0e58301, commit 158dfef, commit 8fb8e3f, commit a849735, commit 825ed4d, commit 768d796 (02 Jul 2018) by Jameson Miller (jamill). (Merged by Junio C Hamano -- gitster -- in commit ae533c4, 02 Aug 2018)

block alloc: allocate cache entries from mem_pool

When reading large indexes from disk, a portion of the time is dominated in malloc() calls.
This can be mitigated by allocating a large block of memory and manage it ourselves via memory pools.

This change moves the cache entry allocation to be on top of memory pools.

Design:

The index_state struct will gain a notion of an associated memory_pool from which cache_entries will be allocated from.
When reading in the index from disk, we have information on the number of entries and their size, which can guide us in deciding how large our initial memory allocation should be.
When an index is discarded, the associated memory_pool will be discarded as well - so the lifetime of a cache_entry is tied to the lifetime of the index_state that it was allocated for.

In the case of a Split Index, the following rules are followed.
1st, some terminology is defined:

Terminology:

  • 'the_index': represents the logical view of the index
  • 'split_index': represents the "base" cache entries. Read from the split index file.

'the_index' can reference a single split_index, as well as cache_entries from the split_index. the_index will be discarded before the split_index is.
This means that when we are allocating cache_entries in the presence of a split index, we need to allocate the entries from the split_index's memory pool.

This allows us to follow the pattern that the_index can reference cache_entries from the split_index, and that the cache_entries will not be freed while they are still being referenced.

Managing transient cache_entry structs:

Cache entries are usually allocated for an index, but this is not always the case. Cache entries are sometimes allocated because this is the type that the existing checkout_entry function works with.
Because of this, the existing code needs to handle cache entries associated with an index / memory pool, and those that only exist transiently.
Several strategies were contemplated around how to handle this.

Chosen approach:

An extra field was added to the cache_entry type to track whether the cache_entry was allocated from a memory pool or not.
This is currently an int field, as there are no more available bits in the existing ce_flags bit field.
If / when more bits are needed, this new field can be turned into a proper bit field.

We decided tracking and iterating over known memory pool regions was less desirable than adding an extra field to track this state.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250