2

I have an application that mmaps a large read-only data structure into memory and reads parts of it. After some time, around 1 GB of physical memory is filled with pages loaded from this data structure. When the amount of free memory on that machine drops below a certain threshold (~750 MB or so), the kernel seems to release all (or almost all) pages loaded through mmap.

How does Kernel decide when to start unloading mmapped pages? Is there a way to control that threshold? (e.g. anything in /proc/sys/vm or cgroups)

Pavel
  • 1,038
  • 1
  • 11
  • 30
  • https://stackoverflow.com/questions/26381256/when-system-run-out-of-memory-the-mmap-memory-is-swapped-to-swap-area-or-the-ma – c4f4t0r May 22 '19 at 22:53

3 Answers3

1

From manual, the munmap() system call deletes the mappings for the specified address range, causing further references to addresses within the range to generate invalid memory references.

DIRTY PAGE HANDLING How munmap() handles a dirty page, depends on what type of memory is being unmapped:

 [Anonymous]        If the memory is anonymous memory and if the last reference is going away, then the contents are discarded by definition of anonymous memory.
 [System V Shared]  If the memory mapping was created using System V shared memory, then the contents persist until the System V memory region is destroyed or the sys-                        tem is rebooted.
 [File mapping]     If the mapping maps data from a file (MAP_SHARED), then the memory will eventually be written back to disk if it's dirty.  This will happen automat-                        ically at some point in the future (implementation dependent).  Note: to force the memory to be written back to the disk, use msync(2).
 If there are still other references to the memory when the munmap is done, then nothing is done to the memory itself and it may be swapped out if need be. The memory
 will continue to persist until the last reference goes away
asktyagi
  • 2,860
  • 2
  • 8
  • 25
0

I don't think there is an individual control for each mmaped region: when the kernel decides it needs more memory, some pages will be "swapped out". But since these days nobody uses swap anymore, the only pages that can be safely freed are the ones in filesystem buffer cache (from which files are mmaped, but also just cache, for when a process needs a file, it will already be in memory). I don't think the heuristics for choosing which pages to free cares about what pages are mapped or not, only about its likelyhood to be needed in a near future.

What you can do is mlock() the pages you don't want the kernel to release, so that they are never "swapped out".

lvella
  • 314
  • 2
  • 13
0

Tweaking the thresholds is a bit tricky. There always needs to be some buffer of free memory, for good performance. And personally, I don't fully understand virtual memory tuning with regards to mmap pages vs anonymous.

For more of the working set in memory, add RAM to the host. Or maybe a faster storage system, to read parts of the file in.

John Mahowald
  • 32,050
  • 2
  • 19
  • 34