22

I have a LINUX server running a process with a large memory footprint (some sort of a database engine). The memory allocated by this process is so large that part of it needs to be swapped (paged) out.

What I would like to do is to lock the memory pages of all the other processes (or a subset of the running processes) in memory, so that only the pages of the database process get swapped out. For example I would like to make sure that i can continue to connect remotely and monitor the machine without having the processes impacted by swapping. I.e. I want sshd, X, top, vmstat, etc to have all pages memory resident.

On linux there are the mlock(), mlockall() system calls that seem to offer the right knob to do the pinning. Unfortunately, it seems to me that I need to make an explicit call inside every process and cannot invoke mlock() from a different process or from the parent (mlock() is not inherited after fork() or evecve()).

Any help is greatly appreciated. Virtual pizza & beer offered :-).

psmears
  • 26,070
  • 4
  • 40
  • 48
pol lol
  • 281
  • 1
  • 2
  • 5
  • mlockall(2) is a very very very bad idea to do this. 1. You need do munlockall for all other process otherwise the risky is OOM (But you shouldn't do this if the process has called mlockall by itself) – snyh Dec 12 '18 at 01:50
  • 2. The number of other process is varying. 3. Some special process can't be ptrace or it will change the original program's behavior subtly -------------- Just simply use cgroup to control you database RAM usage, or use systemd to manage it. – snyh Dec 12 '18 at 02:28

4 Answers4

15

It has been a while since I've done this so I may have missed a few steps.

Make a GDB command file that contains something like this:

call mlockall(3)
detach

Then on the command line, find the PID of the process you want to mlock. Type:
gdb --pid [PID] --batch -x [command file]

If you get fancy with pgrep that could be:
gdb --pid $(pgrep sshd) --batch -x [command file]

Zan Lynx
  • 53,022
  • 10
  • 79
  • 131
  • 1
    ....... Don't put a script like that where any of your users can find it... – twalberg Sep 20 '12 at 20:45
  • 3
    @twalberg: Unless the user is root, the amount of memory that can be locked is limited. Usually to something small like 64KB. There is an rlimit setting in most Linux systems for non-root mlock which allows it to be used for security purposes, to keep passwords and encryption keys out of swap files for example, but not for denial of service attacks. – Zan Lynx Sep 20 '12 at 20:48
  • Still... 100 users reserving 64KB for each of their 50 processes adds up to ... a lot... Maybe no one user can cause a DOS, but the cumulative effect can be decidedly negative. – twalberg Sep 20 '12 at 20:52
  • @twalberg: That would require all 100 users to agree to screw with the server all at the same time. Unlikely. – Zan Lynx Sep 21 '12 at 14:54
  • Nice idea! Will definitely try it even it there might be other ways. Too cool not to :-). Regarding DoS attacks, I'm not that worried but thank you for the heads up. This server has a lot of RAM (192GB) so 64kB waste per process is not an issue. Especially as no users connect directly to it. – pol lol Sep 21 '12 at 20:40
  • 1
    `call (int) mlockall(1)` worked for me in testing of block i/o contribution to latency. Very effective! – wick May 30 '21 at 17:50
7

Actually locking the pages of most of the stuff on your system seems a bit crude/drastic, not to mention being such an abuse of the mechanism it seems bound to cause some other unanticipated problems.

Ideally, what you probably actually want is to control the "swappiness" of groups of processes so the database is first in line to be swapped while essential system admin tools are the last, and there is a way of doing this.

Community
  • 1
  • 1
timday
  • 24,582
  • 12
  • 83
  • 135
  • 1
    Hi Timday, you are right about being too drastic. The memory cgroups seem to be _exactly_ what I need (at first sight at least, did not get to read all the docs). Very cool! I did not know such functionality exists. Thank you! – pol lol Sep 21 '12 at 20:32
  • 2
    Note that with cgroups v2 (which is slowly starting to become the default), there is also `memory.low` (and `memory.min`) setting that defines how much memory will not be swappped (if possible). I'm not sure how this works in detail, though. – Matthijs Kooijman Jun 09 '20 at 12:58
3

While searching for mlockall information I ran across this tool. You may be able to find it for your distribution. I only found the man page.

http://linux.die.net/man/8/memlockd

Zan Lynx
  • 53,022
  • 10
  • 79
  • 131
  • Hi Zan, thank you for the pointer. From what I understand mlockall() locks files in memory (rather than processes) but it is another tool that can come in handy. Thanks again! – pol lol Sep 21 '12 at 20:35
  • @pollol: Did you read the man page for `mlockall()`? It is not files, it is process memory. As a side effect, if you map a file into memory and lock it, it is locked into cache also. – Zan Lynx Sep 21 '12 at 20:40
  • My post was bad on several levels. First I was referring to memlockd rather than mlockall(). Second, I think I misunderstood the man page for memlockd. It says: "lock system programs and config files in memory". From this i understood that the executable of a program plus additional libraries are lock in the cache rather than the whole RSS of a process. – pol lol Sep 23 '12 at 15:05
  • 2
    Just in case somebody reads this: I checked out `memlockd`s source code, all it does is map all given process executables into memory `mmap()` and then locking that process binary down in memory `mlock()` (relevant source code: http://pastebin.com/t2csPhHP). It's NOT a replacement for `mlockall()`! – ntninja Oct 11 '13 at 19:43
1

Nowadays, the easy and right way to tackle the problem is cgroup.

Just restrict memory usage of database process:

1. create a memory cgroup
    sudo cgcreate -g memory:$test_db -t $User:$User -a $User:$User

2. limit the group's RAM usage to 1G. 
    echo 1000M > /sys/fs/cgroup/memory/$test_db/memory.limit_in_bytes
    or 
    echo 1000M > /sys/fs/cgroup/memory/$test_db/memory.soft_limit_in_bytes

3. run the database program in the $test_db cgroup
   cgexec -g memory:$test_db $db_program_name
peterh
  • 11,875
  • 18
  • 85
  • 108
snyh
  • 1,225
  • 14
  • 19
  • 4
    **Not an answer**, the goal is not to limit the memory usage, but to make processes exempt from swapping. – peterh Dec 12 '18 at 15:27
  • When the database process has been restrict the RAM usage(only the database process will be swapped out), Then the other process can be avoid swapping if the total RAM is enough. – snyh Dec 13 '18 at 03:23
  • @peterh the cgroup has the softlimit option which meanings the process(database) can beyond the limit memory restriction(1G in example) if the system can meet other process's RAM requirement. – snyh Dec 13 '18 at 03:26