Disable Linux kernel from killing postgresql process

Question

I have standard 'out-of-the-box' installation of

Linux version 3.0.1.stk64 (dfn@localhost.localdomain) (gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC) ) #1 SMP Sat Aug 13 12:53:46 EDT 2011

It has postgresql 8.4 installed as (start script)

/etc/init.d/postgresql

Data directory

/etc/postgresql/8.4/main/

My problem is that sometimes the Kernel decides to kill some of the Postgresql processes at times of low memory. I would like to inform kernel that Postgresql should not be chosen to be killed. I read from postgresql documentation (http://www.postgresql.org/docs/9.1/static/kernel-resources.html) that a command line echo -17 > /proc/self/oom_adj can be used to avoid the kill.

I tried to add this commandline to /etc/init.d/postgresql script but don't really know where to put it there.

Any pointers how to go about it? The scipt (/etc/init.d/postgresql) by the way is:

#!/bin/sh
set -e

### BEGIN INIT INFO
# Provides:             postgresql
# Required-Start:       $local_fs $remote_fs $network $time
# Required-Stop:        $local_fs $remote_fs $network $time
# Should-Start:         $syslog
# Should-Stop:          $syslog
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    PostgreSQL RDBMS server
### END INIT INFO

# Setting environment variables for the postmaster here does not work; please
# set them in /etc/postgresql/<version>/<cluster>/environment instead.

[ -r /usr/share/postgresql-common/init.d-functions ] || exit 0

. /usr/share/postgresql-common/init.d-functions

# versions can be specified explicitly
if [ -n "$2" ]; then
    versions="$2 $3 $4 $5 $6 $7 $8 $9"
else
    get_versions
fi

case "$1" in
    start|stop|restart|reload|status)
        for v in $versions; do
            $1 $v
        done
        ;;
    force-reload)
        for v in $versions; do
            reload $v
        done
        ;;
    *)
        echo "Usage: $0 {start|stop|restart|reload|force-reload|status} [version ..]"
        exit 1
        ;;
esac

exit 0

Note that nowadays (year 2020) postgres should default to guarding postgres main process from OOM Killer. However, if system is running out of memory and one of the postgres worker processes needs to be killed, the main process will restart automatically because Postgres cannot guarantee that shared memory area is not corrupted. As a result, you'll experience denial-of-service in any case. The only real way to avoid this is to not run out of memory. Solve that issue if you want real fix. — Mikko Rantalainen, Sep 11 '20 at 10:22

score 3 · Answer 1 · answered Oct 25 '12 at 16:16

3

You should adjust oom_adj directly to postmaster pid, add a lines like this (NOT TESTED):

pid=`cat $PGDATA/postmaster.pid | head -1`
echo -17 > /proc/$pid/oom_adj

answered Oct 25 '12 at 16:16

Andrei Mikhaltsov

3,027
1
23
31

I can confirm that /proc/$postmasterpid/oom_adj default value is zero whereas sshd has -17 on redhat and debian. Shouldn't -17 be the default and a feature request be done? – Oct 25 '12 at 16:25
-17 is default in postgres-9.0 (and 9.1), i think version 8 is using too old init script to adjust oom-killer parameters. – Andrei Mikhaltsov Oct 25 '12 at 16:40
BTW oom_adj is obsolete, http://kernel.org/doc/Documentation/ABI/obsolete/proc-pid-oom_adj – Andrei Mikhaltsov Oct 25 '12 at 16:42
Yes oom_score_adj should be used instead, but you need a recent kernel (not avail in rhel5). – Oct 25 '12 at 16:52
NOTE: I am not a Linux guru so I did not understand a single word of advice here .. what can I do to make it work with 8.4 asap and preferably so that it is part of the startup script(s) and works like a charm at automatic startup? – Martin Oct 26 '12 at 04:35
try adding two lines from the answer, if it doesn help (check by running command: service postgresql restart) then try this: find pidfile by running command "find / -name postmaster.pid", replace $PGDATA/postmaster.pid with full path of found file. – Andrei Mikhaltsov Oct 26 '12 at 11:37
Add the lines where (what row?)? I assume adding the lines would require the postgresql to be already running, which part of the script would best suit that? How can I test if it works, is there some evidence for example after reboot that I can check that the setting is activated? – Martin Oct 27 '12 at 00:12
I added those lines just before exit 0 at the end, hope it works. Is there a way to test it works? – Martin Oct 28 '12 at 01:40
yes, before exit is right position, just try to restart postgres by command: service postgresql restart and then look for errors and see the value of oom setting in /proc/PID/oom_adj (find postmaster pid in 'ps ax' command) – Andrei Mikhaltsov Oct 29 '12 at 10:32

Disable Linux kernel from killing postgresql process

1 Answers1