Linux SIGINT passive capture

Question

Is there a place where the Linux kernel passively logs SIGKILL (kill -9) shutdown requests?

I have a JVM running that is arbitrarily being shut down and I suspect that, based on the evidence available, is being shut down by a stray process that is somehow issuing a shutdown of the JVM process. I have robust logging in place but in order to confirm my suspicion, I'd have to turn up the logging level to overwhelming levels.

I've researched exhaustively through /var/log and can't seem to find any place that might capture and log these SIGKILL events. Any ideas where I might find these events, if they exist?

Answer 1

Option 1:

If your kernel has ftrace support (very likely) try the killsnoop tool from Brendan Gregg's perf-tools:

wget https://raw.githubusercontent.com/brendangregg/perf-tools/master/killsnoop
chmod +x killsnoop
sudo ./killsnoop -s

More usage examples in the killsnoop_example.txt file.

Option 2: (passive capture)

If your kernel has no ftrace support you can use the kernel-siglog kernel module from https://github.com/nfedera/kernel-siglog :

git clone https://github.com/nfedera/kernel-siglog.git

cd kernel-siglog/
make

sudo insmod siglog.ko

Once inserted the siglog kernel module will record the last 10,000 signals in /proc/siglog

I had a similar issue and found the culprit using this kernel module. I had it inserted on a customer's server for some weeks and when the service was killed I logged in, did a cat /proc/siglog and found that my service was killed by a customer's own buggy watchdog script.