Parent bash script not receiving 'trap' despite process still running

Question

What I'm actually trying to achieve:

I'm trying to get a custom daemon working on a system that uses SysVinit. I have the bootstrapper /etc/init.d/xyz script already, which calls my daemon, but it doesn't automatically place it in the background. This is similar to how services like nginx behave: the binary backgrounds itself - i.e. it's not the responsibility of the /etc/init.d/nginx script to daemonise the process, so if you ran /opt/nginx/sbin/nginx directly you would also experience daemonised/background execution.

The Problem

My problem is that using my current method, the daemon doesn't terminate with the parent process (which is what gets terminated when you call service xyz stop).

I'm using a parent launcher.sh script that runs a daemon.sh & script. However, when I kill launcher.sh the daemon.sh continues to run, despite my best efforts with trap (it simply never gets called):

-> launcher.sh

#!/bin/bash

function shutdown {
    # Get our process group id
    PGID=$(ps -o pgid= $$ | grep -o [0-9]*)

    echo THIS NEVER GETS CALLED!

    # Kill process group in a new process group
    setsid kill -- -$$
    exit 0
}

trap "shutdown" SIGTERM

# Run daemon in background

./daemon.sh &

-> daemon.sh

#!/bin/bash

while true
do
    sleep 1
done

To run & kill:

./launcher.sh

<get PID for launcher>

kill -TERM 123 # PID of launcher.sh... which _is_ still running and has its own PID.

Result: daemon.sh still running and the shutdown function never gets called - I've confirmed this before by placing an echo here in the function body.

Any ideas?

EDIT: The launcher.sh script is being run using daemon launcher.sh, where daemon is a function provided by Amazon Linux's init.d/functions file (see here: http://gist.github.com/ljwagerfield/ab4aed16878dd9a8241b14bc1501392‌f).

The script `launcher.sh` in OP exits after starting `daemon.sh`. Assuming that 123 is the PID of `launcher.sh`, it is surprising that `kill -TERM 123` works. (You mention in a comment that "the [launcher?] script exits but the process continue to run", which doesn't make sense- what is that process actually doing?!) As such, I believe your question is missing some crucial detail. E.g.: How exactly do you get the PID of `launcher.sh`? Are you placing `launcher.sh` in the background? Are you piping any of the fds of `launcher.sh` into anything not shown here? — Matei David, Sep 15 '16 at 15:04

score 1 · Accepted Answer · answered Sep 15 '16 at 12:26

The trap command only works as long as the script is running.

The way this is normally done is that, when the daemon is forked off, it writes its PID into a file. The init script then either uses that file to determine what process to kill, or calls your launcher script to kill the process.

For the first instance:

launcher.sh:

/path/to/daemon.sh &
echo "$!" > /var/run/xyz.pid

A simple and somewhat naieve version of /etc/init.d/xyz:

# ... pull in functions or sysconfig files ...
start() {
    # ... do whatever is needed to set things up to start ...
    /path/to/launcher.sh
}
stop() {
    # ... do whatever is needed to set things up to stop ...
    kill `cat /var/run/xyz.pid`
}
# ... other functions ...

A non-naive startup script will depend on which version of linux you're running; I would suggest looking at other examples in /etc/init.d to see how they do this.

score 0 · Answer 2 · answered Sep 15 '16 at 12:16

It doesn't make sense to me why you'd want to have two scripts do this. Can you just call daemon.sh & in your init script? Or perhaps you can use the daemon command.

NAME
       daemon - turns other processes into daemons

SYNOPSIS
        usage: daemon [options] [--] [cmd arg...]

If you need to use trap, perhaps you can use it in daemon.sh for a clean shutdown. It's hard to tell if these are your real scripts or just examples.

Part of the problem with launcher.sh is that it exits... there's nothing keeping it running, so you can't kill it - it's already gone. I'm not just saying this, I actually tested your script to make sure before I answered. See my comments added to your script.

#!/bin/bash

function shutdown {
    # Get our process group id
    PGID=$(ps -o pgid= $$ | grep -o [0-9]*)

    echo THIS NEVER GETS CALLED!

    # Kill process group in a new process group
    setsid kill -- -$$
    exit 0
}

trap "shutdown" SIGTERM

# Run daemon in background *** script keeps running ***

./daemon.sh &

# It exits here
echo "Exiting... bye!"

Re. 1st comment: the script gets launched using `daemon xyz` - if I make it `daemon xyz &` the output is a little funny, implying that's not how it's supposed to be called. Re. 2nd comment: I'm using `daemon` in my `init.d` script already, but unfortunately it doesn't run the script in the background (see Amazon Linux `init.d/functions` file here: https://gist.github.com/ljwagerfield/ab4aed16878dd9a8241b14bc1501392f). Re 3rd comment: while the script definitely exits, the process continues to run... I can see it in `ps aux` and I can `kill` it successfully, but `daemon.sh` remains. — Lawrence Wagerfield, Sep 15 '16 at 12:28
@LawrenceWagerfield You should add that missing detail to your question, it changes things. But it still doesn't explain why you need to do it this way. You don't need two scripts, unless you've left out more information about your requirements. — Ryan Babchishin, Sep 15 '16 at 12:56

Parent bash script not receiving 'trap' despite process still running

2 Answers2