1

How to have Linux wait till my C++ program completes its cleanup routine. The program initially calls function sigaction(2) to register a custom SIGTERM handler. If test the handler by running kill -s TERM $(ps -C a.out -o pid=), it would have no problem. However, in the real shutting down is another case. Sometimes the handler can get its job all done, but sometimes not. Apparently there is a race condition when the machine is shutting down. Does anyone know how to make the system wait a little bit longer so as to avoid the race condition? Thanks.

Cody
  • 609
  • 4
  • 21
  • 4
    This depends on what kind of `init` system your Linux distribution uses: classic SysVinit, `upstart`. `systemd` or perhaps some more uncommon one? If you don't know, please tell the name and version of your Linux distribution at least. Generally, if you create the proper configuration to start your program automatically at boot, it will also include a way to specify how long to wait when shutting it down. If you just start your program manually and let it die when shutting down, it will be handled by "general treatment for loose processes", which may be too strict for your program. – telcoM May 21 '19 at 16:06
  • @telcoM So the quick answer is "No, you can't, unless you put something into /etc/rc6.d/ to close your application with IPC and wait for it to finish."? – Cody May 22 '19 at 15:33
  • 1
    When using `systemd`, you could specify your kill command as the `ExecStop=` command, and then specify a `TimeoutStopSec=` value to adjust the timeout. With `SysVinit`, the killing of loose processes at shutdown usually happens very late, and certainly only after any service-specific shutdown scripts are completed, so just having a startup/shutdown script that sends the signal at an earlier phase of the shutdown process and optionally uses `sleep` or something to give your service a bit more time before your program's shutdown script exits might be enough. – telcoM May 24 '19 at 05:11
  • In other words, the quick answer is "Yes, but the recommended ways to achieve the desired behavior depend on the init system used." – telcoM May 24 '19 at 05:14
  • @telcoM Yesterday I tried /etc/init.d/ and related /etc/rc?.d/ but in vain, because the application was killed (without properly releasing its resource) before the shell script in /etc/init.d/ had a chance to run. Any suggestion? By the way, I added an entry in "Session and Startup" (Xfce4) for the application to run automatically every time the system boots up. Are there any mysterious mechanism behind the scene for the applications listed in "Session and Startup" to die that kills the application (improperly) before /etc/init.d/ has any chance to do it? Thanks. – Cody May 24 '19 at 08:37
  • 1
    Maybe; it depends on which Linux distribution you're using and what init system it has. In RHEL 6.x, there is an extra lock file requirement for `init.d` scripts, which causes the shutdown scripts to skip things that have been started manually, without using a startup script. In `systemd`, similar behavior is inherent to the way `systemd` works. **Could you please disclose the Linux distribution you're asking about?** – telcoM May 24 '19 at 16:40
  • @telcoM My OS is MX 18.2. Sorry I forgot to mention it. – Cody May 24 '19 at 16:59
  • @Cody I’ve faced exactly same issue, have you find the solution to this problem ? – atari83 Aug 04 '21 at 23:14

1 Answers1

0

In the comments you said you use MX Linux version 18.2. It seems to be based on Debian 9, which uses systemd by default, but still has the option to revert to classic SysVinit if desired. The web pages of MX Linux seem to emphasize the UI and do not mention anything special about the init system, so I'll assume it uses systemd too.

With systemd, a mechanism called control groups (cgroups for short) is in play: when systemd starts a service, it will also place its process in a special cgroup. This cgroup is automatically inherited by any process started by the service. When systemd is stopping a service, it will first execute any custom ExecStop actions, if any are defined for the service, wait for TimeoutStopSec, and if there are then any processes left in the service's cgroup, systemd will send them a SIGTERM, wait for another TimeoutStopSec, and then will send a SIGKILL for any left-over processes in the cgroup.

The thing tripping you up is probably the fact that user sessions are also encapsulated in a cgroup: anything you start manually, with e.g. sh /etc/init.d/yourservice start will still count as part of your user session, even if it performs all the actions needed to classically daemonize. And so, when you initiate a shutdown, the first action is to log out any user sessions... which causes your manually-started service to first receive a SIGHUP, then SIGTERM after a short delay, and possibly a SIGKILL after another short delay. Once the user sessions are cleaned up, the rest of the shutdown process will happen.

In order to use init.d scripts successfully with systemd, you'll need to know a few things.

systemd's compatibility mechanism for init.d scripts works by automatically generating a native systemd .service unit file for every init.d script, and then using those unit files just like native systemd services. This causes three requirements you might be unfamiliar with:

  • your init.d script should have a Linux Standard Base comment block before any non-comment lines in the script that describes the dependencies to any other services. It should look somewhat like this (example from Dovecot IMAP server):
### BEGIN INIT INFO
# Provides:          dovecot
# Required-Start:    $local_fs $remote_fs $network $syslog $time
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Should-Start:      postgresql mysql slapd winbind nslcd
# Should-Stop:       postgresql mysql slapd winbind nslcd
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Dovecot init script
# Description:       Init script for dovecot services
### END INIT INFO

  • After placing your init.d script, you should run systemctl daemon-reload to trigger the re-run of the systemd-sysv-generator that produces the unit file that will call your script.

  • After placing the script as /etc/init.d/yourservice and running systemctl daemon-reload, you should start the service using either systemctl start yourservice or service yourservice start. Only these methods cause systemd to place your service to its own control group, which will be important for the orderly shutdown your service needs. Running sh /etc/init.d/yourservice start will not do that.

You can use systemctl cat yourservice to view the resulting auto-generated yourservice.service unit file.

It might be a better idea to just write a native systemd service unit file for your service. You can find the distribution's standard unit files at /lib/systemd/system/ directory; you can use them as examples, but you should place your custom unit file into /etc/systemd/system instead, so your unit file won't be overridden by any package updates.

telcoM
  • 229
  • 3
  • 12