Process is not respawning when an associated post-stop process is killed

Question

I have a process xyz, whose upstart script is as below

description "Run the xyz daemon"
author      "xyz"

import SETTINGS

start on (
  start-ap-services SETTINGS
)
stop on (
  stop-ap-services SETTINGS
) or stopping system-services

respawn
oom score 0

script
  . /usr/share/settings.sh

  # directory for data persisted between boots
  XYZ_DIR="/var/lib/xyz"
  mkdir -p "${XYZ_DIR}"
  chown -R xyz:xyz "${XYZ_DIR}"

  if [ "${SETTING}" = 1 ]
      ARGS="$ARGS --enable_stats=true"
  fi

    # CAP_NET_BIND_SERVICE, CAP_DAC_OVERRIDE.
    exec /sbin/minijail0 -p -c 0x0402 -u xyz -g xyz \
      -G /usr/bin/xyz ${ARGS}
  else
    exec sleep inf
  fi
end script

# Prevent the job from respawning too quickly.
post-stop exec sleep 3

Now, due to OOM issue. xyz is killed based on it's OOM score and it gets respawned as expected. After a several restart of xyz, the post-stop sleep is killed after which xyz is never respawned.

How can this be prevented or Is there any solution to this?

Note: Name xyz is a dummy process name used only to mention my actual doubt.

I haven't worked on upstart scripts before. Any help would be of greater help.

I think you made an error when copying your configuration. There is not post-stop stanza in the config you posted. — CameronNemo, Apr 22 '19 at 23:19
I did not see that, and I was confused about the dangling `else ... fi` statement after the exec. — CameronNemo, Apr 26 '19 at 07:39

score 1 · Answer 1 · answered Apr 26 '19 at 07:16

Upstart can get confused when post-stop, pre-start, and post-start sections remain running across respawns.

I prefer to keep any command that takes longer than a few hundred milliseconds in a main job section, using auxiliary jobs if necessary.

For example, this will stall a job xyz that is being respawned or otherwise stopped:

start on stopping xyz RESULT='ok'
task
exec sleep 3

This has the same effect as your post-stop stanza, except that Upstart can better handle the state tracking for the simplified main job.

Process is not respawning when an associated post-stop process is killed

1 Answers1