4

I have a program that consist of a runner (which is the main thread) that creates 1 or more child threads which mainly use subprocesses to trigger 3rd party apps.

I wanted to be able to gracefully terminate all threads once a SIGINT is received, hence I defined a handler in my main thread as following:

signal.signal(signal.SIGINT, handler)

I initially though that once a SIGINT is received then it will affect only my main thread and then I'll be able to manage the child threads to terminate,.

However what I'm actually observing is that pressing control+c is also affecting my child threads (I'm seeing that the subprocess within the child thread is raising exception with RC 512 once I press control+c).

Can someone advise if it is possible that only the main thread will detect this signal without affecting the child threads?

  • Possible duplicate : [python-prevent-signals-to-propagate-to-child-threads](https://stackoverflow.com/questions/38596069/python-prevent-signals-to-propagate-to-child-threads) – stovfl Oct 11 '18 at 16:41
  • That isn't a threading problem. When you type ^C into a terminal window, Linux doesn't send the SIGINT to a specific process, it sends the signal to all of the members of a _[process group](https://en.wikipedia.org/wiki/Process_group)_. Each of the child processes that your program creates will be members of the same group as the parent, unless you take _[special steps to detach them](http://man7.org/linux/man-pages/man2/getpgrp.2.html)_. (p.s., I'm not a Python wizard, so I'm not sure what library calls you can make to achieve that.) – Solomon Slow Oct 11 '18 at 16:47
  • @stovfl I sow this topic, however couldn't find the answer there unfortunately. What I'm seeing is that the control+c can target the child thread first and not directly the handler that I defined – user3019483 Oct 11 '18 at 21:26
  • The dup tells otherwise. It's time to show a [mcve] why a `signal.signal(signal.SIGINT, signal_handler)` will not work for you. Relevant: [catch-keyboardinterrupt-or-handle-signal-in-thread](https://stackoverflow.com/questions/30267647/catch-keyboardinterrupt-or-handle-signal-in-thread) – stovfl Oct 12 '18 at 05:50

1 Answers1

5

If you use subprocess.Popen() to create child processes, and you do not want them to be killed by a SIGINT signal, use the preexec_fn argument to set the SIGINT signal disposition to ignored before the new binary is executed:

child = subprocess.Popen(...,
                         preexec_fn = lambda: signal.signal(signal.SIGINT, signal.SIG_IGN))

where ... is a placeholder for your current parameters.

If you use actual threads (either threads or threading module), Python's signal module sets everything up so that only the main/initial thread can receive signals or set signal handlers. So, proper threads are not really affected by signals in Python.

In the subprocess.Popen() case, the child process initially inherits a copy of the process, including signal handlers. This means that there is a small window during which the child process may catch a signal, using the same code as the parent process; but, because it is a separate process, only its side effects are visible. (For example, if the signal handler calls sys.exit(), only the child process will exit. The signal handler in the child process cannot change any variables in the parent process.)

To avoid this, the parent process can temporarily switch to a different signal handler, that only remembers if a signal is caught, for the duration of the subprocess creation:

import signal

# Global variables for sigint diversion
sigint_diverted   = False     # True if caught while diverted
sigint_original   = None      # Original signal handler

def sigint_divert_handler():
    global sigint_diverted
    sigint_diverted = True

def sigint_divert(interrupts=False):
    """Temporarily postpone SIGINT signal delivery."""
    global sigint_diverted
    global sigint_original
    sigint_diverted = False
    sigint_original = signal.signal(signal.SIGINT, sigint_divert_handler)
    signal.siginterrupt(signal.SIGINT, interrupts)

def sigint_restore(interrupts=True):
    """Restore SIGINT signal delivery to original handler."""
    global sigint_diverted
    global sigint_original
    original = sigint_original
    sigint_original = None
    if original is not None:
        signal.signal(signal.SIGINT, original)
        signal.siginterrupt(signal.SIGINT, interrupts)
    diverted = sigint_diverted
    sigint_diverted = False
    if diverted and original is not None:
        original(signal.SIGINT)

With the above helpers, the idea is that before creating a child process (using subprocess module, or some of the os module functions), you call sigint_divert(). The child process inherits a copy of the diverted SIGINT handler. After creating the child process, you restore SIGINT handling by calling sigint_restore(). (Note that if you called signal.siginterrupt(signal.SIGINT, False) after setting your original SIGINT handler, so that its delivery won't raise IOError exceptions, you should call here sigint_restore(False) instead.)

This way, the signal handler in the child process is the diverted signal handler, which only sets a global flag and does nothing else. Of course, you still want to use the preexec_fn = parameter to subprocess.Popen(), so that SIGINT signal is ignored completely when the actual binary is executed in the child process.

The sigint_restore() not only restores the original signal handler, but if the diverted signal handler caught a SIGINT signal, it is "re-raised" by calling the original signal handler directly. This assumes that the original handler is one you've already installed; otherwise, you could use os.kill(os.getpid(), signal.SIGKILL) instead.


Python 3.3 and later on non-Windows OSes exposes signal masks, which can be used to "block" signals for a duration. Blocking means that the delivery of the signal is postponed, until unblocked; not ignored. This is exactly what the above signal diversion code tries to accomplish.

The signals are not queued, so if one signal is already pending, any further signals of that same type are ignored. (So, only one signal of each type, say SIGINT, can be pending at the same time.)

This allows a pattern using two helper functions,

def block_signals(sigset = { signal.SIGINT }):
    mask = signal.pthread_sigmask(signal.SIG_BLOCK, {})
    signal.pthread_sigmask(signal.SIG_BLOCK, sigset)
    return mask

def restore_signals(mask):
    signal.pthread_sigmask(signal.SIG_SETMASK, mask)

so that one calls mask = block_signals() before creating a thread or a subprocess, and restore_signals(mask) afterwards. In the created thread or subprocess, the SIGINT signal is blocked by default.

Blocked SIGINT signal can also be consumed using signal.sigwait({signal.SIGINT}) (which blocks until one is delivered), or signal.sigtimedwait({signal.SIGINT}, 0) which returns immediately with the signal if one is pending, and None otherwise.


When a subprocess manages its own signal mask and signal handlers, we cannot make it ignore a SIGINT signal.

On Unix/POSIXy machines, we can stop the SIGINT from being sent to the child process, however, by detaching the child process from the controlling terminal, and running it in its own session.

There are two sets of changes needed in subprocess.Popen():

  • Execute the command or binary under setsid: either [ "setsid", "program", args.. ] or "setsid sh -c 'command'", depending on whether you supply the binary to be executed as a list or as a string.

    setsid is a command-line utility that runs the specified program with the specified arguments in a new session. The new session does not have a controlling terminal, which means it will not receive a SIGINT if the user presses Ctrl+C.

  • If the parent does not use a pipe for the subprocess' stdin, stdout, or stderr, they should be explicitly opened to os.devnull:

    stdin=open(os.devnull, 'rb'),
    stdout=open(os.devnull, 'wb'),
    stderr=open(os.devnull, 'wb')

    This ensures that the subprocess does not fall back under the controlling terminal. (It is the controlling terminal that sends the SIGINT signal to each process when user presses Ctrl+C.)

If the parent process wants to, it can send a SIGINT signal to the child process using os.kill(child.pid, signal.SIGINT).

Nominal Animal
  • 38,216
  • 5
  • 59
  • 86
  • Well I'm using Python 3.3+, I tried the solution you offered however subprocesses are still being affected by SIGINT. I see that my main thread create worker threads, and these worker threads trigger "Popen" frequently (SSH commands). Before "Popen" I called "mask = block_signals()", and after Popen I called "restore_signals(mask)", also the "preexec_fn" was define as you suggested. however when I start pressing control+c I still see that "Popen" subprocesses raise exceptions (As a result of RC!=0) instead of just triggering the SIGINT handler that was defined in the main thread. Any idea? – user3019483 Oct 29 '18 at 14:12
  • In addition, when using threads only (No sub-processes involved), the mechanism works exactly as expected (SIGINT/control+c trigger only the handler that was defined in the main thread) – user3019483 Oct 29 '18 at 14:13
  • @user3019483: Some processes, including `ssh`, install their own signal handlers and signal masks. The parent process cannot stop them from handling a SIGINT signal; it can only make it so that unless the child decides otherwise, SIGINT is ignored. So, to shield them from Ctrl+C pressed on the keyboard, they need to be detached from the terminal. That also means they cannot read from the keyboard, or write to the terminal; only to/from their parent process, or files or sockets. Can you use a non-Windows solution? – Nominal Animal Oct 29 '18 at 14:22
  • Yes, I mainly need a solution for linux systems. Any idea how this can be achieved in such cases were the process has its own signal handler? – user3019483 Oct 29 '18 at 14:30
  • @user3019483: Yes, appended to my answer. It boils down to running the subprocess under `setsid`, and making sure it does not have any handles open to the controlling terminal (by supplying handles to all standard streams in the subprocess.Popen() call). – Nominal Animal Oct 29 '18 at 15:02
  • @NominalAnimal Am I understanding this correctly that the shell decides whether or not SIGINT should propagate to child processes (in a process group) or not? Could you elaborate on how this decision-making process works in detail and why shells tend to send SIGINT to child processes by default? Also, why do open handles (stdin/stdout/stderr) play a role here? (It would make sense to me if Ctrl+C were sent to `stdin` but it is not and, instead, translated by the shell to SIGINT.) – balu Dec 22 '20 at 13:26