Hanging parent process after running a timed-out subprocess and piping results

Question

I wrote some code to run a script (via a subprocess) and kill the child process after a certain timeout. I'm running a script called "runtime_hang_script.sh" that just contains "./runtime_hang," which runs an infinite loop. I'm also redirecting stdout to a pipe -- I plan to write it to both sys.stdout and to a file (aka I'm trying to implement tee). However, my code hangs after the subprocess times out. Note that this ONLY hangs when running "sh runtime_hang_script.sh" and not "./runtime_hang." Also, this doesn't hang when I try piping directly to a file or when I don't read from the pipe.

I've tried other implementations of creating a timed subprocess, but I keep on getting the same issue. I've even tried raising a signal at the end of the problem -- for some reason, the signal is raised earlier than anticipated, so this doesn't work either. Any help would be appreciated. Thanks in advance!

process = None

def run():
    global process
    timeout_secs = 5
    args = ['sh', 'runtime_hang_script.sh']
    sys.stdout.flush()

    process = subprocess.Popen(args, stdout=subprocess.PIPE, bufsize=1)

    with process.stdout:
        for line in iter(process.stdout.readline, b''):
            sys.stdout.write(line.decode('utf-8'))
        sys.stdout.flush()

    process.wait()


proc_thread = threading.Thread(target=run)
proc_thread.start()
proc_thread.join(5)

print(proc_thread.is_alive())

if proc_thread.is_alive():
    process.kill()

Is `process.kill()` killing only the shell and not the hanging process? — Davis Herring, Jul 02 '19 at 01:39
Yeah, I think this might be the problem. Is there a way to kill the hanging process as well besides calling popen on kill -9? — ElizWang, Jul 02 '19 at 16:21
You can always use `os.kill`—no reason for the complexity of starting yet more processes. Use `exec` in the shell script to avoid having two processes to kill. — Davis Herring, Jul 02 '19 at 17:25

score 0 · Answer 1 · answered Jul 02 '19 at 04:41

Assuming you are using Python 3.3 or newer, you can use the timeout argument of the subprocess.communicate() method to implement your 5-second timeout:

import subprocess
import sys

timeout_secs = 5
args = ['sh', 'runtime_hang_script.sh']

process = subprocess.Popen(args, stdout=subprocess.PIPE, bufsize=1)
try:
   print("Waiting for data from child process...")
   (stdoutData, stderrData) = process.communicate(None, timeout_secs)
   print("From child process:  stdoutData=[%s] stderrData=[%s]" % (stdoutData, stderrData))
except subprocess.TimeoutExpired:
   print("Oops, child process took too long!  Now it has to die")
   process.kill()

print("Waiting for child process to exit...")
process.wait()
print("Child process exited.")

Note that spawning a child thread isn't necessary with this approach, since the timeout can work directly from the main thread.

Thanks for the response. Unfortunately, I'm not using Python 3.3+, so this doesn't work for me. Thanks again for the help though. — ElizWang, Jul 02 '19 at 16:25
Note that this doesn’t guarantee getting all the data the child did write, although that may not matter. — Davis Herring, Jul 02 '19 at 18:42

Hanging parent process after running a timed-out subprocess and piping results

1 Answers1