1

I am running a program from a python script (currently with os.system). However, sometimes the program hangs at a point and I would like to kill it if nothing is written to stdout or stderr after a certain time interval. A simple timeout on the program won't work, because this code typically runs for a very long time (many hours to days), and sometimes it hangs before it still has a long way to go.

It seems that subprocess.Popen is the way to go, but I haven't found a good example on how to do this. I also wanted to write the stdout/stderr to a file.

Based on some examples I was thinking about something like this:

p = Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None)

while True:
    line = p.stdout.readline()
    outfile.write(line)
    # save current time or something, compare to time of
    # previous loop, if larger than timeout, kill process

But I'm not sure how to implement the time loop, and how to make sure that the while doesn't run forever when the process eventually terminates on its own (and doesn't hang). Any pointers would be much appreciated.

tiago
  • 22,602
  • 12
  • 72
  • 88
  • Why not fix the bug in the child process so it doesn't hang? – John Zwinck Mar 17 '13 at 13:45
  • @JohnZwinck, yes that is of course the best solution. However, in this case this is a rare occurrence and really hard to replicate. In some cases it also hangs for reasons outside the process (e.g. filesystem, etc.). – tiago Mar 17 '13 at 14:02

2 Answers2

2

Try using signal.alarm to set a timer after each line is received, and then handle SIGALRM by checking if too much time has passed since the last line.

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • Thank you. `signal.alarm` is really cool. I managed to implement what I wanted using it. – tiago Mar 19 '13 at 07:40
2

For completeness, here's the code I ended up using, making use of the suggested signal.alarm:

import time
import shlex
import subprocess

logfile = open(log, 'w', 1)
# cmd is command to run
args = shlex.split(cmd)   # tokenise args list
p = subprocess.Popen(args, shell=False, bufsize=0, stdin=None,
                     stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

def _handler(signum, frame):
    print('Timeout of %s min reached, stopping execution' % timeout)
    p.kill()
    time.sleep(30)  # to ensure no ghost process is left running
    raise RuntimeError('Timeout')

signal.signal(signal.SIGALRM, _handler)
try:
    while True:
        signal.alarm(int(timeout))
        inline = p.stdout.readline()
        if not inline:
            break
        logfile.write(inline)
        signal.alarm(0)
except RuntimeError:
    logfile.close()
    return 0

p.communicate()   # wait for process to finish, get return code
logfile.close()
return p.returncode
tiago
  • 22,602
  • 12
  • 72
  • 88