Capture output from subprocess.call that I have no control over

Question

I'm testing a piece of Python code that uses subprocess.call(), so I have no control over that function call. I need to capture the output from that system call to do assertions. I tried to set os.stdout to a StringIO object, but that doesn't capture the system call outputs. How do I solve this problem? Here's my code so far:

Code to test (I have no control over this):

def runFile(filename):
    check_call("./" + filename)

My attempt to capture system call output:

import StringIO
oldout = sys.stdout
try:
    sys.stdout = StringIO.StringIO()
    runFile("somefile")
    output = sys.stdout.getvalue()
    assert output == "expected-output"
finally:
    sys.stdout = oldout

score 1 · Accepted Answer · answered Jun 02 '12 at 10:31

1

You could redirect your own stdout by

rd, wr = os.pipe()
oldstdout = os.dup(1)
os.dup2(wr, 1)
os.close(wr)

and then read from rd while the external process is running.

This might not work, as the external process might write more than fits in the pipe buffer. In this case, you would have to spawn a reading thread.

Afterwards, you restore the old state with

os.dup2(oldstdout, 1)
os.close(oldstdout)
os.close(rd)

and continue normally.

answered Jun 02 '12 at 10:31

glglgl

89,107
13
149
217

I do like this approach, but is there a way to make the current process not block if there's a lot of output? – Addison Jun 09 '12 at 03:49
If you can't change the function call to `check_call()`, then no. If you can change it, you could either give it a `stdout=subprocess.PIPE` and poll that `stdout` in another thread, or you could replace that `check_call()` with a `Popen`, a `stdout.read()` on that object and a `wait()`. But if you can't change, it's the only option (besides patching subprocess). – glglgl Jun 09 '12 at 08:08

score 0 · Answer 2 · answered Jun 02 '12 at 09:15

The subprocess module directly writes to the output stream using os.write, so changing sys.stdout won't do anything.

That's why you generally need to specify the output stream used with the subprocess module using the ...([cmd, arg, ...], stdout=output_file, ...) notation, and why you can't use a StringIO as output file as it has no fileno to write to.

So the only way to achieve what you want to do ist o monkeypatch subproces.check_call before importing the module you need to test, so that it works a little more like check_output.

For example:

import subprocess

def patched_call(*popenargs, **kwargs):
    if 'stdout' in kwargs:
        raise ValueError('stdout argument not allowed, it will be overridden.')
    process = subprocess.Popen(*popenargs, stdout=subprocess.PIPE, **kwargs)
    output, unused_err = process.communicate()
    # do something with output here
    retcode = process.poll()
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        raise CalledProcessError(retcode, cmd, output=output)
    return retcode

subprocess.check_call = patched_call

This seems like a good solution, but it doesn't capture all output. This approach would necessitate having a patch for each variation of the call() function, e.g. check_call(), output_call(), call(), Popen(), etc. — Addison, Jun 09 '12 at 03:51

Capture output from subprocess.call that I have no control over

2 Answers2