In a Python 2.4 script, I would like to execute a os system call `ls -l` or `curl` for example and capture the output in a variable. How to do this?

Question

I am writing a python script on a remote server with an old version of python 2.4. In the script I want to issue commands like curl -XPUT 'http://somerul/_search' -d file.txt or an ls -ltrh and capture the outputs of these commands into a variable. For the curl command the output will be a json format that I will parse (please advise if an old json parser is available for me to use)..

How can I make these kinds of system calls in the python script and capture the output into a variable?

Thank you in advance!

Answer 1

You can't do it directly with os.system or anything else in os. As the documentation* tells you, it just executes the command in a subshell. The command's stdout will be the shell's stdout which is your stdout, so there's nothing you can do to capture it.

That's why the docs explicitly tell you to use subprocess instead. And, if you read the docs, there's a nice section on Replacing Older Functions with the subprocess Module. Which includes exactly what you're trying to do as the first example.

Unfortunately, what you really want here is check_output, which is unfortunately 2.7+. If you had that, it would just be:

out = subprocess.check_output(['ls', '-ltrh'])

---

The simplest way to get it is to install subprocess32 from PyPI, which backports everything from Python 3.2's subprocess module to 2.4-2.7.

If you want to stick with the stdlib, as you can see from the 2.7 source, this is a trivial function to write on top of pieces that were already there in 2.4:

def check_output(*popenargs, **kwargs):
    if 'stdout' in kwargs:
        raise ValueError('stdout argument not allowed, it will be overridden.')
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
    output, unused_err = process.communicate()
    retcode = process.poll()
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        raise CalledProcessError(retcode, cmd, output=output)
    return output

Or, if you just want to do this one time, you don't need the wrapper:

p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out, _ = p.communicate()
do_something_with(out)

---

For your parenthetical extra question about a JSON parser:

simplejson is the parser that became the stdlib json module in 2.6. The latest versions only work with 2.5+, but older versions (I believe everything before 3.0, but don't quote me on that) were for 2.4+.

---

* Note that I linked to 2.7, because 2.4 is harder to find online. But they didn't remove any features from os.system or subprocess between 2.4 and 2.7, and they do carefully note what new features were added or changed.

Answer 2

Here's how you do it in Python 2.7. I don't have the 2.4 interpreter, so I can't test it out for you:

import subprocess
cmd = ['ls', '-ltrh']
p = subprocess.Popen(cmd, stdout = subprocess.PIPE)
result = p.stdout.read()
retval = p.wait()

Note that for subprocess to properly work, you have to give the command as a list of arguments.