Python's Subprocess.Popen With Shell=True. Wait till it is completed

Question

Submitting a complex cmd string made of a full file path to an executable, the multiple flags, arguments, parameters, inputs and outputs seems to require me to set shell=True otherwise subprocess.Popen is not able understand anything more complex than just a simple path to executable (with no spaces in a filepath).

In my example I have quite a long cmd:

cmd = " '/Application/MyApp.app/Contents/MacOS/my_executable' '/Path/to/input/files' -some -flags -here -could -be -a -lot '/full/path/to/output/files' "

Submitting this cmd to subprocess.Popen " results to an error that complains on something about the path and not being able to find it.

So instead of using :

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

check_call seems workings quite well:

proc = subprocess.check_call(cmd, shell=True)

Interesting, only after shell is set to True

shell=True

the subprocess.check_call works with a supplied cmd.

The side effect is that the rest of the code seems proceeds running without waiting for subprocess.check_call(cmd, shell=True) to finish first.

The code is designed the way that the rest of the execution is dependent on a result of subprocess.check_call(cmd, shell=True).

I wonder if there is anyway to enforce the code execution to wait till subprocess.check_call(cmd, shell=True) is finished. Thanks in advance!

Have you tried converting the `cmd` into a list when running with `shell=False` and `subprocess.Popen`? You could use the `shlex.split()` function to do that, like in the example code provided in the [documentation of the Popen constructor](http://docs.python.org/2/library/subprocess.html#popen-constructor). — miikkas, Dec 08 '13 at 08:24

Kobi K · Accepted Answer · 2013-12-08T08:38:43.120

2

As @mikkas suggest just use it as a list here is a working example:

mainProcess = subprocess.Popen(['python', pyfile, param1, param2], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# get the return value from the method
communicateRes = mainProcess.communicate()

stdOutValue, stdErrValue = communicateRes

You are calling python.exe pyfile param1 param2

By using communicate() you can get the stdout and stderr as a Tuple

You can use python method split() to split your string to a list for example:

cmd = "python.exe myfile.py arg1 arg2"

cmd.split(" ")

Output:

['python.exe', 'myfile.py', 'arg1', 'arg2']

edited Dec 08 '13 at 08:38

answered Dec 08 '13 at 08:33

Kobi K

7,743
6
42
86

2

Avoid `cmd.split(' ')` because it cannot handle quotes. Use `shlex.split` instead, which knows the syntax used for shell's commands and thus it can handle things like `'program "a single argument with quotes"'` as expected. – Bakuriu Dec 08 '13 at 12:22
Nice didn't know that – Kobi K Dec 09 '13 at 07:38

score 1 · Answer 2 · answered Dec 08 '13 at 07:59

1

I think the check_call function should wait for the command to finish.

See the docs here http://docs.python.org/2/library/subprocess.html

answered Dec 08 '13 at 07:59

Aaron

2,354
1
17
25

I need to read the stdout of the process running. Does subprocess.check_call allow to read its stdout? If so would you please clarify how to accomplish this. – alphanumeric Dec 08 '13 at 08:14
1

why don't you try using subprocess.check_output instead? – Aaron Dec 08 '13 at 08:15

Sameer · Answer 3 · 2019-10-18T09:06:05.417

Check call does not wait. You need to do a process.wait() and check the return code explicitly to get the functionaly you want.

Process = subprocess.Popen('%s' %command_string,stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
            Process.wait()
            if Process1.returncode!=0:
                    print Process1.returncode
                    sendMail()
                    return
            else:
                    sendMail()

Python's Subprocess.Popen With Shell=True. Wait till it is completed

3 Answers3

Linked