4

I know there are some similar questions, here Invoking C compiler using Python subprocess command and subprocess, invoke C-program from within Python but I believe my question is in some sense different.

I need to compile a c++ program which uses some ROOT libraries so I need to add some flags and link some libraries for the compilation. Therefore my compilation line on the normal shell is:

> $($ROOTSYS/bin/root-config --cxx) $($ROOTSYS/bin/root-config --cflags --glibs) Analysis.cxx -o analysis.exe

which works nicely. I want to do this compilation from my python script. I have read the documentation for the subprocess module but I could not get a solution without using shell=True in the call of subprocess.Popen and I do not really undestand the difference. If I use:

process = Popen(["$($ROOTSYS/bin/root-config --cxx) $($ROOTSYS/bin/root-config --cflags --glibs) Analysis.cxx -o analysis.exe"], shell=True)

does the job. However, this:

process = Popen(["$($ROOTSYS/bin/root-config --cxx)", "$($ROOTSYS/bin/root-config --cflags --glibs)", "Analysis.cxx", "-o", "analysis.exe"])

I got the following:

    Traceback (most recent call last):
  File "make_posanalysis.py", line 45, in <module>
    "Analysis.cxx", "-o", "analysis.exe"])
  File "Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I would like to understand the difference between using/not using shell=True since it seems to be the reason behind making the script work or not. Or, is there something else I am missing?

pseyfert
  • 3,263
  • 3
  • 21
  • 47
Arturo Rodriguez
  • 85
  • 8

1 Answers1

2

From the documentation:

If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

So it'e executing something equivalent to:

/bin/sh -c '$($ROOTSYS/bin/root-config --cxx)' '$($ROOTSYS/bin/root-config --cflags --glibs)' "Analysis.cxx", "-o", "analysis.exe"

This isn't what you want, because it only performs $(...) expansion in the first argument; everything else is taken literally, and become the positional arguments if the command in the first argument refers to $1, $2, etc.

If you want everything parsed by the shell, just give a single string.

Community
  • 1
  • 1
Barmar
  • 741,623
  • 53
  • 500
  • 612
  • So if I got it right, there is no way to do it without the shell=True because I need to perform two `$(...)` expansions? – Arturo Rodriguez Feb 22 '19 at 20:04
  • Yes, that's correct. You could execute those commands yourself, save the output, and then build the command from that. – Barmar Feb 22 '19 at 20:05
  • So basically the only truly shell interaction (call the shell command, not sure if I can called it "interaction") happens with the first argument if I pass the command as a list. Thanks a lot! – Arturo Rodriguez Feb 22 '19 at 20:44
  • 1
    If you pass the command as a list, the first element is treated as the shell command, the rest are arguments. If you pass it as a string, the whole string is a shell command. – Barmar Feb 22 '19 at 21:14