Python subprocess block

Question

I'm having a problem with the module subprocess; I'm running a script from Python:

subprocess.Popen('./run_pythia.sh', shell=True).communicate()

and sometimes it just blocks and it doesn't finish to execute the script. Before I was using .wait(), but I switched to .communicate(). Nevertheless the problem continues.

First the script compiles a few files, then it execute into a file:

run_pythia.sh:

#!/bin/bash
#PBS -l walltime=1:00:00

./compile.sh
./exec > resultado.txt

compile.sh:

O=`find ./ -name "*.o" | xargs`

# LOAD cernlib2005
module load libs/cernlib/2005

# Compile and Link
FC=g77
CERNLIBPATH="-L/software/local/cernlib/2005/lib -lpacklib"

$FC call_pyth_mix.f analise_tt.f $O $CERNLIBPATH -o exec

Do you need to interact with the process? Or are you simply reading the output? You should use `.communicate()` as it'll prevent any deadlock issues. However, if you need to interact with the process by writing to stdin and reading from stderr/stdout, that's a whole new ball of wax. — dlamotte, May 05 '10 at 13:11

score 3 · Answer 1 · answered May 04 '10 at 23:49

Is the script you execute, is run_pythia.sh guaranteed to finish executing? If not, you might not want to use blocking methods like communicate(). You might want to look into interacting with the .stdout, .stderr, and .stdin file handles of the returned process handle yourself (in a non-blocking manner).

Also, if you still want to use communicate(), you need to have had passed subprocess.PIPE object to Popen's constructor arguments.

Read the documentation on the module for more details.

score 0 · Answer 2 · edited Aug 23 '12 at 20:51

0

Maybe you can try to do a trace on it:

import pdb; pdb.set_trace()

edited Aug 23 '12 at 20:51

Peter Mortensen

30,738
21
105
131

answered May 05 '10 at 10:01

krisdigitx

7,068
20
61
97

Python subprocess block

2 Answers2