OS system call to Sicstus hangs indefinitely using Python

Question

I'm trying to write a proofchecking application that receives proofs from a user on a website and sends it through to a Prolog script to check its validity.

I'm using Django, Python 2.7 and Sicstus. In my server "view.py" file, I call a python script "checkProof.py", passing it the raw text form of the proof the user submits. Inside of that file I have the following function:

def checkProof(pFile, fFile):
    p = subprocess.Popen(['/bin/bash', '-i', '-c', 'sicstus -l ProofServer/server/proofChecker.pl -- %s %s' % (pFile, fFile)],
        stdout=subprocess.PIPE)
    p.communicate() # Hangs here.

proofChecker.pl receives a modified version of the proof (pFile), analyses it and outputs feedback into a feedback file (fFile). The Python script loops until the feedback file is generated, and returns this to the rest of the server.

The first time I call this function, everything works fine and I get the expected output. The second time I call this function, the program hangs indefinitely at "p.communicate()".

This means that, currently, only one proof can be checked using the application between server restarts. The server should be able to check an indefinite number of proofs between restarts.

Does anyone know why this is happening? I'd be happy to include additional information if necessary.

Update

Based on advice given below, I tried three different kinds of calls to try to determine where the problem lies. The first is what I'm trying to do already - calling Sicstus on my real proofchecking code. The second was calling a very simple Prolog script that writes a hardcoded output. The third was a simple Python script that does the same:

def checkProof(pFile, fFile):
  cmd1 = 'sicstus -l ProofServer/server/proofChecker.pl -- %s %s' % (pFile, fFile)
  cmd2 = 'sicstus -l ProofServer/server/tempFeedback.pl -- %s %s' % (pFile, fFile)
  cmd3 = 'python ProofServer/server/tempFeedback.py %s %s' % (pFile, fFile)
  p = subprocess.Popen(['/bin/bash', '-i', '-c', cmd3],
      stdout=subprocess.PIPE)
  p.communicate() # Hangs here.

In all three cases, the application continues to hang on the second attempted call. This implies that the problem is not with calling Sicstus, but just with the way I'm calling programs in general. This is a bit reassuring but I'm still not sure what I'm doing wrong.

Before calling it the second time, make a `ps wwaux|grep sicstus` to see if the first process is still running... — false, Jun 01 '14 at 13:29
@false, I tried that. I found three processes related to sicstus, two that were clearly based on the code I wrote, and one that I didn't recognise. I was able to kill the first two processes, but not the third as its pid kept jumping around whenever I tried. After killing the first two processes, a second call to the subprocess still hangs indefinitely, and the two processes reappear. — Tagc, Jun 01 '14 at 13:35
When I start up sicstus (4.3b7) with `-f` (so nothing extra), I see only one process **with above method**. But `pstree -palhup $SICSPID` shows me three extra processes. Have you tried replacing sicstus by dummy script that emulates sicstus for a very specific task? — false, Jun 01 '14 at 13:53
I'll admit that what I actually tried was ps aux|grep sicstus, not ps wwaux|grep sicstus last night. I'm not sure if that makes any difference. I'll try the latter when I get into labs later today. "Have you tried replacing sicstus by dummy script that emulates sicstus for a very specific task?" As in, replacing all the Prolog part of the project? That wouldn't be feasible. Prolog is actually very ideal for what I'm trying to do, and I have to write about it in my report. I also don't have a great deal of time left for this. — Tagc, Jun 01 '14 at 13:56
My suggestion was just a means to narrow down the culprit. So for the moment, you accept only one program and produce one output. — false, Jun 01 '14 at 13:58
Updated. Also, I'm an idiot and the third process I was thinking of from last night was simply "grep --color=auto sicstus". The relevant process is "/bin/bash -i -c sicstus -l ProofServer/server/proofChecker.pl -- knowledgeBase.pl proofFeedback" as expected. — Tagc, Jun 01 '14 at 16:06

score 1 · Accepted Answer · answered Jun 04 '14 at 15:16

I managed to fix this issue, eventually.

I think the issue was that appending the -i (interactive) flag to bash meant that it expected input, and when it didn't get that input it suspended the process on the second call. This is what was happening when trying to replicate the process with something simpler.

I got rid of the -i flag, and found that it now raised the error "/bin/bash: sicstus: command not found", even though sicstus is on my server's PATH and I can call it fine if I ssh into the server and call it directly. I fixed this by specifying the full path. I can now check proofs an indefinite number of times between server restarts, which is great. My code is now:

def checkProof(pFile, fFile):
  cmd = '/usr/local/sicstus4.2.3/bin/sicstus -l ProofServer/server/proofChecker.pl -- %s %s' % (pFile, fFile)
  p = subprocess.Popen(['/bin/bash', '-c', cmd])
  p.communicate()

OS system call to Sicstus hangs indefinitely using Python

1 Answers1