3

I've got an Apache2/web2py server running using the wsgi handler functionality. Within one of the controllers, I am trying to run an external executable to perform some processing on 2 files.

My approach to this is to use the subprocess module to kick off the executable. I have simplified the code to a bare-bones implementation with little success.

from subprocess import *
p = Popen(("echo", "Hello"), shell=False)
ret = p.wait()
print "Process ended with status %s" % ret

When running the above code on its own (create new file and running via python command line), it works exactly as expected.

However, as soon as I place the exact same code into my web2py controller, the external process stops working. Instead of the process returning with code 0 as is expected in the above example, it always returns -6 and "Hello" is not printed to stdout.

After doing some digging, I found that negative results from p.wait() implies that a signal caused the process to end abnormally. And, according to some docs I found, -6 corresponds to the SIGABRT signal.

I would have expected this signal to be a result of some poorly executed code in my child process. However, since this is only running echo (and since it works outside of web2py) I have my doubts that the child process is signalling itself.

Is there some web2py limitation/configuration that causes Popen() requests to always fail? If so, how can I modify my logic so that the controller (or whatever) is actually able to spawn this external process?

** EDIT: Looks like web2py applications may not like the subprocess module. According to a reply to a message reply in the web2py email group:

"You should not use subprocess in a web2py application (if you really need too, look into the admin/controllers/shell.py) but you can use it in a web2py program running from shell (web2py.py -R myprogram.py)."

I will be checking out some options based on the note here and see if any solution presents itself.

zashu
  • 747
  • 1
  • 7
  • 22
  • I'm having the same problem [here](http://stackoverflow.com/questions/25501052/python-subprocess-module-failing-to-execute-bitcoind). Your answer is nice but I would like to know what could be intercepting the process with `SIGABRT` and how can I identify it? – derrend Sep 01 '14 at 04:44
  • @derrend - Honestly not sure what causes it. My guess is that it has something to do with the way that web2py hijacks the python runtime. From what I remember look through the source (this was a while ago), web2py effectively runs a giant `exec` on your code within its own, specialized scope. I don't know the details, but maybe this scope is hostile towards `Popen`? If you are really curious, I'd recommend the mailing list. The web2py creator is pretty active there. – zashu Sep 02 '14 at 18:20
  • @derrend - Just realized that you are on Django instead of web2py. I'm not familiar with the Django internals, but I figure it could either be that Django operates similarly to web2py, or perhaps that there is an issue starting a subprocess from a process spun up by a webserver. For what it's worth, I was originally running my application via Apache/wsgi. Maybe that's the culprit? – zashu Sep 02 '14 at 19:58

1 Answers1

2

In the end, the best I was able to come up with involved setting up a simple XML RPC server and call the functions from that:

my_server.py

#my_server.py
from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
from subprocess import *

proc_srvr = xmlrpclib.ServerProxy("http://localhost:12345")

def echo_fn():
    p = Popen(("echo", "hello"), shell=False)
    ret = p.wait()
    print "Process ended with status %s" % ret
    return True  # RPC Server doesn't like to return None

def main():
    server = SimpleXMLRPCServer(("localhost", 12345), ErrorHandler) 
    server.register_function(echo_fn, "echo_fn")
    while True:
        server.handle_request()

if __name__ == "__main__":
    main()

web2py_controller.py 

#web2py_controller.py

def run_echo():
    proc_srvr = xmlrpclib.ServerProxy("http://localhost:12345")
    proc_srvr.echo_fn()

I'll be honest, I'm not a Python nor SimpleRPCServer guru, so the overall code may not be up to best-practice standards. However, going this route did allow me to, in effect, call a subprocess from a controller in web2py.

(Note, this was a quick and dirty simplification of the code that I have in my project. I have not validated it is in a working state, so it may require some tweaks.)

zashu
  • 747
  • 1
  • 7
  • 22