3

I want to provide a service on the web that people can test out the performance of an algo, which is written in python and running on the linux machine

basically what I want to do is that, there is a very trivial PHP handler, let's say start_algo.php, which accepts the request coming from browser, and in the php code through system() or popen() (something like exec( "python algo.py" ) ) to issue a new process running the python script, I think it is doable in this part

problem is that since it is a web service, surely it has to serve multiple users at the same time, but I am quite confused by the Global Interpreter Lock GIL http://wiki.python.org/moin/GlobalInterpreterLock that the 'standard' CPython has implemented, does it mean, if I have 3 users running the algo now (which means 3 separated processes, correct me if I am wrong plz), at a particular moment, there is only one user is being served by the Python interpreter and the other 2 are waiting for their turns?

Many thanks in advance

Ted

Ted Xu
  • 1,095
  • 1
  • 11
  • 20
  • Are you using Apache and `mod_wsgi`? If so, concurrent requests are handled at that level well outside Python. If not, please explain why you're not using Apache (or nginx) and mod_wsgi. – S.Lott Sep 13 '11 at 18:10
  • @S.Lott thx, but the problem is not about the concurrent REQUESTS, it is about concurrently running the python script cuz those script will take a long time to run whereas the php code will simply return after issuing the "python algo.py" cmd – Ted Xu Sep 14 '11 at 01:06
  • The question conflates concurrent requests with each request running a script. The question is confusing. Any time you mention the GIL in the context of Apache and mod_wsgi, you have to explain what -- precisely -- you think is going on. Apache separates the requests. Each request forks a **separate** process. Please be perfectly clear that you understand that each separate request (made separate by Apache and mod_wsgi) actually submits a separate subprocess. Please be very clear -- in the question -- that you understand this. – S.Lott Sep 14 '11 at 01:58

2 Answers2

10

If you are opening each script by invoking a new process; you will not run afoul of the GIL. Each process gets its own interpreter and therefore its own interpreter lock.

SingleNegationElimination
  • 151,563
  • 33
  • 264
  • 304
7

The GIL is per-process. If you start multiple python processes, each will have its own GIL that prevents the interpreter(s) in this specific process from running more than one thread at a time. But independent processes can run at the same time.

Also, multiple threads inside one Python process do take turns on running (rather frequently, IIRC once per hundred opcode instructions or a few dozen milliseconds depending on the version), so it's not like the GIL prevents concurrency at all - it just prevents multi-threading.

Community
  • 1
  • 1
  • thank you a lot delnan, very detailed explanation, I should do more homework before asking, similar question has been asked on this site already – Ted Xu Sep 13 '11 at 16:26