14

To achieve something similar to google app engines 'deferred calls' (i.e., the request is handled, and afterwards the deferred task is handled), i experimented a little and came up with the solution to spawn a thread in which my deferred call is handled.

I am now trying to determine if this is an acceptable way.

Is it possible (according to the WSGI specification) that the process is terminated by the webserver after the actual request is handled, but before all threads run out?

(if there's a better way, that would be also fine)

davidism
  • 121,510
  • 29
  • 395
  • 339
keppla
  • 1,753
  • 2
  • 15
  • 29
  • 3
    You'll probably want to have some sort of task queue + thread pool arrangement, both for performance, and to avoid having so many threads that they starve each other. – Marcin Jul 05 '11 at 07:57
  • @Marcin: good point, when it shows the thread approach is feasible, i will implement such an infrastructure – keppla Jul 05 '11 at 08:02
  • I'm disappointed you haven't had any answers yet. – Marcin Jul 05 '11 at 13:57
  • Maybe a bad title or phrasing? any ideas for improvement? – keppla Jul 05 '11 at 14:03
  • You might add the name of some wsgi frameworks, maybe have a more task-oriented title (e.g. "spawn threads to create deferred calls in WSGI web applications"). – Marcin Jul 05 '11 at 18:04

2 Answers2

13

WSGI does not specify the lifetime of an application process (as WSGI application is a Python callable object). You can run it in a way that is completely independent of the web server, in which case, only you control the lifetime.

There is also nothing in the WSGI that would prohibit you from spawning threads, or processes, or doing whatever the hell you want.

Cat Plus Plus
  • 125,936
  • 27
  • 200
  • 224
7

FWIW, also have a read of:

http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode

The hooking of actions to close() of iterable is the only way within context of the WSGI specification itself for doing deferred work. That isn't in a separate thread though and would occur within the context of the actual request, albeit after the response is supposed to have been flushed back to the client. Thus your deferred action will consume that request thread until the work is complete and so that request thread would not be able to handle other requests until then.

In general, if you do use background threads, there is no guarantee that any hosting mechanism would wait until those background threads complete before shutting process down. In fact, can't even think of any standard deployment mechanism which does wait. There isn't really even a guarantee that atexit handlers will be called on process shutdown, something that the referenced documentation also briefly talks about.

Graham Dumpleton
  • 57,726
  • 6
  • 119
  • 134