5

I am running a Flask webapp running behind uwsgi (2 processes). A part of my code involves pinging a remote resource, seeing if it has been modified (If-Modified-Since), and updating a local copy of that resource on the webserver if modified.

That update also sends myself an email of the diff. I'm concerned that this takes a long time, causing user requests to time out while sending the email.

Is the Python threading library the right way to tackle this? (spawn a thread and send the email there?) Will this interfere with uwsgi's processes at all?

Thanks for the help!

(on a side note: I am also a bit concerned about the 2 uwsgi processes bumping heads if they both try to update the resource on the local copy... I wonder if the threading module's lock capabilities is the right thing to look at for this problem as well?)

EDIT: To clarify, my primary concern is that the email task is part of the code execution. It takes a long time and runs before the return_template call, therefore holding up the response to the user. Is the Python threading library the right way to tackle this problem, given the Flask/uwsgi environment?

VMDX
  • 665
  • 1
  • 6
  • 9
  • 1
    You might want to [search](http://stackoverflow.com/search?q=%2Bwsgi+%2Bseparate+%2Bthread&submit=search) for _+wsgi +separate +thread_. – Piotr Dobrogost Sep 16 '11 at 16:45
  • Hey thanks! The [first result](http://stackoverflow.com/questions/6579467/is-it-ok-to-spawn-threads-in-a-wsgi-application) there helps. – VMDX Sep 16 '11 at 19:38

2 Answers2

6

The best solution for this kind of tasks is using the uWSGI spooler. If you want to run the tasks at specific interval you can use the @timer decorator and let the spooler to the hard work:

from uwsgidecorators import *

#this will execute the_task() every 30 seconds in the spooler
@timer(30, target='spooler')
def the_task(signum)
    do_the_long_task()
roberto
  • 76
  • 1
  • Cool, good to know. I'll have to look into uwsgidecorators. But this email task is spawned by an user request, not executing through a timed poll. – VMDX Sep 16 '11 at 08:23
  • @VMDX - I've got a similar problem now (ie need to start a long running task based on user request from the web), how did you invoke the separate thread in the end? – Ammar Akhtar Jun 02 '15 at 16:51
2

If you try to make a server independent app (for instance, if you have in mind replacing uWSGI with Gunicorn later), I would recommend using Celery.

Here are the first steps: http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html

Marc de Verdelhan
  • 2,501
  • 3
  • 21
  • 40
  • And just to update you don't even need to use Flask-Celery as it works out of the box. :) – Raj Jan 17 '14 at 02:31