Celery and Django - Running worker as Daemon (Mac OS X)

Question

I have a Django project in Mac OS X (10.8.4) with an application that intends to model an airport. In the database, SQLite, I have several tables and one of them is for flight delays (flight, date, new_hour, cause).

My objective is to use Celery to perform certain tasks like notify users for delays on their flights and delete delays which are no longer valid. For this, I thought that I would run the Celery worker as a daemon but I have encountered huge problems when trying to read the documentation on how to do this.

So I have followed the instructions on how to configure my Django project to use Celery, defining and calling tasks, starting the worker process and calling the task. I have an extra application on my project called celerytest and a tasks.py file that looks like this:

from celery import task
from flughafen.models import Country, Airline, Aircraft, Airport, Flight, Reservation, CheckIn, Delay

@task()
def delete_delays():
    # code to retrieve delays which are no longer valid and to delete them

I can call this task based on the instructions given here. However, when I read the instructions for Running the worker as a daemon it is not clear for me what to do.

The documentation mentions a link for Mac OS X which redirects me to a github page where there are some plist files which I am supposed to load with launchctl. I have downloaded them and loaded them, but they appear with an exit status:

octavio:daemon ohd$ launchctl load org.celeryq.celerybeat.plist
octavio:daemon ohd$ launchctl load org.celeryq.celeryd.plist
octavio:daemon ohd$ launchctl load org.celeryq.celerymon.plist
octavio:daemon ohd$ launchctl list | grep celery
-   2   org.celeryq.celerymon
-   2   org.celeryq.celeryd
-   2   org.celeryq.celerybeat

I guess I am missing some configuration like this or this, but I am not sure where should it be located.

Could anyone please provide me with advise?

Thanks.

score 2 · Accepted Answer · answered Sep 19 '13 at 23:01

I was struggling with the issue. While I didn't go the launchd route, I eventually got daemonized workers using the daemonize utility.

http://software.clapper.org/daemonize/

You can easily install it with brew.

Then using Fabric I can restart the workers like so:

def celeryd():
    with cd('/usr/local/Cellar/daemonize/1.7.4/sbin'):
        # Kill existing workers
        sudo('ps auxww | grep celeryd | grep -v "grep" | awk \'{print $2}\' | xargs kill')
        # Create new workers
        sudo('daemonize -u pipeadmin %s/manage.py celery worker' % siteDir)

Celery and Django - Running worker as Daemon (Mac OS X)

1 Answers1