Stopping celery task gracefully

Question

I'd like to quit a celery task gracefully (i.e. not by calling revoke(celery_task_id, terminate=True)). I thought I'd send a message to the task that sets a flag, so that the task function can return. What's the best way to communicate with a task?

score 11 · Accepted Answer · edited Mar 15 '16 at 16:55

11

Use signals for this. Celery's revoke is the right choice; it uses SIGTERM by default, but you can specify another using the signal argument, if you prefer.

Just set a signal handler for it in your task (using the signal module) that terminates the task gracefully.

edited Mar 15 '16 at 16:55

Stephen Fuhry

12,624
6
56
55

answered May 11 '13 at 04:41

Cairnarvon

25,981
9
51
65

1

Would it be possible to catch an arbitrary signal and do some clean up action? Also, is signal platform independent? – orange May 12 '13 at 10:20
Python's `signal` module papers over the platform differences, except that the only supported signals on Windows are SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, or SIGTERM, so you should probably stick to those. All of those can be caught. It's not possible to set a handler that will trigger whenever any signal at all is caught, if that's what you're asking. – Cairnarvon May 12 '13 at 10:30
I finally got around implementing this and it works fine on OSX and Linux. However, on Windows (8), I don't seem to be able to catch the signal. I tried SIGABRT and SIGTERM, but none of them are getting caught. I install the listener in the `@task` annotated function. Is there anything else, I need to do? – orange Jun 10 '13 at 23:57
I'm sorry, I can't help you with that; Windows isn't my forte. The semantics *should* be the same, though. – Cairnarvon Jun 11 '13 at 00:03
I've ran some tests and tried to catch some signals (SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV and SIGTERM) on Windows without Celery. It seems that none of them can be caught which is quite disappointing. Looking at another solution now using the RabbitMQ message bus... Thanks for your help anyway. – orange Jun 13 '13 at 12:28

score 11 · Answer 2 · edited May 09 '16 at 14:07

11

Also you can use an AbortableTask. I think this is the best way to stop task gracefully.

http://docs.celeryproject.org/en/latest/reference/celery.contrib.abortable.html

from celery.contrib.abortable import AbortableTask
from proj.celery import app

@app.task(bind=True, base=AbortableTask)
def abortable_task(self):
    while not self.is_aborted():
       print 'I am running'

    print 'I was aborted!'

If you save id task in somewhere you can call this whenever you want.

from celery.contrib.abortable import AbortableAsyncResult

abortable_task = AbortableAsyncResult(task_id)
abortable_task.abort()

edited May 09 '16 at 14:07

mastov

2,942
1
16
33

answered May 09 '16 at 13:46

Antonio Cabanas

111
1
5

i used this approach, but for me as soon as i start the task, worker cpu goes to 100% i also added time.sleep(1) after the print statement inside while loop. then the task gets aborted but still the worker is consuming 100% cpu – vk-code Jun 13 '18 at 19:23
celery logs for above comment `[2018-06-13 19:13:05,518: INFO/MainProcess] Received task: callmodel.tasks.abortable_task[c7cc4eff-dc6b-482a-a3a9-a63cf30a8974] [2018-06-13 19:13:06,659: INFO/MainProcess] Received task: callmodel.tasks.abortable_task[17c00f10-e341-449b-b80a-864e11cacd50] [2018-06-13 19:13:10,679: INFO/ForkPoolWorker-1] Task callmodel.tasks.abortable_task[c7cc4eff-dc6b-482a-a3a9-a63cf30a8974] succeeded in 5.1599816720008675s: '{"status": "now aborted!"}'` – vk-code Jun 13 '18 at 19:24

Stopping celery task gracefully

2 Answers2

Linked