54

I have a Django project on an Ubuntu EC2 node, which I have been using to set up an asynchronous using Celery.

I am following this along with the docs.

I've been able to get a basic task working at the command line, using:

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ celery --app=myproject.celery:app worker --loglevel=INFO

To start a worker. I have since made some changes to the Python, but realized that I need to restart a worker.

From the command line, I've tried:

 ps auxww | grep 'celery worker' | awk '{print $2}' | xargs kill -9

But I can see that the worker is still running.

How can I kill it?

edit:

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ sudo ps auxww | grep celeryd | grep -v "grep" | awk '{print $2}' | sudo xargs kill -HUP
kill: invalid argument H

Usage:
 kill [options] <pid> [...]

Options:
 <pid> [...]            send signal to every <pid> listed
 -<signal>, -s, --signal <signal>
                        specify the <signal> to be sent
 -l, --list=[<signal>]  list all signal names, or convert one to a name
 -L, --table            list all signal names in a nice table

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see kill(1).

edit 2:

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ ps aux|grep celery
ubuntu    9756  0.0  3.4 100868 35508 pts/6    S+   15:49   0:07 /home/ubuntu/.virtualenvs/env1/bin/python3.4 /home/ubuntu/.virtualenvs/env1/bin/celery --app=tp.celery:app worker --loglevel=INFO
ubuntu    9760  0.0  3.9 255840 39852 pts/6    S+   15:49   0:05 /home/ubuntu/.virtualenvs/env1/bin/python3.4 /home/ubuntu/.virtualenvs/env1/bin/celery --app=tp.celery:app worker --loglevel=INFO
ubuntu   12760  0.0  0.0  10464   932 pts/7    S+   19:04   0:00 grep --color=auto celery
user1592380
  • 34,265
  • 92
  • 284
  • 515

5 Answers5

79

Try this in terminal

ps aux|grep 'celery worker'

You will see like this 

username  29042  0.0  0.6  23216 14356 pts/1    S+   00:18   0:01 /bin/celery worker ...

Then kill process id by

sudo kill -9 process_id # here 29042

If you have multiple processes, then you have to kill all process id using above kill commmand

sudo kill -9 id1 id2 id3 ...

From the celery doc

ps auxww | grep 'celery worker' | awk '{print $2}' | xargs kill -9

OR if you are running celeryd

ps auxww | grep celeryd | awk '{print $2}' | xargs kill -9

Note

If you are running celery in supervisor, even though kill the process, it automatically restarts(if autorestart=True in supervisor script).

Mandar Pande
  • 12,250
  • 16
  • 45
  • 72
itzMEonTV
  • 19,851
  • 4
  • 39
  • 49
  • Thanks for looking at this. I tried (env1)ubuntu@ip-172-31-22-65:~/projects/tp$ ps aux|grep celery worker grep: worker: No such file or directory – user1592380 Mar 27 '15 at 18:42
  • (env1)ubuntu@ip-172-31-22-65:~/projects/tp$ ps aux|grep 'celery worker' -> ubuntu 12436 0.0 0.0 10460 936 pts/7 S+ 18:45 0:00 grep --color=auto celery worker – user1592380 Mar 27 '15 at 18:52
  • 1
    That means no celery worker is running?.Try `ps aux|grep celery` – itzMEonTV Mar 27 '15 at 18:59
  • Thanks for your help on this, and the detailed explanation. although I'm still working on my problem, you've given me some good information. – user1592380 Mar 28 '15 at 15:13
  • still problem? `sudo kill -9 9756 9760` ? – itzMEonTV Mar 28 '15 at 15:19
  • I asked a new question at http://stackoverflow.com/questions/29319103/trouble-killing-celery-process – user1592380 Mar 28 '15 at 15:47
  • @itzMEonTV can you provide insight on how to perform these action remotely, also what if workers are running on Windows OS. – Saurabh Jun 30 '17 at 20:56
52
pkill -f "celery worker"

easy to kill process by string patterns

alan_wang
  • 785
  • 8
  • 9
41

If the celery worker is running on a machine you do not have access to, you can use Celery "remote control" to control workers through messages sent via the broker.

celery control shutdown

This will kill all workers immediately. Depending on your setup, you might have to use -A myProject, like with Django.

Documentation here.

Navid Khan
  • 979
  • 11
  • 24
Paulo Peres Junior
  • 549
  • 4
  • 6
  • Just FYI: This answer was flagged by someone as "low-quality" because of its length or content. This may be because there is no explaining text to accompany your suggested solution. – Morten Jensen Jan 26 '18 at 19:07
  • I don't get what `proj` is? – Kyle Bridenstine Jul 24 '18 at 15:37
  • 5
    @KyleBridenstine proj is the name of you Application/Project, For instance if you application is Called DjangoApp the command wouyld be `celery -A DjangoApp control shutdown` from the root of your pjoject – Paulo Peres Junior Jul 25 '18 at 19:03
  • This one worked for me. I tried killing celery workers by several other methods here. But `ps -eaf | grep celery` tells me that they didn't really work. – mrpandey Aug 15 '20 at 20:28
4
ps auxww | grep 'celery worker' | grep -v " grep " | awk '{print $2}' | xargs kill -9

this one is very similar to one presented before but improved because avoid the error that shows when attempt to kill the grep process..

walter
  • 539
  • 5
  • 5
2

In case someone's looking to shutdown their celery app programmatically, the same thing can be done in python with: celery_app.control.shutdown()

D4nt3
  • 96
  • 1
  • 4