1

I am using celery in an uncommon way - I am creating custom process when celery is started, this process should be running all the time when celery is running. Celery workers use this process for their tasks (details not needed).

I run celery from command line and everything is ok:

celery -A celery_jobs.tasks.app worker -B --loglevel=warning

But when I use celeryd to daemonize celery, there is no way to stop it. Command celeryd stop tries to stop celery but never ends.

When I check process trees in both cases, there is the difference - when running from command line, the parent is obviously celery process (main process which has celery workers as childs). Killing (stopping) the parent celery process will stop all celery workers and my custom process.

But when running with celeryd, my custom process has parent /sbin/init - and calling celeryd stop is not working - seems like main celery process is waiting for something, or is unable to stop my custom process as it is not child process of celery.

I don't know much about processes and it is not easy to find information, because I don't know what I should search for, so any tips are appreciated.

misoK
  • 415
  • 1
  • 3
  • 15

2 Answers2

2

I have had the same problem. I needed a quick solution, so I wrote this bash script

#/bin/bash

/etc/init.d/celeryd stop
sleep 10
export PIDS=`ps -ef | grep celery |grep -v 'grep' | awk '{print $2}'`
for PID in $PIDS; do kill -9 $PID; done;

If the process doesn't stop after 10 seconds, it's a long-time-to-stop candidate, so i decided to stop abruptly

xecgr
  • 5,095
  • 3
  • 19
  • 28
0

I assume your custom process is not a child of any of your pool worker processes and need not be so.

I use supervisord instead of celeryd to daemonize workers. It can be used to daemonize other processes as well. Such as your custom processes.

In your case your supervisord.conf can have multiple sections. One for each celery worker node and one (or more) for your custom process(es).

When you kill the supervisord process (with -TERM) it will take care of terminating all the workers and your custom process as well. If you use -TERM, then you will need to make sure your custom processes handle them.

ksrini
  • 1,412
  • 2
  • 19
  • 33