0

I'm using celery with systemd. I noticed that most times on restart, I lose the workers mid-task. From the celery multi documentation, it seems like the celery multi stopwait should be waiting for the tasks to finish.

Got the following error on restart:

Process "ForkPoolWorker-10" pid:16902 exited with "signal 15 (SIGTERM)"

celery.conf

[Unit]
Description=Celery background worker
After=network.target

[Service]
Type=forking
User=celery
Group=celery
WorkingDirectory=/src

ExecStart=celery multi start worker -A main.celery -Q celery --logfile=/data/celery.log --loglevel=info --concurrency=10 --pidfile=/var/run/celery/%%n.pid

ExecStop=celery multi stopwait worker --pidfile=/var/run/celery/%%n.pid

[Install]
WantedBy=multi-user.target

I also read the systemd documentation, we should at least be waiting 90 seconds for the task to be completed before sending out a SIGTERM. I receive this error in less than 10 seconds of running the restart command.

What am I doing wrong?

Using celery version: 5.2.2 (dawn-chorus)

Ankit
  • 3,878
  • 5
  • 35
  • 51
  • I’m voting to close this question because From the tag: systemd questions should be for *programming questions* using systemd or its libraries. Questions about *configuring the daemon* (including writing unit files) are better directed to Unix & Linux: https://unix.stackexchange.com. – Rob Oct 20 '22 at 09:49
  • IMO, this question requires very specific understanding of celery, that would not be justified for unix community. I don't think there's any bug in systemd config, but shared it for completeness. I can also try posting it on unix community. – Ankit Oct 20 '22 at 22:46
  • Do not cross post. Not allowed. – Rob Oct 20 '22 at 23:05
  • So you want to close the question here and not let me post the question in the other channel? – Ankit Oct 21 '22 at 01:13
  • To me everything seems normal here. Are you sure you are loosing the workers mid-task? `celery multi stopwait` will just send SIGTERM to every worker process and wait for them to finish. So seeing `exited with "signal 15 (SIGTERM)"` should be expected. – Garuno Oct 28 '22 at 11:48

0 Answers0