Supervising celerybeat with supervisor and virtualenv

Question

My celerybeat.conf

[program:celerybeat]
command=/path/app/env/bin/celery beat -A project.tasks --loglevel=INFO
environment=PYTHONPATH=/path/app/env/bin

user=nobody
numprocs=1
stdout_logfile=/var/log/celeryd.log
stderr_logfile=/var/log/celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
killasgroup=true
priority=998

When I starting supervisor I receive an error:

pidfile_fd = os.open(self.path, PIDFILE_FLAGS, PIDFILE_MODE)
celery.platforms.LockFailed: [Errno 13] Permission denied: '/celerybeat.pid'

Any idea how to solve this?

score 16 · Answer 1 · answered Nov 26 '13 at 00:05

The problem is that you have not specified any directory in the config file and the default directory then is '/' (root) which your user does not have permissions to write.

Setting the user as root solved your problem because now you had permission to write to '/' however it might not be the best solution. There are multiple ways you can solve this by including:

Add a directory variable in the config and provide a path that your user has permissions to write to.
```
directory=<path>
```
Provide a pidfile argument to the celery command that you are using to start celery. Make sure you have write permissions to the path you specify for the pidfile.
```
command=/path/app/env/bin/celery beat -A project.tasks --loglevel=INFO --pidfile=/tmp/celerybeat-myapp.pid
```

Works fine after custom add "--pidfile" to tmp directory. – Joni Apr 02 '16 at 08:02 — Joni, Apr 02 '16 at 08:02

score 1 · Answer 2 · answered Sep 03 '13 at 07:45

1

Here is my (working) version for Celere beat:

[program:celery_periodic]
command=<venv_path>/bin/python <path>/manage.py celery worker --loglevel=info -c 1 -E -B -Q celery_periodic -f <log_folder>/celery_periodic.log -n periodic_worker
directory=<path>
user=<some_user>
group=<some_user>
autostart=true
autorestart=true
redirect_stderr=True
daemon = False
debug = False
stdout_logfile = NONE
stderr_logfile = NONE
loglevel = "info"

May be this helps.

Also check permissions on folder where you create pid file.

answered Sep 03 '13 at 07:45

Artem Mezhenin

5,539
6
32
51

It is not recommended to start beat inside the celery worker for production http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html – RunLoop Feb 27 '16 at 06:52
this comment refers to some 2nd version of Celery. At that time it was a standard way of doing this. Not sure how thing are going now. – Artem Mezhenin Feb 29 '16 at 17:38

score 0 · Answer 3 · answered Sep 02 '13 at 20:55

0

I solve my problem by setting user=root, but I think this is bad way...

answered Sep 02 '13 at 20:55

Gr1N

1,337
3
14
19

4

Don't do that! Use root as sparingly as possible. Better follow @sanchitarora's answer and write the pid file in a directory you have write access to. `--pidfile=/tmp/celerybeat-myapp.pid` – j7nn7k Nov 01 '14 at 09:20

score 0 · Answer 4 · answered Sep 03 '19 at 22:37

I ran into a similar issue. It was fixed by changing 2 setting:

as mentioned above: adding --pidfile=/tmp/celerybeat-myapp.pid
user = "an actual non-root user"

The default provided in some documentation "user = nobody" seem to cause extra permission errors with celerybeat.conf (not in celery.conf though).

Supervising celerybeat with supervisor and virtualenv

4 Answers4