16

My question is a duplicate of this one, but more detailed.

The problem is that I have a BROKER_URL set in my Celery config file, but that isn't reflected in and I am loading the config: I checked, and it is being loaded - in fact, other constants defined there are being set, just not BROKER_URL.

This appears to be a bug, but I wanted to be sure.

celeryconfig.py:

BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"

CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True

(JSON is being used as the serializer, not Pickle, so I know this is working.)

app.py:

from celery import Celery

app = Celery('tasks', broker='amqp://guest@localhost//')
app.config_from_object('celeryconfig')

Invoking the Worker:

celery -A app.app worker -l info

But then I get this:

[2013-11-12 11:20:51,610: INFO/MainProcess] consumer: Connected to amqp://guest@127.0.0.1:5672//.

I tried breaking up BROKER_URL, but to no avail:

BROKER_TRANSPORT = 'amqp'
BROKER_USER = 'user'
BROKER_PASSWORD = 'password'
BROKER_HOST = 'remote.server.com'
BROKER_PORT = 5672
BROKER_VHOST = '/vhost'

Interestingly, it does work when I explicitly set the BROKER_URL in app.py:

from celery import Celery

app = Celery('tasks', broker='amqp://guest@localhost//')
app.config_from_object('celeryconfig')
app.conf.BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"
Community
  • 1
  • 1
wolverdude
  • 1,583
  • 1
  • 13
  • 20
  • Note that breaking up the broker url is deprecated, and will be removed in celery 4: http://celery.readthedocs.org/en/latest/internals/deprecation.html – michel.iamit May 06 '15 at 20:01

4 Answers4

20

Of course, I realized what I'd done wrong immediately after finishing this question, but I still posted it because someone might find it useful.

My problem is that I copied and pasted code from the tutorial (*facepalm).

I'm overriding the config file when I define the app with a broker arg:

app = Celery('tasks', broker='amqp://guest@localhost//')

Simply remove that:

app = Celery('tasks')

Tada! Everything works fine... and I learned a valuable lesson.

wolverdude
  • 1,583
  • 1
  • 13
  • 20
8

Just for clarification because you are using this:

app.config_from_object('django.conf:settings', namespace='CELERY')

You must add the CELERY_ prefix to the BROKER_URL entry in your celeryconfig.py:

CELERY_BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"

CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True
Pablo Guerrero
  • 936
  • 1
  • 12
  • 22
user2471214
  • 729
  • 9
  • 17
2

Just for anyone else with a similar issue, I ran into a similar problem with a different cause.

I'm using celery with django on heroku, but in my dev environment I am using django-environ to populate environment variables from a file.

I forgot to preface my celery worker command with DJANGO_READ_DOT_ENV_FILE=1

So I went from:

celery -A myapp worker -l info

to

DJANGO_READ_DOT_ENV_FILE=1 celery -A myapp worker -l info

Silly, but a good reminder. Hope it saves someone some time.

David
  • 53
  • 1
  • 7
0

I was going absolutely crazy on this. Celery was ignoring my Django settings even though I followed the docs to the t.

I ran

celery -A celery worker -l info

I put a breakpoint() in my common/celery.py file and realised that it was not the celery app that was being run. I had to run the below instead:

celery -A my_project_name worker -l info
M3RS
  • 6,720
  • 6
  • 37
  • 47