Error on using IronMQ as celery broker in Heroku

Question

Currently we have a Python Django application hosted in Heroku cloud that uses Celery workers for background processing.

Celery workers were using django DB as the broker. Recently we decided to use IronMq as the broker since it’s an enterprise solution.

The packages we are using are

Python 2.7
Django 1.3
Celery 3.0.16
Django-celery 3.0.11
Iron-celery 0.3.1

We are using two workers. One worker is for the queue ‘Normal priority’ and another worker is for the queue HighPriority’. But when the application tired to start the worker for ‘HighPriority’ queue an error was triggered and as a result both the workers where crashed.

Following are additional settings we are using to configure IronMQ and celery queues.

BROKER_URL='ironmq://xxxxx:xxxxx@'
CELERY_RESULT_BACKEND='ironcache://xxxxx:xxxxx@'

import os
import djcelery
import iron_celery
from kombu import Exchange, Queue
djcelery.setup_loader()


CELERY_DEFAULT_QUEUE = 'NormalPriority'
CELERY_QUEUES = {
                CELERY_DEFAULT_QUEUE: {
                                      'exchange': CELERY_DEFAULT_QUEUE,
                                      'binding_key': CELERY_DEFAULT_QUEUE,
                                      }
                }


CELERY_CREATE_MISSING_QUEUES=True
CELERY_ROUTES = {'tasks.retrieveDocketUsingWorker': {'queue': 'HighPriority'},
                'tasks.retrieveDocketFromDdUsingWorker': {'queue': 'HighPriority'}
                }

I have added a small part of the error log below. Some one please have a look into this.

 2013-03-14T09:21:17+00:00 app[celeryd1.1]:     self.queue_declare(nowait, passiv
   e=False)
 2013-03-14T09:21:17+00:00 app[celeryd1.1]:   File "/app/.heroku/python/lib/pytho
 n2.7/site-packages/kombu/entity.py", line 497, in queue_declare
 2013-03-14T09:21:17+00:00 app[celeryd1.1]:     nowait=nowait)
 2013-03-14T09:21:17+00:00 app[celeryd1.1]:   File "/app/.heroku/python/lib/pytho
 n2.7/site-packages/kombu/transport/virtual/__init__.py", line 401, in queue_decl
 are
  2013-03-14T09:21:17+00:00 app[celeryd1.1]:     return queue, self._size(queue),
   0
    2013-03-14T09:21:17+00:00 app[celeryd1.1]:   File "/app/.heroku/python/lib/pytho
     n2.7/site-packages/iron_celery/iron_mq_transport.py", line 78, in _size
    2013-03-14T09:21:17+00:00 app[celeryd1.1]:     return details["size"]
 2013-03-14T09:21:17+00:00 app[celeryd1.1]: **TypeError: string indices must be int
      egers**
2013-03-14T09:21:17+00:00 app[celeryd1.1]: [2013-03-14 05:21:17,631: INFO/MainPr
ocess] Celerybeat: Shutting down...
2013-03-14T09:21:18+00:00 heroku[celeryd1.1]: Process exited with status 0
2013-03-14T09:21:18+00:00 heroku[celeryd1.1]: State changed from up to crashed
2013-03-14T09:21:23+00:00 heroku[router]: at=info method=POST path=/getDocketDow
nloaderWorkerStatus/ host=www.irelanddev.com fwd="115.119.213.182" dyno=web.1 qu
eue=0 wait=0ms connect=1ms service=2447ms status=200 bytes=60
2013-03-14T09:21:26+00:00 heroku[router]: at=info method=POST path=/getDocketDow
 nloaderWorkerStatus/ host=www.irelanddev.com fwd="115.119.213.182" dyno=web.1 qu
eue=0 wait=0ms connect=1ms service=23ms status=200 bytes=60
2013-03-14T09:21:31+00:00 heroku[router]: at=info method=POST path=/getDocketDow
nloaderWorkerStatus/ host=www.irelanddev.com fwd="115.119.213.182" dyno=web.1 qu
eue=0 wait=0ms connect=1ms service=196ms status=200 bytes=60
2013-03-14T09:21:36+00:00 heroku[router]: at=info method=POST path=/getDocketDow

Thanks in advance.

score 0 · Answer 1 · answered Mar 15 '13 at 17:55

0

I believe this is a similar issue to one I've seen before. Please upgrade iron_core (pip install iron-core -U). It should be version 1.0.2.

answered Mar 15 '13 at 17:55

Paddy

2,793
1
22
27

Thanks Paddy. I will upgrade the iron core version. – SkariaArun Mar 18 '13 at 04:33
@SkariaArun I assume you're the one I'm emailing from the Iron.io support system. :) If you make sure iron-core and requests are both up to date (sometimes multiple versions of requests being installed confuse Python, somehow) this shouldn't be an issue. :) Sorry for the problems! Feel free to hop on get.iron.io/chat if you need help. – Paddy Mar 19 '13 at 19:11
Thanks Paddy for your valuable support.Its me whom you were mailing from iron.io support. Currently we have rescheduled the implementations to sometime later.This is because we got other urgent fixes to be implemented . I will definitely upgrade the request module. Will getback to you when we start the implementation of ironMQ. – SkariaArun Mar 20 '13 at 09:31

Error on using IronMQ as celery broker in Heroku

1 Answers1