5

Am running Celery 3.1.16 with a RabbitMQ 3.4.1 back end and using Flower 0.7.3 on Python3.4 to monitor my celery tasks. I have several tasks running and I can view their results in the task tab of Celery Flower.

In the monitor tab, there are 4 sections. Succeeded tasks, failed tasks, task times, and broker. Of these 4, only the Broker view is showing a 'traffic' graph. Is there a setting to enable the other graphs show some statistics?

flowerconfig.py

# Broker settings
BROKER_URL = 'amqp://guest:guest@localhost:5672//'

# RabbitMQ management api
broker_api = 'http://guest:guest@localhost:15672/api/'

#Port
port = 5555

# Enable debug logging
logging = 'INFO'

Supervisor: flower.conf

[program:flower]
command=/opt/apps/venv/my_app/bin/celery flower --app=celery_conf.celeryapp --conf=flowerconfig
directory=/opt/apps/my_app/celery_conf
user=www-data
autostart=true
autorestart=true
startsecs=10
redirect_stderr=true
stderr_logfile=/var/log/celery/flower.err.log
stdout_logfile=/var/log/celery/flower.out.log

While were at it, in the Broker graph, I have two queues one green the othe red. However, the one that shows in the graph is the red one yet both are running and I can view their results from the Tasks window.

I've noticed something peculiar in the Config Tab under Workers Tab in Flower. The CELERY_ROUTE and CELERY_QUEUES are showing as empty lists while all other fields look like they picked the correct data out of the celeryconfig file

BROKER_URL  amqp://guest:********@localhost:5672//
CELERYBEAT_SCHEDULE {}
CELERYD_PREFETCH_MULTIPLIER 0
CELERY_ALWAYS_EAGER False
CELERY_AMQP_TASK_RESULT_EXPIRES 60
CELERY_CREATE_MISSING_QUEUES    False
CELERY_DEFAULT_EXCHANGE default
CELERY_DEFAULT_QUEUE    default
CELERY_DEFAULT_ROUTING_KEY  ********
CELERY_IMPORTS  ['student.admission', 'student.schedule']
CELERY_INCLUDE  ['celery.app.builtins', 'student.schedule', 'student.admission']
CELERY_QUEUES   [{}, {}, {}, {}, {}]     #<==== Should it show an empty list?
CELERY_RESULT_BACKEND   amqp://guest:guest@localhost:5672//
CELERY_ROUTES   [{}, {}, {}, {}]     #<==== Should it show an empty list?
CELERY_STORE_ERRORS_EVEN_IF_IGNORED True
CELERY_TASK_RESULT_EXPIRES  3600

The celeryconfig.py looks like below:

BROKER_URL = 'amqp://guest:guest@localhost:5672//'
CELERY_RESULT_BACKEND = 'amqp://guest:guest@localhost:5672//'

#Task settings
CELERY_TASK_RESULT_EXPIRES = 3600
CELERY_AMQP_TASK_RESULT_EXPIRES = 60
CELERYD_PREFETCH_MULTIPLIER = 0 
CELERY_ALWAYS_EAGER = False
CELERY_CREATE_MISSING_QUEUES = False
CELERY_STORE_ERRORS_EVEN_IF_IGNORED = True

#Scripts to be imported 
CELERY_IMPORTS=('student.admission', 'student.schedule')

#Celery Exchanges, Queues, Routes
default_exchange = Exchange('default', type='direct')
student_admission_exchange = Exchange('student_admission_exchange', type='direct', durable=False)

CELERY_QUEUES = (
    Queue('default', default_exchange, routing_key='default'),
    Queue('student_admission_queue', student_admission_exchange, routing_key='admission', durable=False),
)
CELERY_ROUTES = (
                 {'student.admission.admit': {'queue': 'student_admission_queue','routing_key': 'admission'}},
                     )
CELERY_DEFAULT_QUEUE = 'default'
CELERY_DEFAULT_EXCHANGE = 'default'
CELERY_DEFAULT_ROUTING_KEY = 'default'

Edit

As I see am not the only one stuck on this, I though I include the screenshot of the "missing" graphs as a guide.

Celery: Uncharted Graphs

lukik
  • 3,919
  • 6
  • 46
  • 89
  • I think the best thing you could is to open an issue on the Flower GitHub issue tracker. I don't think that the missing CELERY_ROUTES and CELERY_QUEUES settings have anything to do with this; grepping through the source doesn't show them being used anywhere. Source: I have the same problem. – jvkersch Jan 09 '15 at 13:23

1 Answers1

1

In my case this was not a problem with Flower itself, but with the fact that the timestamps on my tasks were not accurate (as exhibited by the message "Substantial drift from *** may mean clocks are out of sync" in my Celery logs). Fixing the clock may be the answer.

Flower figures out whether an event is new (and hence needs to be plotted) by comparing the timestamp on the event with the timestamp of when it last updated the plot (see https://github.com/mher/flower/blob/master/flower/views/monitor.py#L47). In my case, that comparison was always False, so no events were being plotted.

jvkersch
  • 575
  • 1
  • 4
  • 9
  • Which clock should I change? Celery, Flower are in one machine and if a run `date` it shows me `Sat Jan 10 19:49:34` which is the correct date. If I go to Tasks Tab in Celery I see the last ran task its showing `Received 2015-01-10 19:49:46.709552` and `Started 2015-01-10 19:49:46.711021` both of which are correct. So which clock are we talking about here? – lukik Jan 10 '15 at 16:51
  • I don't know (I ended up doing the quick'n'dirty and adjusting the 'if' clause in Flower because I had only limited time and I couldn't figure out the underlying cause). I'll update my answer if I know more. In the meantime I would suggest printing out the timestamps on the events in one of the monitors and seeing if anything is off. – jvkersch Jan 10 '15 at 21:57
  • The timestamp is generated on javascript, which I feel is an error, why not generate it on python which uses the server clock? This happens on Flower 0.9.1 which is the last version to date. – Hassek Apr 14 '16 at 02:39