Celery task history

Question

I am building a framework for executing tasks on top of Celery framework.

I would like to see the list of recently executed tasks (for the recent 2-7 days).

Looking on the API I can find app.backend object, but cannot figure out how to make a query to fetch tasks.

For example I can use backends like Redis or database. I do not want to explicitly write SQL queries to database.

Is there a way to work with task history/results with API?

I tried to use Flower, but it can only handle events and cannot get history before its start.

check out django-celery, I think it already provides an admin for tasks — Ale, Mar 01 '16 at 15:39
It seems you might be right. I did not want to use django in this project, but it is likely I need the same functionality that djcelery provides. — baldr, Mar 01 '16 at 15:47

score 6 · Answer 1 · edited Aug 04 '16 at 10:36

6

You can use the persisent option，eg: flower -A ctq.celery --persistent=True

edited Aug 04 '16 at 10:36

Louis

answered Aug 04 '16 at 10:08

杨冠宇

Why was that down voted? It seems reasonable. Whoever down voted please clarify because I'd like to know why that is a bad idea. – digitaldavenyc Apr 18 '17 at 15:05
this is not bad idea. The down vote because I used Chinese! 23333 – 杨冠宇 May 16 '17 at 08:26

score 4 · Answer 2 · edited Jun 10 '22 at 15:07

4

You need to keep the task results in a backend, for example Redis. The Celery documentation contains information about how to do this here:

Also you want to set the CELERY_TASK_RESULT_EXPIRES configuration parameter, because by default the results are discarded after one day.

If you do this, then Flower will show you the history of the task execution, regardless from when it started.

edited Jun 10 '22 at 15:07

nicorellius

answered Mar 01 '16 at 15:50

mavroprovato

Yes, I have tried that way, but it does not work. That's why I am asking. In the Flower's sources I have found that it works with events only. With backend api it is possible to get past task's results, but only knowing it's ID. It is likely `djcelery` would be the best for my needs. – baldr Mar 01 '16 at 16:18
@Kapucko, I left an idea to solve this with celery. And I create a new record in the database when a task starts. I finally switched the project to django. – baldr Feb 08 '17 at 09:58
@baldr thank you for your response. Could you tell me what advantages does django provide in this case? – Kapucko Feb 09 '17 at 09:10
@Kapucko, not only this exact case, but I prefer Django for its flexibility and I like it's project structure. Managing database with models and migrations is pretty mature and easy. – baldr Feb 09 '17 at 09:19
Just wanted to note that Celery introduced new lowercase settings as of version 4.0, so CELERY_TASK_RESULT_EXPIRES is now configured with [result_expires](https://docs.celeryproject.org/en/latest/userguide/configuration.html#std-setting-result_expires). – aaronbriel Oct 14 '20 at 13:48

2 Answers2