22

I use:

  1. Celery
  2. Django-Celery
  3. RabbitMQ

I can see all my tasks in the Django admin page, but at the moment it has just a few states, like:

  • RECEIVED
  • RETRY
  • REVOKED
  • SUCCESS
  • STARTED
  • FAILURE
  • PENDING

It's not enough information for me. Is it possible to add more details about a running process to the admin page? Like progress bar or finished jobs counter etc.

I know how to use the Celery logging function, but a GUI is better in my case for some reasons.

So, is it possible to send some tracing information to the Django-Celery admin page?

honk
  • 9,137
  • 11
  • 75
  • 83
derevo
  • 9,048
  • 2
  • 22
  • 19

4 Answers4

34

Here's my minimal progress-reporting Django backend using your setup. I'm still a Django n00b and it's the first time I'm messing with Celery, so this can probably be optimized.

from time import sleep

from celery import task, current_task
from celery.result import AsyncResult

from django.http import HttpResponse, HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.utils import simplejson as json
from django.conf.urls import patterns, url


@task()
def do_work():
    """ Get some rest, asynchronously, and update the state all the time """
    for i in range(100):
        sleep(0.1)
        current_task.update_state(state='PROGRESS',
            meta={'current': i, 'total': 100})


def poll_state(request):
    """ A view to report the progress to the user """
    if 'job' in request.GET:
        job_id = request.GET['job']
    else:
        return HttpResponse('No job id given.')

    job = AsyncResult(job_id)
    data = job.result or job.state
    return HttpResponse(json.dumps(data), mimetype='application/json')


def init_work(request):
    """ A view to start a background job and redirect to the status page """
    job = do_work.delay()
    return HttpResponseRedirect(reverse('poll_state') + '?job=' + job.id)


urlpatterns = patterns('webapp.modules.asynctasks.progress_bar_demo',
    url(r'^init_work$', init_work),
    url(r'^poll_state$', poll_state, name="poll_state"),
)
Florian Sesser
  • 5,972
  • 1
  • 25
  • 26
  • 3
    On newer versions I think the idiom is to do `@task(bind=True)` and then call `self.update_state`. Performance-wise am not sure which is better. – dashesy Dec 25 '14 at 00:15
10

I am starting to try figuring this out myself. Start by defining a PROGRESS state exactly as explained on the Celery userguide, then all you need is to insert a js in your template that will update your progress bar.

John
  • 6,701
  • 3
  • 34
  • 56
Marconius
  • 711
  • 1
  • 6
  • 15
7

Thank @Florian Sesser for your example!

I made a complete Django app that show the progress of create 1000 objects to the users at http://iambusychangingtheworld.blogspot.com/2013/07/django-celery-display-progress-bar-of.html

Everyone can download and use it!

Trinh Nguyen
  • 1,445
  • 1
  • 14
  • 22
6

I would recommend a library called celery-progress for this. It is designed to make it as easy as possible to drop-in a basic end-to-end progress bar setup into a django app with as little scaffolding as possible, while also supporting heavy customization on the front-end if desired. Lots of docs and references for getting started in the README.

Full disclosure: I am the author/maintainer of said library.

Cory
  • 22,772
  • 19
  • 94
  • 91
  • Can you please provide a working example? I'm stuck with celery-progress – DevLoverUmar Apr 15 '20 at 05:24
  • 1
    @MuhammadUmarFarooq feel free to file an issue in celery-progress with the specific problem you're facing and I'll take a look as soon as I can – Cory Apr 16 '20 at 06:04
  • Sir, instead of resolving individual issues, can you please provide a github demo. So that, a person can checkout a desired commit as per libraries versions. Thanks – DevLoverUmar Apr 16 '20 at 06:12
  • Or please just mention the versions on which you built your demos, I'm not sure but I think django/celery latest versions are causing the problem. – DevLoverUmar Apr 16 '20 at 09:22
  • 1
    The latest demo is running on Django 3.0.5 and Celery 4.4.0 which I believe are the latest as of this date. @MuhammadUmarFarooq – Cory Apr 19 '20 at 13:11