How to programmatically generate celerybeat entries with celery and Django

Question

I am hoping to be able to programmatically generate celerybeat entries and resync celerybeat when entries are added. The docs here state

By default the entries are taken from the CELERYBEAT_SCHEDULE setting, but custom stores can also be used, like storing the entries in an SQL database.

So I am trying to figure out which classes i need to extend to be able to do this.

I have been looking at celery scheduler docs and djcelery api docs but the documentation on what some of these methods do is non-existent so about to dive into some source and was just hoping someone could point me in the right direction.

I guess a high level of what I'm doing would be helpful... As a user I need to be able to select from a predefined set of tasks and provide a way for a user to select some sort of custom schedule for it to execute, like every day/week/month and what day and time.

Also this is in Django with djcelery.

UPDATE

I see the code for the djcelery admin but am not clear on how that data is being persisted. I currently have a generic addTask view that looks like this :

def addTask(request):

intervalSchedule = IntervalSchedule.from_schedule(schedule(timedelta(seconds=10)))
intervalSchedule.save()
modelData = dict(
    name="dcTestPersist",
    task="technologytrackerapi.views.createRecord",
    schedule=intervalSchedule,
)
periodicTask = PeriodicTask(**modelData)
periodicTask.save()
return render_to_response('taskView.html')

The data looks correct in the db but when the daemon runs it has this error :

[2012-03-06 00:23:07,926: WARNING/Beat] Process Beat:
[2012-03-06 00:23:07,926: WARNING/Beat] Traceback (most recent call last):
[2012-03-06 00:23:07,926: WARNING/Beat] File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
[2012-03-06 00:23:07,926: WARNING/Beat] self.run()
[2012-03-06 00:23:07,927: WARNING/Beat] File "/home/dchesterman/Documents/PythonDev/.virtualenvs/ros/local/lib/python2.7/site-packages/celery/beat.py", line 464, in run
[2012-03-06 00:23:07,927: WARNING/Beat] self.service.start(embedded_process=True)
[2012-03-06 00:23:07,927: WARNING/Beat] File "/home/dchesterman/Documents/PythonDev /.virtualenvs/ros/local/lib/python2.7/site-packages/celery/beat.py", line 403, in start
[2012-03-06 00:23:07,927: WARNING/Beat] interval = self.scheduler.tick()
[2012-03-06 00:23:07,927: WARNING/Beat] File "/home/dchesterman/Documents/PythonDev/.virtualenvs/ros/local/lib/python2.7/site-packages/celery/beat.py", line 194, in tick
[2012-03-06 00:23:07,927: WARNING/Beat] next_time_to_run = self.maybe_due(entry, self.publisher)
[2012-03-06 00:23:07,927: WARNING/Beat] File "/home/dchesterman/Documents/PythonDev/.virtualenvs/ros/local/lib/python2.7/site-packages/celery/beat.py", line 170, in maybe_due
[2012-03-06 00:23:07,927: WARNING/Beat] is_due, next_time_to_run = entry.is_due()
[2012-03-06 00:23:07,928: WARNING/Beat] File "/home/dchesterman/Documents/PythonDev/.virtualenvs/ros/local/lib/python2.7/site-packages/djcelery/schedulers.py", line 54, in is_due
[2012-03-06 00:23:07,928: WARNING/Beat] return self.schedule.is_due(self.last_run_at)
[2012-03-06 00:23:07,928: WARNING/Beat] AttributeError: 'NoneType' object has no attribute 'is_due'

I'm not sure why my schedule does not use the default is_due()

score 6 · Accepted Answer · answered Apr 25 '12 at 16:50

6

This is what ended up working for me :

def addTask(request):

  intervalSchedule = IntervalSchedule.from_schedule(schedule(timedelta(seconds=10)))
  intervalSchedule.save()

  modelData = dict(
      name="dcTestPersist",
      task="technologytrackerapi.tasks.createRecord",
      interval_id=intervalSchedule.pk,
  )

  periodicTask = PeriodicTask(**modelData)
  periodicTask.save()

  me = ModelEntry(periodicTask)

  try:
      me.save()

  except:
    from django.db import connection
    print connection.queries
    raise

  return render_to_response('taskView.html')

I had to wrap the Periodic Task in a ModelEntry.

answered Apr 25 '12 at 16:50

Dustin

729
1
10
25

For some reason I don't understand I followed this creating my scheduled task and did not need to wrap with `ModelEntry`. Don't know why or what `ModelEntry` is. Random note - I was also setting `expires` and was perplexed that it has no effect. It *does* have an effect, but on `celeryd` not `celerybeat` - `celerybeat` continues to trigger indefinitely, but after `expires` `celeryd` ignores it (status `revoked`). – Chris Apr 28 '16 at 18:52
Its part of djcelery which is an integration with django. Not sure if you're using that. https://github.com/celery/django-celery I would doubt it. Looks deprecated. This is 4 years old. – Dustin Apr 28 '16 at 21:12
@ Dustin sorry wasn't so much asking as making a note for others that may come along afterwards. Your answer is the simplest example of scheduling a PeriodicTask for CeleryBeat programmatically on the internet! The rest use the config file or give only part of the answer. (I am using djcelery - I think that's where the database back-end come from, and I `from djcelery.models import *`.) – Chris Apr 29 '16 at 12:39
..although sorry should clarify `PeriodicTask` is in both celery and djcelery, as you imply - mine worked with the djcelery version. – Chris May 02 '16 at 16:59

score 1 · Answer 2 · answered Mar 04 '12 at 06:22

I think what you want to do is add PeriodicTasks to the database. Looks like the bottom section of https://github.com/ask/django-celery/blob/master/djcelery/admin.py is how they add in tasks in the admin -- you'll need to offer something similar on the front end.

How to programmatically generate celerybeat entries with celery and Django

2 Answers2

Linked