Stack Overflow
Stack Overflow

Questions tagged [celerybeat]

The Celery periodic task scheduler

Celery beat is a scheduler for periodic tasks in Celery. It provides useful functions to create and execute tasks on certain intervals, with support for fail-over, retries, status tracking etc. For more info look at project documentation: http://celery.github.com/celery/internals/reference/celery.beat.html

514 questions
14
votes
4 answers

Supervising celerybeat with supervisor and virtualenv

My celerybeat.conf [program:celerybeat] command=/path/app/env/bin/celery beat -A project.tasks…
Gr1N
  • 1,337
  • 3
  • 14
  • 19
13
votes
4 answers

Django Celery results set task id to something human readable?

After many days I have a working celery and celery beat task list, and the results are stored using django_celery_results. However when I look at the table record, it hasn't got any useful information in there. is it possible to set the task id to…
AlexW
  • 2,843
  • 12
  • 74
  • 156
13
votes
2 answers

Multiple instances of celerybeat for autoscaled django app on elasticbeanstalk

I am trying to figure out the best way to structure a Django app that uses Celery to handle async and scheduled tasks in an autoscaling AWS ElasticBeanstalk environment. So far I have used only a single instance Elastic Beanstalk environment with…
wuser92
  • 479
  • 3
  • 16
13
votes
2 answers

Celery beat schedule multiple tasks under same time-interval group

I'm trying to set up two tasks that both run every minute. Is there any way to group them in one to run? I specified CELERYBEAT_SCHEDULE in my celeryconfig.py as following: CELERYBEAT_SCHEDULE = { 'every-minute': { 'task': 'tasks.add', …
xbb
  • 2,073
  • 1
  • 19
  • 34
12
votes
2 answers

How to programmatically generate celerybeat entries with celery and Django

I am hoping to be able to programmatically generate celerybeat entries and resync celerybeat when entries are added. The docs here state By default the entries are taken from the CELERYBEAT_SCHEDULE setting, but custom stores can also be used,…
Dustin
  • 729
  • 1
  • 10
  • 25
12
votes
2 answers

How can to start tasks with celery-beat?

Why i can't run a periodic tasks? proj/settings.py REDIS_HOST = 'localhost' REDIS_PORT = '6379' CELERY_BROKER_URL = 'redis://localhost:6379' BROKER_URL = 'redis://' + REDIS_HOST + ':' + REDIS_PORT CELERY_BEAT_SCHEDULE = { 'task-first': { …
user11301070
12
votes
4 answers

Correct setup of django redis celery and celery beats

I have being trying to setup django + celery + redis + celery_beats but it is giving me trouble. The documentation is quite straightforward, but when I run the django server, redis, celery and celery beats, nothing gets printed or logged (all my…
Alejandro Veintimilla
  • 10,743
  • 23
  • 91
  • 180
12
votes
1 answer

Setting up periodic tasks in Celery (celerybeat) dynamically using add_periodic_task

I'm using Celery 4.0.1 with Django 1.10 and I have troubles scheduling tasks (running a task works fine). Here is the celery configuration: os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myapp.settings') app =…
Marc
  • 1,340
  • 2
  • 14
  • 23
12
votes
4 answers

How to dynamically add a scheduled task to Celery beat

Using Celery ver.3.1.23, I am trying to dynamically add a scheduled task to celery beat. I have one celery worker and one celery beat instance running. Triggering a standard celery task y running task.delay() works ok. When I define a scheduled…
Jakub Czaplicki
  • 1,787
  • 2
  • 28
  • 50
12
votes
3 answers

celerybeat - multiple instances & monitoring

I'm having application built using celery and recently we got a requirement to run certain tasks on schedule. I think celerybeat is perfect for this, but I got few questions: Is it possible to run multiple celerybeat instances, so that tasks are…
DmitrySemenov
  • 9,204
  • 15
  • 76
  • 121
12
votes
5 answers

Celery Beat: Limit to single task instance at a time

I have celery beat and celery (four workers) to do some processing steps in bulk. One of those tasks is roughly along the lines of, "for each X that hasn't had a Y created, create a Y." The task is run periodically at a semi-rapid rate (10sec). The…
rwe
  • 632
  • 1
  • 8
  • 13
11
votes
2 answers

Celery beat not starting EOFError('Ran out of input')

Everything worked perfectly fine until: celery beat v3.1.18 (Cipater) is starting. __ - ... __ - _ Configuration -> . broker -> amqp://user:**@staging-api.user-app.com:5672// . loader -> celery.loaders.app.AppLoader .…
Benjamin Toueg
  • 10,511
  • 7
  • 48
  • 79
11
votes
1 answer

Django/djcelery 1.8.2 AppRegistryNotReady: Translation infrastructure cannot be initialized

I am getting the following error: File "/Library/Python/2.7/site-packages/Django-1.8.2-py2.7.egg/django/utils/translation/trans_real.py", line 164, in _add_installed_apps_translations "The translation infrastructure cannot be initialized before the…
Manas
  • 399
  • 1
  • 4
  • 13
10
votes
1 answer

Celery and Redis keep running out of memory

I have a Django app deployed to Heroku, with a worker process running celery (+ celerycam for monitoring). I am using RedisToGo's Redis database as a broker. I noticed that Redis keeps running out of memory. This is what my procfile looks like: web:…
Bilal and Olga
  • 3,211
  • 6
  • 30
  • 33
10
votes
1 answer

How to set up different weekday/weekend schedules for celery beat in Django?

How do I go about scheduling my tasks differently for weekdays and weekends in celery beat? The schedule is set as follows in my settings.py file { "task_weekday": { "task": "tasks.my_regular_task", "schedule":…
ChickenNugget
  • 101
  • 3
1
2
3
…
34 35