celery-task is a building block of Celery distributed task queue system
celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.
Questions tagged [celery-task]
727 questions
4
votes
2 answers
Django Celery delay() always pushing to default 'celery' queue
I'm ripping my hair out with this one.
The crux of my issue is that, using the Django CELERY_DEFAULT_QUEUE setting in my settings.py is not forcing my tasks to go to that particular queue that I've set up. It always goes to the default celery queue…
Kyle O'Brien
- 932
- 1
- 11
- 28
4
votes
2 answers
Why when I try to use models in celery tasks django raises "Apps aren't loaded yet" error?
I work with django 1.10.5 and celery 4.0.2. I have the structure like this.
-proj
-application
__init__.py
celery.py
celery_conf.py
settings.py
tasks.py
urls.py
wsgi.py
-extuser (application with extended UserModel)
…
Dmitriy
- 53
- 8
4
votes
1 answer
Calling task chunk with keyword arguments
How do I chunk tasks with keyword arguments? For example, this task:
@app.task
def add(x, y, multiply=1, unit="GB"):
return '%s %s' % ((x + y) * multiply, unit)
I can call this task as usual with add.apply_async(args=(1, 2), kwargs={'unit':…
Oskar Persson
- 6,605
- 15
- 63
- 124
4
votes
2 answers
Celery beat serving old (removed) task
Under supervisor, celery beat serves periodic tasks to celery workers for a Django app of mine. I have 4 tasks, task1, task2, task3, and task4. Recently I made a 5th task: task5.
My problem is that I commented out task5 from my workers, removed its…
Hassan Baig
- 15,055
- 27
- 102
- 205
4
votes
3 answers
Python celery - how to wait for all subtasks in chord
I am unit testing celery tasks.
I have chain tasks that also have groups, so a chord is resulted.
The test should look like:
run celery task ( delay )
wait for task and all subtasks
assert
I tried the following:
def wait_for_result(result):
…
dynamicmindset
- 41
- 1
- 1
- 5
4
votes
1 answer
The right way to perform a task asynchronously in Django, via Celery
I have a Django app which mimics Instagram, in that users upload photos or memes, and then their fans get notified of the said photo.
Currently, in order to send notifications, as soon as a user uploads a photo, I iterate through every fan the…
Hassan Baig
- 15,055
- 27
- 102
- 205
4
votes
1 answer
Collect results from parallel Celery task executions
I am trying to achieve a rather simple Celery workflow, where I receive the result of multiple parallel invocations of the same task as a tuple (or list).
@app.task
def add(x, y):
return x + y
@app.task
def master():
return group(add.s(1,…
Igor Raush
- 15,080
- 1
- 34
- 55
4
votes
1 answer
Celery: @shared_task and non-standard BROKER_URL
I have a Celery 3.1.19 setup which uses a BROKER_URL including a virtual host.
# in settings.py
BROKER_URL = 'amqp://guest:guest@localhost:5672/yard'
Celery starts normally, loads the tasks, and the tasks I define within the @app.task decorator…
Falk Schuetzenmeister
- 1,497
- 1
- 16
- 35
4
votes
1 answer
Make supervisor stop Celery workers correctly
I meet a lot weird thing when using celery. Such as, I update tasks.py, supervisorctl reload(restart), but tasks is wrong. Some tasks seems disappear and so on.
Today I found that because supervisorctl stop all can not stop all celery workers. And…
Mithril
- 12,947
- 18
- 102
- 153
4
votes
1 answer
How to access the orm with celery tasks?
I'm trying to flip a boolean flag for particular types of objects in my database using sqlalchemy+celery beats. But how do I access my orm from the tasks.py file?
from models import Book
from celery.decorators import periodic_task
from application…
user805981
- 9,979
- 8
- 44
- 64
4
votes
1 answer
Celery rate_limit not respected when scheduling via 'eta' option
I'd like to schedule a task to run at various times in the future. I'm trying to use the eta option when calling the task to do this. This part works fine on its own, but at the same time, I'd like to have a rate limit on the task. I'm not able to…
apatel
- 83
- 3
4
votes
1 answer
Celery + Gevent pool hangs after 1000+ tasks execution
We have 8 Core, 16 GB memory, Linux server running celery,
It is running a celery worker queue myQueue, and running with 1000 concurrency under gevent pool.
After executing tasks for around 1 hour, worker suddenly freezes, it is not accepting new…
Vimal
- 189
- 1
- 3
- 13
4
votes
0 answers
Node.js script (node-celery) call to celery task handles 'self' argument improperly
I created a celery task script as follows:
from celery import Task
from celery.contrib.methods import task
from celery.contrib.methods import task_method
from pipelines.addsub import settings
from pipelines.addsub.log import…
Ed Johns
- 211
- 3
- 11
4
votes
0 answers
Celery workers stop working with amazon sqs
I am using celery 3.1.9 with sqs. I have worker common_w running as daemon. It works with common queue on sqs.
A worker unexpectedly stop processing tasks. No exceptions and errors.
Last logs with option -l DEBUG:
[2014-09-03 21:01:14,766:…
willfill
- 294
- 2
- 9
4
votes
0 answers
Use multiple message brokers with celery and django
I am using celery with django.
I'm have two celery apps running on the same server but using different message brokers.
One is using redis another is using django-orm.
The app using the redis backend is configured to receive tasks from only a…
Shaumux
- 735
- 11
- 25