celery-task is a building block of Celery distributed task queue system
celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.
Questions tagged [celery-task]
727 questions
5
votes
1 answer
Deploying Celery Tasks from Asyncio Event Loop
I have a backend application that is currently written with asyncio: fastapi for a web server, sqlalchemy 1.4 + asyncpg for an async database driver. I have a need to deploy tasks to workers that will run and update the host application. Currently I…
Goodies
- 4,439
- 3
- 31
- 57
5
votes
0 answers
Celery revoke old task if new one is created with same arguments
I'm having trouble understanding the Celery queue and revokes. My main question is how to revoke (stop from executing) tasks that are already in the queue if a new task with the same name and arguments is created. so in…
quqa123
- 605
- 6
- 15
5
votes
1 answer
How to pause and stop celery task from django template
I am working on a django application that uses celery to run tasks asynchronously. Right now a user can submit a form from the webpage to start a celery task. But there is no way to pause or stop the task on the click of a button inside a django…
Sashaank
- 880
- 2
- 20
- 54
5
votes
1 answer
Upload files to s3 using celery tasks
I am trying to upload video file to s3 but after putting in task queue with celery. While the video is being uploaded, user can do other things.
My views.py to call celery tasks
def upload_blob(request, iterator, interview_id, candidate_id,…
Somnath Das
- 163
- 2
- 10
5
votes
2 answers
How can I test on_failure in celery
My celery task has a base class were an on_failure method is implemented.
In my test, I patched one of the methods that the task is calling to, to raise an exception but on_faliure is never called.
Base class
class BaseTask(celery.Task):
…
ericman
- 153
- 2
- 9
5
votes
1 answer
How to register Celery task to specific worker?
I am developing web application in Python/Django, and I have several tasks which are running in celery.
I have to run task A one at a time so I have created worker with --concurrency=1 and routed task A to that worker using following command.
celery…
Darshit
- 420
- 2
- 11
5
votes
1 answer
How to correctly connect to mongodb in Celery worker?
Learning Celery, read Celery Best Practices, and have quite a simple question about database usage with Celery.
Deni Bertovic says:
You shouldn't pass Database objects (for instance your User model) to
a background task because the serialized…
sortas
- 1,527
- 3
- 20
- 29
5
votes
2 answers
Celery add_periodic_task blocks Django running in uwsgi environment
I have written a module that dynamically adds periodic celery tasks based on a list of dictionaries in the projects settings (imported via django.conf.settings).
I do that using a function add_tasks that schedules a function to be called with a…
Tim
- 1,272
- 11
- 28
5
votes
3 answers
Celery beat not sending crontab task when hour is set
I'm using celery 4.1 and all my periodic tasks work correctly except where I set the hour in a crontab task. I was thinking it had something to do with the timezone setting, but I can't seem to work out where the problem…
Drew
- 491
- 5
- 9
5
votes
2 answers
Celery 4.0.2 AsyncResult.then not working
I would like to adopt the promise protocol as shown in the docs.
The example provided there works but the promise is handled on the worker side.
Instead I would like to get notified on the client.
Here is my test.py:
from celery import Celery
app =…
Fabbree
- 51
- 2
5
votes
2 answers
Celery : understanding the big picture
Celery seems to be a great tool, but I have hard time understanding how the various Celery components work together:
The workers
The apps
The tasks
The message Broker (like RabbitMQ)
From what I understand, the command line:
celery -A…
Archimondain
- 374
- 2
- 17
5
votes
1 answer
Are prefetched tasks in celery being acknowledged?
I have following setup:
RabbitMQ broker + Django
Celery with CELERYD_PREFETCH_MULTIPLIER=32 (I have a lot of small task
thus prefetching them makes a lot of sense from performance standpoint)
CELERY_ACKS_LATE=False (tasks are not idempotent)
I…
Dmytriy Voloshyn
- 1,032
- 12
- 27
5
votes
1 answer
periodic task in celery using python without django
Hello all i am newbie to celery and python. i am creating a simple task using rabbitmq-server. But i have no idea how to achieve periodic task using celery beat in python. I search but every where i get periodic task with django.
i use this code as…
Manish Yadav
- 435
- 2
- 8
- 24
5
votes
1 answer
Groups of chains with positional arguments in partial tasks using Celery
I am writing an application which will execute a group of several synchronous chains of tasks asynchronously.
In other words, I might have the pipeline foo(a,b,c) -> boo(a,b,c) for some list of bs.
My understanding is to create a chain of…
erip
- 16,374
- 11
- 66
- 121
5
votes
2 answers
Celery Flower dashboard remove worker
I recently installed flower 0.8.4, I created an snapshot in Digital Ocean and then launch a new server using this snapshot. Now flower is still showing the name of the old worker. How can I delete this worker while preserving existing worker…
gogasca
- 9,283
- 6
- 80
- 125