Stack Overflow
Stack Overflow

Questions tagged [celery-task]

celery-task is a building block of Celery distributed task queue system

celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.

727 questions
3
votes
1 answer

Celery single task persistent data

Lets say a single task is enough for a machine to stay very busy for a few minutes. I want to get the result of the task, then depending on the result, have the worker perform the same task again. The question I cannot find an answer to is this:…
phil0stine
  • 303
  • 1
  • 13
2
votes
1 answer

Celery task state does not update

I'm seeing a bizarre problem with a Django app which uses Celery as as the task queue: I kick off a task. During execution, the task calls update_state(...) to set the task state to indicate progress. The task completes. I verify this through…
Bilal and Olga
  • 3,211
  • 6
  • 30
  • 33
2
votes
1 answer

Stop the currently running task by celery worker in Django

I have a Django application and React as my frontend framework and I am using celery for running lengthy tasks which do take a lot of time and everything is working fine. Also I am providing real time status of the each task running on the celery…
Om Kashyap
  • 118
  • 8
2
votes
0 answers

Celery only one physical server

Details: We have multiple jobs runnig in crontab. Those jobs call a set a apis say 10 and do some calculation and send reports. Some times same cron job calls the same api again. My task is i need to move this to celery. Problem is there is only one…
Amardeep Singh
  • 21
  • 2
2
votes
1 answer

Celery: custom Task/Request attribute shared with Queue

There is a tracker class, that just counts success, failed, pending, and started tasks via redis. The goal is to extend Celery, so its workers can access the group_id and keep statistics for the group. I expect an interface similar to: def…
Vova
  • 21
  • 2
2
votes
1 answer

Custom Celery task initialisation

The docs say that custom tasks are instantiated only once, and that it is a good way to cache needed data like database connections for all task invocations. But it seems it occurs both on the worker, and the caller. A task MCVE: # tasks.py from…
Amadan
  • 191,408
  • 23
  • 240
  • 301
2
votes
0 answers

Limit different number of workers for different groups of celery tasks

For different celery job groups, how can we limit the number of workers asynchronously differently. Here is a good reference on how to group different celery tasks together : https://sayari3.com/articles/18-chains-groups-and-chords-in-celery/ I need…
2017561-1
  • 149
  • 6
2
votes
2 answers

How to use async function inside celery task?

I've found some similar questions but couldn't find what I want. I have async function which is I want to use it inside my celery task but cannot call it with await inside task. Is there any way to do it? db.py async def select_users(): sql =…
mirodil
  • 422
  • 2
  • 8
  • 21
2
votes
1 answer

kombu.exceptions.EncodeError: Object of type set is not JSON serializable in celery

I am having a fast API where I am using celery for task queue predictions. whenever I am doing predictions and trying to return a response I am getting 500 Undocumented Error: Internal Server Error and receiving an error from docker…
user18842848
2
votes
0 answers

How to execute nested tasks with Celery?

I need to run multiple tasks (task3) for a list of users and the execution time should be as short as possible. So I want to group the tasks and run them in parallel. The problem is that two other tasks (task1 and task2) must be executed…
Florent
  • 1,791
  • 22
  • 40
2
votes
1 answer

Django celery page is giving 404

I'm trying to acces https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html but getting 404. Is celery not officially available?
Piyush Kumar
  • 191
  • 1
  • 10
2
votes
2 answers

Celery runing subtasks asynchronously within a task and waiting for results

I have a celery task that launches three other celery tasks. I want these tasks to execute asynchronously and wait for them to finish before i resume the parent task. However the child tasks are running synchronosly and I don't know why. The problem…
iam.mattevans
  • 51
  • 5
2
votes
0 answers

celery multi start PermissionError: [Errno 13] Permission denied: '/var/run/celery'

I'm trying to run celery multi workers for a specific group of task with the command celery multi start proj.main -E but when I execute it got me this permission error celery multi start proj.main -E celery multi v5.2.3 (dawn-chorus) Traceback…
WhiteAB
  • 162
  • 3
  • 11
2
votes
0 answers

How to quit selenium webdriver when celery task is revoked in Python

I'm running a selenium webdriver in a celery task. However, in some cases, I need to revoke/terminate the celery task. I successfully terminate the celery task using the following >>> from proj.celery import app >>> app.control.revoke(task_id,…
Muhammad Zubair
  • 466
  • 5
  • 17
2
votes
1 answer

Celery. Running single celery beat + multiple celery workers scale

Having single celery beat running by: celery -A app:celery beat --loglevel=DEBUG and three workers running by: celery -A app:celery worker -E --loglevel=ERROR -n n1 celery -A app:celery worker -E --loglevel=ERROR -n n2 celery -A app:celery worker…
user3583807
  • 766
  • 1
  • 8
  • 26
1 2 3
…
48 49