Stack Overflow
Stack Overflow

Questions tagged [celery-task]

celery-task is a building block of Celery distributed task queue system

celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.

727 questions
5
votes
0 answers

Get the exception object thrown from celery group tasks

I have a group object. I wan't to wait until all of the tasks finish and print their results. This is my code: tasks = [my_task.s(some_val=val).set(queue='different_q') for val in val_list] job = group(tasks) job_result = job.apply_async() results =…
Mr T.
  • 4,278
  • 9
  • 44
  • 61
5
votes
3 answers

Celery (Django + Redis) task fails: "No connection could be made because the target machine actively refused it"

UPDATE: I decided to try using Django as the broker for simplicity, as I assumed I did something wrong in the Redis setup. However, after making the changes described in the docs I get the same error as below when attempting to run a Celery task…
dkhaupt
  • 2,220
  • 3
  • 23
  • 37
5
votes
2 answers

How do I override the backend for celery tasks

we're using Redis as our result backend. However for one task we'd like to override this to use RabbitMQ instead. The documentation for Task.backend says: The result store backend to use for this task. Defaults to the CELERY_RESULT_BACKEND…
scytale
  • 12,346
  • 3
  • 32
  • 46
5
votes
0 answers

Celery - Get all the fail tasks from a particular queue

On my application layer, I've a single worker containing 8 queues, each queue contain task from different category. Each celery task saves record in a distributed DB on servers categorized by different categories. For eg:- Task for Category A will…
Praful Bagai
  • 16,684
  • 50
  • 136
  • 267
5
votes
1 answer

Celery - running a set of tasks with complex dependencies

In the application I'm working on, a user can perform a "transition" which consists of "steps". A step can have an arbitrary number of dependencies on other steps. I'd like to be able to call a transition and have the steps execute in parallel as…
ajd
  • 982
  • 1
  • 8
  • 19
5
votes
1 answer

Executing Python Script as Celery Task

I am trying to run a python script as a celery task with Django. The issue I am having is that the task thinks it is complete as soon as the script begins running. I initially used subprocess.popen() in the tasks.py file, but realized this would…
Jason Hawkins
  • 625
  • 1
  • 9
  • 24
5
votes
1 answer

Celery Result error "args must be a list or tuple"

I am running a Django website and have just gotten Celery to run, but I am getting confusing errors. Here is how the code is structured. In tests.py: from tasks import * from celery.result import AsyncResult project = Project.objects.create() #…
DoctorWizard
  • 303
  • 1
  • 3
  • 11
5
votes
2 answers

crontab not working with celery multi start

I am trying to get Celery work for awhile now. All my crontabs works just fine when I test it synchronously sudo celery -A testdjango worker --loglevel=DEBUG --beat but when I do celery multi start -A testdjango w1 -l info none of my crontabs…
Jonathan
  • 2,728
  • 10
  • 43
  • 73
5
votes
0 answers

How to get the execution time for a task using celery?

I want to fetch the execution time for any task, assuming I have task_id with me. Is there a way to get the execution time? I know the time will be logged by celery, but I don't want to parse the log file. On similar lines, is there a way to get…
Simpson
  • 292
  • 3
  • 13
5
votes
1 answer

Best way to map a generated list to a task in celery

I am looking for some advice as to the best way to map a list generated from a task to another task in celery. Let's say I have a task called parse, which parses a PDF document and outputs a list of pages. Each page then needs to be individually…
chaimp
  • 16,897
  • 16
  • 53
  • 86
5
votes
1 answer

received unregistered task of type

I am trying to run tasks which are in the memory . registerd tasks on worker [2012-09-13 11:10:18,928: WARNING/PoolWorker-1] [u'B.run', u'M1.run', u'M11.run', u'M22.run', u'M23.run', u'M24.run', u'M25.run', u'M26.run', u'M4.run',…
Prayag Gandhi
  • 51
  • 3
4
votes
1 answer

Celery worker variable sharing issues

I am using Python and celery in a project. In the project, I have two files: celeryconfig.py BROKER_URL = "amqp://guest:guest@localhost:5672//" CELERY_RESULT_BACKEND = "amqp" CELERY_IMPORTS = ("example",) CELERYD_CONCURRENCY = 2 and example.py from…
enderskill
  • 7,354
  • 3
  • 24
  • 23
4
votes
1 answer

Celery kombu.exceptions.ContentDisallowed in Docker

I am using celery with a fastAPI. Getting Can't decode message body: ContentDisallowed('Refusing to deserialize untrusted content of type json (application/json)') while running in docker. When running the same in local machine without docker there…
Nithin Varghese
  • 893
  • 1
  • 6
  • 28
4
votes
1 answer

How to make a celery worker stop receiving new tasks (Kubernetes)

So we have a kubernetes cluster running some pods with celery workers. We are using python3.6 to run those workers and celery version is 3.1.2 (I know, really old, we are working on upgrading it). We have also setup some autoscaling mechanism to add…
Antonio Gamiz Delgado
  • 1,871
  • 1
  • 12
  • 33
4
votes
1 answer

upgrading celery 4.x.x to 5.x.x in Django app - execute_from_commandline() replacement

the usage in 4.x.x was as following: from tenant_schemas_celery.app import CeleryApp class TenantCeleryApp(CeleryApp): def create_task_cls(self): return self.subclass_with_self('...', abstract=True, name='...',…
user2552806
1 2 3
…
48 49