Stack Overflow
Stack Overflow

Questions tagged [celery-task]

celery-task is a building block of Celery distributed task queue system

celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.

727 questions
8
votes
2 answers

Storing a task id for each celery task in database

I have using celery with MySQL. I want to to store the task id as a plain integer in a database or in a variable of celery task. How can I do that?
pynovice
  • 7,424
  • 25
  • 69
  • 109
7
votes
1 answer

python celery invalid value for -A unable to load application

I have a following project directory: azima: __init.py main.py tasks.py task.py from .main import app @app.task def add(x, y): return x + y @app.task def mul(x, y): return x * y @app.task def xsum(numbers): return…
Azima
  • 3,835
  • 15
  • 49
  • 95
7
votes
1 answer

Flask Error: Unable to load celery application

Please help me to get out of this problem I am getting this when I am running celery -A app.celery worker --loglevel=info Error: Unable to load celery application. The module app.celery was not found. My code is-- # Celery Configuration from…
Mohd Mujtaba
  • 169
  • 1
  • 2
  • 8
7
votes
1 answer

Using Celery queues with multiple apps

How do you use a Celery queue with the same name for multiple apps? I have an application with N client databases, which all require Celery task processing on a specific queue M. For each client database, I have a separate celery worker that I…
Cerin
  • 60,957
  • 96
  • 316
  • 522
7
votes
1 answer

Django + Celery tasks on multiple worker nodes

I've deployed a django(1.10) + celery(4.x) on the same VM, with rabbitmq being the broker(on the same machine). I want to develop the same application on a multi-node architecture like I can just replicate a number of worker nodes, and scale the…
Vatsal Parekh
  • 248
  • 4
  • 12
7
votes
1 answer

Celery Tasks Lost after sending to rabbitmq broker

I am having a strange issue where some of my tasks are getting dropped after sent to broker. This happens for around 1 out of 10 tasks. I have checked that there is no old celery worker consuming the task. I used database backed backend and flower…
Anurag
  • 1,521
  • 2
  • 17
  • 34
7
votes
3 answers

Flask Celery update_state from inside another function

I'd like to update the state of my Celery task from another function. Here's what I have now: The Route @app.route('/my-long-function', methods=['POST']) def my_long_function(): param1 = request.form['param1'] param2 =…
vandernath
  • 3,665
  • 3
  • 15
  • 24
7
votes
3 answers

Tasks being repeated in Celery

After a couple days, my celery service will repeat a task over and over indefinitely. This is somewhat difficult to reproduce, but happens regularly once a week or more frequently depending on the tasks volume being processed. I will appreciate any…
rodorgas
  • 962
  • 2
  • 12
  • 29
7
votes
1 answer

Celery is rerunning long running completed tasks over and over

I've a python celery-redis queue processing uploads and downloads worth gigs and gigs of data at a time. Few of the uploads takes upto few hours. However once such a task finishes, I'm witnessing this bizarre celery behaviour that the celery…
user2252999
  • 183
  • 2
  • 10
7
votes
2 answers

Celery - Activate a task via command line or HTTP requests

I have a predefined celery task in my code, say my_proj.tasks.my_celery_task I want to activate the task via the command line/HTTP request (not via my application). I searched the documents (saw flower and curl options) but there isn't a real good…
Lin
  • 2,445
  • 5
  • 27
  • 37
7
votes
1 answer

Get current celery task id anywhere in the thread

I'd like to get the task id inside a running task, without knowing which task I'm in. (That's why I can't use https://stackoverflow.com/a/8096086/245024) I'd like it to be something like this: @task def my_task(): foo() def foo(): …
lev
  • 3,986
  • 4
  • 33
  • 46
7
votes
2 answers

Stopping celery task gracefully

I'd like to quit a celery task gracefully (i.e. not by calling revoke(celery_task_id, terminate=True)). I thought I'd send a message to the task that sets a flag, so that the task function can return. What's the best way to communicate with a task?
orange
  • 7,755
  • 14
  • 75
  • 139
7
votes
0 answers

accessing celery task results in chain within a group

My workflow with celery is: group1 = (task1, task2) chain2 = (task3, group1) group3 = (task4, task5, chain2) When I start group3, everything goes fine: all the task are executed with the "dependency" that I need. The tasks perform some operation…
Desmond
  • 567
  • 1
  • 7
  • 17
7
votes
1 answer

python celery max-tasks-per-child-setting default

I am using celery and i want to use the max-tasks-per-child-setting because some of the celery process get high memory usage. I am trying to find the default value for this setting before i change but i cant find that information. I looked here but…
yossi
  • 12,945
  • 28
  • 84
  • 110
6
votes
2 answers

Prevent celery task from starting until a different task with similar arguments is done

Let's say I have a celery task which takes two arguments: X(a,b) I need to implement custom concurrency logic with the following two rules: Instances of X can run concurrently if they have different values for a. That is, if X(a=1,b=10) is running…
Johnny Metz
  • 5,977
  • 18
  • 82
  • 146
1 2 3
…
48 49