Stack Overflow
Stack Overflow

Questions tagged [celery-task]

celery-task is a building block of Celery distributed task queue system

celery-task is a building block of Celery distributed task queue system. Basically, it is a python class, but it can be created out of any callable by @celery.task decorator.

727 questions
3
votes
2 answers

Celery worker which directory to start from

I need some help regarding Celery workers. I'm especially not able to understand where (which directory) does the celery worker command needs to be fired from and what is the concept behind it and some things around imports. So say I have the…
qre0ct
  • 5,680
  • 10
  • 50
  • 86
3
votes
1 answer

How to add celery task to top of queue?

I use celery for doing some tasks, all tasks added by .apply_async and my script do it automatically, depends on some external conditions. I want to get result of tasks not in direct order but in reverse. For example, I add task1 after that task2…
Mihail Salnikov
  • 137
  • 2
  • 11
3
votes
1 answer

ModuleNotFoundError: No module named 'social.models' when running celery worker

I have the following project structure: SocialRating - accounts -- __init__.py -- models.py -- tasks.py - instagram -- __init__.py -- crawler.py - social -- __init__.py -- models.py # Not Django models - celery_worker.sh accounts/models.py…
artem
  • 16,382
  • 34
  • 113
  • 189
3
votes
1 answer

In Celery tasks, do retry_backoff and retry_backoff_max affect the manual self.retry() calls?

So according to the docs, default_retry_delayspecifies the time in seconds before a retry of the task should be executed. I just want to be sure that this only affects the manually called self.retry()calls and not the autoretries triggered by Celery…
Alec Gerona
  • 2,806
  • 1
  • 24
  • 24
3
votes
0 answers

celery interval scheduled tasks are not executing after 24 hours

I'm using django-celery-beat https://django-celery-beat.readthedocs.io/en/latest/ to manage celery tasks in my django project. Code Snippets: creating Interval time def get_or_create_interval(interval_time): schedule,created =…
Arbazz Hussain
  • 1,622
  • 2
  • 15
  • 41
3
votes
1 answer

Question: Usage of django celery.backend_cleanup

There is not much documentation available for the actual usage of django celery.backend_cleanup Let's assume i have following 4 tasks scheduled with different interval Checking DatabaseScheduler Logs I had found that only Task1 is executing on…
Arbazz Hussain
  • 1,622
  • 2
  • 15
  • 41
3
votes
2 answers

Celery distributing Queues and Workers

I am new to Celery and I am trying to understand how queues work. If I have two tasks say task1 and task2 and I place them in different queues and on task1 I use only a single worker and on task2 I use multiple works, will task1 only run one at a…
Max Powers
  • 1,119
  • 4
  • 23
  • 54
3
votes
1 answer

How to call a Celery shared_task?

I'm trying to use stream_framework in my application (NOT Django) but I'm having a problem calling the stream_framework shared tasks. Celery seems to find the tasks: -------------- celery@M3800 v3.1.25 (Cipater) ---- **** ----- --- * *** * --…
QLands
  • 2,424
  • 5
  • 30
  • 50
3
votes
2 answers

Celery tasks through view (Django) returned as pending, but okay via Terminal

This issue has been discussed before and looking over numerous posts, I am so far unable to find a solution to this problem. I'm new to celery so my learning curve is still fairly steep. Below my current scripts: myapp.__init__.py from __future__…
pymat
  • 1,090
  • 1
  • 23
  • 45
3
votes
1 answer

Celery | Flask error: expected a bytes-like object, AsyncResult found

I am currently working on a flask application (Python 3.6) and wanted to integrate celery because I have multiple long running background tasks. Edit: Celery 4.1 The integration was no problem and the celery tasks are executed correctly, but I…
Martin Pichler
  • 185
  • 2
  • 10
3
votes
0 answers

Group or Chain tasks from different REMOTE celery applications/workers

I want to experiment on grouping tasks from different celery applications, running on different/remote servers/machines. So for example, in application 1, running in server 1, I have this task: @app.task(name='func1', bind=True, queue='func1') def…
jaysonpryde
  • 2,733
  • 11
  • 44
  • 61
3
votes
0 answers

Celery Beat not Working

Celery beat is not scheduling task properly and freezes. Celery Configuration 4.1.0 CELERY_BROKER_URL = 'redis://127.0.0.1:6379' CELERY_ACCEPT_CONTENT = ['application/json'] CELERY_RESULT_SERIALIZER = 'json' CELERY_TASK_SERIALIZER =…
shuboy2014
  • 1,350
  • 2
  • 18
  • 44
3
votes
2 answers

is Celery Task initialized per each worker process, or once per app?

I have a heavy external library class which takes time to initialize and consumes a lot of memory. I want to create it once per task instance, at minimum. class NlpTask(Task): def __init__(self): print('initializing NLP parser') …
davka
  • 13,974
  • 11
  • 61
  • 86
3
votes
0 answers

celery set visibility_timeout not working

When running django celery tasks, i see that my task re execute after every hour, maybe by default visibility_timeout setting, so i try to change visibility_timeout = 120 for re execute task every 120 seconds in celery.py config like…
Thong Nguyen
  • 405
  • 1
  • 4
  • 10
3
votes
3 answers

Django celery and celery-beat daemonizing script error

Im going to set periodic tasks with celery for what im trying to daemonize a django celery process in a test server using a script (referenced on the official website): Init-script: celery So, this is my celeryd file: # Names of nodes to start # …
Brian Ocampo
  • 1,345
  • 1
  • 12
  • 22
1 2 3
…
48 49