1

I'm currently running on docker-compose on an 8 core instance (docker_serivce, dramatiq_service, rabbitmq_service, postgres_service, etc): django_apscheduler==0.5.2 dramatiq [rabbitmq, watch]==1.8.1 RabbitMQ (rabbitmq: 3.8-management-alpine image)

The bug is that many processes are executing the same task. The task is created only once (apscheduler BackgroundScheduler), but several processes try to execute it, and some threads are the same between different processes.

Here is an example. The job is in the queue once but when it's executed, many processes take it. 139698903181056 is the thread and 21 the process

"INFO | 10/06/2021 10:30:09 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:09 | 139698894788352 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:08 | 139698911573760 | app.orders.tasks | tasks | tasks.py | 18 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:07 | 139699123865344 | app.orders.tasks | tasks | tasks.py | 19 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:03 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:02 | 139698374702848 | app.orders.tasks | tasks | tasks.py | 16 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 19 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698903181056 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:01 | 139698307593984 | app.orders.tasks | tasks | tasks.py | 21 | 39 | send_menu | Sending menu #87..."
"INFO | 10/06/2021 10:30:00 | 139698861217536 | app.orders.tasks | tasks | tasks.py | 22 | 39 | send_menu | Sending menu #87..."
Rodrigo Suarez
  • 184
  • 6

1 Answers1

0

The problem was related to Django-apscheduler.

The dramatiq container (8 workers) started with the scheduler paused, but in one part of my code, there was a piece of code that resumes the scheduler, so all 8 workers started triggering the tasks at the same time, causing the repeated tasks even though the job was created only once.

Moral: always check if your scheduler starts only once.

Rodrigo Suarez
  • 184
  • 6