What is the best Way to run a CronJob in containerized django application with 2 or more Replicas

Question

I am working on a project which is deployed on docker swarm as a service with 3 replicas. I want to run a simple management command to delete some rows from a table if the date has passed. I have written a django command for it, but want to make the run automated using cron job. I do want to make sure the Job is run only once a day from any of the container which is part of my service. On the internet I found some packages built for running cron jobs for Django application, but none of them considers more than one containers. Some packages have lock based approach but it were file based locks and not a shared lock. I did not want to celery for this simple task.

Following is glimpse of my command:


class Command(BaseCommand):
    """Command to clear user subscription if end_date has passed"""

    def handle(self, *args, **options):
        try:
            deleted_count, relative_deleted = MyModel.delete_inactive_instances()
        except Exception:
            raise CommandError('Could Not Remove Inactive Subscriptions From DB')
        else:
            self.stdout.write(self.style.SUCCESS('Successfully Removed Inactive Subscriptions %s ' % deleted_count))

I am currently running a command each day by docker exec:

python manage.py delete_inactive_instances

Following is my docker-stack file:

services:
  production_app:
    image: {id}.dkr.ecr.{region}.amazonaws.com/xxxxxx:latest
    expose:
      - 8000
    deploy:
      replicas: 2
    command: >
        sh -c "python manage.py migrate && gunicorn app.wsgi:application --workers 3 --bind 0.0.0.0:8000"
    env_file:
      - .prod.env

  nginx:
    image: {id}.dkr.ecr.{region}.amazonaws.com/nginx:latest
    ports:
      - 80:80

you can put your jobs in an async message broker and write/deploy workers to pick them up and run them. but this would be an overkill for most of the projects. an entrypoint script file which schedules cron jobs and runs the server after that would suffice. — Mahdi Sadeghi, Oct 10 '20 at 11:59
Yes, thought of entrypoint scripts, but in that case, they would be scheduled in each replicated container(currently 2). I want the above command to run only once a day. And message queue as you mentioned is overkill for running just a single command. — Aniket Singla, Oct 10 '20 at 14:19

score 1 · Answer 1 · edited Feb 14 '23 at 21:42

You can use Celery and the Celery Beat scheduler. It integrates perfectly with Django. In your docker swarm configuration you will add the following services :

a celery beat scheduler: 1 replica
a celery worker: as many replicas as you want
redis (for message broker)

If you don't want to add additionnal services to your swarm you can also run celery workers as a background task within your django container. However you should only run one instance of the beat scheduler to prevent task duplication. The beat scheduler is the one that controls when task should run, but they are actually executed by the celery workers

You can find an example of working setup with docker compose configuration on this project: https://github.com/testdrivenio/django-celery-beat

As an alternative you can just define an additional service in your stack configuration with the same image but a different entrypoint: this service will just be in charge of running the cron tasks that are defined by the entrypoint

What is the best Way to run a CronJob in containerized django application with 2 or more Replicas

1 Answers1