0

I did quite get the problem that arises by sharing a job store across multiple schedulers in APScheduler.

The official documentation mentions

Job stores must never be shared between schedulers

but doesn't discuss the problems related to that, Can someone please explain it?

and also if I deploy a Django application containing APScheduler in production, will multiple job stores be created for each worker process?

kethan bravo
  • 93
  • 2
  • 6

1 Answers1

1

There are multiple reasons for this. In APScheduler 3.x, schedulers do not have any means to signal each other about changes happening in the job stores. When the scheduler starts, it queries the job store for jobs due for execution, processes them and then asks how long it should sleep until the next due job. If another scheduler adds a job that would be executed before that wake-up time, the other scheduler would happily sleep past that time because there is no mechanism with which it could receive a notification about the new (or updated) job.

Additionally, schedulers do not have the ability to enforce the maximum number of running instances of a job since they don't communicate with other schedulers. This can lead to conflicts when the same job is run on more than one scheduler process at the same time.

These shortcomings are addressed in the upcoming 4.x series and the ability to share job stores could be considered one of its most significant new features.

Alex Grönholm
  • 5,563
  • 29
  • 32
  • If a new job is added by, let's say, scheduler2 then scheduler2 knows about that job, and it will trigger the job at the right time. But I see a problem of execution of a job multiple times if multiple schedulers are querying a single job store. – kethan bravo Jan 29 '21 at 16:43
  • So does this answer your question? Please mark the answer as such if it does. – Alex Grönholm Jan 31 '21 at 11:05