How do you manage drupal's cron when spread on multiple servers?

Question

What's the best way to stop the machines fighting, while keeping them identical to keep things simple and reduce single points of failure?

Can you clarify what the cron jobs do and why they are "fighting"? — sciurus, Sep 01 '11 at 16:13
What is your configuration and what resource(s) (is|are) being fought over? — Bart Silverstrim, Sep 01 '11 at 16:21
You might get better results if this was asked on http://drupal.stackexchange.com/. — Zoredache, Sep 01 '11 at 16:23

score 2 · Accepted Answer · answered Sep 01 '11 at 16:24

Check out this article which suggests using Hudson (now Jenkins) for the job.

Excerpt:

For remote jobs, Hudson can sign onto systems with SSH, copy over its own runtime, and run whatever you’d like on the remote system. This means that, no matter how many servers in a cluster need scheduled jobs, Hudson can schedule, run, and log them from one server. Hudson can distribute the jobs dynamically based on which machines are already busy, or it can bind jobs to specific boxes.

Cheers

score 1 · Answer 2 · answered Sep 01 '11 at 16:04

The best way is to maintain some kind of global lock right in the script. This lock should be reachable from every server so there are two obvious choices here: using a DB (and you already have it) and using a distributed file system (and you don't have it probably). And of course you will have to modify behaviour of the script to set and to remove the lock.

How do you manage drupal's cron when spread on multiple servers?

2 Answers2