Solutions for scheduling cronjob

Question

I have setup a list of cron, Some of the cron script takes a long time to run (like 1-5 hours, and they are increasing every day). I do not want to run two cron scripts at the same time, as I do not have the resources or need for it. I need to find a solution so that the scheduled scripts will not start until the other previous script has finished. I have 10-15 cron job in the list, where I don't want to overlap 5 of them.

Does anyone have any suggestions on what kind of solution I should be looking more closely at to achieve my goal?

score 2 · Answer 1 · answered Dec 10 '12 at 12:12

2

A. Create temporary lockfile which will be used by the script while it's running, if new script will find this file - it will stop.

i.e.

if [ -a lockfile ]; then
exit 0
else
touch lockfile
fi
... rest of code...
rm -f lockfile

B. Check with the script if another one is already running like ps auxwf|grep blabla.

answered Dec 10 '12 at 12:12

GioMac

4,544
4
27
41

Hi thanks, but do you know any API that handle the things? like i see some of them like resque-scheduler (ruby on rail) / Celery (python). is there anything like this in PHP? – Shamsul Dec 10 '12 at 12:36
@Shamsul No reason to overcomplicate things. Lock files work just fine for your use case and take about a minute to build in. – ceejayoz Dec 10 '12 at 14:56
Agree with ceejayoz. No need to have API and complicate things, ex: file locking is used to for this: http://php.net/manual/en/function.flock.php – GioMac Dec 11 '12 at 21:46

score 1 · Answer 2 · answered Dec 10 '12 at 14:41

1

Using lock file approach may lead to situation wher daily cronjob wont start. Have you considered running those jobs from one cronjob sequentally

#!/bin/bash/
sh /jobs/somejob1
sh /jobs/somejob2

I recommend running i/o heavy apps sequentally from one job

answered Dec 10 '12 at 14:41

simplexio

146
2

No, let me cleat the things a little more.. The corn list are like i have scheduled 1st corn at 8:00 PM every day, and then 2nd at 11:00, 3rd at 1:00 AM, so if the 1st job stop by 11:00 that's good but the problem is its sometime takes 12:00 to finish and my 2nd job starts and i do not want it to start at 11 it will start at 12:00 but 3rd job will also depends start on finishing on the 2nd job. I think its more clear now. – Shamsul Dec 10 '12 at 14:58

score 0 · Answer 3 · answered Dec 10 '12 at 14:28

You can work with lockfiles in PHP too. I solved exactly the same problem with code:

$lockfile='/tmp/'.__CLASS__.'.lock'; // the lockfile should be different for each script
$lock=fopen($lockfile, 'w');
if(!flock($lock, LOCK_EX|LOCK_NB)){
//  $this->log("This process is already started, i'm exiting.");
    exit;
}

Place this at the beginning of you job script.

score 0 · Answer 4 · answered Dec 10 '12 at 15:04

0

There is a package called lockfile-progs available in any debian-derived (and probably other) distribution, and it works.

answered Dec 10 '12 at 15:04

rackandboneman

2,577
11
8

score 0 · Answer 5 · answered Dec 10 '12 at 15:43

0

The very same topic came up on Sysadvent earlier this month: http://sysadvent.blogspot.co.uk/2012/12/day-4-zookeeper-for-distributed.html

You can use Zookeeper as a distributed lock manager for lockfiles across a cluster of machines.

answered Dec 10 '12 at 15:43

Tom O'Connor

27,480
10
73
148

Solutions for scheduling cronjob

5 Answers5