PHP Advance Job Queue

Question

I am making a script with 2000 jobs in a day using cron (means that is server side and automatically do all jobs.)

but the job require to run simultaneously 10 (or specified no. of jobs ) jobs .

like if u see IDM(internet download manager ) there is a queue function it run multiple jobs at a time and if any complete it start another . i want something like this ..

how can i do this ?

score 4 · Accepted Answer · edited May 23 '17 at 11:47

4

You can either go ahead and write your own custom job queue handler. Spawn a separate process per job, and keep collecting the response in the parent process. Restart new jobs when previous one's have finished.

Or alternately you can dig into using Gearman (specially if you have multiple boxes running jobs in parallel). Also do view solutions proposed here on asynchronous-processing-with-php-one-worker-per-job

edited May 23 '17 at 11:47

Community

1
1

answered Apr 15 '11 at 05:13

Abhinav Singh

2,643
1
19
29

By the way, I've written my own job manager/queue, which was nice, but did not handle multiple servers very well. Now I'm using gearman instead. – Peter Lindqvist Apr 15 '11 at 08:28
I am newbe in PHP and i dont know how to make php job handler . I will be thankful if will give some more help abt how to make it or any tutorial link also dont idea abt ur Gearman how to use it . – Soumo Gorai Apr 15 '11 at 18:22
php is not a language of choice when it comes to writting job processors and managers which handle jobs across a cluster.... gearman should be ur first choice and you should only use php to instruct it.... I agree with Nick, in future you will soon face unknown and unexpected issues with your own custom solutions, that too if you choose php for such a task. – Abhinav Singh Apr 16 '11 at 09:34

score 2 · Answer 2 · answered Apr 15 '11 at 07:17

I disagree with making your own job queue handler. You're going to run into problems down the line that you didn't anticipate and that existing projects have already met and dealt with.

I'd go with something like beanstalkd, build a generic script to handle the jobs in the queue, then spawn $x child processes to go through and process them.

score 1 · Answer 3 · answered Mar 23 '12 at 16:08

1

There's also the wordpress job queue - see http://code.trac.wordpress.org/wiki/JobsDocs (I've not used it, so can't vouch for it)

answered Mar 23 '12 at 16:08

David Goodwin

4,184
1
22
12

score -1 · Answer 4 · answered Apr 15 '11 at 05:40

-1

You would be well off writing your own job queue handler.
You could make php ignore_user_abort and make it a daemon process too...

But make sure you have some control over it before you do that.

answered Apr 15 '11 at 05:40

infinite

7
2

PHP Advance Job Queue

4 Answers4

Linked