Scale number of workers according to number of jobs in gearman

Question

I am using Gearman as my job queue manager. Basically i use gearman to manage mail jobs (fire & forget approach). I have a single worker to handle those mail sending jobs. When my application is going to send large amount (>1000) of emails, i can see the queue is growing and it takes some time to process the complete task.

Can i add more workers to pick jobs from the queue?. Can Gearman handle that worker management or should i intentionally add workers by tracking the queue size. Can anybody help me?

score 0 · Answer 1 · answered Nov 29 '13 at 02:48

0

Gearman cannot automatically launch new workers on demand. You can use GermanManager with different configs to launch required number of jobs for you when the queue is getting large.

answered Nov 29 '13 at 02:48

mym

252
2
9

is there any way by which we can analyze system cpu/capacity and launch more worker accordingly by writing script or code ? any idea or guidance will be really helpful – Rishabh Agrawal Oct 22 '17 at 19:21
also how can we identify that which worker performed particular job ? – Rishabh Agrawal Oct 22 '17 at 19:21

Scale number of workers according to number of jobs in gearman

1 Answers1