Gearman Worker with REST API

Question

I wanted to check this concept and what everyone thinks of it. I want to use Gearman to run a couple tasks in the background (mainly data gathering and processing).

I want to set up Gearman around a PHP Framework (in my case, it is CakePHP2). As far as I understand Gearman, the workers are simply a PHP daemon running a single php script (ie worker.php).

I am trying to figure out how to bring the logic/code that I have already built in the PHP Framework into that single worker script. Or else I find I may have to rebuild a whole lotta stuff like the models.

So my solution is that to keep the worker.php light and instead have the worker script simply initiate REST API calls via CURL. Does ths sound like a good option to go for?

score 1 · Accepted Answer · answered Oct 03 '12 at 18:03

Making a cURL call sounds a bit of an overhead to me.. I'd first explore the following:

Load a part of the framework in the worker (cheaper than to make a call & then invoke the framework)
Make a special worker for "calling back". E.g., the real workers would do the real work and at completion would submit a result to the "callback" queue. That callback worker then would be accepting the jobs from Gearman and taking them from there (just like a REST API endpoint would take it from there).

In any way, unless you are splitting the work between machines, it's not a good idea to make extra HTTP calls. Why involve Apache, if you can just work directly with PHP?

Thanks, I had not considered the overheard from doing the cURL calls. — MechaStorm, Oct 05 '12 at 01:03

score 0 · Answer 2 · answered Oct 05 '12 at 01:09

I am adding this as an addition to the answer but reply was too short.

I have been able to do the Gearman workers as a CakePHP console and wrap it around a supervisord config

// Add the example configs below
[program:my-gearman-test]
command= /path/to/cakephp/app/Console/cake gearman test
process_name= %(process_num)g-gearman-test
numprocs=1 
directory=/path/to/cakephp/app
autostart=true
autorestart=true
user=www-data
stdout_logfile=/path/to/logs/worker_stdout.log
stdout_logfile_maxbytes=1MB
stderr_logfile=/path/to/logs/worker_stderr.log
stderr_logfile_maxbytes=1MB

Then I run supervisord, and it will start initializing the Cake console command. Now with the worker part of the CakePHP framework, I get all the goodness that comes from using a framework instead of doing a standalone worker.php script

Gearman Worker with REST API

2 Answers2