Sharing curl handler among gearman workers

Question

I've a gearman worker (called "manager") which reads a list of urls and then calls, for each of them, another gearman worker (called "dependent") which fetches the content of passed url and does some work on it - indexing, and so on. Each url belongs to the same domain (eg: www.test.com). When every worker calls the curl_init(), a new connection is made to www.test.com.

I'm looking to a method to share the curl's resource handler among workers in order to reuse the same connection. In my case the manager could create the resource handler, then pass it to dependents as a workload parameter.

score 0 · Answer 1 · answered Feb 26 '13 at 15:08

0

You can use shared memory for store a variable with semaphore functions, http://www.php.net/manual/en/book.sem.php.

Or you can use APC, and dump cached variables into file. But I'am not sure how it will works.

answered Feb 26 '13 at 15:08

Kirill Zorin

239
1
5

Ofcourse, it doesn't work if your workers placed at different servers – Kirill Zorin Feb 26 '13 at 15:12
Could I use a centralized redis store? Redis could hold something as: – fdellutri Feb 26 '13 at 15:17
1

What for? And why you want to share cURL resourse instead of to create new? – Kirill Zorin Feb 26 '13 at 15:22

Sharing curl handler among gearman workers

1 Answers1