running into speed issues with multi curl. I am using multi curl to grab XML from various urls, all with response times of under 300ms. But my multi curl function is taking over 1 second to grab these URLS (About 10-15 URLS only). Below is the code I am using:
function multiRequest($data, $options = array()) {
// array of curl handles
$curly = array();
// data to be returned
$result = array();
// multi handle
$mh = curl_multi_init();
// loop through $data and create curl handles
// then add them to the multi-handle
foreach ($data as $id => $d) {
$curly[$id] = curl_init();
$url = (is_array($d) && !empty($d['url'])) ? $d['url'] : $d;
curl_setopt($curly[$id], CURLOPT_URL, $url);
curl_setopt($curly[$id], CURLOPT_HEADER, 0);
curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curly[$id], CURLOPT_NOSIGNAL, 1);
curl_setopt($curly[$id], CURLOPT_TIMEOUT_MS, 750);
curl_multi_add_handle($mh, $curly[$id]);
}
// execute the handles
do {
curl_multi_select($mh, .01);
$status = curl_multi_exec($mh, $running);
} while ($status === CURLM_CALL_MULTI_PERFORM || $running);
// get content and remove handles
foreach($curly as $id => $c) {
if(curl_errno($c) == 0)
$result[$id] = curl_multi_getcontent($c);
curl_multi_remove_handle($mh, $c);
}
// all done
curl_multi_close($mh);
return $result;
}
Is there anything I should be doing to speed this up? My clients throw away the request if it takes over 500ms to complete, so I want to get it to run as long as the longest request takes. I have the timeout set to 750ms, because even though the request times are less then 300ms, my function returns no results if below 750ms.
