How to access PHP multi cURL handle response as soon as it happens

Question

I'm requesting multiple addresses with different response time:

$urlList = [
    'https://httpstat.us/200?sleep=3000',
    'https://httpstat.us/200?sleep=1000'
];

$mh = curl_multi_init();

$curlHandlerList = [];
foreach ($urlList as $i => $url) {
    $ch = $curlHandlerList[$i] = curl_init();

    curl_setopt_array($ch, [
        CURLOPT_URL => $url,
        CURLOPT_HEADER => false,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_SSL_VERIFYHOST => false,
        CURLOPT_SSL_VERIFYPEER => false,
    ]);

    curl_multi_add_handle($mh, $ch);
}

while (CURLM_OK == curl_multi_exec($mh, $active) && $active) {
    usleep(10**4); // 10**4 microseconds => 0.01 seconds
    curl_multi_select($mh, .01);
}

foreach ($curlHandlerList as $ch) {
    $html = curl_multi_getcontent($ch);

    echo "$html
";
}

Is it possible to access the output of already completed curl handlers as soon as it happens?

You can set write callback per transfer, with `CURLOPT_WRITEFUNCTION` — Daniel Stenberg, Mar 05 '19 at 09:06
@DanielStenberg You're right, this would be preferred solution. — Lukas, Mar 05 '19 at 12:43

Lukas · Answer 1 · 2019-03-05T08:07:21.600

Answering my own question since I could not find the answer anywhere. Instead of creating one multi cURL handler I've created one for each of the addresses that need to be requested. See code below:

$urlList = [                                                            
    'https://httpstat.us/200?sleep=3000',                               
    'https://httpstat.us/200?sleep=1000'                                
];                                                                      

processList($urlList, function($key, $output) {                         
    echo "Output from {$key}\n{$output}\n\n";                             
});                                                                     

function processList($urlList, $closure) {                              

    $curlHandlerList = $multiCurlHandlerList = [];                      

    foreach ($urlList as $i => $url) {                                  
        $ch = $curlHandlerList[$i] = curl_init();                       
        $mh = $multiCurlHandlerList[$i] = curl_multi_init();            

        curl_setopt_array($ch, [                                        
            CURLOPT_URL => $url,                                        
            CURLOPT_HEADER => false,                                    
            CURLOPT_RETURNTRANSFER => true,                             
            CURLOPT_SSL_VERIFYHOST => false,                            
            CURLOPT_SSL_VERIFYPEER => false,                            
        ]);                                                             

        curl_multi_add_handle($mh, $ch);                                
    }                                                                   

    while (count($multiCurlHandlerList)) {                              
        foreach ($multiCurlHandlerList as $i => $mh) {                  
            if (CURLM_OK != curl_multi_exec($mh, $active) || !$active) {

                $html = curl_multi_getcontent($curlHandlerList[$i]);    

                $closure($i, $html);                                    

                curl_multi_remove_handle($mh, $curlHandlerList[$i]);    
                unset($multiCurlHandlerList[$i], $curlHandlerList[$i]); 
            }                                                           
        }                                                               
        usleep(10**4); // 10**4 microseconds => 0.01 seconds            
    }                                                                   
}

Do note that usleep is a crappy replacement for doing the correct wait there. You'll wait too much when there's work to do and too little when there's nothing to do... — Daniel Stenberg, Mar 05 '19 at 09:07
By the correct way, you mean using curl_multi_select? It only accepts one resource so I don't think it can be implemented in my example? — Lukas, Mar 05 '19 at 12:50
Hm you're right. PHP has no way to do that "properly" (libcurl itself has APIs that allow it). I'd say that's another reason to solve it with a write callback... — Daniel Stenberg, Mar 05 '19 at 14:41

How to access PHP multi cURL handle response as soon as it happens

1 Answers1