Add Delay/Sleep in between url requests in multi cutl php

Question

I am fetching json data from api which allows me to hit 1 request per minute. Currently using multi curl but don't have idea how to execute curl after every 60sec so that i can get the 10 json file from 10 different url/api request in 10 minutes. Pls check the code and help me out.

function multi_thread_curl($urlArray, $optionArray, $nThreads) {

//Group your urls into groups/threads.
$curlArray = array_chunk($urlArray, $nThreads, $preserve_keys = true);

//Iterate through each batch of urls.
$ch = 'ch_';
foreach ($curlArray as $threads) {

    //Create your cURL resources.
    foreach ($threads as $thread => $value) {

        ${$ch . $thread} = curl_init();

        curl_setopt_array(${$ch . $thread}, $optionArray); //Set your main curl options.
        curl_setopt(${$ch . $thread}, CURLOPT_URL, $value); //Set url.

    }

    //Create the multiple cURL handler.
    $mh = curl_multi_init();

    //Add the handles.
    foreach ($threads as $thread => $value) {

        curl_multi_add_handle($mh, ${$ch . $thread});

    }

    $active = null;

    //execute the handles.
    do {

        $mrc = curl_multi_exec($mh, $active);

    } while ($mrc == CURLM_CALL_MULTI_PERFORM);

    while ($active && $mrc == CURLM_OK) {

        if (curl_multi_select($mh) != -1) {
            do {

                $mrc = curl_multi_exec($mh, $active);

            } while ($mrc == CURLM_CALL_MULTI_PERFORM);
        }

    }

    //Get your data and close the handles.
    foreach ($threads as $thread => $value) {

        $results[$thread] = curl_multi_getcontent(${$ch . $thread});

        curl_multi_remove_handle($mh, ${$ch . $thread});

    }

    //Close the multi handle exec.
    curl_multi_close($mh);

}


return $results;

}


$optionArray = array(
    CURLOPT_RETURNTRANSFER => TRUE,
    CURLOPT_TIMEOUT => 20
);

$nThreads = 1;

$urlArray = array("url1", "url2","url3", "url10");

$results = multi_thread_curl($urlArray, $optionArray, $nThreads);

score 0 · Answer 1 · answered Feb 06 '23 at 22:56

I would recommend using a cron job to schedule the call to the API for one url every minute, instead of a single script / function that is going to take 60s/url. You mentioned that you get the json response of 10 different urls, so that's 10 minutes for the script to be running, that's likely above the max_execution_time set by php.ini.

You could increase that by using set_time_limit, and just add a sleep(60) in your script, which will cause PHP to pause the script at that line for 60 seconds...

Add Delay/Sleep in between url requests in multi cutl php

1 Answers1