PHP - foreach loop run all array instantly?

Question

I am trying to loop some functions in loop with "foreach" and array. but it takes very long time to get information. is it possible to run all array loop elements at the same time?

ini_set("max_execution_time", 0);
function bhttpGet($burl)
{
    global $data;
    global $makaronas;
    global $makaronas2;
    $data = array("clientType" => "Android", "os" => "21", "screenType" => "xxhdpi", "version" => "3");                                                                    
    $data_string = json_encode($data);  
    $bch = curl_init();  

    curl_setopt($bch,CURLOPT_URL,$burl);
    curl_setopt($bch,CURLOPT_RETURNTRANSFER,true);
    curl_setopt($bch,CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($bch,CURLOPT_HEADER, false);
    curl_setopt($bch, CURLOPT_POSTFIELDS, $data_string);
    curl_setopt($bch, CURLOPT_HEADER, 1);
    curl_setopt($bch, CURLOPT_RETURNTRANSFER, 1);

    curl_setopt($bch, CURLOPT_HTTPHEADER, array(
    'Accept-Language: en',
    'Content-Type: application/json; charset=UTF-8',
    'User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.0.2; HTC One Build/LRX22G)',
    'Host: mobileapi.turkishairlines.com',
    'Connection: Keep-Alive',
    'Accept-Encoding: gzip',
    'Cookie: JSESSIONID=5096E068E5B2E11E4EC96A6B725C5BB1; AWSELB=A3F31BC902ED4282BBCBA2D31E20E7210081D104D0AC59E7291F2B1692E6E27BB90F1D78BBC7CFBAD0D14DA4FFA5A59F223573F1C428362BCD544E72A28094F07742E08B3292F74F8BE17859D515D4392E4D9C052C'
    ));

    $result = curl_exec($bch);
    curl_close($bch);
    preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $result, $matches);
    $cookies = array();
    foreach($matches[1] as $item) {
        parse_str($item, $cookie);
        $cookies = array_merge($cookies, $cookie);
    }
    $makaronas = $cookies['JSESSIONID'];
    $makaronas2 = $cookies['AWSELB'];
}

function httpGet($url){
    global $makaronas3;
    global $makaronas4;
    $ch = curl_init();  

    curl_setopt($ch,CURLOPT_URL,$url);
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
    curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false); 
    curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    'Accept-Language: en',
    'User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.0.2; HTC One Build/LRX22G)',
    'Host: mobileapi.turkishairlines.com',
    'Connection: Keep-Alive',
    'Accept-Encoding: gzip',
    "Cookie: JSESSIONID={$makaronas3}; AWSELB={$makaronas4}"

    ));
    $output2=curl_exec($ch);
    curl_close($ch);
 }

 function chttpGet($kurl){
    global $makaronas3;
    global $makaronas4;
    global $output3;
    $kch = curl_init();  

    curl_setopt($kch,CURLOPT_URL,$kurl);
    curl_setopt($kch,CURLOPT_RETURNTRANSFER,true);
    curl_setopt($kch,CURLOPT_SSL_VERIFYPEER, false); 
    curl_setopt($kch, CURLOPT_HTTPHEADER, array(
    'Accept-Language: en',
    'User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.0.2; HTC One Build/LRX22G)',
    'Host: mobileapi.turkishairlines.com',
    'Connection: Keep-Alive',
    'Accept-Encoding: gzip',
    "Cookie: JSESSIONID={$makaronas3}; AWSELB={$makaronas4}"
    ));

    $output3=curl_exec($kch);
    curl_close($kch);
}

function between($src,$start,$end){
    $txt=explode($start,$src);
    $txt2=explode($end,$txt[1]);
    return trim($txt2[0]);
  }
$departuredatearray= array("01.08.2016","02.08.2016","03.08.2016","04.08.2016","05.08.2016","06.08.2016","07.08.2016","08.08.2016","09.08.2016","10.08.2016","11.08.2016","12.08.2016","13.08.2016","14.08.2016","15.08.2016","16.08.2016","17.08.2016","18.08.2016","19.08.2016","20.08.2016","21.08.2016","22.08.2016","23.08.2016","24.08.2016","25.08.2016","26.08.2016","27.08.2016","28.08.2016","29.08.2016","30.08.2016","31.08.2016");

    foreach ($departuredatearray as $lopas) {
    $initialize="https://mobileapi.turkishairlines.com/init.json";
    $initialize2 = "https://mobileapi.turkishairlines.com/booking/internationalFlights.json?cabinType=economy&adultCount=1&childCount=0&infantCount=0&isOneWay=true&departureAirportCode=VNO&arrivalAirportCode=IST&flightDate={$lopas}&isReturn=false";
    $initialize3 = "https://mobileapi.turkishairlines.com/booking/internationalFare.json?departureFlightIndex=0&returnFlightIndex=-1&isDeparturePromotion=false&isReturnPromotion=false";
$mano=bhttpget($initialize);
echo "$mano";
$makaronas3=$makaronas;
$makaronas4=$makaronas2;
$mano2=httpget($initialize2);

echo "$mano2";
$mano4 = chttpget($initialize3);

echo "$mano4";
echo "<BR>";
echo "<table border='1' align='center' style='text-align:center;'>";
    echo "<tr>";
        echo "<td>flight Number</td>";
        echo "<td>departure Airport Code</td>";
        echo "<td>arrival Airport Code</td>";
        echo "<td>departure Date</td>";
        echo "<td>departure Time</td>";
        echo "<td>arrival Date</td>";
        echo "<td>arrival Time</td>";
        echo "<td>duration</td>";
        echo "<td>total Price</td>";
        echo "<td>Curency</td>";
    echo "</tr>";
    echo "<tr>";
        echo "<td>";
        echo between($output3, '"flightNumber":"', '","departureAirportCode');
        echo "</td>";
        echo "<td>";
        echo between($output3, 'departureAirportCode":"', '","departureAirportName');
        echo "</td>";
        echo "<td>";
        echo between($output3, 'arrivalAirportCode":"', '","arrivalAirportName');
            echo "</td>";          
            echo "<td>";
            echo between($output3, 'departureDate":"', '","formattedDepartureDate');
            echo "</td>";   
            echo "<td>";
            echo between($output3, 'departureTime":"', '","arrivalDate');
            echo "</td>";   
            echo "<td>";
            echo between($output3, 'arrivalDate":"', '","formattedArrivalDate');
            echo "</td>"; 
            echo "<td>";
            echo between($output3, 'arrivalTime":"', '","duration');
            echo "</td>";  
            echo "<td>";
            echo between($output3, 'duration":"', '","stopTimeAfterArrival');
            echo "</td>";
            echo "<td>";
            echo between($output3, 'totalPrice":"', '","currency":"');
            echo "</td>";
            echo "<td>";
            echo between($output3, 'currency":"', '","totalTax');
            echo "</td>";

echo "</table>";
};

Welcome to SO, please be a bit more specific when asking question: what have you tried, what do you expect, etc. See [how to ask](http://stackoverflow.com/help/how-to-ask) — Nehal, Apr 22 '16 at 07:13

edhurtig · Answer 1 · 2016-04-22T08:49:05.340

Your issue here is the time each iteration of the foreach is spending blocked on the HTTP request. This is an issue with the API latency and it typically dealt with by processing the portions of HTTP requests concurrently.

For example, if each HTTP request takes 3 seconds to complete, and you can process 10 HTTP requests concurrently, then you have made your script roughly 10x faster

 100 Requests w/o concurrency (3 seconds each, 1 at a time) = 300 seconds
 100 Requests w/ concurrency (3 seconds each, 10 at a time) =  30 seconds

There is a pretty great way of parallelizing HTTP requests the curl_multi* suite of functions. It's builtin to the cURL library and it is pretty easy to use.

Tutorial: http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/

PHP Manual: http://php.net/manual/en/function.curl-multi-exec.php

It is important to note that there are real limits to the number of HTTP requests that you can and should perform concurrently. curl_multi* defaults to 10 concurrent HTTP requests which is a pretty good limit. Increasing this limit must be done with care, respect, and responsibility it can do serious harm to other services.

To increase this limit (for example if you are making requests against your own servers) you can use curl_multi_setopt with the CURLMOPT_MAXCONNECTS option

$new_limit = 15;
curl_multi_setopt ( $mh, CURLMOPT_MAXCONNECTS, $new_limit );

Also note that you can not make a single request go any faster than it already does unless there is something wrong with your server/internet connection. Concurrency will not help you make individual requests go any faster (just thought I'd make that clear).

You can also use threads if you like, but I think the builtin cURL approach is going to be fastest, easiest, and the most maintainable given your existing code

this is a good suggestion, although hope the OP understands that you cannot run all cases of a foreach simultaneously and "there is no such thing as a free meal" — Elzo Valugi, Apr 22 '16 at 07:39
the script in itself is not slow, foreach is a very fast function, it can loop through 10 million entries in fraction of seconds. in your case the problem is the API latency, the response is slow(ish) and each request has to wait for the previous one to finish. A lot of slowish request sum up to a slow script. What @edhurtig is suggesting (and myself) is to parallelize these requests. — Elzo Valugi, Apr 22 '16 at 07:48
@user3237444 Have a look at the tutorial I linked to, It has a great example showing you how to implement concurrent HTTP requests — edhurtig, Apr 22 '16 at 08:01
How about using [mpyw/co](https://github.com/mpyw/co)? With this lib, you don't have to write annoying `curl_multi_exec`+`curl_multi_select` loop anymore. — mpyw, Aug 05 '16 at 21:33

score 0 · Answer 2 · answered Apr 22 '16 at 07:46

0

In the same line of thought that @edhurtig started, you may want to take a look at threading in PHP. I think you stated somehow the problem incorrectly. You need to do several independent request and they don't need to be serial, which happens within a foreach loop. Making parallel API calls is possible by opening a new thread for each request, thus executing them in parallel.

answered Apr 22 '16 at 07:46

Elzo Valugi

27,240
15
95
114

Yes, I think you understand me correctly. But i am new in programming, maybe you can post any example? – Ernest Devenson Apr 22 '16 at 07:54
1

@user3237444 I am not trying to underestimate you or you will to learn programming in any way, and I like that you are trying complicated stuff. But these concepts are sortof complex and I would recommend you to start with simpler stuff and after you understand those go to more complex things. You have examples and the whole documentation of threading in the link I gave you and PHP documentation is very very good. Still, I would recommend you to understand the basics of the language like `control structures` way before starting on something like multi threading. – Elzo Valugi Apr 22 '16 at 08:11

PHP - foreach loop run all array instantly?

2 Answers2