0

How could I reduce the number of loops in my code. Current code is a dummy example but as I am scraping content from a URL and it contains around 6K-7K li traversal to scrap data so speed is a concern in my app.

1) I am scraping data from a URL using Simple PHP DOM parser, so my first concern is whether it is a relevant way to traverse DOM for scraping data(around 6K-7K li elements) or file_get_contents() will be an appropriate.

2) How do i reduce the number of loops in the below mentioned code, I tried but fails-

My whole code

<?php
$arrunsort     = array('ab','ad','ar','cd','cb','sc','si','wa','za');
$prevLabel     = array();
$currLabelarr  = array();
$newarr        = array();
foreach( $arrunsort as $sortelem )
{
    $currLabel = substr($sortelem, 0, 1);
    if( $currLabel !== $prevLabel )
    {
        //echo '<h2>',strtoupper($currLabel),'</h2>','<br>';
        $currLabelarr[] = $currLabel;
        $prevLabel = $currLabel;
    }
}

foreach($currLabelarr as $arrkeymatch) 
{
    $newarr[$arrkeymatch] = array();
    $i = 0;
    foreach($arrunsort as $value) 
    {
        if( $arrkeymatch == (substr($value, 0, 1)) )
        {
            //echo '<li>',$value,'</li>';
            $newarr[$arrkeymatch][$i] = $value;
            $i++; 
        }   
    }
}


foreach($newarr as $key=> $val)
{
    echo "<h3>",$key,'</h3>';
    echo '<ul>';
    foreach($val as $myval)
    {
        echo '<li>',$myval, '</li>';
    }
    echo '</ul>';
}

Now my tries to reduce the number of loops-

  • Comment the last foreach loop
  • Uncomment echo '<h2>',strtoupper($currLabel),'</h2>','<br>'; & echo '<li>',$value,'</li>';

But the li items are coming below, let me know how could I solve this to minimize loops.

EDIT

In the meantime I checked Caching but as a noob I am not very sure if it a genuine method of caching in my case.

Community
  • 1
  • 1
Trialcoder
  • 5,816
  • 9
  • 44
  • 66
  • Why aren't you just using `DomDocument` to load all the items you want? – Rob W Jun 27 '13 at 15:28
  • 1
    3 loops is nowhere near "excessive" when you're dealing with parsing... – Marc B Jun 27 '13 at 15:29
  • @MarcB so the above code is ok to go with it ...or it can to optimized to reduce page load time? – Trialcoder Jun 27 '13 at 15:31
  • @MarkB my another thought is to save the data in a text file as web page content is not going to change too frequently (may be an year) but the data is too much on the page I am scrapping so I am looking for every option as page load time is litle bit long – Trialcoder Jun 27 '13 at 15:33
  • well, don't load the remote content you're parsing on EVERY request, if that content doesn't change "frequently". e.g. if it only changes once an hour, doing a full fetch/parse run on it is rather pointless. fetch it once and cache it locally. – Marc B Jun 27 '13 at 15:33
  • @MarcB yeah I too have heard about caching results but never implemented it yet..can u plz pass any relevant link for this topic..will going to greatly help me a lot – Trialcoder Jun 27 '13 at 15:35
  • to even go further with Marc B's concept of fetching once and caching locally, you can also do your looping once and then store the result and load that rather than reparsing the data every time. You can also then attach an expire date to it at which point you can do it all again. – Rooster Jun 27 '13 at 16:08
  • @Rooster can u plz pass any relevant link I never tried this..so i m really confused whr n how to start..plz help me with this – Trialcoder Jun 27 '13 at 16:12
  • @Trialcode 1. Scrape a website(cUrl is better option than file_get_contents by the way). 2. Run your processing stuff(your foreachs). 3. Store the processed data. 4. Set up steps 1-3 as a cron job and pick an interval upon which you want to update the stored data(once a day for example). 5. Wherever your displaying the product of your foreachs now, pull in the stored value. ----------Just a quick example – Rooster Jun 27 '13 at 16:28

0 Answers0