0

I am trying to achieve following:

  1. Parse XML sitemap of website (working)
  2. Download source of every single page
  3. Echo something like "Done" or url of website
  4. Sleep for few seconds to avoid crashing server

I don't know why, but after about 10 minutes I am always getting 500 ERROR.

<?php
error_reporting( E_ALL );
ini_set('display_errors', 1);


ob_start(); 


$urls = array();  

$DomDocument = new DOMDocument();
$DomDocument->preserveWhiteSpace = false;
$DomDocument->load('http://elody.cz/sitemap.xml');
$DomNodeList = $DomDocument->getElementsByTagName('loc');


//parsovani xml, vkladani linku do pole
foreach($DomNodeList as $url) {
    $urls[] = $url->nodeValue;
}

echo 'Loading XML done!<br />';                  


foreach ($urls as $url) {     
      $data = file_get_contents($url);      
      echo $url."<br />";
      ob_flush();
      flush();
      unset($data);
      unset($var);      
      sleep(1);

} 

ob_end_flush();     
?>

Do you have idea how I could find out where is the problem? :)

Thank you Filip

Filip
  • 95
  • 10
  • Check your server's error log file to see what caused this error. – Rajdeep Paul Jun 18 '16 at 16:51
  • Check your PHP error log (`phpinfo()`s `error_log`) and your web server's error log. The location and availability of those depend on your host's configuration, so you may have to contact them. I'm wondering if it could simply be the `max_execution_time` which is set to 10 minutes. – Julie Pelletier Jun 18 '16 at 16:53
  • Thank you very much. How can I force script to write in error log? I enabled error log on server, but it is still empty. – Filip Jun 18 '16 at 17:28

0 Answers0