-1

I have a site that that scrapes off of it's sister sites but for reporting reason I'd like to be able to work out how long the task took to run. How would I approach with this with PHP, Is it even possible?

In an ideal world if the task couldn't connect to actually run after 5 seconds I'd like to kill the function from running and report the failure.

Thank you all!

David
  • 10,435
  • 3
  • 17
  • 11

1 Answers1

2

If you use cURL for scraping, you can use the timeout function like this

// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options including timeout
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);  // capture the result in a string
curl_setopt($ch, CURLOPT_TIMEOUT, 5);  // The number of seconds to wait while trying to connect.
// grab the info
if (!$result = curl_exec($ch))
{
    trigger_error(curl_error($ch));
}

// close cURL resource, and free up system resources
curl_close($ch);

// process the $result
Jannie Theunissen
  • 28,256
  • 21
  • 100
  • 127
  • Could I run this directly in a PHP File? – David May 21 '12 at 17:18
  • Sure, you just need the php_curl extension enabled and set the $url="http://site_i_want_to_grab.com/some_page/some_id" – Jannie Theunissen May 21 '12 at 17:23
  • How would I go about adding a conditional to this? For example in my case I'd like to: if (you-can-find-the-url) {do this}else{do that and record the error} – David May 29 '12 at 17:50