Zend Http Client : Invalid URI Supplied Error for Urls that redirect

Question

I want to get html of a URL. Basically I am doing this :

$client = new Zend_Http_Client($url);           
$client->setConfig(array('strictredirects' => true, 'timeout'=> 100, 'storeresponse' => true));
$response = $client->request();
$html = $response->getBody();

For some urls that redirect, i am getting the following error

Invalid URI supplied

For example if you consider the following URL:

http://www.hiexpress.com/redirect?path=hd&brandCode=ex&hotelCode=housl&regionCode=1&localeCode=en&cm_mmc=mdpr--GoogleMaps--ex-_-housl

It redirects to another URL. When i try to get the lastResponse, it gives me nothing. HOw do i get the html for this URL?? I tried the config option strictredirects but still its giving the same error. HOw do i solve this??

can you try using `strictredirects = false` in config – Nandakumar V Sep 11 '13 at 11:55 — Nandakumar V, Sep 11 '13 at 11:55

score 0 · Answer 1 · answered Sep 11 '13 at 09:21

Try this put this into your controller

  // @$uri your url
  $client = new Zend_Http_Client($uri, array(
    'maxredirects' => 2,
    'timeout'      => 10,
  ));

  // Try to mimic the requesting user's UA
  $client->setHeaders(array(
    'User-Agent' => $_SERVER['HTTP_USER_AGENT'],
    'X-Powered-By' => 'Zend Framework'
  ));

  $response = $client->request();

  $body = $response->getBody();
  $body = trim($body);
// Get DOM
if( class_exists('DOMDocument') ) {
  $dom = new Zend_Dom_Query($body);
} else {
  $dom = null; // Maybe add b/c later
}

$title = null;
if( $dom ) {
  $titleList = $dom->query('title');
  if( count($titleList) > 0 ) {
    $title = trim($titleList->current()->textContent);
    $title = substr($title, 0, 255);
  }
}
$this->view->title = $title;//Title of the page
$description = null;
if( $dom ) {
  $descriptionList = $dom->queryXpath("//meta[@name='description']");
  // Why are they using caps? -_-
  if( count($descriptionList) == 0 ) {
    $descriptionList = $dom->queryXpath("//meta[@name='Description']");
  }
  if( count($descriptionList) > 0 ) {
    $description = trim($descriptionList->current()->getAttribute('content'));
    $description = substr($description, 0, 255);
  }
}
$this->view->description = $description;// Description of the page

score 0 · Accepted Answer · answered Sep 11 '13 at 13:32

For some reason Zend Http Client did not work, had to use CURL to get the work done. To get HTML :

$headers = array( "User-Agent:MyAgent/1.0\r\n");
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION,  TRUE);
curl_setopt($curl, CURLOPT_MAXREDIRS, 50);
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($curl, CURLOPT_TIMEOUT, 20);
$html = curl_exec($curl);
curl_close($curl);
echo $html;

To get the Effective Url / redirected url :

$headers = array( "User-Agent:MyAgent/1.0\r\n");
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION,  TRUE);
curl_setopt($curl, CURLOPT_MAXREDIRS, 50);
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($curl, CURLOPT_TIMEOUT, 20);
$content = curl_exec($curl);
$redirectedUrl = curl_getinfo($curl, CURLINFO_EFFECTIVE_URL);
curl_close($curl);
echo $redirectedUrl;

Zend Http Client : Invalid URI Supplied Error for Urls that redirect

2 Answers2