7

I am using function file_get_contents to get contents from web page. Some website working well but most of them give me this error

failed to open stream: HTTP request failed! HTTP/1.1 503 Service Temporarily Unavailable

Here is my simple code

echo file_get_contents("url");

When i run this url in browser it works fine.what can be the issue?

  • 1
    You might not have a required request header in order to receive a valid response. – Ryan Jun 12 '13 at 09:16
  • what does that mean?what can i do?is it not possible to do for those sites? –  Jun 12 '13 at 09:17

2 Answers2

7

503 means the functions are working and you're getting a response from the remote server denying you. If you ever tried to cURL google results the same thing happens, because they can detect the user-agent used by file_get_contents and cURL and as a result block those user agents. It's also possible that the server you're accessing from also has it's IP address blackballed for such practices.

Mainly three common reasons why the commands wouldn't work just like the browser in a remote situation.

1) The default USER-AGENT has been blocked.
2) Your server's IP block has been blocked.
3) Remote host has a proxy detection.
Hiren Soni
  • 574
  • 3
  • 11
  • I have update my answer, by the way, any chance the url you're trying to access is a https url? – Hiren Soni Jun 12 '13 at 10:09
  • I do not know much about,let me give you and example...try this site you will see...http://dubai.dubizzle.com/ –  Jun 12 '13 at 10:17
  • I have checked that url with CURL as wel, they have keep some redirection script to prevent proxy or some external access.. So you can't access this URL with file_get_contents or even with cURL as well. – Hiren Soni Jun 12 '13 at 10:36
  • here is my curl code, $ch = curl_init(); $timeout = 5; curl_setopt($ch,CURLOPT_URL,$url); curl_setopt($ch,CURLOPT_RETURNTRANSFER,1); curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout); $data = curl_exec($ch); if(!curl_errno($ch)){ echo $data; }else{ echo 'Curl error: ' . curl_error($ch); } curl_close($ch); – Hiren Soni Jun 12 '13 at 10:37
  • what can i do so that i can use sites like this?what is the solution –  Jun 12 '13 at 10:39
  • Not sure, But need to spend time to know what logic they have used for that, than try to creak it... Else you can contact them if they can help you....:( – Hiren Soni Jun 12 '13 at 10:59
  • Please let me know if you found any solution for same – Hiren Soni Jun 13 '13 at 06:40
2

With this script most Websites think you are a Browser:

    <?php
        getWebsite('http://mywebsite.com');

        function getWebsite($url='http://mywebsite.com'){
        
        $ch = curl_init();
        $user_agent='Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/'.rand(8,100).'.0';
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($ch, CURLOPT_AUTOREFERER, false);
        curl_setopt($ch, CURLOPT_VERBOSE, 1);
        curl_setopt($ch, CURLOPT_HEADER, 0);

        curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_SSLVERSION,CURL_SSLVERSION_DEFAULT);
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        $webcontent= curl_exec ($ch);
        $error = curl_error($ch); 
        curl_close ($ch);
        return  $webcontent;

        }
    ?>
dazzafact
  • 2,570
  • 3
  • 30
  • 49
  • 1
    Not really fool proof. Google will refuse the request if you make too many requests, even with a browser user agent string. I have tried to randomly select from an array of different strings per each cURL, but it's too late. I only hope that it resets after a certain amount of time. – TARKUS Aug 11 '17 at 23:20
  • Google can't be fetched. its nearly Impossible. Use the regular API – dazzafact Aug 22 '22 at 15:09