0

I need to check some proxies for web site, so I use Curl multi interface to do it simultaneously. Using existing examples and hints I wrote this code:

public function process(){
        // *** Multi handle initialized and easy handles added
        $active = null;

        do {
            $mrc = curl_multi_exec($this->cmh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);

        while ($active && ($mrc == CURLM_OK)){
            if (curl_multi_select($this->cmh, 0.1) == -1) {
                usleep(100);
            }

            do {
                // stucked here
                $mrc = curl_multi_exec($this->cmh, $active);

                while (false != ($mhinfo = curl_multi_info_read($this->cmh))){
                    if ($mhinfo['msg'] == CURLMSG_DONE) {

                        $handle = $mhinfo['handle'];
                        $result = $mhinfo['result'];

                        // *** Result analysis

                        curl_multi_remove_handle($this->cmh, $handle);
                        curl_close($handle);
                    }
                }

            } while ($mrc === CURLM_CALL_MULTI_PERFORM);


        }


        if ($mrc != CURLM_OK)
            throw new CurlError ($this, $this->cmh, $mrc, curl_multi_strerror ($mrc));

        // *** MULTI HANDLE WILL BE CLOSE LATER
    }

Easy handles initalized here:

    public function getCurlHandle() {
        $ch = curl_init ("https://www.avito.ru/");

        $headers = array
        (
            'Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1',
            'Accept-Language: ru-RU,ru;q=0.9,en;q=0.8',
            'Accept-Encoding: gzip, deflate, sdch'
        );

        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
        curl_setopt($ch, CURLOPT_TIMEOUT, 20);
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($ch, CURLOPT_FRESH_CONNECT, TRUE);
        curl_setopt($ch, CURLOPT_ENCODING , "gzip");
        curl_setopt($ch, CURLOPT_PROXY, (string)$this->proxy);
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.65 Safari/537.36");

        return $ch;
    }

The problem is that sometimes it hangs on curl_multi_exec and falls with exceeded timeout. There are php version and curl version information:

PHP 5.5.12-2ubuntu4.3 (cli) (built: Mar 16 2015 20:50:26) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies

php > print_r (curl_version());
Array
(
    [version_number] => 468225
    [age] => 3
    [features] => 50877
    [ssl_version_number] => 0
    [version] => 7.37.1
    [host] => x86_64-pc-linux-gnu
    [ssl_version] => OpenSSL/1.0.1f
    [libz_version] => 1.2.8
    [protocols] => Array
        (
            [0] => dict
            [1] => file
            [2] => ftp
            [3] => ftps
            [4] => gopher
            [5] => http
            [6] => https
            [7] => imap
            [8] => imaps
            [9] => ldap
            [10] => ldaps
            [11] => pop3
            [12] => pop3s
            [13] => rtmp
            [14] => rtsp
            [15] => smtp
            [16] => smtps
            [17] => telnet
            [18] => tftp
        )

)
Roman Khotsyn
  • 11
  • 3
  • Maybe increasing the timeout will help? You may want to look at RollingCurl implementation. – frz3993 Apr 19 '15 at 19:50
  • I have a big timeout and it won't help, I used their implementation, not helped. I've tried C libcurl and faced the same problems. – Roman Khotsyn Apr 20 '15 at 23:02
  • Solution to problem can be found [here](http://stackoverflow.com/questions/29760065/curl-with-multi-interface-for-many-connections-with-proxy), but only for C (there can be some problems with multithreaded PHP and HTTPS) – Roman Khotsyn Apr 27 '15 at 22:58

0 Answers0