1

Here's the the simple demonstrative snippet:

 <?php 
class WorkerThreads extends Thread
{
    private $workerId;
    private $url;

    public function __construct($string)
    {
        $this->url = $string;
    }

    public function run()
    {

        $curl = curl_init();

        curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($curl, CURLOPT_URL, $this->url);

        if ($this->post) {
            curl_setopt($curl, CURLOPT_POSTFIELDS, $this->post);
        }

        $response = curl_exec($curl);
        //echo $response;
        curl_close($curl);
    }
}

$data = file('urls.txt'); 
$data_size = count($data);
// Worker pool
$workers = [];
$t_count = 100;
$flag = 1;
$k = 0;

while($flag === 1)
{   

    $c_w = count($workers);

    if($c_w < $t_count)
    {
        for($i = $c_w; $i<$t_count - $c_w;$i++)
        {
            if($k >= $data_size)
            {
                $flag = 0;
                break;
            }   

            $url = explode("|",$data[$k]);
            $workers[$i] = new WorkerThreads($url[0]);
            echo $k." ".$url[0]."\n";
            $workers[$i]->start();
            $k++;   
        }

    }

    $c_w = count($workers);
    for($i=0;$i<$c_w;$i++)
    {
        if($workers[$i]->join())
        {
            //echo "joining $i\n";
            unset($workers[$i]);

        }
    }


}

?>

The problem that I can't understand is that on php 7x86 I always getting this error:

Fatal error: Uncaught RuntimeException: cannot start WorkerThreads, out of reso
rces in C:\Users\alex\Desktop\test.php:56
Stack trace:
#0 C:\Users\alex\Desktop\test.php(56): Thread->start()
#1 {main}
  thrown in C:\Users\alex\Desktop\test.php on line 56

For some reason this not appear in php 7 x64 version (with same ini settings). Why is that happening? Thanks!

upd: Also there is no info about this case on google. Asked it in few communities, but no one seems to know about how to fix this or even why exception raises.

Александр Пушкин
  • 329
  • 4
  • 16
  • 32-bit processes have much more limited address space than 64-bit processes do. You should probably just limit the number of threads you create to some reasonable number, say 100. – David Schwartz Jan 31 '16 at 12:06
  • It doesn't even works with 50 threads. 100 is desirable value for me at the moment. – Александр Пушкин Jan 31 '16 at 12:08
  • What resource limit have you set in `php.ini`? You probably need at least 1MB per thread. – David Schwartz Jan 31 '16 at 12:09
  • I've got 3000mb set in php.ini: memory_limit=3000M – Александр Пушкин Jan 31 '16 at 12:10
  • You need to chunck your pool batches according to the nb of available cpu threads. This is especialy tru with pthreads v3. After each join batch complited, unset the pool. – cpugourou Feb 09 '16 at 11:56
  • @cpugourou, but why this works okay on x64? – Александр Пушкин Feb 09 '16 at 12:30
  • 1
    Ressouces allocation is different. Also pthreads uses massive ram ressources , especialy when you are dealing with big arrays inside them. Avoid as much a possible to pass arrays to the workers and use variables mostly. Execute queries, curl etc inside them. If your workers generates arrays, chunck you pool and never go over your cpu threads, unset and force garbage collection after each batch . This is good for 32 or 64. More you apply those rules, faster your pthreads workers will be. 10 batches of 12 threads (6 cores) are much faster than throuing 120 in one pool. – cpugourou Feb 10 '16 at 11:56
  • @cpugourou, I'am using threads actually, and it consumes much less resources that pc has. The x64 php 7 version runs near 150 curl threads fine, but x86 can't run even 50. That's a sad limitation. – Александр Пушкин Feb 11 '16 at 18:37
  • it s nothing to do with 32 or 64. first you d better use multicurl instead of pthreads and as you are using php7 go to this thread where we have discussed it already. http://stackoverflow.com/questions/34857779/why-not-all-threads-are-completed/34888049#34888049 – cpugourou Feb 12 '16 at 17:27
  • @cpugourou, I've already created several threads at SO and created an issue. For my needs I must use pthreads for paralleling several tasks. However, I still got no answer from the developer about that behaviour of pthreads on php 7 x86 version. – Александр Пушкин Feb 12 '16 at 18:05

0 Answers0