How to use perl to run the same php script parallel?

Question

Ik have a php script that grabs content from another website. I want that same script to run at the same time, so parallel (with different input parameters).

There is a function pcntl that can be used to multithread processes within php, however the manual says it is not recommended to use it in a web server environment. So I decided not to do that since my script is running on a server.

I decided to use threading in perl to call the php function. When I call two different php scripts from Perl they are executed simultaneously, but when I use the same php script the first script has to finish before the second script can start.

It also seems to make no difference if I use thread in perl or not because when using the same script the first session has to finish before it will start with the next. The php scripts are also executed simultaneously without using threads, see script below.

Any help would be very very welcome!

File---> test.pl



#!C:/wamp/bin/perl/bin/perl.exe

print "Content-type: text/html\n\n";
print '' . "\n
";
print '' . "\n
";

use Config;
$Config{useithreads} or die('Recompile Perl with threads to run this program.');

use threads;
my $Param3 = 'foo';
my $thr1 = threads->create(\&sub1, 'Param 1', 'Param 2', $Param3);
my @ParamList = (42, 'Hello', 3.14);
my $thr2 = threads->create(\&sub1, @ParamList);
my $thr3 = threads->create(\&sub1, qw(Param1 Param2 Param3));
sub sub1 {
my @InboundParameters = @_;
print("In the thread\n");
print '
' . 
print('Got parameters >', join('<>', @InboundParameters), "<\n");}

test1.php:

<?php echo "PHP generated this, this is script test1"; sleep(5); ?>

test2.php:

<?php echo "PHP generated this, this is script test2"; sleep(5);?></pre></code>

It appears like your solution itself is flawed, you are likely to get better answers posting your actual problem rather than asking about problems with your solution. — Murali VP, Jul 02 '11 at 12:38
My problem is that I want to run the same php script at the same time. (see title, I think it should be done with perl) I thought just stating my problem would seem like I didn't try myself! — Bastiaan, Jul 02 '11 at 13:02
That's not your problem, that's how you think you want to solve your problem. You could solve your problem by finding a solution that fetches the pages in parallel. http://search.cpan.org/perldoc?LWP::Parallel::UserAgent is a parallel UserAgent, but it's all in Perl. Since you weren't opposed to using Perl anyway, you might consider going further down that road. Otherwise, look for a parallel request agent in a PHP library. — DavidO, Jul 03 '11 at 02:10
Thanks! I found a way in php to do the same, which is working great! — Bastiaan, Jul 03 '11 at 13:40
http://www.developertutorials.com/tutorials/php/parallel-web-scraping-in-php-curl-multi-functions-375/ — Bastiaan, Jul 03 '11 at 13:41

score 0 · Answer 1 · edited May 23 '17 at 12:20

There is a very real danger when mixing and matching in threads, where you can trip over all sorts of 'non thread safe' behavior.

However, perl is perfectly capable of dealing with parallel stuff without needing to 'call out' to PHP.

E.g. referring to: Perl daemonize with child daemons

#!/usr/bin/perl

use strict;
use warnings;

use threads;
use Thread::Queue;

my $nthreads = 5;

my $process_q = Thread::Queue->new();
my $failed_q  = Thread::Queue->new();

#this is a subroutine, but that runs 'as a thread'.
#when it starts, it inherits the program state 'as is'. E.g.
#the variable declarations above all apply - but changes to
#values within the program are 'thread local' unless the
#variable is defined as 'shared'.
#Behind the scenes - Thread::Queue are 'shared' arrays.

sub worker {

    #NB - this will sit a loop indefinitely, until you close the queue.
    #using $process_q -> end
    #we do this once we've queued all the things we want to process
    #and the sub completes and exits neatly.
    #however if you _don't_ end it, this will sit waiting forever.
    while ( my $server = $process_q->dequeue() ) {
        chomp($server);
        print threads->self()->tid() . ": pinging $server\n";
        my $result = `/bin/ping -c 1 $server`;
        if ($?) { $failed_q->enqueue($server) }
        print $result;
    }
}

#insert tasks into thread queue.
open( my $input_fh, "<", "server_list" ) or die $!;
$process_q->enqueue(<$input_fh>);
close($input_fh);

#we 'end' process_q  - when we do, no more items may be inserted,
#and 'dequeue' returns 'undefined' when the queue is emptied.
#this means our worker threads (in their 'while' loop) will then exit.
$process_q->end();

#start some threads
for ( 1 .. $nthreads ) {
    threads->create( \&worker );
}

#Wait for threads to all finish processing.
foreach my $thr ( threads->list() ) {
    $thr->join();
}

#collate results. ('synchronise' operation)
while ( my $server = $failed_q->dequeue_nb() ) {
    print "$server failed to ping\n";
}

You can hopefully see that by doing this - you can 'queue up' a list of URLs, then fetch it via LWP within the worker thread. Or if you really must, embed your PHP there.

How to use perl to run the same php script parallel?

1 Answers1