I am trying to send multiple HTTP get requests using Perl. I need to send those request using sock proxy.
If I use following code, I am able to send a request with sock proxy
#!/usr/bin/perl
use strict;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
agent => q{Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; YPC 3.2.0; .NET CLR 1.1.4322)},
);
$ua->proxy([qw/ http https /] => 'socks://localhost:9050'); # Tor proxy
#$ua->cookie_jar({});
$a = 10;
while ( $a < 20 ) {
my $rsp = $ua->get('http://example.com/type?parameter=1¶meter=2');
print $rsp->content;
$a = $a + 1;
}
It works successfully but I need to use AnyEvent to send multiple GET requests in parallel
#!/usr/bin/perl
use strict;
use AnyEvent;
use AnyEvent::HTTP;
use Time::HiRes qw(time);
use LWP::Protocol::socks;
use AnyEvent::Socket;
my $cv = AnyEvent->condvar( cb => sub {
warn "done";
});
my $urls = [
"http://url-1-withallparameters",
"http://url-2-withallparameters",
"http://url-3-withallparameters",
];
my $start = time;
my $result;
$cv->begin(sub { shift->send($result) });
for my $url ( @$urls ) {
$cv->begin;
my $now = time;
my $request;
$request = http_request(
GET => $url,
timeout => 2, # seconds
sub {
my ($body, $hdr) = @_;
if ($hdr->{Status} =~ /^2/) {
push (@$result,
join("\t",
($url,
" has length ",
$hdr->{'content-length'},
" and loaded in ",
time - $now,
"ms"))
);
}
else {
push (@$result,
join("",
"Error for ",
$url,
": (",
$hdr->{Status},
") ",
$hdr->{Reason})
);
}
undef $request;
$cv->end;
}
);
}
$cv->end;
warn "End of loop\n";
my $foo = $cv->recv;
print join("\n", @$foo), "\n" if defined $foo;
print "Total elapsed time: ", time-$start, "ms\n";
This is working fine but I am not able to send these requests using sock proxy.
Even in the terminal, if I export proxy commands like curl and wget they work fine with sock proxy, but when I use a Perl command to execute this script as a sock proxy it does not work.
I could integrate it with LWP::UserAgent but it is not working with AnyEvent.
I have used
proxy => [ $host, $port ],
below
GET => $url,
but it works for HTTP and HTTPS proxy only, not for sock proxy. This url is working for HTTP/HTTPS proxy but not for sock proxy.
