Stack Overflow
Stack Overflow

Questions tagged [lwp-useragent]

The LWP::UserAgent is a class implementing a web user agent in the libwww-perl library.

The LWP::UserAgent is a class implementing a web user agent in the libwww-perl library by Michael Schilli. This class provides the option to setup various configuration options when making HTTP requests, allowing control over headers, cookies, credentials, protocols, proxies and many others.

For example, using a combination of these settings allow the request sender to pretend he is a certain user with certain characteristics - for example, a Windows user that uses Mozilla Firefox 19.0, has certain cookies in his browser and has obtained HTTP auth credentials.

225 questions
0
votes
3 answers

LWP::UserAgent set ip of requested url so LWP doesn't have to do dns lookup

I'm using LWP::UserAgent to request a lot of page content. I already know the ip of the urls I am requesting so I'd like to be able to specify the ip address of where the url I am requesting is hosted, so that LWP does not have to spend time doing a…
srchulo
  • 5,143
  • 4
  • 43
  • 72
0
votes
1 answer

Login to Way2sms through LWP failing

I am trying to login to Way2sms through LWP. The source code of the page says that the login form (which, in this case, appears to be the root node of everything within the body) has an action attribute of '', which I read in some other…
SexyBeast
  • 7,913
  • 28
  • 108
  • 196
0
votes
3 answers

Why don't I get a defined value from LWP::UserAgent->new()?

I get this error while using cookie_jar method: Can't call method cookie_jar on an undefined value Here is my code: my $cookie_jar= new HTTP::Cookies; my $ua=new LWP::UserAgent; my %cookies= fetch CGI::Cookie; my…
Sam
0
votes
1 answer

how to download `decoded_content`

***UPDATED CODE with resume functionality** my $ua = LWP::UserAgent->new; $ua->credentials('$ip:80', 'Realm', 'username', 'password'); my $response = $ua->mirror($url,$newfile); if ($response->is_success) { print "Download…
gio_Beginer
  • 71
  • 8
0
votes
1 answer

LWP::UserAgent `:content_cb`: does additional code in the callback slow down the downlaod?

Does additional code in the :content_cb-callback slow down the download? Supposed the additional code would take 1_000/1_000_000 seconds to run and the callback gets called 1_000 times, would that slow down the download for 1_000/1_000_000 * 1_000…
sid_com
  • 24,137
  • 26
  • 96
  • 187
0
votes
1 answer

POST array to LWP: only first entry getting posted

Here's section of code and i am trying to POST whole array using LWP but server is receiving only first row of array(0 index) while others are not getting sent to server, please guide what i am doing wrong $data_post[0] = "text1"; $data_post[1] =…
neo
  • 89
  • 4
  • 11
0
votes
1 answer

LWP::UserAgent get(): is there a default size for the `read_size_hint` option

When using the get method from LWP::UserAgent with :content_file and :content_cb set but without setting :read_size_hint: will there be a constant default value or could the chunk-size change?
sid_com
  • 24,137
  • 26
  • 96
  • 187
-1
votes
1 answer

Does perl-5.16.0-LWP-Protocol-Connect-6.09.tgz supports TLS 1.2?

I am kind of new to perl world but my script fails when loaded via SFTP with below error , IN SUBROUTINE: CSRF TOKEN DECODED CONTENT: error while setting up ssl connection (SSL connect attempt failed with unknown error error:14077410:SSL…
Don Jackson Prabhu
  • 1
-1
votes
1 answer

How can I use Perl's LWP::UserAgent to fetch the same URL with different query strings?

I have a running LWP::UserAgent that should be applied on following URL: http://dms-schule.bildung.hessen.de/suchen/suche_schul_db.html?show_school=5503 This runs with many many similar targets see the following…
zero
  • 1,003
  • 3
  • 20
  • 42
-1
votes
1 answer

Can't connect to api-ssl.bitly.com:443

Since two days i can't generate bitly url. Here is the error i am having: $VAR1 = bless( { '_content' => 'Can\'t connect to api-ssl.bitly.com:443 LWP::Protocol::https::Socket: SSL connect attempt failed with unknown error…
mamesaye
  • 2,033
  • 2
  • 32
  • 49
-1
votes
2 answers

Perl LibXML parse scaler from LWP HTTPS data

I am trying to get text from a website that is HTTPS. I have made this work with LWP, but I need to parse the information because it is XML. I think I have found out how to do what I want with XML::LibXML but I cannot access the data from…
CircuitB0T
  • 465
  • 2
  • 6
  • 19
-2
votes
1 answer

Perl Web Navigation through URL

i'm trying to navigate through this site http://irl.worldfootball.net/ to get to player's pages. I want to be able to just take the player name variables i have and insert them in the url to get to each player's respective page, but i can't quite…
conorw89
  • 45
  • 1
  • 8
-3
votes
1 answer

Why does LWP::Simple::get slow down a subsequent regular expression?

While benchmarking using a perl regex on a string If I get the string using a shell call it works great. But If I get using LWP::Simple the subsequent regex get slowed down With lwp the regex takes 13s with wget it works is less than 4s Why ?…
Ram
  • 1,155
  • 13
  • 34
-3
votes
1 answer

Loop in Perl - with LWP::UserAgent

Here the code that has needs to be build in loop - to make USER-Agent to iterate over a bunch of targets. for my $i (1..10000) { my $request = HTTP::Request->new(GET => sprintf("http://www.example.com,%d", $i)); Is this the correct way to…
zero
  • 1,003
  • 3
  • 20
  • 42
-4
votes
1 answer

Downloading text files: Perl

The files are not downloading, please help. #!/usr/bin/perl -w require HTTP::Response; require LWP::UserAgent; open (INPUT, "ndb_id_file.txt") or die "can't open ndb_id_file.txt"; @input = foreach $line(@input) { $ua =…
phani
  • 1
1 2 3
…
14
15