Stack Overflow
Stack Overflow

Questions tagged [lwp]

The World-Wide Web library for Perl - LWP is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web.

The main focus of the library is to provide classes and functions that allow you to write WWW clients. The library also contain modules that are of more general use and even classes that help you implement simple HTTP servers.

Most modules in this library provide an object oriented API. The user agent, requests sent and responses received from the WWW server are all represented by objects. This makes a simple and powerful interface to these services. The interface is easy to extend and customize for your own needs.

The main features of the library are:

  • Contains various reusable components (modules) that can be used separately or together.
  • Provides an object oriented model of HTTP-style communication. Within this framework we currently support access to http, https, gopher, ftp, news, file, and mailto resources.
  • Provides a full object oriented interface or a very simple procedural interface.
  • Supports the basic and digest authorization schemes.
  • Supports transparent redirect handling.
  • Supports access through proxy servers.
  • Provides parser for robots.txt files and a framework for constructing robots.
  • Supports parsing of HTML forms. Implements HTTP content negotiation algorithm that can be used both in protocol modules and in server scripts (like CGI scripts).
  • Supports HTTP cookies.
  • Some simple command line clients, for instance lwp-request and lwp-download.

More information on the CPAN website.

365 questions
3
votes
1 answer

How do I create a gzip compressed HTTP::Response?

I need to create an HTTP::Response with compressed data. How do I go about making the content gzipped? Do I just add the appropriate headers and compress it myself using Compress::Zlib? Or does any of the LWP modules provide a method for handling…
mrburns
  • 453
  • 1
  • 8
  • 17
3
votes
2 answers

How to send HTTP POST data in multipart/form-data to REST API in Perl

I am sending POST HTTP request to a website REST API from my Perl script using multipart/form-data which requires basic authentication. I am passing some params as key value pairs in Content along with image file to be uploaded. I passed credentials…
P_Z
  • 199
  • 2
  • 8
3
votes
1 answer

How to login in Perl using LWP::UserAgent and can check cookies value using HTTP::Cookies?

I'm trying to automate a test-case in perl using LWP::UserAgent. I need to check the cookie value post login to the application. Have tried the following sample perl script, but while printing I'm getting output in HASH value : my $ua =…
Pooja
  • 481
  • 1
  • 8
  • 15
3
votes
1 answer

Can I call the Perl LWP Post function with a string parameter

I have a problem supplying a parameter list to the UserAgent Post function (using the https module). The examples are of the form: my $ua = LWP::UserAgent->new(); my $response = $ua->post( $url, { 'param1', 'value1', 'param2', 'value2' } ); The API…
kaj66
  • 153
  • 1
  • 11
3
votes
3 answers

Loggin into website with LWP and Perl

Somewhat inexperienced programmer here trying to write a program to log into my courses site and download all the content (lectures homeworks etc). Obviously it is a password protected site so I have to give it that. I understand LWP::UserAgent and…
msikd65
  • 427
  • 1
  • 5
  • 11
3
votes
1 answer

Perl post request to send zip file as base64?

I have a Perl script trying to send a zip file like so with LWP UserAgent module my $req = POST $url, Content_Type => 'form-data', Content => [ submit => 1, upfile => [ $fname ] ]; where $fname is the path of the file.…
user391986
  • 29,536
  • 39
  • 126
  • 205
3
votes
0 answers

Is it possible to add some debug output when LWP::UserAgent connected to proxy?

I have a very simple example Perl script $ cat proxy.pl use strict; use warnings; use LWP::UserAgent (); my $ua = LWP::UserAgent->new; $ua->timeout(10); $ua->show_progress(1); $ua->proxy([ [ 'http' ] =>…
Paul Serikov
  • 2,550
  • 2
  • 21
  • 36
3
votes
3 answers

set the download speed with LWP - Perl

I am developing an application which download files from internet; the files are mainly in ftp servers, I am using LWP::Simple and the getstore function to retrieve the files. But I would like to limit the speed of download, like in wget ... have…
Juan
  • 1,520
  • 2
  • 19
  • 31
3
votes
2 answers

Querying a website with Perl LWP::Simple to Process Online Prices

In my free time, I've been trying to improve my perl abilities by working on a script that uses LWP::Simple to poll one specific website's product pages to check the prices of products (I'm somewhat of a perl noob). This script also keeps a very…
Cooper
  • 679
  • 3
  • 9
3
votes
1 answer

How can I share OpenSSL sessions between Perl processes?

I'm using Perl to connect to some (very) low-powered hardware devices with TLS. The first handshake can take around 10-15 seconds to complete! Reusing the session (from the same Perl process) is much faster but when the next job comes around to…
wes
  • 7,795
  • 6
  • 31
  • 41
3
votes
1 answer

Override DNS For Specific Domains Like A Hosts File, But Without Using Hosts file

I need to issue a series of parallelized web requests from the same server to a specific domain, but control what IP address these requests actually go to. Originally, I came up with a scheme where I would request the IP I wanted specifically, and…
Dereleased
  • 9,939
  • 3
  • 35
  • 51
3
votes
1 answer

How to read a https JSON stream with perl?

Ok I'll try and be as detailed as possible. I need to connect to a https url that streams JSON data, and search for keywords via REGEX as the data streams then take the matching JSON element decode it and insert it into a database. The REGEX on is…
Sean Smith
  • 85
  • 5
3
votes
2 answers

How do I do a chunked transfer-encoding upload with WWW:Mechanize?

I'm attempting to use a particular web service, and I can successfully perform the upload with the following command: curl -X POST --header "Transfer-Encoding: chunked" -d @Downloads/file.pdf https://some.webservice/upload I get back a json…
John O
  • 4,863
  • 8
  • 45
  • 78
3
votes
2 answers

Perl HTTPS over proxy using LWP::UserAgent

I wish to request a URL via a HTTPS proxy using perl's LWP::UserAgent module. There is quite a few reference around this, but nothing could help me get it work. #!/usr/bin/perl use strict; use warnings; use LWP::UserAgent; use Data::Dumper; BEGIN…
alpha_cod
  • 1,933
  • 5
  • 25
  • 43
3
votes
2 answers

Visit Half Million Pages with Perl

Currently I'm using Mechanize and the get() method to get each site, and check with content() method each mainpage for something. I have a very fast computer + 10Mbit connection, and still, it took 9 hours to check 11K sites, which is not…
snoofkin
  • 8,725
  • 14
  • 49
  • 86
1 2 3
…
24 25