Stack Overflow
Stack Overflow

Questions tagged [lwp]

The World-Wide Web library for Perl - LWP is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web.

The main focus of the library is to provide classes and functions that allow you to write WWW clients. The library also contain modules that are of more general use and even classes that help you implement simple HTTP servers.

Most modules in this library provide an object oriented API. The user agent, requests sent and responses received from the WWW server are all represented by objects. This makes a simple and powerful interface to these services. The interface is easy to extend and customize for your own needs.

The main features of the library are:

  • Contains various reusable components (modules) that can be used separately or together.
  • Provides an object oriented model of HTTP-style communication. Within this framework we currently support access to http, https, gopher, ftp, news, file, and mailto resources.
  • Provides a full object oriented interface or a very simple procedural interface.
  • Supports the basic and digest authorization schemes.
  • Supports transparent redirect handling.
  • Supports access through proxy servers.
  • Provides parser for robots.txt files and a framework for constructing robots.
  • Supports parsing of HTML forms. Implements HTTP content negotiation algorithm that can be used both in protocol modules and in server scripts (like CGI scripts).
  • Supports HTTP cookies.
  • Some simple command line clients, for instance lwp-request and lwp-download.

More information on the CPAN website.

365 questions
2
votes
1 answer

WWW::Mechanize error: Error GETing url Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)

I am very new to perl. I am using it at work, and I got the following error yesterday, for a script that was running just fine on the previous day. Error GETing https://www.someurl.com:443: Protocol scheme 'https' is not supported…
Amruta
  • 1,295
  • 1
  • 13
  • 24
2
votes
1 answer

Perl LWP::Simple get fails from Ubuntu server, but not local machine

I'm trying to get something from a webpage using something like this #!/usr/bin/perl use LWP::Simple; use LWP::Protocol::https; $url =…
caceves
  • 23
  • 6
2
votes
0 answers

How do I specify SSL CipherString with Perl LWP?

I access several websites for various reasons using Perl LWP. It has been working great for years until I upgraded my server from Ubuntu 18.04 to 20.04 recently, and then LWP stopped working for one particular website. It seems that is a very old…
James B
  • 51
  • 4
2
votes
3 answers

PHP Simple HTML DOM or Python-BSoup: which one is the easier approach?

i am currently working on a approach to parse a site that contains datas on Foundations in Europe. http://www.foundationfinder.ch/ which has a dataset of 790 foundations. All the data are free to use - with no limitations copyrights on it. What is…
zero
  • 1,003
  • 3
  • 20
  • 42
2
votes
0 answers

Issues connecting to server with sslv3 disabled in legacy perl

As the title says, I'm attempting to POST some JSON to a REST API using a pretty old version of perl, specifically 5.10.1. I'm working in a pretty old legacy codebase on RHEL 6. The handshake fails with error 500 because the server has sslv3…
ragingSloth
  • 1,094
  • 8
  • 22
2
votes
4 answers

AWS SES certificate verify failed

I have set up SES successfully on one AWS instance. Now I am trying to use it on a second (not cloned) instance and when I run any of the SES scripts, I get an error: ses-get-stats.pl -k aws-credentials -q I get: Can't connect to…
Mitch
  • 2,350
  • 7
  • 29
  • 48
2
votes
1 answer

Unable to get page via HTTPS with LWP::Simple in Perl

I try to download a page from an HTTPS URL with Perl: use LWP::Simple; my $url = 'https://www.ferc.gov/xml/whats-new.xml'; my $content = get $url or die "Unable to get $url\n"; print $content; There seems to be a problem. Just can't figure out the…
wiedhas
  • 57
  • 7
2
votes
1 answer

How to explicitly set cookie using LWP

I want to set some cookies for use in a HTTP::Request later. This is my code (shortest form that already exhibits the problem): #!/usr/bin/perl -w use strict; use LWP::UserAgent; use HTTP::Cookies; use Data::Dumper; # DEBUG my $ua =…
gpvos
  • 2,722
  • 1
  • 17
  • 22
2
votes
3 answers

HTTP::Request and literal %2B

I am making a script that POST some XML to another server but I have problems with a plus singn (+). Here is my code: #!/usr/bin/perl use strict; use warnings; use LWP::UserAgent; my $XML = qq|
Welcho
  • 35
  • 6
2
votes
1 answer

How can cpanm retrieve settings for a proxy that requires username and password

I'm working with CPANM to interact with a proxy that requires username and password. I specified the settings when running "o conf init /proxy/ under cpan". My perception is the variables used in a unix environment for specifying proxies are not…
ffejrekaburb
  • 656
  • 1
  • 10
  • 35
2
votes
1 answer

Create Google Team Drive with Perl module LWP::Authen::OAuth2

I'm trying to use Perl with LWP::Authen::OAuth2 to perform google team drive creation. Understand to create google team drive using google Drive API, it requires 1 param to be posted that is requestId and another json body name (reference:…
Armitage
  • 33
  • 2
2
votes
2 answers

LWP Won't Run in CGI Script

I have a CGI script to load publications from BibBase: #!/usr/bin/perl use LWP::UserAgent; my $url = 'https://bibbase.org/show?bib=http://www.example.com/pubs.bib'; my $ua = LWP::UserAgent->new; my $can_accept = HTTP::Message::decodable; my…
yakatz
  • 2,142
  • 1
  • 18
  • 47
2
votes
3 answers

LWP::Simple runs very fine: how to store 6000 ++ records in a file and do some cleanup?

good evening dear community! i want to process multiple webpages, kind of like a web spider/crawler might. I have some bits - but now i need to have some improved spider-logic. See the target-url…
zero
  • 1,003
  • 3
  • 20
  • 42
2
votes
1 answer

LWP, HTML::TableExtract and the output with Text::CSV - how to add attributes here?

I have a little parser that parses a site - with 6150 records. But I need to have this in a CSV-format. First of all see here the target site: http://192.68.214.70/km/asps/schulsuche.asp?q=a&a=50&s=1750 I need all the data - with separation in the…
zero
  • 1,003
  • 3
  • 20
  • 42
2
votes
2 answers

Perl LWP Post form-data with my boundary

I want to post data with content type multipart/form-data: use strict; use warnings; use LWP::UserAgent; my $ua = LWP::UserAgent->new; $ua->post ( 'http://127.0.0.1:12555', 'Content-Type' => 'form-data', Content => { 'data1' =>…
BingLi224
  • 452
  • 2
  • 6
  • 12
1 2 3
…
24 25