6

I am running the following Perl snippet on Debian using Perl v5.14.2 and libwww-perl v6.04-1

my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new("GET", "https://google.com/");
my $rep = $ua->request($req);
print $rep->status_line;

This instantly returns "500 Can't connect to google.com:443". I have tried using LWP::Simple, Net::SSLeay, Crypt::SSLeay, etc. without any success.

Oddly enough, executing the same code on another Debian system running exactly the same Perl and LWP versions works.

So I thought, there is some error with the underlying system, but other applications - like cURL for any browser - are working fine.

Also, openssl s_client -connect google.com:443 returns Verify return code: 20 (unable to get local issuer certificate) on both systems.

Has anyone ever encountered this phenomenon and has a solution?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Tomas
  • 161
  • 1
  • 4
  • Are both debians on same network? – mpapec Aug 15 '13 at 10:05
  • No. I tried this with several systems on different networks, also non-Debian and other perl versions, it seems that there is an issue with the first system. – Tomas Aug 15 '13 at 10:07
  • 2
    try `wget https://google.com/` to make sure that network is not blocking traffic – mpapec Aug 15 '13 at 10:19
  • 1
    This works fine. As noted, it also works with other programming languages or software. It seems perl-specific. Is there any way to make LWP more verbose? – Tomas Aug 15 '13 at 10:21
  • `LWP::Debug` but it is deprecated – mpapec Aug 15 '13 at 11:10
  • I tried that, too. "deprecated" is nicely put, "broken" fits better... – Tomas Aug 15 '13 at 11:45
  • Try reading `perldoc LWP::Debug` instead :) and try `lwp-request -UuSsEed https://google.com` – optional Aug 15 '13 at 12:25
  • Nice, I didn't know about lwp-request. However, it gives the same error message and also `Client-Warning: Internal response`. – Tomas Aug 15 '13 at 12:28
  • 1
    surely it gives **more** than that. Do you have LWP::Protocol::https? – optional Aug 15 '13 at 12:34
  • Click [here](http://pastebin.com/GJgFzzF1) to see the full output. And yes, I have installed the latter. By the way, setting `verify_hostname` to zero makes no difference. – Tomas Aug 15 '13 at 12:37
  • 3
    **next step** is flip on https debugging options `use IO::Socket::SSL qw(debug3); / $ENV{HTTPS_DEBUG} ` ... pastebin? its like 7 lines, FWIW for future reference, update your post instead :) – optional Aug 15 '13 at 12:53
  • 4
    HTTPS Debugging gave me the solution, a SSL version error related to a [bug](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=679911), whose workaround I applied in IO::Socket::SSL long time ago, which I had forgotten. Reinstalling yielded the desired result. Thank you! Maybe you should put the hint `use IO::Socket::SSL qw(debug3);` as answer, which surely will help in the general case. – Tomas Aug 15 '13 at 13:33

3 Answers3

7

Instead of this:

$ua = LWP::UserAgent->new;

Try to use this:

$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0 });
Tiago
  • 321
  • 3
  • 4
  • Thanks for your answer, but as I noted in the question and the comments "setting verify_hostname to zero makes no difference". It may help in the general case, but not in this specific one, which has been solved differently as you can observe if you read the discussion in the comments. – Tomas Feb 07 '14 at 16:20
3

FYI for others battling LWP 500 erors:

Some LWP 500 errors are apparently caused by another type of SSL problem (which isn't solved with the verify_hostname setting). Instead, LWP may be communicating with the HTTPS server, but the response is not what LWP expects. In my case, the solution was to force SSLv3, by the following means:

my %ssl_options = (SSL_version => 'SSLv3');
$ua = LWP::UserAgent->new(ssl_opts => \%ssl_options),

Also, for anyone trying to figure out what kind of 500 error you are getting, output this along with your error:

print $response->as_string;

For my problem (solved by setting SSLv3), my $response->as_string output included:

"SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message LWP"

I'll also say that my code ran fine for years on an earlier version of Ubuntu. This problem only appeared once I upgraded Ubuntu. I've concluded that there are probably multiple ways that newer versions of LWP differ from older ones, so developers beware!

Good luck solving your LWP problem!

Brett G
  • 31
  • 1
1

I had this issue on Windows Server 2003 with Strawberry Perl. The error 500 was a problem with IO::Socket::IP.

Performing a re-install of the module (in cpan, force install IO::Socket::IP) resolved the issue.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Anonymous
  • 11
  • 1