Get raw response headers from LWP?

Question

Is there a way to grab raw, unmodified response headers from an HTTP request made with LWP? This is for a diagnostic tool that needs to identify problems with possibly malformed headers.

The closest thing I've found is:

use LWP::UserAgent;
my $ua = new LWP::UserAgent;
my $response = $ua->get("http://somedomain.com");
print $response->headers()->as_string();

But this actually parses the headers, and then reconstructs a canonicalized, cleaned-up version of them from the parsed data. I really need the entire header text in exactly the form in which it was returned by the server, so anything malformed or non-standard will be clearly identifiable.

If it turns out there is no way to do this with LWP, is there perhaps some other Perl module that can do this?

Answer 1

Net::HTTP provides lower level access with less processing. Since it is a subclass of IO::Socket::INET you can read directly from the object after making the request.

use Net::HTTP;

# Make the request using Net::HTTP.
my $s = Net::HTTP->new(Host => "www.perl.com") || die $@;
$s->write_request(GET => "/", 'User-Agent' => "Mozilla/5.0");

# Read the raw headers.
my @headers;
while(my $line = <$s>) {
    # Headers are done on a blank line.
    last unless $line =~ /\S/;
    push @headers, $line;
}
print @headers;

Answer 2

Based on an inspection of the HTTP::Response object (and the HTTP::Headers object it contains), the headers are discarded as they're parsed.

I would recommend you try WWW::Curl instead.

EDIT Snippet using WWW::Curl:

use WWW::Curl::Easy;

my ($header, $body);

my $curl = WWW::Curl::Easy->new;
$curl->setopt(CURLOPT_URL, $url_to_get); # get this URL
$curl->setopt(CURLOPT_WRITEHEADER, \$header); # save header text in this var
$curl->setopt(CURLOPT_WRITEDATA, \$body); # save body text in this var

my $code = $curl->perform;
if (0 == $code) {
  # header text is in $header, body text in $body 
} else {
  print $curl->strerror($code).": ".$curl->errbuf."\n";
}