How do I change the order of HTTP request headers sent by Perl's LWP?

Question

For a test i need to do a get requets to a website - unfortunatly when using perl lwp the "connection" appears in the header b4 the host. As a result the request gets filtered by the web application firewall. All i need is to remove or move down the connection line in the header. When i do the requets with my script:

use warnings;
use IO::Socket;
use LWP::UserAgent;
use LWP::Protocol::http;
use HTTP::Request;
my $ua = LWP::UserAgent->new();
push(@LWP::Protocol::http::EXTRA_SOCK_OPTS, SendTE => 0, PeerHTTPVersion => "1.1");
$ua->default_header(Cookie => 'XXX', User-Agent => 'whateva');
my $request = $ua->get('https://www.test.com/test.html?...');
....

The header looks like this:

GET /test.html?... HTTP/1.1
Connection: close
Host: www.test.com
User-Agent: whateva
Cookie: XXXX

BUT it should look like this to work (conenction comes after host):

GET /test.html?... HTTP/1.1
Host: www.test.com
Connection: close
User-Agent: whateva
Cookie: XXXX

How do i get rid of that connection line in LWP? I just need to re-oder it....Its not that it needs to be completly removed; I am happy to add it later in there again as

# $userAgent->default_header ("Connection" => "keep-alive");..

Thx a lot in advance!

ikegami · Answer 1 · 2012-12-10T22:52:45.973

To work around the bug in your firewall*, change

return _bytes(join($CRLF, "$method $uri HTTP/$ver", @h2, @h, "", $content));

in Net/HTTP.pm to

my @h3 = ( @h2, @h );
if (my ($idx) = grep /^Host:/, 0..$#h3) {
    unshift(@h3, splice(@h3, $idx, 1));
}

return _bytes(join($CRLF, "$method $uri HTTP/$ver", @h3, "", $content));

* — According to the HTTP/1.1 spec, RFC 2616, "The order in which header fields with differing field names are received is not significant."

How do I change the order of HTTP request headers sent by Perl's LWP?

1 Answers1