1

I have written a Perl CGI script which fetches an html page from a site using POST, then does some processing on the html data and prints output html content.

My script url: http://mysite.com/cgi-bin/p.pl

Site URL: http://site.com/interesting.asp

#!/usr/bin/perl
# p.pl
...

sub post_url {
    my( $url, $formref ) = @_;
    my $ua = new LWP::UserAgent(timeout => 300);
    my $response = $ua->post($url, $formref );

if( $response->is_success ){
    return $response->content;
} else {
    return undef;
}
}

my $url = 'url: http://site.com/interesting.asp';
my %param = ('eno' => '1234', 'submit' => 'Submit');
my $htmlcontent = post_url( $url, \%param );    # <--- Not working

... do some processing on $htmlcontent ...

... print out html page ...

The script runs fine when run from the command line but fails to fetch the HTML page from the site when run from a Webserver. Is it because the script it trying to access a page from a different domain/IP address? Can anybody suggest a workaround?

Girish
  • 925
  • 3
  • 10
  • 20

3 Answers3

2

There could be many causes for this. Assuming that the cgi script is actually called (i.e., that it is installed in the proper location for your web server), I would ensure that LWP is actually installed on your server. Indeed, in various occasions I had to install it manually, YMMV.

It would be really helpful to output some diagnostics where the script fails... Does new LWP::UserAgent succeed? if so, what is $response->status_line after the post?

EDIT:

Since your error message is "500 permission denied", it is likely that your web server has blocked all outgoing connection. You should check whether your provider allows outgoing http connection through some proxy and configure that into LWP::UserAgent. If there is no proxy available, then possibly there is no way to go out. We cannot help here without knowing your hosting provider settings, though; maybe you can ask your hosting provider support...

sergio
  • 68,819
  • 11
  • 102
  • 123
  • Ah! $response->status_line reads: "500 Can't connect to site.com:80 (connect: Permission denied)" – Girish Jul 03 '11 at 08:25
  • What do you think might be the problem? – Girish Jul 03 '11 at 08:34
  • 2
    @Girish: It looks like the `connect` system call is forbidden to the webserver. For instance, SELinux supports such setup. You can try `use IO::Socket::INET; my $sock = IO::Socket::INET->new(PeerAddr=>'www.perl.org', PeerPort => 'http(80)', Proto => 'tcp') or die "socket: $!";` to check whether it's true. – Dallaylaen Jul 03 '11 at 09:05
  • i am getting "socket: Permission denied at ...". so, i guess the web server is preventing scripts from connecting to any site except the one its running on. – Girish Jul 03 '11 at 09:17
0

SELINUX enabled systems will not allow an outgoing connection from a web agent (httpd).

On CentOS 6.3 you can use this command to enable outbound web connections from Apache in a Perl script:

# setsebool -P httpd_can_network_connect on

This page can tell you more about SELinux and HTTPD settings: http://wiki.centos.org/TipsAndTricks/SelinuxBooleans

Lance Cleveland
  • 3,098
  • 1
  • 33
  • 36
0

You should remove "url: " from $url:

my $url = 'http://site.com/interesting.asp';

Serguei Trouchelle
  • 1