mojolicious script works three times, then crashes

Question

The following script should demonstrate a problem I'm facing using Mojolicious on OpenBSD5.2 using mod_perl.

The script works fine 4 times being called as CGI under mod_perl. Additional runs of the script result in Mojolicious not returning the asynchronous posts. The subs that are usually called when data is arriving just don't seem to be called anymore. Running the script from command line works fine since perl is then completely started from scratch and everything is reinitialized, which is not the case under mod_perl. Stopping and starting Apache reinitializes mod_perl so that the script can be run another 4 times.

I only tested this on OpenBSD5.2 using Mojolicious in the version that's provided in OpenBSDs ports tree (2.76). This is kinda old I think but that's what OpenBSD comes with.

Am I doing something completely wrong here? Or is it possible that Mojolicious has some circular reference or something which causes this issue?

I have no influence on the platform (OpenBSD) being used. So please don't suggest to "use Linux and install latest Mojolicious version". However if you are sure that running a later version of Mojolicous will solve the problem, I might get the permission to install that (though I don't yet know how to do that).

Thanks in advance!

T.

Here's the script:

#!/usr/bin/perl

use diagnostics;
use warnings;
use strict;
use feature qw(switch);
use CGI qw/:param/;
use CGI qw/:url/;
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
use Mojo::IOLoop;
use Mojo::JSON;
use Mojo::UserAgent;

my ($activeconnections, $md5, $cgi);

my $ua = Mojo::UserAgent->new;
$ua->max_redirects(0)->connect_timeout(3)->request_timeout(6); # Timeout 6 seconds of which 3 may be connecting
my $delay = Mojo::IOLoop->delay();

sub online{
 my $url = "http://www.backgroundtask.eu/Systeemtaken/Search.php";
 $delay->begin;
 $activeconnections++;
 my $response_bt = $ua->post_form($url, { 'ex' => $md5 }, sub {
    my ($ua, $tx) = @_;
    my $content=$tx->res->body;
    $content =~ m/(http:\/\/www\.backgroundtask\.eu\/Systeemtaken\/taakinfo\/.*$md5\/)/;
        if ($1){
                print "getting $1\n";
                my $response_bt2 = $ua->get($1, sub {
                        $delay->end();
                        $activeconnections--;
                        print "got result, ActiveConnections: $activeconnections\n";
                        ($ua, $tx) = @_;
                        my $filename = $tx->res->dom->find('table.view')->[0]->find('tr.even')->[2]->td->[1]->all_text;
                        print "fn = " . $filename . "\n";
                }
            )
        } else {
                print "query did not return a result\n";
                $activeconnections--;
                $delay->end;
        }
 });
}

$cgi = new CGI;
print $cgi->header(-cache_control=>"no-cache, no-store, must-revalidate") . "\n";

$md5 = lc($cgi->param("md5") || ""); # read param
$md5 =~ s/[^a-f0-9]*//g if (length($md5) == 32); # custom input filter for md5 values only

if (length $md5 != 32) {
        $md5=lc($ARGV[0]);
        $md5=~ s/[^a-f0-9]*//g;
        die "invalid MD5 $md5\n" if (length $md5 ne 32);
}

online;

if ($activeconnections) {
        print "waiting..., activeconnections: $activeconnections\n" for $delay->wait;
}

print "all pending requests completed, activeconnections is " . $activeconnections . "\n";
print "script done.\n md5 was $md5\n";
exit 0;

Answer 1

Well I hate to say it, but there's a lot wrong here. The most glaring is your use of ... for $delay->wait which doesn't make much sense. Also you are comparing numbers with ne rather than !=. Not my-ing the arguments in the deeper callback seems problematic for async style code.

Then there are some code smells, like regexing for urls and closing over the $md5 variable unnecessarily.

Lastly, why use CGI.pm when Mojolicious can operate under CGI just fine? When you do that, the IOLoop is already running, so some things get easier. And yes I understand that you are using the system provided Mojolicious, however I feel I should mention that the current version is 3.93 :-)

Anyway, here is an example, which strips out a lot of things but still should do pretty much the same thing as the example. Of course I can't test it without a valid md5 for the site (and I also can't get rid of the url regex without sample data).

#!/usr/bin/perl

use Mojolicious::Lite;
use Mojo::UserAgent;

my $ua = Mojo::UserAgent->new;
$ua->max_redirects(0)->connect_timeout(3)->request_timeout(6); # Timeout 6 seconds of which 3 may be connecting

any '/' => sub {
  my $self = shift;
  $self->res->headers->cache_control("no-cache, no-store, must-revalidate");

  my $md5 = lc($self->param("md5") || ""); # read param
  $md5 =~ s/[^a-f0-9]*//g if (length($md5) == 32); # custom input filter for md5 values only

  if (length $md5 != 32) {
    $md5=lc($ARGV[0]);
    $md5=~ s/[^a-f0-9]*//g;
    die "invalid MD5 $md5\n" if (length $md5 != 32);
  }

  $self->render_later; # wait for ua

  my $url = "http://www.backgroundtask.eu/Systeemtaken/Search.php";
  $ua->post_form($url, { 'ex' => $md5 }, sub {
    my ($ua, $tx) = @_;
    my $content=$tx->res->body;
    $content =~ m{(http://www\.backgroundtask\.eu/Systeemtaken/taakinfo/.*$md5/)};
    return $self->render( text => 'Failed' ) unless $1;
    $ua->get($1, sub {
      my ($ua, $tx) = @_;
      my $filename = $tx->res->dom->find('table.view')->[0]->find('tr.even')->[2]->td->[1]->all_text;
      $self->render( text => "md5 was $md5, filename was $filename" );
    });
  });

};

app->start;