2

I would like to implement light job queue using Redis (minion requires Postgresql). I can successfully run a minimal example but I have problem with my full Mojolicious application.

In the startup routine and the call look as follows.

sub startup{
    ...
    ###################### Redis part
    $self->helper(
        redis => sub {
            state $redis = Mojo::Redis2->new(server => '127.0.0.1:6379'); 
        }
    );

    $self->redis->on(message => sub {
        my ($redisself, $message, $channel) = @_;

        # PROBLEM: on morbo this is blocking
        # PROBLEM: on hypnotoad this works only sometimes 
        # - e.g., for the first time after hot deployment restart

        say "DEQUEUE: $message @ $channel";
        sleep (10); # replaces long-running operation
        say "FINISHED processing: $message @ $channel";
    });

    $self->redis->subscribe(['long_running_tasks'] => sub {
        my ($rself, $err, $res) = @_;
        say "SUBSCRIBED to channel long_running_tasks";       
    });

    $self->redis->on(connection => sub { 
        my ($self, $info) = @_;  
        say "got redis connection id: $info->{id}, group: $info->{group}, nblocking: $info->{nb}.";
    });

    $self->redis->on(error => sub {
        my ($self, $err) = @_;
        say "got redis error $err";
    });
    ######################## Redis part END
    ... 

    # route
    $manager->get('/regenerate')->to('controller#regenerate');
}


#######################
# in the Controller.pm
#######################
sub regenerate {
    my $self = shift;

    $self->redis->publish("long_running_tasks" => "regenerate");

    $self->flash(
        msg_type => 'info',
        msg      => 'Regeneration of will be executed in background.'
    );
    $self->redirect_to($self->get_referrer());
}

I run Redis 3.2.5 (homebrew) on my localhost. The problems concern nondeterministic behavior of the code.

  • the minimal example
    • runs fine on morbo
    • the publish action is executed 3 times on hypnotoad
  • the big example
    • on morbo, the publish action is blocking, but runs fine
    • on hypnotoad, the publish action is executed in a non-blocking fashion but the execution happens only in for some calls (e.g., some of the first calls after a hot deployment restart)
      • subscribe action is always successful

Where are the problems?

vikin9
  • 497
  • 4
  • 16
  • If you could write the tests, it could simplify problem detection. – mpapec Nov 20 '16 at 12:21
  • I think, I am first struggling with the understanding how it should work. It is difficult to write a test if a basic case fails your expectations. I must have done some basic mistake here, but I can't see it. – vikin9 Nov 20 '16 at 16:24

0 Answers0