I would like to implement light job queue using Redis (minion requires Postgresql). I can successfully run a minimal example but I have problem with my full Mojolicious application.
In the startup
routine and the call look as follows.
sub startup{
...
###################### Redis part
$self->helper(
redis => sub {
state $redis = Mojo::Redis2->new(server => '127.0.0.1:6379');
}
);
$self->redis->on(message => sub {
my ($redisself, $message, $channel) = @_;
# PROBLEM: on morbo this is blocking
# PROBLEM: on hypnotoad this works only sometimes
# - e.g., for the first time after hot deployment restart
say "DEQUEUE: $message @ $channel";
sleep (10); # replaces long-running operation
say "FINISHED processing: $message @ $channel";
});
$self->redis->subscribe(['long_running_tasks'] => sub {
my ($rself, $err, $res) = @_;
say "SUBSCRIBED to channel long_running_tasks";
});
$self->redis->on(connection => sub {
my ($self, $info) = @_;
say "got redis connection id: $info->{id}, group: $info->{group}, nblocking: $info->{nb}.";
});
$self->redis->on(error => sub {
my ($self, $err) = @_;
say "got redis error $err";
});
######################## Redis part END
...
# route
$manager->get('/regenerate')->to('controller#regenerate');
}
#######################
# in the Controller.pm
#######################
sub regenerate {
my $self = shift;
$self->redis->publish("long_running_tasks" => "regenerate");
$self->flash(
msg_type => 'info',
msg => 'Regeneration of will be executed in background.'
);
$self->redirect_to($self->get_referrer());
}
I run Redis 3.2.5 (homebrew) on my localhost. The problems concern nondeterministic behavior of the code.
- the minimal example
- runs fine on morbo
- the publish action is executed 3 times on hypnotoad
- the big example
- on morbo, the publish action is blocking, but runs fine
- on hypnotoad, the publish action is executed in a non-blocking fashion but the execution happens only in for some calls (e.g., some of the first calls after a hot deployment restart)
- subscribe action is always successful
Where are the problems?