What happens with the QueueWorker when TTR ran out?

Question

This relates to laravel 5.3, beanstalk, ttr and timeout working with Queue's and QueueWorkers. TTR: https://github.com/kr/beanstalkd/wiki/faq

If I understand correctly a job from the Queue gets the state reserved when a QueueWorker is picking it. This job state will be changed back to ready when the ttr runs out. But what happens with the QueueWorker?

Let's say the QueueWorker has a timeout set to 600 by the following command:

php artisan queue:work --tries=1 --timeout=600 --sleep=0

ttr is, as default, set to 60 seconds.

During the job a request is done to another site and it takes 120 seconds till response. After 60 seconds the job is set back to the ready state because the TTR. Will the QueueWorker keep working on the job till response has been received, maximum of 600 seconds? Or will the QueueWorker stop working on the job when TTR has been reached?

score 4 · Accepted Answer · edited Jul 13 '18 at 14:13

Actually, the QueueWorker will run till the job is completed. When you run queue worker without the daemon flag, it will run the code below:

return $this->worker->pop(
    $connection, $queue, $delay,
    $this->option('sleep'), $this->option('tries')
);

Reference: https://github.com/laravel/framework/blob/5.2/src/Illuminate/Queue/Console/WorkCommand.php#L123

What this code does is pop its job from the queue and fire that job as a command:

public function process($connection, Job $job, $maxTries = 0, $delay = 0)
{
    if ($maxTries > 0 && $job->attempts() > $maxTries) {
        return $this->logFailedJob($connection, $job);
    }

    try {
        $job->fire();

        $this->raiseAfterJobEvent($connection, $job);

        return ['job' => $job, 'failed' => false];
    } catch (Exception $e) {
        if (! $job->isDeleted()) {
            $job->release($delay);
        }

        throw $e;
    } catch (Throwable $e) {
        if (! $job->isDeleted()) {
            $job->release($delay);
        }

        throw $e;
    }
}

Reference: https://github.com/laravel/framework/blob/5.2/src/Illuminate/Queue/Worker.php#L213

Digging in the source for more information: https://github.com/laravel/framework/tree/5.2/src/Illuminate/Queue

Does this mean the QueueWorker will keep working on the job till timeout has been reached, even when ttr ran out? — MmynameStackflow, Jan 13 '17 at 20:15
Correct. for more details about how TTR works https://github.com/kr/beanstalkd/wiki/faq#how-does-ttr-work — Sanath Samarasinghe, Jan 16 '17 at 10:21
What's the point then of setting a timeout higher then 60 seconds? Right now, a second QueueWorker will pick the job because after TTR ran out and the job will have the ready state which causes problems because the first QueueWorker cannot find the job anymore when trying to delete it. When you use a timeout higher then the TTR it is a must to make use of "touching" the job once in a while, preventing TTR from running out of time till timeout has been reached. Am I correct? — MmynameStackflow, Jan 16 '17 at 11:03

What happens with the QueueWorker when TTR ran out?

1 Answers1