5

Configuration

I'm using Laravel 8 with sentry/sentry-laravel plugin.

There is a Job that works just fine 99% of time. It retries N times in case of any problems due to:

public $backoff = 120;

public function retryUntil()
{
    return now()->addHours(6);
}

And it simply calls some service:

public function handle()
{
    // Service calls some external API
    $service->doSomeWork(...);
}

Method doSomeWork sometimes throws an exception due to network problems, like Curl error: Operation timed out after 15001 milliseconds with 0 bytes received. This is fine due to automatic retries. In most cases next retry will succeed.

Problem

Every curl error is sent to Sentry. As an administrator I must check every alert, because this job is pretty important and I can't miss actually failed job. For example:

  1. There is some network problem that is not resolved for an hour.
  2. Application queues a Job
  3. Every 2 minutes application generates similar message to Sentry
  4. After network problems resolved job succeeds, so no attention required
  5. But we are seing dozens of errors, that theoretically could be ignored. But what if there an actual problem in that pile and I will miss it?

Question

How to make that only "final" job fail would send a message to Sentry? I mean after 6 hours of failed retries: only then I'd like to receive one alert.

What I tried

There is one workaround that kind of "works". We can replace Exception with SomeCustomException and add it to \App\Exceptions\Handler::$dontReport array. In that case there are no "intermediate" messages sent to Sentry.

But when job finally fails, Laravel sends standard ... job has been attempted too many times or run too long message without details of actual error.

frutality
  • 345
  • 1
  • 3
  • 13

1 Answers1

1

I was facing this same issue, you should be able to handle the error and replicate the behavior of the framework by using $this->release().

In the code snippet below, I am using the inherited properties $tries and $backoff that the framework would use, but have added my own logic to take these into account even when the exception from the job is handled. When the number of attempts made is less than the tries, release()ing the job tells the queue to re-process it. In the case where the tries are exceeded, re-throwing the exception should cause it to send to your error reporting system and fail the job as expected.

public $tries = [n];

public $backOff = [m]; 

public function handle(): void
{
    try {
        ...
    } catch (Throwable $e) {
        $this->handleException($e);
    }
}

private function handleException(Throwable $throwable): void
{
    if ($this->attempts() < $this->tries) {
        $this->release($this->backOff);
        return;
    }

    throw $throwable;
}
Andrew
  • 11
  • 1