Consumer Error Handling in Symfony Messenger / RabbitMQ

Question

I'm using the new Symfony Messenger Component 4.1 and RabbitMQ 3.6.10-1 to queue and asynchronously send email and SMS notifications from my Symfony 4.1 web application. My Messenger configuration (messenger.yaml) looks like this:

framework:
    messenger:
        transports:
            amqp: '%env(MESSENGER_TRANSPORT_DSN_NOTIFICATIONS)%'

        routing:
            'App\NotificationBundle\Entity\NotificationQueueEntry': amqp

When a new notification is to be sent, I queue it like this:

use Symfony\Component\Messenger\MessageBusInterface;
// ...
$notificationQueueEntry = new NotificationQueueEntry();
// [Set notification details such as recipients, subject, and message]
$this->messageBus->dispatch($notificationQueueEntry);

Then I start the consumer like this on the command line:

$ bin/console messenger:consume-messages

I have implemented a SendNotificationHandler service where the actual delivery happens. The service configuration:

App\NotificationBundle\MessageHandler\SendNotificationHandler:
    arguments:
        - '@App\NotificationBundle\Service\NotificationQueueService'
    tags: [ messenger.message_handler ]

And the class:

class SendNotificationHandler
{
    public function __invoke(NotificationQueueEntry $entry): void
    {
        $this->notificationQueueService->sendNotification($entry);
    }
}

Until this point, everything works smoothly and the notifications get delivered.

Now my question: It may happen that an email or SMS cannot be delivered due to a (temporary) network failure. In such a case, I would like my system to retry the delivery after a specified amount of time, up to a specified maximum number of retries. What is the way to go to achieve this?

I have read about Dead Letter Exchanges, however, I could not find any documentation or example on how to integrate this with the Symfony Messenger Component.

score 3 · Answer 1 · edited Jan 24 '19 at 15:18

3

What you need to do is tell RabbitMQ, that the message is rejected instead of acknowledged. By default the messenger will take care of this inside the AmqpReceiver. As you can see there, if you throw an exception that implements the RejectMessageExceptionInterface inside your handler, the message will automatically be rejected for you.

You could also "simulate" this behaviour with custom middleware. I created something like it, in a small demo application. The mechanism consists of a middleware that wraps the (serialized) original message inside a new RetryMessage and sends it via a custom message bus to a different queue, used as a dead letter exchange. The handler for that message will then unpack the RetryMessage (getting the original message and deserializing it) and transmit it over the default bus:

See:

This is a basic setup which rejects the message and allows you to consume it again instantly(!). You probably want to add additional information such as headers for timestamps when delaying the consumption to improve on this. For this you should look at writing your own receiver, middleware and/or handler.

edited Jan 24 '19 at 15:18

Nek

2,715
1
20
34

answered Oct 30 '18 at 17:40

dbrumann

16,803
2
42
58

For the headers that are useful when dealing with dead-lettered messages you can check: https://www.rabbitmq.com/dlx.html – dbrumann Oct 30 '18 at 17:42
Two questions: 1.) What exactly will happen in RabbitMQ when I reject the message? How do I configure in which queue it will be pushed, in which interval and how many times it will be redelivered? 2.) When I throw an exception that implements, the `RejectMessageExceptionInterface`, the `bin/console messenger:consume-messages` command exits because the exception is caught and then immediately thrown again (as you can see in `AmqpReceiver`. How will I make it stay alive? – thomaskonrad Oct 31 '18 at 06:30
1) In order to declare an exchange as DLX in RabbitMQ you can use the cli-tool. As far as I know the routing key, i.e. the queue name will be kept, so you have the same queue under a different exchange, but you can modify this by setting a routing key like x-dead-letter-routing-key (?). 2) That the command exits should not be an issue, as you should frequently restart it anyway. You could use a process manager like supervisord for that. If you don't want this behavior, you will have to write your own middleware or adapt the SendMessageMiddleware and disable the default. – dbrumann Oct 31 '18 at 07:22

Consumer Error Handling in Symfony Messenger / RabbitMQ

1 Answers1