1

I'm currently integrating PubSub using the following approach https://docs.spring.io/spring-cloud-gcp/docs/1.2.4.BUILD-SNAPSHOT/reference/html/#spring-integration

Basically I'm subscribed to a topic, I receive the message and do something with it.

I would like to, in case of any error scenario of my business rules (like not accept the message because X,Y,Z reason) send it to a dlq.

I see that just a month and a half ago Google came out with DLQ for PubSub: https://cloud.google.com/pubsub/docs/dead-letter-topics

But I'm not sure what the right way to integrate it with the spring integration approach should be.

Columb1a
  • 463
  • 2
  • 11
  • 25

1 Answers1

4

The built-in DLQ support will work automatically -- all you have to do is turn on Dead Lettering (through add/edit subscription screen in Cloud Console), setting the "Maximum delivery attempts" field to the number of attempts. After your application nack()s a message this number of times, Cloud Pub/Sub will redirect it to the topic you set up as the DLQ.

The built-in support works great for environmental, retryable causes of message delivery failure. However, it has a minimum of 5 retries before it will send the message to DLQ. In your case of business rules validation, you may prefer to mimic DLQs with a custom topic and Spring Integration redirection, since after failing on attempt #1, the message won't suddenly become valid, and the remaining attempts #2 - #5 are a waste of resources.

To mimic the DLQ that redirects after a single failure, you would create a new topic, let's call it validation-dlq, wire it up as a Spring Integration channel, run custom validation checks on each message, and if a message fails, ack() the original message on the source subscription, and publish the message to the validation-dlq topic instead.

Elena Felder
  • 456
  • 3
  • 8
  • Awesome. I think after what you said, it's clear that send messages to DLQ should happen only under retryable causes of message delivery failures. Any other particular scenario, should be handled by topics. – Columb1a Jul 21 '20 at 18:06