0

I have a topic that delivers to a push subscription with ordering keys enabled. The retry limit is about 15 with exponential backoff up to 10 minutes.

It looks like some messages are being processed successfully by the main consumer, but the same messages also get pushed to the dead letter queue of that consumer.

There were no errors or timeouts in the push subscriber's logs, but there were a few 5xx errors (less than 0.01 per second) shown in the pubsub metrics board.

Is this possible due to how messages with ordering keys would be retried or something, or are the messages necessarily failing to deliver as much as their retry limit would allow?

Gavin Haynes
  • 1,721
  • 11
  • 21

1 Answers1

1

It wouldn't be the case that it would affect ordered delivery more than unordered delivery, but it is still possible for messages to get processed successfully and end up in the dead letter queue. Dead lettering is best effort and so there are different scenarios that could occur:

  1. Message failed to be delivered and was to be dead lettered, but dead lettering did not appear to happen (say, there was a deadline exceeded error) and then the state for the message also got reset, resulting in a redelivery to the subscriber that was processed successfully.

  2. The message was not acknowledged before the ack deadline expired. In such a case, it may have been delivered multiple times and acked multiple times, but if all acks were slow, then it would still ultimately end up in the dead-letter topic.

If it is happening frequently and doesn't appear to be for something like the second reason, it would be best to reach out to support with the information about your subscription and ideally the IDs of some messages affected so it can be investigated.

Kamal Aboul-Hosn
  • 15,111
  • 1
  • 34
  • 46