3

For various reasons my consumer on AWS sometimes reads a few messages from an SQS queue and decides to put some of them back in the queue to be processed later.

The way I do that is by setting their VisibilityTimeout to 0 which makes them immediately visible to other consumers. This is documented here.

The problem is that after doing that a few times, the message's ReceiveCount reaches the maxReceiveCount which causes the message to be moved to the DLQ. I'm wondering if I can somehow reset the message's ReceiveCount to avoid that.

The only option I can currently think of is to just send a copy of the message back to the beginning of the queue and deleting the original message.

user972014
  • 3,296
  • 6
  • 49
  • 89
  • i think something is missing with your request, can you explain why you want to push back the messages? – Liran May 30 '21 at 15:31

2 Answers2

3

This all depends on your workload. It's not uncommon to have a lambda on your DLQ that will write messages back into the primary queue for retry. However, you often transform the message a little and add (for instance) a key to the json packet like dlq_retry_count. Then your DLQ lambda can retry it if below a certain threshold or delete the message.

If you just keep taking DLQ messages and pumping them back into the queue with no means of tracking you could end up with a queue full of poison pill messages.

3

I'm wondering if I can somehow reset the message's ReceiveCount to avoid that.

Sadly, you can't reset it as you its automatically managed by SQS. You have no control over it.

If you don't want to use multiple SQS, one for different purpose and consumer, the only solution I can think of is to simply increase maxReceiveCount to account for those extra reads. So if you know that your messages will be read and returned to the queue, let say 5 times, then increase your maxReceiveCount to 8 for instances. This will mean that if the message is read 3 times more then expected, there is something wrong with it.

Marcin
  • 215,873
  • 14
  • 235
  • 294