33

I'm trying to use the AWS SQS FIFO service together with an Elastic Beanstalk worker environment.

Let's say I send a message with MessageDeduplicationId test, if I continue sending this exact message in the next 5 minutes, the message will be ignored, correct?

What happens if I send a message with MessageDeduplicationId test , the consumer processes the message and deletes it, then, in about 1 minute, I send the exact same message again. Will this message be ignored?

My question is, does deduplication occur as long as the same MessageDeduplicationId is still in queue/flight? Or is the id banner forever, no other message with the same id can be sent.

Thanks.

BinaryButterfly
  • 18,137
  • 13
  • 50
  • 91
tehmaestro
  • 1,010
  • 2
  • 11
  • 21

2 Answers2

20

What happens if I send message with MessageDeduplicationId test , the consumer processes the message and the deletes it, then, in about 1 minute, I send the exact same message again. Will this message be ignored?

Answer seems to be: Yes

Amazon SQS continues to keep track of the message deduplication ID even after the message is received and deleted.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html

antak
  • 19,481
  • 9
  • 72
  • 80
  • 1
    I can confirm this. In a 5 minute interval, it will ignore the message stack completely. (First paragraph of the shared link) – Jose A Apr 10 '20 at 02:37
9

I continue sending this exact messages in the next 5 minutes, the message will be ignored, correct?

Correct. Any longer than 5 minutes, though, and it will be accepted again and delivered again and then ignored for another 5 minutes.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427
  • 2
    Ok, but what if the message is processed and deleted from the queue in, lets say 5 seconds after it was sent. What happens if I send the same deduplication id in the next 4 minutes and 55 seconds? – tehmaestro Dec 05 '17 at 07:47
  • 11
    The message would be accepted by SQS and then discarded because the purpose of the deduplication ID is to help ensure "exactly once" delivery -- to allow you to safely resubmit a message to the queue if you are unsure whether you submitted it without error, before. Example: you submit a message to the queue and while the `200 OK` response is on the wire on its way back to you, you lose your Internet link. The queue has the message, but you have no way to confirm that, as a producer. So, you submit it again, knowing it will only be delivered once. – Michael - sqlbot Dec 05 '17 at 16:18