1

According to https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting they recommend to not process more records if you fail to process a record because it will preserve the order of the messages in an FIFO queue.

But does that mean I should add all subsequent MessageIds (the ones Im skipping) to the BatchItemFailures-collection?

taracus
  • 393
  • 1
  • 5
  • 19

1 Answers1

2

Yes, if you are skipping any messages, you should return their IDs. Any messages passed into your function that you did not successfully processed should be returned in BatchItemFailures.

AWS doesn't know that you wrote some code in your Lambda function to skip those messages. They are just making a recommendation that you write that code. The only way AWS knows you didn't successfully process those messages is if your function returns all those IDs you didn't process.

Mark B
  • 183,023
  • 24
  • 297
  • 295
  • I know Amazon recommend this too, but it confuses me. If you mark messages as 'failed' just because they were unfortunate enough to part of a batch that partially failed, then isn't there a risk of them getting dead-lettered despite them not being a problem? – Barguast Feb 10 '23 at 11:40
  • @Barguast If you don't do that, then they get marked as "successful" even though they were never processed at all, and they just disappear forever. Isn't that worse than going into a DLQ? – Mark B Feb 10 '23 at 16:06
  • Absolutely, but it feels like their ought to be a way to mark a batch as successful up to a certain point so whatever is interpreting it can distinguish between 'this message failed' [retry count + 1] and 'batch aborted before this message wasn't handled' [retry count is unaffected]. – Barguast Feb 11 '23 at 19:36