According to Implementing partial batch responses AWS doc:
If you're using this feature with a FIFO queue, your function should stop processing messages after the first failure and return all failed and unprocessed messages in batchItemFailures. This helps preserve the ordering of messages in your queue.
I am processing a SQS FIFO queue following AWS documentation mentioned above. My batch size is 10. Since the queue is not high-scale, the batch often contains messages with different message group IDs. When a failure occurs, the rest of the processing is stopped and all records left are returned as failures to the queue. So I often end-up with valid records in my dead-letter queue just because they were processed in a batch containing an unrelated invalid one.
Would it be valid, after a failure, to return all remaining records in the batch but only the ones with the same group ID? According to the doc, current recommendation is to preserve the ordering of messages in your queue. I am not seeing why processing other records with a different group ID would go against that.
I posted this question in AWS's powertools-lambda-python repository, since I am using their implementation SqsFifoPartialProcessor. This implementation strictly follows AWS documentation.
They did not have knowledge of whether I could override the current behaviour to solve my issue.
