Number of deleted messages is twice as many as received messages

Question

I created SQS service with terraform

resource "aws_sqs_queue" "ses_queue" {
  name                       = "ses_queue"
  message_retention_seconds  = 86400
  receive_wait_time_seconds  = 1
  visibility_timeout_seconds = 15
}

resource "aws_lambda_event_source_mapping" "send_email_message" {
  event_source_arn = aws_sqs_queue.ses_queue.arn
  function_name    = aws_lambda_function.send_email_message.function_name
  batch_size       = 5
}

I am sending emails using lambda function

for record in event.get("Records"):
    receipt_handle = record.get("receiptHandle", "")
    request_body = record.get("body")
    response = send_email(request_body)
    if response:
        sqs_client.delete_message(QueueUrl=constants.SES_QUEUE_URL, ReceiptHandle=receipt_handle)

I am wondering why number of deleted messages is twice as many as received messages

just wondering whether it's because you delete the message within lambda and also because SQS will delete the message automatically after lambda returns? — brushtakopo, Jan 05 '23 at 15:19

score 2 · Answer 1 · answered Jan 05 '23 at 15:31

The Lambda Event Source Mapping already deletes the messages from the Queue if your Lambda Function terminates without errors:

If your function successfully processes the batch, Lambda deletes the messages from the queue. — Source

In your Function Code you explicitly delete the messages you processed as well, this means the delete happens twice.

score 0 · Accepted Answer · answered Jan 09 '23 at 10:19

Fixed it with reporting batch item failures

terraform

resource "aws_lambda_event_source_mapping" "send_email_message" {
  event_source_arn         = aws_sqs_queue.ses_queue.arn
  function_name            = aws_lambda_function.send_email_message.function_name
  batch_size               = 5
  function_response_types  = ["ReportBatchItemFailures"]
}

lambda

reprocess_messages = []
for record in event.get("Records"):
    receipt_handle = record.get("receiptHandle", "")
    message_id = record.get("messageId", "")
    request_body = record.get("body")
    response = send_email(request_body)
    if not response:
        reprocess_messages.append({"itemIdentifier": message_id})

return {"batchItemFailures": reprocess_messages}

Number of deleted messages is twice as many as received messages

2 Answers2