How to block AWS SQS FIFO while having a message in the deadletter queue?

Question

Imagine the following lifetime of an Order.

Order is Paid

Order is Approved

Order is Completed

We chose to use an SQS FIFO to ensure all these messages are processed in the order they are produced, to avoid for example changing the status of an order to Approved only after it was Paid and not after has been Completed.

But let's say that there is an error while trying to Approve an order, and after several attempts the message will be moved to the Deadletter queue.

The problem we noticed is the subsequent message, that is "Order is completed", it is processed, even though the previous message, "Approved", it is in the deadletter queue.

How we should handle this?

Should we check the contents of deadletter queue for having messages with the same MessageGroupID as the consuming one, assuming we could do this?

Is there a mechanism that we are missing?

Why would `"Order is completed"` msg be even produced if `Order is Approved` action hasn't been successfully completed? — Marcin, Jul 12 '22 at 10:04
@Marcin Basically we are using sqs FIFO with dlq , we notice that if there's any failure in any message of the three messages the sqs will send the message that have that failure to the dlq and the sqs will continue with the other two message so imagine we have three message " Order is Paid" "Order is Approved" "Order is Completed" and the failure happens in the first one and then it is sent to the dlq the sqs will continue with the other two but what I want to do is to block these two messages that belong to the same group id? is there's any way or any workaround to do so? — Mikhail Fayez, Jul 12 '22 at 11:38
There is nothing SQS can do here. You need to fix your flow. Either there is some bug in the flow or the flow is not correct; therefore invalid state transition is happening. I didn't understand the flow but you can implement the state transition validation and reject the other messages as well, so all the messages of that group will go to DLQ and then handel — Amolpskamble, Jul 13 '22 at 12:42
@MikhailFayez instead of adding the three messages to the queue, add just the first one `Order is Paid`. If it succeeds, make the `orderIsPaid` handler itself add a new message to the queue `Order Is Approved`, then if this one also succeeds, make it add the last message `Order Is Completed`. You don't need to add all messages ahead of time, add them as your flow succeeds. — Alisson Reinaldo Silva, Aug 16 '22 at 15:56

score 1 · Answer 1 · answered Aug 16 '22 at 16:16

You don't have to block the queue, but rather only add messages when they are good to be processed. If you have a flow in which certain steps (messages to be processed) depend on the success of previous steps, you should make these messages only be added after the previous step (message) succeeds.

Let's say you have an SQS queue with a handler to process the message, it could look something like below.

Since you're using the same FIFO queue for all steps, I'm using the STEP as the MessageGroupId to allow messages of different steps to be processed in parallel (as they could belong to different orders), but the steps of one particular order are always processed in sequence and require the previous one to succeed.

On a side note, you shouldn't need FIFO queues with the approach below, and you could have separate queues with separate handlers for each step.

const sqs = new AWS.SQS();

const STEPS = {
  ORDER_PAID: "ORDER_PAID",
  ORDER_APPROVED: "ORDER_APPROVED",
  ORDER_COMPLETED: "ORDER_COMPLETED",
};

async function sendMessage(step: string, orderId: string) {
  return sqs
    .sendMessage({
      QueueUrl: process.env.YOUR_QUEUE_URL || "",
      MessageGroupId: step,
      MessageBody: JSON.stringify({
        currentStep: step,
        orderId,
      }),
    })
    .promise();
}

exports.handler = async function (event: any, context: any) {
  for (const message of event.Records) {
    const { currentStep, orderId } = JSON.parse(message.body);
    if (currentStep === STEPS.ORDER_PAID) {
      // process order paid, then add next step back to queue
      await sendMessage(STEPS.ORDER_APPROVED, orderId);
    }
    if (currentStep === STEPS.ORDER_APPROVED) {
      // process order approved, then add next step back to queue
      await sendMessage(STEPS.ORDER_COMPLETED, orderId);
    }
    if (currentStep === STEPS.ORDER_COMPLETED) {
      // process order completed
    }
  }
  return context.logStreamName;
};

How to block AWS SQS FIFO while having a message in the deadletter queue?

1 Answers1