How to test localstack with jest? (trigger sqs messages asynchronously)

Question

We have a Nestjs app using Localstack to develop locally sending emails with SES, and SQS. We are writing an e2e test, which basically sends a message to queue A, SES receives the message, publishes it, and sends an event (like delivered, bounced) to another queue B.

We are struggling to model this in a jest test, since they are two separate events. It's not that one is a callback of the other. They are two separate queues, so we have the problem that if we ask for a message in queue B, it can be that message is still not there.

We manage to make it work using a delay(1000), but this makes our test flaky. Here is the code.

This is basically a helper for the test, it uses the aws-sdk v2:

import * as AWS from 'aws-sdk';

export class SqsHelper {
  sqs: AWS.SQS;

  constructor() {
    this.sqs = new AWS.SQS({
      region: configuration.aws.region,
      endpoint: configuration.localstackUrl,
    });
  }

and here is the actual publishing of the message to a queue:

  async sendMessageToRequestEmailQueue(
    emailAddress: string,
    request: SendEmailDTO,
  ): Promise<void> {
    const messageData = { ...request };
    messageData.sender.address = emailAddress;
    messageData.identifier = faker.datatype.uuid();

    await this.sqs
      .sendMessage({
        MessageBody: JSON.stringify(messageData),
        QueueUrl: `${configuration.localstackUrl}/000000000000/email_delivery_request`,
      })
      .promise();

    return delay(1000); //needs to wait for the email to be sent
  }

The delay method is just a setTimeout within a promise:

const delay = (ms: number) => {
  return new Promise<void>((resolve, _reject) => {
    setTimeout(() => {
      resolve();
    }, ms);
  });
};

export default delay;

So we had to use the delay(1000) in our tests, otherwise test don't pass.

This is how the test looks like:

First we instantiate the helper

sqs = new SqsHelper();

and then we use the methods that trigger the calls to localstack:

it('publishes a "DELIVERED" status change if the message is successfully delivered', async () => {
    await sqs.sendMessageToRequestEmailQueue(emailAddress, standardEmailDTO);

    const messages = await sqs.consumeMessageFromQueue(queueUrl);

    const message = messages.Messages[0];

    expect(message.Body).toContain('DELIVERED');
  });

the consumeMessageFromQueue(queueUrl) it's within the SqsHelper class, it's just another promise retrieving the messages in queue B:

  async consumeMessageFromQueue(
    queueUrl: string,
  ): Promise<PromiseResult<ReceiveMessageResult, AWS.AWSError>> {
    console.log('### 2');
    const params: AWS.SQS.ReceiveMessageRequest = { QueueUrl: queueUrl };
    return this.sqs.receiveMessage(params).promise();
  }

This works, the messages.Messages array exists and it is populated.

The problem arises when we get rid of the delay(1000), test breaks:

async sendMessageToRequestEmailQueue(
    emailAddress: string,
    request: SendEmailDTO,
  ): Promise<PromiseResult<SendMessageResult, AWS.AWSError>> {
    const messageData = { ...request };
    messageData.sender.address = emailAddress;
    messageData.identifier = faker.datatype.uuid();

    return this.sqs
      .sendMessage({
        MessageBody: JSON.stringify(messageData),
        QueueUrl: `${configuration.localstackUrl}/000000000000/email_delivery_request`,
      })
      .promise();
  }

The test fails, the messages are not there. Since messages are there when using delay(1000), we suspect it has to do with the event loop and jest. We tried several options:

a) use the delay(1000) in the test before retrieving messages (test don't pass):

it('publishes a "DELIVERED" status change if the message is successfully delivered', async () => {
    await sqs.sendMessageToRequestEmailQueue(emailAddress, standardEmailDTO);

    delay(1000);
    const messages = await sqs.consumeMessageFromQueue(queueUrl);

    const message = messages.Messages[0];

    expect(message.Body).toContain('DELIVERED');
  });

Here comes a big question, why the delay in the code actually passes the test, but not within the test? what is the difference in the event loop / stack queue world?

b) we also tried to use this library: wait-for-expect

it.only('publishes a "DELIVERED" status change if the message is successfully delivered', async () => {
    await sqs.sendMessageToRequestEmailQueue(emailAddress, standardEmailDTO);

    console.log('### waitForExpect', waitForExpect);
    await waitForExpect(async () => {
      const messages = await sqs.consumeMessageFromQueue(queueUrl);
      console.log('### messages', messages);

      if (messages.Messages) {
        const message = messages.Messages[0];
        expect(message.Body).toContain('DELIVERED');
      }
    });
  });

But messages are not there. We don't understand why this library doesn't work. It should keep trying until expectation it's fulfilled. The problem is, it times out without receiving the messages.

c) we tried other options like flushing promises etc.

d) we tried to chain promises too:

it.only('publishes a "DELIVERED" status change if the message is successfully delivered', async () => {
    const promise = sqs.sendMessageToRequestEmailQueue(
      emailAddress,
      standardEmailDTO,
    );

    promise.then(async (data) => {
      console.log('### c data', data);
      const messages = await sqs.consumeMessageFromQueue(queueUrl);
      console.log('### messages', messages);

      if (messages.Messages) {
        const message = messages.Messages[0];
        expect(message.Body).toContain('DELIVERED');
      }
    });
  });

but console.log('### messages', messages) never appears in the test.

We suspect that jest is not waiting for the promises to be resolved. It only does so when we use the delay(1000) in the actual code, but we can not understand why, and how to solve this issue, we can not rely on flaky tests.

How to test localstack with jest? (trigger sqs messages asynchronously)

0 Answers0