SQS FIFO Listener Receives Messages Before Previous One Finished

Question

I have a spring-boot application with aws sqs fifo, which need to process messages in fifo manner. Lets say there are 3 messages (m1,m2,m3) under same group id in fifo queue. order of execution should be m1,m2 and m3.

m2 should wait until m1(take more time than m2 and m3 to execute) finish it execution totally and m3 should wait until both m1 and m2 finish their execution totally.

sqs client class :

import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.WebIdentityTokenCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;

@Configuration
public class AmazonSQSConfiguration {
@Primary
@Bean("sqsClient")
public AmazonSQSAsync buildAmazonSQSAsync() {
    return AmazonSQSAsyncClientBuilder
            .standard()
            .withRegion(Regions.AP_SOUTHEAST_1)
            .withCredentials(awsCredentialsProvider())
            .build();
}

public AWSCredentialsProvider awsCredentialsProvider() {
    return WebIdentityTokenCredentialsProvider.builder()
            .roleSessionName("test-order")
            .build();

}

Dependencies,

  <!--aws related dependencies-->
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-aws</artifactId>
        <version>2.2.6.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-aws-messaging</artifactId>
        <version>2.2.6.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.12.161</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-configuration-processor</artifactId>
        <optional>true</optional>
    </dependency>

Since I want to process m2 and m3 after m1 totally process,(which means not right after m1 received from listener, m2 should wait until m1 totally processed) What I have done is setting deletion Policy to never and manually remove m1 from queue once it finished it execution. So m2 will get eligible to fetch from my application. As per the AWS doc, messages behind prior message are not consuming until prior message remove from the group. so my lister looks like this,

 @SqsListener(value = "${queue.name}", deletionPolicy = SqsMessageDeletionPolicy.NEVER )
 public void processQueue(@Headers Map<String, String> header, @Payload String message)

But It seems like right after m1 received, m2 and m3 also receiving(may be in fifo order too). Since m1 is heavy weight task, before m1 finished, m2 and m3 getting completed. This totally violating my requirement to use fifo queues.

my logs confirm that,

message	publish time	listen time
m1	August 19, 2022 3:16:44.131 PM	August 19, 2022 3:26:40.684 PM
m2	August 19, 2022 3:16:44.137 PM	August 19, 2022 3:26:40.684 PM
m3	August 19, 2022 3:16:44.139 PM	August 19, 2022 3:26:40.686 PM

Normally m1 takes 5 seconds to execute. but it seems like m2 is also fetching in same time.

SQS configurations, I really don't understand what's wrong. Please give suggestions?

cliffdevs · Answer 1 · 2022-10-31T19:22:43.217

I also recently ran into this. There's a bug in handling fifo messages in order by groupId within a batch that was fixed in later versions of the library: https://github.com/awspring/spring-cloud-aws/pull/40

From the PR description of the issue being resolved:

For FIFO queues the AsynchronousMessageListener groups messages with same messageGroupId into so called MessageGroups. The MessageExecutor (renamed to MessageGroupExecutor) handles the messages within those groups sequentially.

Messages from non-FIFO queues are handled as before with the only difference that they are also wrapped in a MessageGroup. Each separate message belongs to its own MessageGroup.

It looks like you'll need to consume at least version 2.3.x for spring-cloud-aws-messaging to get these changes, which sadly will result in some compatibility issues with your current spring boot versions. Good luck!

Thanks for the comment. please refer my answer as well. – kevin_Aberathna Nov 03 '22 at 09:58 — kevin_Aberathna, Nov 03 '22 at 09:58

score 0 · Answer 2 · answered Nov 03 '22 at 09:58

I found the issue in my case. I just have to add below bean to application.

  @Bean
    public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync sqsClient) {
        SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
        factory.setAmazonSqs(sqsClient);
        factory.setAutoStartup(true);
        factory.setMaxNumberOfMessages(1);
        return factory;
    }

We have to set MaxNumberOfMessages to 1.

SQS FIFO Listener Receives Messages Before Previous One Finished

2 Answers2