Trying to consume from SQS as fast and reliable as possible using the @SqsListener annotation.
Here is my code:
@SqsListener(value = "${sqs-url}", maxConcurrentMessages = "${maxConcurrentMessages:30}",
maxMessagesPerPoll = "${maxMessagesPerPoll:15}", pollTimeoutSeconds = "${pollTimeoutSeconds:5}")
public void listenEvent(List<Message<String>> messages, BatchAcknowledgement ack) {
var successfullyProcessed = new HashSet<>();
// https://docs.awspring.io/spring-cloud-aws/docs/3.0.0/reference/html/index.html#batchacknowledgement
messages.forEach(message -> {
try {
// ... my logic ....
successfullyProcessed.add(message);
} catch (Exception e) {
log.error("Failed message={}", message, e);
}
});
ack.acknowledge(successfullyProcessed);
log.info("Acknowledged {} messages", successfullyProcessed.size());
}
extra config for manual acknowledge:
//...
// https://docs.awspring.io/spring-cloud-aws/docs/3.0.0/reference/html/index.html#acknowledging-messages
factory.configure(options -> options
.maxDelayBetweenPolls(Duration.ofSeconds(5))
.acknowledgementMode(AcknowledgementMode.MANUAL)
.acknowledgementInterval(Duration.ofSeconds(5))
.acknowledgementThreshold(10)
.acknowledgementOrdering(AcknowledgementOrdering.PARALLEL));
I believe maxConcurrentMessages is the number of threads to run in parallel, but not sure. If it is, Then don't want to have that number to high as I have limited CPU. Problem is that maxMessagesPerPoll cannot be higher than maxConcurrentMessages and therefore I cannot do polls for 100 messages using 8 threads. How can I make this efficient using low maxConcurrentMessages?
