-1

Current-approach: For Spring Boot app, in AWS environment:

  1. The service-layer spawns 5 threads, using ExecutorService --> Callable-Future for concurrent processing.
  2. On completion, each callable-future task, returns a response.
  3. The 5 responses, from the 5 threads, are received by a single calling class using future.get() and are then analyzed further.
    All this works fine.

Questions:
In AWS, how can AWS-SQS/SNS be used to implement the above ?

Issue using SQS: Per My Analysis
A) SNS or SQS can process a message, but cannot respond with a value.
B) The only way it can send back a response is by posting the response, to another queue or using a callback
C) With the 5 separate callbacks, it is difficult to combine the 5 responses for further analysis .

Any thoughts please, on whether, technically, there is a way to use SQS or even Kafka for this?

Additional-Info:

  1. Threadpool is set to 15 for testing
  2. Each thread takes around 1 sec to 3 sec to process
  3. Frequency is 30-API calls a minute thanks

1 Answers1

0

SNS and SQS are queue systems so you can use them but Is it necessary for this case I don't know details of your structure.

Also how can you wait queue systems you need to wait a thread for each request and retrieve it and process it.

My opinion is if you are in heavy load or you don't want to lose your data you can use them because they are async.

SQS has order and SNS is simple queue which do not hold order. You can wait for 5 messages to be consumed and then process it.

IMHO : If you don't have a problem with performance and other stuff callables are enough for this implementation.

Question:

1- Are you using thread pool?

2- How many process do you need to perform?

Gurkan İlleez
  • 1,503
  • 1
  • 10
  • 12
  • thanks Gurkan, - THe order of the callback would not be reliable, as some threads can take some more time. - API call itself is ASync and system could receive upto 30 APIcalls a minute - Each API call results in 5 threads. – Prashant-M Nov 01 '21 at 13:13