We have a requirement in our project where there are list of long running tasks to be performed. We have added this list of tasks to a message queue(SQS).
Now once we start polling messages from the SQS, each message(task) involves updating huge list of database records, we can say that this huge rows of database updates are specific to the polled message.
A pictorial representation is given
M1
/ | \
R1 R2 R3.....
Here M1 is the message polled for which R1,R2,R3.... are the list of database rows that are to be updated.
Now the question is how to tackle this huge list of database row updates for a given message. Do we have to 1)Create ExecutorService(we are using Java for backend), create multiple threads for reading the list of records and update them to the database back again or 2)Just push the row updates as separate messages back to the message queue.
In either approaches, which one is preferred and what would be an elegant scalable solution to handle this problem
