I am extracting a long running data analysis routine out into a microservice and have currently modelled it having:
- a Request channel to receive the data processing request
- a data channel to return periodic snapshots of the analysis so far (the result will get refined over time)
- Completion channel to send a message when the processing has finished/reached its execution timeout
- Error channel for any error scenarios (Spring Integration is used in both the calling process and the microservice with a message broker in between)
The calling process needs to execute additional processes after the the data processing has completed (received a message on the completion channel) and it has received all the data snapshots on the data channel.
What is the best strategy for waiting before continuing the processing on the calling service after receiving the completion message and somehow knowing that all the data messages have been received? I am assuming that as it is asynchronous that it is possible for the completion message to arrive before the final data message(s) have been received and processed.
