I've been trying to read up on the Asynchronous Request-Response pattern, and just wanted to hear some thoughts on when we should choose this pattern over some other communication pattern (eg. synchronous request-response, event-driven, etc.). Let me start with an example.
Example
Suppose you have a FileUploadService which allows Clients to upload files. If the Client uploads a video file, you will first need to compress the video before uploading the file. As part of the file upload process, you will need to maintain a DB record of the file that is being uploaded. There are 2 approaches you can take:
Approach 1: Asynchronous Request-response
Approach 2: Event-driven?
My Thoughts
In general, I feel that Approach 1 (Asynchronous Request-response) is "better" than Approach 2 for the following reasons:
- In some scenarios, the
VideoCompressionServiceitself may not want to expose an API and may prefer to only consume from a queue. Why? Imagine if they exposed an API, and Clients suddenly send a surge of requests - their service might crash due to the resources (eg. CPU) being maxed out. Although this can possibly be mitigated by having a bounded concurrency pool so that only a limited number of requests can be handled at any time, I feel that having a queue is still generally a better approach - the rate of consumption can be controlled, and the queue can act as a buffer so that messages are not dropped. - If the video compression takes very long (eg. 1 day), it may not be wise to keep an open network connection between the
FileUploadServiceConsumerand theVideoCompressionServicefor such a long time (since the number of open network connections is limited, at some point you may reach this limit). Since you don’t want to keep an open connection indefinitely, you will need to set some kind of timeout for that connection. However, it can be difficult to set an appropriate timeout if the time taken for the video compression varies between requests (eg. a 1 hour video takes 10 minutes, while a 24 hour video takes 6 hours) - too short and many of your requests may fail, too long and you may end up with too many open connections.
Questions
- Is my argument for why Approach 1 is better than Approach 2 valid? Are there any other reasons why you would choose Approach 1 over Approach 2 (or the other way round)?
- Are there any other better approaches that could have been used to implement this system?
- What are some other real-life examples of problems you have faced where Asynchronous Request-response was the only sensible solution (such that it simply did not make sense to use event-driven approaches or synchronous request-response to solve the problem)?
