I have a node.js application which has just one route exposed. Lets say it is /myResource. It is deployed in AWS ECS Cluster having 5 instances with good compute capacity (I've monitored these and there is no issue here).
I have an AWS API gateway to which requests from outside are being made and a load balancer in between the ECS Cluster and API Gateway.
It looks something like follows -
API Gateway --> Load Balancer --> ECS Cluster (5 EC2 instances)
Since there is a 30 seconds timeout set on API Gateway this is how I have troubleshooted till now and not able to find any solution.
- Checked the node.js application logs to see if any of the requests is taking more than 30 seconds, in which case it is obvious that API Gateway will wait till 30 secs and throw 504. But that is not the case, all the requests are completed perfectly within 3-5 secs.
- Checked the load balancer logs to see if the load balancer is taking time to schedule the requests to the underlying instances. It is fine too, it is passing the requests to the instances in nanoseconds.
- Now, what is not clear is how the AWS API Gateway behaves in case of concurrent requests?
In the application logs, I can see the requests completing in 3-5 seconds, but between requests there is a delay of more than 30 seconds.
Since application logs are clean, meaning it is not taking more than 3-5 seconds, there is no need to increase the compute capacity. The solution I am trying for now is to increase the instances from 5 to 10 be reducing the compute capacity to half. But thats not the permanent solution.
Any ideas around this?
