We have an application running with two pods , if each pod is running in each nodes then we are facing intermittent timeouts (tried with ALB and also on the node itself) , if the both pods are running on single node then we don't face any issue.
Detailed Scenario:
- We are using EKS Cluster with 2 Nodes
- Using Calico over EKS
Use Case: Everything is working fine
- Hello-word is running on 2 pods (A & B), both pods are running on Node1
- Curl from ALB - 200
- All good
Use Case: 504 timeout
- Hello-word is running on 2 pods (A & B), now pod A is running on Node1 and pod B is running on Node2
- Curl from anywhere - every alternate req 504
- Curl from inside container all 200
Use Case: Everything is working fine
- Hello- word is running with pod1 , all working fine
Summary: App is not working only when 2 pods are running on diff diff nodes