13

I have an ELB and 3 nodes behind it.

Can someone please explain me what will ELB do in these scenarios:

  1. Client Request -> ELB -> Node1 fails in the middle of the request (ELB timeout)
  2. Client Request -> ELB -> Node1 timeouts (Server timeout and health check haven't kicked in yet)

Particularly I'm wondering if ELB retries the request to another node?

I made a test and it doesn't seem to, but maybe there's a setting that I've missed.

Thanks,

sparrovv
  • 6,894
  • 2
  • 29
  • 34
  • 4
    nope. the ELB will not retry. the ELB keeps tabs on what servers are health/number of conns and passed the data back and forth. it does not have any retry logic. – Mircea Aug 04 '15 at 23:24

2 Answers2

13

This may have been a matter of passage of time, but these days ELBs do retry requests that abort:

  • Either because of an Idle Timeout (60s by default);
  • Or because the instance went unhealthy due to failing health checks, with Connection Draining disabled (default is enabled)

However, this holds only if you haven't sent any response bytes yet. If you have sent incomplete headers, you will get a 408 Request Timeout. If you have sent full headers (but didn't send a body or got halfway through the body) the client will get the incomplete response as-is.

The experiments I've performed were all with a single HTTP request per connection. If you use Keep-Alive connections, the behavior might be different.

rix0rrr
  • 9,856
  • 5
  • 45
  • 48
  • 2
    I am trying to find links to the AWS docs that confirm whether ELBs retry requests which fail from Idle timeouts. Would you be able to provide this source @rix0rrr – tr_quest Apr 13 '21 at 00:59
  • We're seeing some evidence that suggests the same. In our case it's using a NLB and related to pod rotation during a K8s deployment. Requests don't fail, but take longer than 60s to succeed. – Federico Jun 28 '22 at 06:43
11

The AWS Elastic Load Balancing service uses Health Checks to identify healthy/unhealthy Amazon EC2 instances. If an instance is marked as Unhealthy, then no new traffic will be sent to that server. Once it is identified as Heathy, traffic will once again be sent to the instance.

If a request is sent to an instance and no response is received (either because the app fails or a timeout is triggered), the request will not be resent nor sent to another server. It would be up to the originator (eg a user or an app) to resend the request.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470