2

I'm working on improve our infrastructure reliability in AWS and one of my tasks is enabling cross-zone balancing on our ELBs. Our ELBs receives around 20K request by second on rush hours as well as all our instances are in us-east-1d. I've followed the below steps:

  1. Modified the ASG to start creating instances in us-east-1c and duplicating the desired capacity
  2. Wait until the new instances created be fully provisioned
  3. Enable us-east-1c zone in ELB.

When I perform the step 3 receive a spike of Sum ELB 5XXs errors.

I guess that these errors are related to that our recently escalated servers are "cold". There are a way to start sending request to them incrementally?

Thanks.

Jose Peinado
  • 35
  • 4

1 Answers1

1

When it comes to pre-warming your load balancer, AWS does not come with any built-in feature for that. They suggest that you do your own "load test" script that increase traffic incrementally OR to contact them and they will have it pre-warmed for you.

When you add another availability zone that your load balancer should include, I believe that the whole load balancer is replaced with a new one behind the scenes (at least that's what happens when you change the AvailabilityZones property through CloudFormation). That means that the newly created load balancer is not warmed up and will need a couple of minutes to scale up.

My suggestion is that you:

  1. Setup a completely new load balancer
  2. Configure it as your old one but with the new availability zone added
  3. Ask Amazon to pre-warm it for you
  4. Make the DNS switch when it's as "hot" as possible
Bazze
  • 1,531
  • 10
  • 11
  • thanks for your response. I guess that pre-warming is not an option since we weren't expecting traffic spikes on our ELB, the traffic spike will go to all servers in the other AZ (us-east-1c). I wanna know how to pre-warm these group of servers or just how to bring traffic to them incrementally. – Jose Peinado Jan 22 '15 at 16:38
  • @JosePeinado: AWS doesn't come with any built in feature for incrementally increasing traffic to the new availability zone you added to the load balancer. When you add another availability zone to an ELB, that ELB is replaced with a new one which isn't as "warm" as the ELB currently in service (which is most likely causing your 5XX errors). My suggestion above should hopefully spare you any glitches migrating to an ELB that also serves the us-east-1c AZ. – Bazze Jan 22 '15 at 19:45
  • you're right, I'll follow up your suggestion, thanks so much :) – Jose Peinado Jan 24 '15 at 20:48