0

Assume we face x requests per seconds, where x is magic number of requests which single load balancer cannot handle.

So we have to use multiple load balancers to handle all requests, yes? But how? I need to load balance the load balancers farm? Poor idea, at the end it also occurs that single load balancer must handle all requests.

So how to use multiple load balancers to handle enormous number of requests?

Szyszka947
  • 111
  • 3
  • There are other load balancing solutions than dedicated load balancers which cannot be cascaded. However your question is quite unclear and needs further details. – Zac67 Jul 21 '23 at 10:03
  • You can utilize Equal-cost multi-path routing (ECMP) to send traffic to multiple load balancers. See for example [booking.com load balancing architecture](https://www.haproxy.com/user-spotlight-series/scaling-the-edge-how-booking-com-powers-a-global-application-delivery-network-with-haproxy) and [github architecture](https://www.haproxy.com/user-spotlight-series/inside-the-github-load-balancer) – AlexD Jul 21 '23 at 10:11
  • 3
    Your false assumption seems to be that all traffic, all requests must first reach a single destination before requests can be distributed evenly (by a loadbalancer). That isn't the case in properly designed solution. You can leverage for example the DNS layer to ensure that the visitors to `www.example.com` will see different IP-address based on their geographic location. And a smart DNS will ensure that vistors from a specific will will be directed to the `www.example.com` instance hosted behind nearby IP-address X while other vistors will see IP-address Y for `www.example.com`. – HBruijn Jul 21 '23 at 10:55
  • 1
    Many other techniques, based on for example routing will do the same: ensure that all traffic does not end up in a single destination. Of course that takes a properly decentralised application design. – HBruijn Jul 21 '23 at 10:56

1 Answers1

4

Instead of passing all the traffic through a single load balancer you can load balance on the network level utilizing Equal-cost multi-path routing (ECMP).

There are two presentations from HAProxy Conf 2019 which describe load balancing architecture of Booking.com and GitHub

Booking.com ECMP architecture

GitHub GLB

AlexD
  • 8,747
  • 2
  • 29
  • 38
  • or just use rrDNS – symcbean Jul 21 '23 at 12:54
  • @symcbean round-robin DNS isn't very good for HA. – AlexD Jul 21 '23 at 13:18
  • Perhaps, but installing an BigIp f5 on the local network of everyone who might visit your site is a little expensive. Also ECMP becomes very difficult to implement across multiple sites/multiple proviers. – symcbean Jul 21 '23 at 13:20
  • ....and I wouldn't expect someone operating on the scale of booking.com or github to base their planning of a new infrastructure solely on discussion like this. – symcbean Jul 21 '23 at 13:23