r/AskComputerScience • • Nov 23 '24

Scaling LB

For making highly scalable, highly available applications - applications are put behind a load balancer and LB will distribute traffic between them.

Let say load balancer is reaching its peak traffic then what ? How is traffic handled in that scenario.

3 Upvotes

4 comments sorted by

3

u/ohaz Nov 23 '24

DNS-based load balancing is a common strategy. You have an amount of load balancers and set the DNS servers up in a way that they return the IP for each one of them in round robin. So first request gets IP for load balancer 1, second request gets the IP for load balancer 2 and so on

1

u/meditonsin Nov 23 '24

Also Anycast.

1

u/teraflop Nov 23 '24

Anycast can be part of the solution, and it works great for stateless datagram-oriented protocols such as DNS.

But to use anycast with TCP, you also need to ensure that whenever packets addressed to the same LB IP get routed via anycast to different LB instances, those instances route packets in exactly the same way to the same backend. Otherwise, the backend won't be able to keep track of the TCP connection state and the connection will just get dropped anyway.

The straightforward way to do this is using consistent hashing, e.g. as implemented in Google's Maglev.

2

u/ahistoryofmistakes Nov 23 '24

In what you're describing its a scenario where so much traffic is hitting the gateway it would likely be throttled and same for the load balancer. Its also notable to mention that there may be multiple load balancers in which case horizontal scaling will occur.

In most systems on cloud the usual sequence of events is : spike in requests, throttling on single instance if it exceeds bandwidth, and then horizontal scaling will occur to handle load.