Load balance the Load balancer

Question

I am looking to get load balancing for my load balancer.

We are looking to create a fully HA network. All of our servers are redundant, the databases are redundant, but above them is a single server running nginx as a reverse proxy to distribute requests.

I'm looking for a way to eliminate this danger hotspot, even though the others are all redundant, if this server goes down, no one will receive requests anymore.

And I'm here to get your help on this.

I have found two ways to solve this problem:

• FloatingIP: Create a load balancer cluster with Kubernetes = I have a unique cluster IP address that allows me to talk to all nodes. I've done this solution before, so I have a cluster of loadbalancers all running the same nginx service, but our router doesn't allow me to write down the target IP address to send requests to, I can only choose a device. Since the cluster IP address is not attached to a physical machine, I can't use it in our router.

• Find a router that directly handles sending a request to multiple servers (like the upstream block in nginx).

I really think that the Floating IP is the best option, but I can’t understand why my router cannot use it, so if you have some name of router capable to use Floating IP I would love it.

For several weeks I've been looking for a router that can do both, or find another solution.

Any help would be welcome

Answer 1

For the sake of explanation let's define the term High Availability which means that we have guarantees that there is always some system that will handle the load. Usually it is implemented by having redundant systems, but it doesn't mean that the load will be distributed between them; it is possible that at every moment of time the single system will process all the load. Load Balancing is a special case of High Availability where the load is distributed between redundant systems, so that could potentially increase the capacity of the service by adding the capacity of all systems in the set.

HAproxy manual states that load balancer typically can outperform around 1000 of application servers it balances (being run on the same hardware), so you don't really have the need to balance it. With that gap in the capacity of the balancer and the service it is hardly useful to load balance the balancer, but that will still increase the complexity of the solution. You'll increase the complexity (increase the cost of the solution and lose some reliability) with no apparent gain.

Usually you balance the load of the servers by making the balancer to know their current load to make educated guesses where to send the next request. The balancer itself is simply made highly available by making it redundant. The floating IP gets assigned to currently performing balancer ("primary") system and if it fails it gets reassigned to the other system ("standby") quickly. All the other time the standby system stands still doing nothing. I understand that the whole system running but doing nothing appears wasteful, but that's not the waste, that's your guarantee that the possible interruption of the service will end very shortly. You may balance some other service on this pair with their roles reversed, so the redundancy will not appear that wasteful, but notice that during outage event both services will be balanced by the same system and therefore it needs the capacity to run them both or both services will degrade, so the "waste" is inevitable. You can also have some randomized load distribution by having two floating IP for the same service and publishing them both in the DNS for the same name, so each client will choose some balancer randomly. Usually you have two balancers and you can manage the floating IPs with the keepalived (which uses VRRP); this is simplest approach with very little setup.

You may also have some static distribution of the load to the balancers by using Netfilter's CLUSTERIP target. In this setup all systems have the same IP address assigned and it's associated with some broadcast MAC address, so all the systems see all the traffic. However, each system only selects a share of the traffic to process and ignores everything else; the selection is made so each request is assigned to one and only one processor. The assignment is made solely on the basis of the source of the request, not making into account the load of systems, so the distribution is "blind". The distribution appears pseudorandom and the it is possible that only one node gets all the load. The algorithm behind it requires that all the participating systems know how many of them are currently online (and each system has an unique individual number) so no request could possibly being lost or processed by more than one system. If any of those clustered processors fails, all other alive systems must reconfigure themselves immediately to account the loss of one processor, otherwise no one of them will consider itself as a valid processor of requests that were previously worked on by that failed system. The Linux's Pacemaker includes the resource agent to use this CLUSTERIP target properly (including the reconfiguration). This configuration is much more complex and therefore it is more brittle and needs much more maintenance and monitoring. It requires Pacemaker which is not very easy to set up, and a cluster messaging layer e.g. Corosync, also proper Pacemaker-based setup needs fencing.

I had a setup with the non-balanced redundant balancers and floating IP managed by the keepalived, I also had a setup with CLUSTERIP managed by the Pacemaker, but I've never tried to marry all of this with Kubernetes.

Answer 2

Using failover IPs is a horrible solution. It's impossible to check if its working without breaking stuff. It only uses half your capacity. It can take an alarmingly long time to detect that the active node has failed.

Forget trying to recover requests that were being processed at the time a node failed. It is possible but it is really, REALLY difficult and expensive.

Use 2 reverse proxy instances and configure round robin DNS. As they read this, many IT professionals will start reaching for their keyboard, thinking that round robin DNS does not provide high availability. While according to the original specs for TCP the client should wait for a timeout - around 5 minutes. However since at least the turn of millenium, browsers have failed over much faster - in 2005, Chrome and Firefox would failover in less than half a second, while MSIE took around 10-20 seconds. I've not tested this more recently.

Concidentally this is also the simplext way to implement network multipathing to your origin server.