I don't understand how to setup a failover for my quite simple scenario. I am building a service gateway for API. What I want to have is two servers hosted in different datacenters. And I simply want the user to be able to access the service even if one of the servers is down. There is no issue with DB sync, I only care about availability of the service.

How do I do that while preventing the user to implement any kind of failover logic on their end? I want the user to be given a single domain or a single IP address and be able to access the service all the time using this single end point.

What I do not understand is how this can be achieved. I know I can setup a network node that will forward the requests to the first or second server, depending on which of those two is currently online. However, I fail to see how this setup solved HA problem as we just introduced a single point of failure to the system - the forwarding node. So, if this node goes down, the service is unavailable.

Could you please explain how to implement this in the real world? Is it possible to achieve this with reasonable cost (i.e. not more than a cost of hosting of the servers themselves).

Edit: It has been suggested that different datacenters requirement is costly. So, feel free to provide suggestions for 2 servers within 1 datacenter.

Edit 2: Feel free to mention what is a reasonable cost for the that setup.

closed as too broad by MadHatter, Iain, GregL, Ward, HBruijn Feb 24 '16 at 16:15

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • In different datacenters? No, it will be expensive. – Michael Hampton Feb 23 '16 at 10:44
  • I think so, because if they are hosted in 1 datacenter, it is easy to imagine that both servers will go down at the same time. I don't say two different companies, but yes, two different datacenters sound reasonable to me - or is it an abnormal request? – Wapac Feb 23 '16 at 10:47
  • 1
    @MadHatter Not exactly. IIRC you can get 99.9% QUITE cheap - that is the cost of replacement hardware. We still talk about 8 hours downtime a year. Every hoster provides that - and if you keep backups and jave not terabytes of data there is plenty of time for that. AFTER THAT - I fully gree. – TomTom Feb 23 '16 at 11:15
  • 1
    @yagmoth555 DNS round robin doesn't work, because of TTL-ignoring cacheing. But even if it did work, RR to what? Still need more hardware, different DC, the services must remain in-sync; it all costs. – MadHatter Feb 23 '16 at 11:37
  • 1
    And I'm warning you that, with intangible costs, it will cost more than that. You can't know how much more until you try it, but in my experience you do need to be prepared for up to an order of magnitude. In any case, I've VTCed, because you could write a book on this; therefore, it's too broad a question. – MadHatter Feb 23 '16 at 11:59

it works quite simple. First rule is you have to have anything more than once. For simplicity I will setup it in one datacenter and with IP addresses owned by this DC (you can do it with your own IP addresses and multiple datacenters, but we're tolking about some multihoming AS stuff, BGP and some other things which are not so cheap and easy to implement).

You will need to have at least 4 servers (you can do it with only two, but it is not good way). 2 for app and 2 for loadbalancing, each servers with multiple network cards.

Basic setup is like this:

       /---\     /------\     /----------\
       | S |-----| LB 1 |-----| SERVER 1 |
--NET--| W |     \------/\   /\----------/
       | I |              \_/
       | T |              / \
--NET--| C |     /------\/   \/----------\
       | H |-----| LB 2 |-----| SERVER 2 |
       \---/     \------/     \----------/

You have two separated lines to net provided by your DC. Both of those lines are in same VLAN and both are connected to switch (best way is 2 switches). 2 loadbalancers are connected to those switches and shares one virtual IP. It is IP which can flow between those two machines. You can use VRRP and keepalived to achieve this pretty well.

Behind those two loadbalancers, two mirrored servers are placed. And here comes magic:

  1. You will point your DNS record to that virtual ip
  2. When someone will come to your app, it will go thru one LB and ends at one server
  3. When one server dies, loadbalancer will notice it with something like healtcheck and disable that server. Every new request will be send to health server.
  4. When one loadbalancer dies, keepalived will notice it (again via some healthcheck) and move that floating IP to health loadbalancer and nobody will notice it.

You should know that HA is expensive way and you cannot do it witl low budget. You need to calculate if outage of your service isn't cheaper that cost of HA, sometimes it is.

You should look on keywords vrrp, keepalived and haproxy for some ideas and ways how to think about it.

  • Thank you, this is informative - I was not aware of the "floating virtual IP". That is what I will have to find more about. – Wapac Feb 23 '16 at 12:27

The usual approach is, of course, using two forwarding (balancing) nodes in some form of HA cluster. The consistency from the point of the outer world is achieved by various forms of shared IP address - VRRP, CARP (same as VRRP, but open implementation), etc. Thus you will have the redundancy on both layers - on the balancing layer and on data/service layer.

The consistency of data/service layer is beyound the scope of this answer, however, usually it's rather simple. You use centralized session store (probably replicated too, like redis or memcached) and replicated set of DBs.

In general this is achievable on only two physical servers, each of them playing dirrerent roles at once: a balancer, a DB server, and so on.

Not the answer you're looking for? Browse other questions tagged or ask your own question.