I am looking for a latency (or geo) load balancing solution combined with dynamic fail overing.

The infrastructure is based on 3 dedicated severs from OVH. The servers are located in 3 different geographical locations (Canada, France and Australia).

There are few things I am trying to achieve:

  1. Client accessing the service are redirected to the closest server, so the response is quick.
  2. If one server is out of service the request will go to the other server and eventually user will get the response.
  3. In failover conditions it is more important that the user get the answer rather the it will be quick response.
  4. I need to have the possibility to restart the server to upgrade or something once in a while. So the magic has to be outside of the main servers.
  5. The traffic goes through HTTPS, but HTTP shall redirect to HTTPS.

Small illustration of the problem

I am interested only in HTTP traffic. Each request has to go to the server, nothing can be cached as the response is very dynamic.

I have already looked at AWS Route 53, which is great but it is just a DNS, so it will not solve the failovering part.

OVH has IP load balancing, but I cannot make it to work and I did't found option to solve problem 1.

I am interested in any hint or solution including the usage of third party services.

I am also considering an option to buy 3 small VPS servers in each location (close to the each one of main servers) which will act as a gateway. In such solution I can use AWS Route 53 to provide latency load balancing to the VPS servers and the servers will handle failovering. By the definition they will be active 100% of the time, while the main servers will have a change to restart without downtime of the service.

  • 1
    It seems that this can be done with Azure Traffic Manager: docs.microsoft.com/en-us/azure/traffic-manager/… – batistuta09 Apr 20 '17 at 11:38
  • Thanks @batistuta09 didn't know that azure has it's DNS. Still it is only DNS, so it solves point 1, but not point 2. Even if it has failovering it is on a DNS level. Consider following situation, one client is making many requests, so the first time it ask DNS for the IP, it return the IP of closes server and is cached. But then this servers goes down and what is happening is that the client is making requests to the dead server until it refresh the entry from DNS. The time of that window may very on many different things. – Arek_24 Apr 20 '17 at 17:03

To update my answer - all of these are optional. Caching can be turned off to allow dynamic responses. and traffic can be served over HTTP only if desired.

One solution that I certainly love is Cloudflare. It pretty much answers all of your problems assuming you are using HTTP / HTTPS and no other protocol not mentioned in your original question.

Addressing the Geolocation based serving of content -

CF basically can direct traffic you send through it with its Traffic Manager. This will find the quickest route to one of your origin servers and maintain that. If however something failed and you needed to therefore server the content from a different location - CF can also then either load balance to multiple servers in one location, or serve from the next closest server.

Addressing the failover aspect -

CF as mentioned previously does an amazing job at pinging your origin servers for a status update and making sure they are fit for purpose. But to expand on that a little, CF does also cache certain types of assets that pass through its proxies. This is their 'always online' feature where even if the origin server is offline it will make an attempt to server that content regardless.

Addressing the updating of servers -

Personally I design all my environments around a Docker containerised design which allows me to do 'rolling updates'. This way I can update, destroy, do whatever within reason to my applications image and know it will be propagated in a safe manner. This obviously can also be rolled back if need be. But again if this is something that isn't possible, CF could allow you to spin up a secondary VM in that geo location to update the image, and hot swap the DNS effectively.

Addressing the rewrites of HTTP -> HTTPS -

CF also allows rules to be applied to traffic. One of these means you can rewrite with a 301 HTTP response all HTTP traffic to HTTPS (can configure however you want though).

(This does assume all your DNS records relating to this go through CF's DNS and are 'orange clouded').

Also some helpful resources to point you in the right direction - Cloudflare Traffic Manager

  • Thank you for the answer. I have updated my post of the missing part you pointed out (as you already saw). I will check out CF and the article you provided. Do you know which plan I will have to choose to get all those features? – Arek_24 Apr 18 '17 at 19:57
  • @Arek_24 You can get away with using just the free package to get all of the above features in one way or another - its what I use for projects I work with. There are some pretty neat features it offers if you want to do more automated optimisation etc - check them out over at the pricing page. You can get headers attached to requests with origin of the requester, or if you want request access to the load balancers (it is still early access) - but thats the only difference. – Harry Traynor Apr 18 '17 at 20:13
  • Ok, thanks I looked around and it looks like the perfect solution, but the most interesting part are behind the early access ;) I will write to CF and see what is possible at the moment. – Arek_24 Apr 18 '17 at 20:54
  • @HarryTraynor - excellent answer - do you have any insight into how you syncronise your "Addressing the updating of servers" - specifically, how best to achieve HA using multiple instances all serving the same replicated content (inlcuding mySQL) - the holy grail :-) – SolaceBeforeDawn Jun 2 '17 at 2:38
  • @Sol when you start looking at clustered environments it is good to have stateless containers - meaning the container becomes disposable and you attach storage in some manner to each one. So when you are upgrading a DB it can spin up and down different containers to change code bases, but just reattach the different volumes so data is not lost. To avoid any black spots of downtime also, you can then use a clustered DB's. Meaning there is more than one node to take load - containers can be updated one at a time etc etc... to give yourself a 100% updated cluster. – Harry Traynor Jun 7 '17 at 22:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.