I have 20 servers in 1 location and I would like to do load balancing. This can go up to 100 in any case. Is there any known method for doing this? I would like also to have some kind of mechanism that when customer queries e.g. service.example.com, he will keep using the same server until the machine is taken out of the cluster because of fail. The client makes a DNS query every minute and let's assume top down DNS record TTL is 30-60 seconds, while others can be well 24h or so, so each session can be valid for up to 24h after initial query, and then the customer will be switched to another server.
I find L4-L7 load balancers quite useless, as I think I could use just the DNS.
The protocols are the binary only based TCP connections as well HTTP ones.
I was thinking that for binary connection (like with no protocol whatsoever), I could use round-robin on the DNS, so each time I query the DNS, I got different response.
For HTTP this is something I dont know. I could put maybe HAProxy and that's it, but for the DNS I am not sure.
I was thinking once about the following method. Send customer to some "master.example.com", which is HTTP server making only redirects, which will generate FQDN with uniqueid, which means that it's like a session id. This way, this FQDN will always resolve to same ip number and can be used only for the ip which queried for it, for the next 24h or forever until the server is switched off.
So this looks like the following:
http://redirect.example.com/resource => http://67hkkdbvh.example.com/resource
Now the session looks like this:
1st minute: http://67hkkdbvh.example.com/resource/1 TTL 60s 2nd minute: http://67hkkdbvh.example.com/resource/2 TTL 60s
If the 67hkkdbvh will die, the customer requests redirect again.
Now I am not sure how I could use HAProxy to help with it???