why no examples of horizontally scalable software load balancers balancing ssl?

Question

I have a bunch of questions with respect to ssl, local sessions, and load balancing which seem to be interconnected, so I apologize in advance for the length of this question.

I have a website which uses file-based sessions. The nature of the site is that most of it is http, but some sections are ssl. Currently, because of the file based sessions, it is necessary for any ssl requests to hit the same server as any previous http requests.

Because of time constraints, I want to do the easiest possible thing to load balance increased http and ssl traffic.

There seems to be 2 options for sticky load balancing algorithms:

• ip based
• cookie based

The ip based solution will probably work, but the hashing algorithm will potentially change the server a user goes to when a server goes down or is added which is undesirable with the current file-based session setup. I also suppose that it is technically possible for a user to legitimately change ips while browsing a website.

The cookie based algorithm seems better, but the inability to inspect the cookie when encrypted by ssl seemingly presents its own problems.

I have been googling for examples on how to load balance ssl, and I cannot seem to find any explicit examples of setups which can do cookie based load balancing AND which can deal with increased ssl load by adding another ssl decoder.

Most of the explicit examples I've seen have the ssl decoder (usually hardware, apache_mod_ssl, or nginx) sitting between the browser client and the load balancer. The examples usually seem to have something like this (modified from http://haproxy.1wt.eu/download/1.3/doc/architecture.txt):

      192.168.1.1    192.168.1.11-192.168.1.14
 -------+-----------+-----+-----+-----+
        |           |     |     |     |       
     +--+--+      +-+-+ +-+-+ +-+-+ +-+-+    
     | LB1 |      | A | | B | | C | | D |    
     +-----+      +---+ +---+ +---+ +---+    
     apache         4 cheap web servers
     mod_ssl
     haproxy 

The ssl decoding part in the above example seems to be a potential bottleneck that is not horizontally scalable.

I've looked at haproxy, and it seems to have a 'mode tcp' option that would allow something like this, which would allow you to have multiple ssl decoders:

              haproxy
                 |
            -------------
            |           |
ssl-decoder-1           ssl-decoder2
            |           |
        -------------------
        |        |        |  
      web1      web2       web3

However, in such a setup, it appears you would lose the client ip because haproxy is not decoding the ssl: https://cloud-support.engineyard.com/discussions/problems/335-haproxy-not-passing-x-forwarded-for

I've also looked at nginx, and I also do not see any explicit examples of horizontally scalable ssl-decoders. There seem to be many examples of people having nginx as a potential bottleneck. And at least this link seems to suggest that nginx doesn't even have the option of the haproxy-like setup where you would lose the ip by saying that nginx "doesn't support transparently passing TCP connections to a backend" How to pass Apache SSL traffic trough nginx proxy? .

Questions:

• Why don't there seem to be more examples of setups adding more ssl decoders to deal with increased traffic?
• Is it because the ssl decoding step is only a theoretical bottleneck, and practically speaking, one decoder will essentially be enough except for sites with ridiculous traffic?
• Another possible solution that comes to mind is perhaps anybody with such increased ssl needs also has a centralized session store, so it doesn't matter which webserver the client hits on sequential requests. Then you could enable mod_ssl or equivalent on every webserver.
• The haproxy solution cited above seems to work (besides the client IP problem), but has anyone come across a sticky cookie based software load balancer solution that would work by increasing the number of decoders while keeping the client IP, or is that perhaps technically not possible (because you have to decode the request to get the client IP, in which case, we have a decoder bottleneck).

Assuming that everything I've said is true, these appear to be my options:

• use ip hashing (bad for users who potentially legitimately change ips, and for server adding and dropping scenarios)
• use nginx or mod_ssl as the 1st program touching the ssl request, this will be a potential ssl decoding bottleneck
• use haproxy as the 1st program touching the ssl request, gaining horizontal ssl scalability, but live with no ips logged at the webserver level for ssl requests (probably temporarily ok)
• over the longer term, move towards a mobile or centralized session store, making sticky sessions unnecessary

Answer 1

The "simplest thing", in all seriousness, is to move to a centralised session store. You've got to setup all this plumbing with load balancers, haproxy, SSL, and the rest of it, when every bit of session-handling code I've ever seen makes it near-trivial to plug in different storage engines, so a bit of code and very, very little extra complexity solves all your problems.

Answer 2

womble is right about the shared session store making things much easier all around. In addition to his answer, I can expand a bit on the load balancing parts of the question:

Why don't there seem to be more examples of setups adding more ssl decoders to deal with increased traffic?

Modern multi-core PC's can do several thousand SSL transactions per second. And if that becomes a bottleneck then a dedicated appliance from F5, Citrix, Cisco or the like can be even faster. So most sites never outgrow a good single-device SSL & load balancing solution.

Assuming that everything I've said is true, these appear to be my options:

There are options for scaling SSL decryption horizontally, if you come to need this.

The common approach is to use DNS Round Robin to highly available SSL accelerator pairs, i.e. publishing multiple IP addresses for the domain, each IP address pointing to a pair of SSL accelerators.

In this case you could worry about DNS TTL timing out in the middle of a user session, bumping the user to another application server. That should not be a common occurrence, but it could happen. A shared session store is the common solution, but it can be handled in other ways.

As one example you could separate the SSL decryption from the application server balancing. SSL handling is more CPU intensive than basic load balancing, thus a single load balancer should be able to saturate a couple of SSL accelerators. Like this:

Internet --> DNS round robin to multiple SSL accelerators --> plain HTTP to a single HTTP load balancer --> plain HTTP to multiple application servers

As mentioned in the beginning, a shared session store simplifies many things, and is almost certainly a better long-term solution than putting lots of complexity into your SSL / load balancing layer.

Answer 3

It's fun to respond to 2-years old questions like this when products have evolved. Right now haproxy supports the PROXY protocol, which allows it to pass the client's IP to the next hop even in pure TCP mode. It also supports native SSL, as well as SSL stickiness if you want to use it as a first layer in front of an SSL farm (possibly made from haproxy servers). So it seems that your request was a bit ahead of time and that implementations have caught up :-)

Answer 4

I agree with womble and Jesper here. The easiest/best route is to fix the code. Of course as sysadmins we often don't have that option, but even in that case there are enough tricks you can pull to get relatively cheap modern hardware to scale far-enough even though not truly horizontally.

I just wanted to post to comment on where you're concerned about losing the client-ip. In any of the major L7/proxy solutions you can insert a X-Forwarded-For (or whatever you want) header in the request. Then on the backend webserver receiving the request you can change the logfile format to log that value in the same space in the file it used to log the layer3 client ip. That way any log parsing software doesn't see the difference (nor do you when tail'ing).

There's tradeoffs to everything and we haven't heard enough about your setup to know, but with the you-can't-go-wrong trio of ha-proxy, nginx, and varnish out there its probably a good idea to move your loadbalancing to a proxy-layer tool. That will solve your ssl problem as well as afford you a host of new options like caching, content switching, and header manipulation.

Answer 5

Some random thoughts ;)

First, shoot the person that decided to use file based session data. There is no way that reading / writing data from a file system is going to be faster than just going back to the source to pull the data you need. This is about the WORST way of going about it.

I personally have never seen a situation where storing data in a session was better than just pulling it directly from the database as necessary. That said, I have seen where using memcache or similar caching strategies can help a site scale to millions of users, but that isn't even in the same ball park as using sessions.

Second, you've just found the number one reason to not use sessions at all: load balancing. FYI - Sticky does not mean Stuck. Even with Sticky sessions turned on you run the very real possibility of the user being shuffled off to another server in the middle of using your app. This will happen at the most inopportune times. Sticky just means the load balancer will try to push the user back to the server they started at, but it is by no means a guarantee.

This point usually leads people to storing session back in the database... Which I believe is a complete fail. For session to work it has to be loaded and written on each page request. When it's stored in a database (necessary for load balanced servers) this requires two server queries: the first to get the data, the second to write any updates.

The fail part is that people usually use sessions so they don't have to go back to the database to pull things like the users name... But if the page has to query the database to load a session then ... well, you should be able to see the logic problem here.

Only it's worse with sessions... because they page processor has to write the session data back to the database at the end of the page lifecycle.. in case anything changed. Which means instead of the one query to pull that user's name you end up with two. For every single page load. Further it means serializing and deserializing the data which has it's own performance impact.

My point is: session is evil and you are usually better off without it. Low traffic sites which only run on one web server don't need the performance boost that can occur; and high traffic sites running on a web farm are limited in scaling due to it.

Answer 6

Rather than use Haproxy on the front you can use round robin DNS to do coarse balancing between multiple SSL decoders then pass it to haproxy for the proper loadbalancing.