I'm interested in building the the architecture in the article referenced below.

I currently have a modestly-priced layer-4 load balancer and my application servers are the SSL endpoints. I want to put an SSL server farm in between my load balancer and my app servers. Then I will put another inexpensive load balancer between the SSL farm and my app servers, to do layer-7 routing.

enter image description here

My web application has a fairly high amount of consumer traffic, that 6 servers can handle at about 50% capacity. Additionally, I have infrastructure traffic that is several orders of magnitude heavier than my consumer traffic. This is data coming in from all over the world that must integrate with my web application in real time. In total I have 18 app servers to handle all the traffic, plus 6 database servers. I will be adding 6 more app servers over the next 2 weeks and another 6 the 2 weeks after that. Conservatively, I estimate I will need to scale to 120 servers by the end of the year.

My motivation right now is to separate the consumer traffic from the infrastructure traffic. The consumer traffic is higher priority than the infrastructure traffic and I cannot allow a stampede on the infrastructure side to take down my consumer-facing servers. Having a website that is always up is the top priority. However if there is a failure in one of the consumer app servers, I want to route that traffic to the servers designated for infrastructure traffic.

The complication is that all the traffic is addressed using the same hostname and is nearly 100% https. The only way in my case to distinguish infrastructure from consumer traffic is by URL (poor architecture I inherited), so I need a layer 7 load balancer to be able to route. However for that to work I need either a fancy hardware-based SSL terminator or an SSL server farm as described above. Because my user base is rapidly scaling, I worry that if I go down the hardware path it will become very expensive very fast, especially since I will need 4 of everything for high availability (2 identical setups in 2 facilities). Meanwhile, the above diagram seems very flexible and more horizontally scalable.

Has anyone built this before? Are there pre-built configurations? What considerations should I make and what software should I use (I've heard of people using apache with mod-ssl, nginx, and stunnel)? Also, when does it make sense to buy an expensive load balancer vs building an SSL server farm?


  • Many people have built these before. But a better question is - why do you? You've mentioned cheap equipment, clusters and the wrong software all in the same question - so its hard to get a handle on what scale you are operating at. Why do you need an SSL unwrapping cluster? What is the purpose of your set-up? More details please ... – Ben Lessani - Sonassi Mar 23 '12 at 1:12
  • @sonassi added details, hope they help. I'm interested in getting your input! – dan Mar 23 '12 at 5:57

For a 120 server cluster, consult a professional. I wouldn't have thought you will get an answer of significant enough detail for your application.

The most complex clustered configuration we have set up was only 20 servers, of which only 12% was HTTPS traffic (14Mbit pure SSL).

Our typical architecture is ...

If it helps, for web clusters, we typically use:

lvs (initial ssl load balancing)
    -> pound (ssl-unwrapping) 
    -> varnish (caching) 
    -> haproxy (load balancing) 
    -> nginx (static content) 
    -> php (dynamic content) 
    -> mysql (db)

We had used stunnel in combination with HAProxy (in place of Pound), but it was causing some complications (with the inability to set headers) further down the chain.


We use this and it works extremely well, so much so that we were unable to push it to its limitations on the hardware we have and volume of traffic being pushed through Apache jMeter during testing.

There's also a note on the homepage of pound regarding performance improvements

If the PCRE, tcmalloc (from the Google perftools package) and/or Hoard are available Pound will link against them. This will provide a significant performance boost and is highly recommended.

But pound is regarded to be good for 'low' traffic applications, but doesn't seem to scale as well as its rivals, which others have documented in benchmarking tests

Software SSL terminators Benchmarks

  1. http://vincent.bernat.im/en/blog/2011-ssl-benchmark-round2.html
  2. http://vincent.bernat.im/en/blog/2011-ssl-benchmark.html
  3. http://barry.wordpress.com/2008/04/28/load-balancer-update/

CPU usage is likely to be the key issue

CPU consumption is going to be your biggest issue, so being smart with the SSL key size is going to help, 1024bit (now deprecated by most SSL authorities) vs 2048bit vs 3072bit keys will increase overheads linearly.

Here's some good reading on general SSL performance, http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html

What you will ultimately find here is that there is no 'right' answer and that only testing, testing again, then testing some more will show you what works best in your scenario.


You can split the infrastructure from consumer traffic by using a Linux firewall. Using the netfilter/iptables string match functionality you can match the traffic based on the url. After the traffic is matched you can use this to perform QoS or forward the traffic differently.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.