Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Currently we have the following setup.

  • Hardware Load Balancer (LB)
  • Box A running Tomcat on 8080 (TA)
  • Box B running Tomcat on 8080 (TB)

TA and TB are running behind LB.

For now it's pretty complicated and manual job to take Box A or Box B out of LB to do the zero downtime deployment.

I am thinking to do something like this:

  • Hardware Load Balancer (LB)
  • Box A running Nginx on 8080 (NA)
  • Box A running Tomcat on 8081 (TA1)
  • Box A running Tomcat on 8082 (TA2)
  • Box B running Nginx on 8080 (NB)
  • Box B running Tomcat on 8081 (TB1)
  • Box B running Tomcat on 8082 (TB2)

Basically LB will be directing traffic between NA and NB now. On each of Nginx's we'll have TA1, TA2 and TB1, TB2 configured as upstream servers. Once one of the upstreams's healthcheck page is unresponsive (shutdown) the traffic goes to another one (HttpHealthcheckModule module on Nginx).

So the deploy process is simple.

Say, TA1 is active with version 0.1 of the app. Healthcheck on TA1 is OK. We start TA2 with Healthcheck on it as ERROR. So Nginx is not talking to it. We deploy app version 0.2 to TA2. Make sure it works. Now, we switch the Healthcheck on TA2 to OK, switch Healthcheck to TA1 to ERROR. Nginx will start serving TA2, and will remove TA1 out of rotation. Done! And now same with the other box.

While it sounds all cool and nice, how do we "starve" the Nginx? Say we have pending connections, some users on TA1. If we just turn it off, sessions will break (we have cookie-based sessions). Not good. Any way to starve traffic to one of the upstream servers with Nginx?


share|improve this question
Why is telling the load balancer to drain connections such a big deal? This is something it should be able to handle easily. – longneck Oct 9 '12 at 2:25
up vote 3 down vote accepted

My opinion is if you're willing to put in such a complicated setup just to get zero-downtime, you should instead be investing your time in getting a better load balancer. By going down the path you suggest, you are trading simplicity for marginal cost savings, which always ends up costing you more than you anticipate.

You're going to need downtime to implement this zero-downtime solution you propose. Take that downtime to instead figure out how to use your existing load balancer, or swap it out for something better.

share|improve this answer

I accomplish basically what your looking for by using HAProxy as my load balancer and having it do frequent checks against a "health check" html page. In my nginx config on the backend web servers, this page doesn't physically exist. During deployment, I create a file on the backend server called "offline.html". In nginx, if this file exists when "check.html" is requested, it serves it, otherwise it serves the real health check page.

During my deployments, I touch the offline.html file, wait a few seconds for the loadbalancer to kick out the server, deploy, then remove the file.

Site fires back up, load balancer adds the server back into the pool, and everything works perfectly...

Side note, I'd try to remove reliance on a session affinity if you can...

share|improve this answer
We have similar way. We just manually redirects the traffic to nodeA while nodeB is going to be upgraded, then after a sanity check on nodeB, we switch all traffic to nodeB and upgrade nodeA. Do not need another balancer layer (nginx). – ruffp Oct 25 '13 at 10:07

Couldn't you implement Spring Session backed by a replicated redis cluster on both Node A and B?

If you then configure both nginx instances to load balance over all 4 tomcat instances then you can shutdown or crash any tomcat server at any time and nginx will just use the next server to serve your request? This is assuming Node A and B can connect to all 4 tomcat instances.

share|improve this answer
2 questions and an assumption ... is this really an Answer? – Pierre.Vriens May 26 at 20:41
I cannot comment or I wouldn't have added this as an answer. However this is very old I didn't notice :-/ – 2Fast2BCn May 26 at 20:57
Ok, I understand. But do you know how easy it is to gain +40 more rep? Eg by suggesting 20 edits that get accepted. You know what: I'll help you a bit to get closer ... – Pierre.Vriens May 26 at 21:08
...on a 4-year old question with an accepted answer. – GregL May 26 at 23:55

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.