Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a pair of tomcat servers sitting behind an apache server running modjk2. the apache server load balances sessions to the tomcat servers (using x.sticky_session=True). This all works fine, however we would like to be able to install new code on the tomcat servers without terminating any sessions.

We are unable to use tomcat's session serialization since not all of the stored objects are serializable and the downtime might be too long.

Currently we bring down TomcatA (effectively killing all of its current sessions), install the new code on it, start it up and then when apache starts routing requests back to it, do the same on TomcatB.

What we would like to do is, disable new sessions on TomcatA (and maybe shorten the lifespan of existing sessions), and then when all of its sessions have expired, bring it down to do the installation .. followed by the same for TomcatB.

Has anyone solved a similar problem, or have any advise regarding this kind of setup? Currently I'm not sure if apache or tomcat should be responsible for the disallow of new sessions...

Regards, Paul.

share|improve this question
    
In short: no. Unless client sessions are explicitly bounded in time (which would mean that all users would be periodically logged out, so it's never done) you could, in theory, have a session that goes on forever as the client keeps making requests. –  womble Jul 13 '11 at 2:11
    
i asking more about the "don't send new sessions to TomcatA" feature, I don't mind kicking a few sessions to get an update done, just not all of them. –  pstanton Jul 13 '11 at 3:17

2 Answers 2

You're after something called "lame ducking".

You have the load-balancers query aliveness by checking a particular URL on the backend. You have a way to tell the backend to start failing that URL while still serving.

Ideally, this URL is different from whatever your watchdog system is using to monitor for whether the process should be automatically restarted. It might be as simple as having added a ?query to the regular health-check URL.

After a short period of time, the loadbalancers will have dropped your victim backend from the serving pool and existing connections will have finished; how you handle buffering in the front-end and slow clients will affect how long this takes, but it is an incentive to have static resources handled by a separate server. :)

You can then update the backend.

share|improve this answer
    
how would the load balancer know to keep serving active sessions from TomcatA, but send new sessions to TomcatB? –  pstanton Jul 13 '11 at 3:19
    
Because any load balancer that implements sticky sessions should implement some way of turning off the sending of new sessions without disabling the whole shebang. If yours doesn't, get a new load balancer. –  womble Jul 13 '11 at 7:06
    
pstanton: The loadbalancer distributes new requests across the "healthy" backends, but should keep an existing request going (until timeout), even if the backend is marked unhealthy. So when TomcatA starts failing the health requests, the pool of healthy backends drops to just TomcatB and only TomcatB gets new requests. –  Phil P Jul 13 '11 at 9:42
up vote 0 down vote accepted

It looks like setting the activation parameter on the worker should do the trick .. setting it to disabled should keep serving existing sessions, but not route new session requests to the disabled worker.

We are going to couple this with a broadcast within our j2ee app to tell users to save their work, log out and log back in.

Once the active session count reaches an acceptably low level we will bring the tomcat instance down for updates, then start it up and repeat for the other tomcat server.

This is just a plan, but sounds doable to me at this point, I'd appreciate your informed opinions before I close of this question however.

Thanks, paul.

share|improve this answer
    
tested, this works. –  pstanton Jul 20 '11 at 12:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.