httpd ProxyPass with sticky sessions and DNS RR

Question

I'm using apache httpd with ProxyPass and DNS RR and sticky Java sessions for Tomcat.

It's not working.

My apache config is:

ProxyPass "/" "balancer://mycluster/" stickysession=JSESSIONID|jsessionid scolonpathdelim=On

<Proxy balancer://mycluster>
    BalancerMember http://stest:8080/
</Proxy>

<Location "/balancer-manager">
    SetHandler balancer-manager
    Require host localhost
</Location>

When I do an nslookup stest on the httpd host I have 5 hosts that come back:

root@sproxy:/app# nslookup stest
Server:     127.0.0.11
Address:    127.0.0.11#53

Non-authoritative answer:
Name:   stest
Address: 10.0.0.7
Name:   stest
Address: 10.0.0.6
Name:   stest
Address: 10.0.0.8
Name:   stest
Address: 10.0.0.2
Name:   stest
Address: 10.0.0.5

Thus, my httpd server should route to one of these IP addresses until a JSESSIONID cookie is set and then it should be sticky for that IP.

The problem is, this does not work. When I do:

while true; do curl -b /tmp/cj.txt -c /tmp/cj.txt -w '\n' localhost/test-session-servlet/json ; done

I get my servlet output, which includes the Tomcat host that I hit and the session id.

I would expect this to only hit one server, since the cookie gets set in /tmp/cj.txt and is passed to every request. I get the same output with the above command as I do with:

while true; do curl -w '\n' localhost/test-session-servlet/json ; done

(note the above command does not use cookies, so it should be load balanced even with sticky sessions because it does not pass the cookie)

In essence, both curl commands send to different servers as if the sticky sessions are not enabled properly, but the first should not do this.

What am I doing wrong here?

Answer 1

Tomcat and httpd are working as designed. The problem is the architecture you have set up is never going to work. You need the route in httpd.conf and the jvmRoute in server.xml to be set and consistent.

One possible solution is to explicitly define each of the stest hosts in your httpd.conf file. Something like:

<Proxy balancer://mycluster>
  BalancerMember http://10.0.0.2:8080/ route=tc01
  BalancerMember http://10.0.0.5:8080/ route=tc02
  BalancerMember http://10.0.0.6:8080/ route=tc03
  BalancerMember http://10.0.0.7:8080/ route=tc04
  BalancerMember http://10.0.0.8:8080/ route=tc05
</Proxy>

You'll also need to set the jvmRoute on the <Engine> element in the server.xml of each Tomcat instance and ensure that it is consistent with the route defined above.