2

I've been struggling to make load balancing and clustering working using these combination:

  • Apache HTTP Server 2.2.21 (httpd-2.2.21-win32-x86-openssl-0.9.8r) using mod_proxy_ajp with sticky session enabled.
  • Apache Tomcat 7.0.23 (apache-tomcat-7.0.23-windows-x64)
  • JDK 7 update 2 (jdk-7u2-windows-x64)
  • Windows 7 64 Bit
  • Spring 3.1

Some links that I've read:

This is my configuration:

httpd.conf

# Required Modules
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule status_module modules/mod_status.so

# Reverse Proxy
<Proxy balancer://mybalancer>
    BalancerMember ajp://localhost:8301 route=s1
    BalancerMember ajp://localhost:8302 route=s2
    BalancerMember ajp://localhost:8303 route=s3
</Proxy>
ProxyPass / balancer://mybalancer/ stickysession=JSESSIONID|jsessionid

# Forward Proxy
ProxyRequests Off

<Proxy *>
    Order deny,allow
    Deny from none
    Allow from localhost
</Proxy>

# Balancer-manager, for monitoring
<Location /balancer-manager>
    SetHandler balancer-manager

    Order deny,allow
    Deny from none
    Allow from localhost
</Location>

server.xml for each tomcat (difference only in port number)

<Server port="8001" shutdown="SHUTDOWN">
    <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
    <Listener className="org.apache.catalina.core.JasperListener" />
    <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
    <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
    <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

    <GlobalNamingResources>
        <Resource name="UserDatabase" auth="Container"
            type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved"
            factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />
    </GlobalNamingResources>

    <Service name="Catalina">
        <Connector port="8101" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8201" />

        <Connector port="8301" protocol="AJP/1.3" redirectPort="8201" />

        <Engine name="Catalina" defaultHost="localhost" jvmRoute="s1">

            <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
                <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" 
                    notifyListenersOnReplication="true" />
                <Channel className="org.apache.catalina.tribes.group.GroupChannel">
                    <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4" port="45564" frequency="500" dropTime="3000" />
                    <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                        <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
                    </Sender>
                    <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" />
                    <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
                    <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" />
                </Channel>
                <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter="" />
                <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve" />
                <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
                <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" />
            </Cluster> 

            <Realm className="org.apache.catalina.realm.LockOutRealm">
                <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" />
            </Realm>
            <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true">
                <Valve className="org.apache.catalina.valves.AccessLogValve"
                    directory="logs" prefix="localhost_access_log." suffix=".txt"
                    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
            </Host>
        </Engine>
    </Service>

</Server>

port numbering (make sure no instances use the same port if deployed in the same server)

  • 80 -> Apache HTTP Server Port
  • 80xx -> Tomcat Server SHUTDOWN Port
  • 81xx -> Tomcat Connector Port (HTTP)
  • 82xx -> Tomcat SSL Redirect Port
  • 83xx -> Tomcat AJP Port
  • 40xx -> Tomcat tcp receive port for NioReceiver

Spring Controller 

@Controller
@RequestMapping("/login")
public class LoginController {
    @RequestMapping(method=RequestMethod.GET)
    public String show(@ModelAttribute("user") User user, HttpServletRequest request) {
        user.setUsername("YUSUF");

        HttpSession session = request.getSession();
        Integer tambah = (Integer) session.getAttribute("tambah");
        if(tambah == null) tambah = new Integer(1);
        else tambah = new Integer(tambah.intValue() + 1);
        session.setAttribute("tambah", tambah);

        return "login";
    }
}

login.jsp 

    <div class="mainFooter">
        Tambah = ${sessionScope.tambah}
        <br>
        ID = ${pageContext.session.id}
    </div>

So far the load balancing part is working, but the session replication is not.
Basically what I want is if I keep pressing refresh on the login page, the variable "tambah" will be incremented and kept in the session. And if the current tomcat node is down, the session will be replicated to the next tomcat node and the data is not gone. But this is what happening:

Login Screen:

Tambah = 39 
ID = C1D59C8CA5D10EB98C1DE08AC618204D.s1

I take the tomcat1 down and keep tomcat2 and tomcat3 running, here is the Login Screen:

Tambah = 1
ID = A83KJFO38FK30FJDL40FLREI39FKDKGD.s2

It seems the failover is not working, the session is not replicated, and the application created a new session. Can someone help point me to the right direction?

Thanks

EDIT: I've solved the question, thanks to @Shane Madden, this is the changes I've made: 

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" />

For each Tomcat, the tcp receiver port must be different (if running on the same server), for example use 4001, 4002, 4003, etc.

Improve this question
7
  • All instances are on the same node? I've never tried this, but I don't think that the other instances of Tomcat on the same node will get the multicast communication that's intended to be between cluster nodes (which are typically different systems).
    – Shane Madden
    Dec 29, 2011 at 1:56
  • @Shane Madden I'm sorry I don't quite understand what you mean? Basically I have 1 httpd server as a load balancer, with 3 tomcat instances behind which I named s1, s2, and s3. I wanted these 3 tomcat's session to be sticky and replication enabled for failover. All of these tomcat instances is in one balancer called "mybalancer". Do I answer your question? Thanks
    – Yusuf S
    Dec 29, 2011 at 2:04
  • What I'm saying is that since all three Tomcat instances are on a single server, the cluster communication (which is built to talk between multiple instances, each on different servers) will not function. The communication is multicast IP traffic, which will send out to the network just fine, but won't come back - it won't work to facilitate communication between multiple instances on the same server.
    – Shane Madden
    Dec 29, 2011 at 2:08
  • 2
    I've solved it! Your comment makes me re-read the tomcat cluster docs, especially in this part -> "If your Tomcat instances are running on the same machine, make sure the tcpListenPort attribute is unique for each instance, in most cases Tomcat is smart enough to resolve this on it's own by autodetecting available ports in the range 4000-4100". I just changed each port to 4001, 4002, 4003 (because the instance is in the same server). And it worked! Thanks very much! I would accept your answer if you put it in the answer section :)
    – Yusuf S
    Dec 29, 2011 at 3:37
  • 1
    Nice, good find! Go ahead and post that as your own answer, since you found that solution yourself (and I was saying that it wouldn't work - and I was wrong!). I believe it won't let you post your own answer until another 4 hours from now, but leave another comment when you do so that it notifies me and I'll vote for it :)
    – Shane Madden
    Dec 29, 2011 at 3:42

2 Answers 2

Reset to default
2

Thanks to @Shane Madden's comments, I re-read the tomcat cluster docs in http://tomcat.apache.org/tomcat-7.0-doc/cluster-howto.html, especially in this part -> "If your Tomcat instances are running on the same machine, make sure the tcpListenPort attribute is unique for each instance, in most cases Tomcat is smart enough to resolve this on it's own by autodetecting available ports in the range 4000-4100".

The changes I've made is in each of the Tomcat instance's server.xml, I make sure each port is different (for example 4001, 4002, 4003) : 

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"  
  address="auto" port="4001" autoBind="100" 
  selectorTimeout="5000" maxThreads="6" />

And voila! load-balancing and clustering is working (with the most basic config of course). I hope this post can help others on initial settings.

Improve this answer
0

I've a similar software stack (apache 2.2.15 and tomcat 7.0.65) but all linux and I ran into the following issue with the balancer status page: Instead of getting into the status page the ProxyPass directive 

ProxyPass / balancer://mybalancer/ stickysession=JSESSIONID|jsessionid

was redirecting the /balancer-manager request to one of the tomcats (and thus getting a tomvat error page). I worked around the problem by setting the ProxyPass to something more restrictive then just /

ProxyPass /myapp balancer://mybalancer/myapp stickysession=JSESSIONID|jsessionid

Not sure if this is the correct way to go about it though ?

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .