2

I run a number of tomcat instances and occasionally some just stop reponding to requests - timeout on every connection.

I'm using AJP with mod_proxy in Apache 2.2.

I get a timeout via Apache/AJP through Tomcat's AJP connector, but also via the direct HTTP connector on 8080.

I have /server-status configured within Apache and it shows 16 requests currently being processed with W, 4 idle requests and 200+ open slots with no connection. My AJP connector is configured as:

<Connector port="8009" address="localhost"
           maxThreads="250" minSpareThreads="5" maxSpareThreads="15"
           connectionTimeout="1000"
           packetSize="16384"
           maxHttpHeaderSize="16384"
           enableLookups="false" redirectPort="8443"
           emptySessionPath="true" URIEncoding="UTF-8" protocol="AJP/1.3"/> 

so it should have plenty of threads to accept new connections.

Using top I see CPU and wait both under 1% and the java process has 80% memory. There is 60M free mem and 200M free swap.

I set up a special threads.jsp page using

SystemThreadList stl = new SystemThreadList();
Thread[] allThreads = stl.getAllThreads();

which gives useful information, but in this state - it doesn't load either.

In catalina.log I see:

Mar 07, 2014 11:53:09 AM org.apache.jk.common.ChannelSocket processConnection
WARNING: processCallbacks status 2

and occassional activity from other web requests, but not mine.

Is there a way from the command line, or using a profiler to get a list of threads and stack traces to find out what is getting stuck?

2

Set up Tomcat to enable RMI with these options:

-Dcom.sun.management.jmxremote.port=12346 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Djava.rmi.server.hostname=myhost

(This is an example, not a statement of best practice for security...)

Use jconsole which is in the JDK and connect to your server.

1

If Tomcat is also unresponsive through the direct HTTP connector, then the issue is not with AJP but something is amiss inside Tomcat.

You can get a full thread dump to be written to catalina.out by sending a

kill -quit <tomcat PID>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.