RHEL 6.4 Kicking off all Terminal Connections

Question

I have a RHEL 6.4 server with 64GB of ram running a java app that loads 60+ war files into tomcat. For some reason, every time it loads every user gets kicked off. This includes a directly connected monitor on the physical server.

The server is not generating any type of crash log but it is not possible to log in. There still appears to be disk activity based on the disk lights.

I'm trying to determine if this application is the cause or merely just a symptom of some underlying hardware/software issue. Anecdotally, I can say it only happens with this app. Even running memtest86 showed no memory errors and a stress test getting the load to 160 with 100% memory usage did not crash.

My two questions are these:

1. Is there a process to monitor that would terminate any and all connections (remember this includes the VGA port on the physical server)
2. Are there any tools that would allow me to "wrap" this application and see exactly what point the terminations occur?

EDIT: Did not find out why connections were being severed. However, I moved the physical drives to another server (same hardware) and the issue does not happen on the new box. Leads me to believe there is hardware issue somewhere.

Answer 1

Try attaching a strace to one of the login shells, and then sending the trace to a file, and throw it in the background. That was it doesn't get caught hopefully.

something like:

PID=''
FILE_OUT=''
strace -D -t -f -v -p ${PID} -o ${FILE_OUT}

This way you can hopefully see what signal/command it's getting that's causing it to shutdown

Now that I think about it, I would throw an strace on one of the logins, but also the java app. You can much up the timestamps to see what happened right before/after in both threads.