I have an application(multithreaded) which is developed in C running in the production server. Recently, my application got into the hang state for 5 hours and we could not notice it.
So, we developed a script which alerts us, takes the information about
pstack
strace
lsof
gcore
top
full processlist ( for mysql ).
and restarts the application.
Hence, we will get a alert as well as the required details why the process got into the hanged stated.
What tools do you use to analyze the problem, when the application is down.