How does one troubleshoot when a system hangs?

Question

On Solaris 10, how do you generally troubleshoot when a system hangs. For example - when i run a command say metastat and hit enter, the system hangs. press control C it comes out and on again running metastat and enter the system hangs. Please let me know how to trouble shoot this and which logs I can check with examples.

Thanks

Answer 1

Truss is a great command for quick troubleshooting, but Solaris has a decisive advantage for this type of work: DTrace. It will take some time to get fully up to speed with it, but never again will you need to wonder "what is it doing?"

Answer 2

I would start with the truss command and the log files, you should monitor the follow files (thy record all system problems like disk errors, swap errors, NFS problems, etc):

tail -f /var/adm/SYSLOG
tail -f /var/adm/messages
tail -f /var/log/syslog

Answer 3

There is a difference between when a command hangs and when a system hangs:

• if the system hangs all processes will hang
• if a command hangs only it is affected

What seems to be your case is a command hang, which can be troubleshot with truss - just run

truss -f command

and you'll see what it is doing and what system call it will hang on. The -f flag will tell truss to also follow child processes.

Since you mention that you are running metastat it is likely to be a disk issue, so looking for information about failed disks in /var/adm/messages would be a place to start.