up vote 0 down vote favorite
share [g+] share [fb]

I'm running CentOS 5.5 on a server. It runs several VMware virtual machines and an NFS server.

Occasionally, like today, it hangs. There's nothing in /var/log/messages that indicates any problem. (I did notice that /var/log/messages is not in time order.)

Any suggestions where to look for the cause?

link|improve this question
What do you mean by "/var/log/messages is not in time order"? The log entries should be (must be?) chronological. There may be some shifts if you are running ntp but those should be small and infrequent. – uesp Mar 8 '11 at 17:23
By "hangs" what do you mean? Does it become unresponsive for a period of time? Does the machine go back to normal after this period of time? – xeon Mar 8 '11 at 19:21
Looks like log out of order was caused by the system clock being off. It reset during the reboot. Unresponsive -- no video, no response to keyboard, ping, ssh. Does not recover. Fixed by reboot. Not CPU/memory/network bound. – Michael Eager Mar 11 '11 at 0:32
feedback

3 Answers

up vote 2 down vote

That's not a lot of information to diagnose by. If the system genuinely hangs - that is, becomes unresponsive on the network and at a local console - and there's nothing in syslog or dmesg to provide a cluse, then I would presume you have encountered a hardware fault, and would begin running diagnostic tools from your hardware vendor. Bad RAM or CPU could certainly cause this type of behaviour.

link|improve this answer
Yes, likely hw problem. The question is which component. I'm looking for diagnostics to set up so that I'll get a clue next time it crashes. – Michael Eager Mar 11 '11 at 0:34
Well, a genuine hardware fault like an intermittent failing CPU isn't gonna give you much in the way of logs down at the OS level. You need to run whatever diagnostic tools your hardware vendor provides to audit the system and determine what's broken. If you have warranty support, now is probably the time to call it in. – Jeff Albert Mar 11 '11 at 18:01
feedback
up vote 0 down vote

If you are running a window manager (gnome or kde) I have seen issues with machines hard locking.

The issue was the gnome screensaver causing some kind of issue where the machine would just completely lock up and stop responding to any connection. After disabling the screensaver the lockups stopped.

Take a look at the xorg logs and the gdm logs (if you are using gnome).

Also, check the timestamps on all of your log files in /var/logs and see if any of the logs are being written to at the time of your lock up.

Have you looked at cron? Could a process be running automagically and causing the lockup?

link|improve this answer
Nothing in Xorg logs, not using Gnome. Cron log suggests the crash was after 07:00. Not sure what that tells me. Only normal cron.hourly being run. – Michael Eager Mar 11 '11 at 0:36
Definitely time to start using the diagnostic tools available from the vendor. I suspect that this could be a hardware issue. If you can push the services of the machine to another, you could then do some real testing on the components. – Mike Mar 11 '11 at 21:34
feedback
up vote 0 down vote

Not necessarily, syslog has the ability to write log messages asynchronously. Also look at sar output to find out what the hang is. It could be I/O waits, the machine could be network bound, memory bound, or CPU bound.

Sar Tutorial

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.