Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

One of our servers froze yesterday, apparently refusing to serve any HTTP requests. The tech guy on site could not connect remotely to the machine, so he rebooted the (virtual) machine from the VMware Infrastructure Client, and everything was up and running again.

Now I want to figure out what went wrong. I looked at a couple of log files, and all just stop logging anything at 5:00am, and start logging again with a boot sequence. I could not find anything suspicious, other then the fact a number of cron jobs ran at 5:00am. These were all fairly simple jobs, not interacting with anything critical, and there was at least some activity after they completed.

The freeze lasted a couple of hours. We did not have any other issues on other virtual machines on the same box, which all have a very similar configuration.

Is there any place I should start looking for clues? What can I tell people to do should this happen again before just resetting the machine? Magic SysRq maybe?

share|improve this question
add comment

2 Answers 2

up vote 1 down vote accepted

My first action would be to take server out of service and run a full Memtest+ run on it to check that memory is not failing. Next check SMART from HDD's for any issues. Next would be to following instructions at http://www.kernel.org/doc/Documentation/networking/netconsole.txt to capture anything like this in future.

share|improve this answer
add comment

Im guessing you've seen this already but how-can-i-use-syslog-to-diagnose-mysterious-crashes dont know if this could help at all, is your servers under stress/serve a great deal of clients

share|improve this answer
    
Thanks, but although we do serve a lot of applications from this server, I did not see any memory allocation errors in the log files. –  Kamiel Wanrooij Oct 18 '10 at 17:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.