3

My ubuntu server (12.04) sometimes restart without notification. How can I check what triggered the server restart?

6

This is an art.

If you can, enable serial console (you'll need BIOS support, may want to enable GRUB support, and run a TTY), and ensure that syslog and the kernel write to console. A screen session capturing this to a file on another server can be useful.

The usual log suspects: /var/log/messages, /var/log/syslog, /var/log/kern.log, /var/log/debug. Note that /var/log/dmesg only captures the kernel ring buffer shortly after boot, so it's probably not going to have too much information.

One of the most useful things to do is to enable netconsole, an in-kernel logging service. You'll need a syslog server somewhere (preferably nearby) to capture the output. kdump can also be useful assuming data may be safely recorded to disk.

If you're still stumped, start looking at hardware components (memory, CPU, and motherboard components are prime suspects), swapping out / disabling some or all of these, etc. Kernel drivers can cause faults, so loading/unloading modules may reveal things.

BIOS configurations can produce issues. E.g.: Dell had a "CSTATE" settings issue a couple of years ago, Intel had a northgate/southgate configuration issue a ways back. Check with your BIOS/system vendor for any suggested settings or firmware updates.

Over the years, I've had random lockups/crashes due to bad capacitors, bad RAM, BIOS, samba drivers, hardware encryption accelerators, power supply/distribution elements, motherboard wiring, mains power disruptions, and various forms of operator error / intervention. Generally, start with easy/loggable stuff, and start eliminating components (hardware or software) from possible candidates.

| improve this answer | |
  • thx. I've put content of those files into gist.github.com/3074912. Can you help to see if there are anything abnormal? BTW, I am talking about a VM – Gelin Luo Jul 9 '12 at 8:13
  • When did the crash(es) happen? Without some time cue, it's hard to say. Best thing to do is grab logs for an appropriate period of time prior to the crash. That said: nothing leaps based on what you posted. – Dr. Edward Morbius Jul 9 '12 at 18:23
2

It's often very difficult.

Some places to look:

  • The output of the dmesg command (May contain information about hardware failures)
  • The contents of /var/log/syslog (Look for a line saying that syslog is starting, then look immediately preceding it)
  • The contents of /var/log/messages (Same as syslog)
  • The contents of /var/log/auth.log (Look for any indication of intentional reboot)
  • Possibly anything else in /var/log/
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.