My ubuntu server (12.04) sometimes restart without notification. How can I check what triggered the server restart?
4
-
Please post output of the dmesg log, and other interesting logs. This make troubleshooting a lot easier.– FrederikJul 8, 2012 at 1:25
-
@FrederikNielsen: I think what he's asking is what are the other interesting logs.– bahamatJul 8, 2012 at 1:48
-
Here is the dmesg log: gist.github.com/3074912– Gelin LuoJul 9, 2012 at 8:01
-
Related: serverfault.com/questions/410992/…– Eduard WirchJan 8, 2013 at 10:36
Add a comment
|
2 Answers
This is an art.
If you can, enable serial console (you'll need BIOS support, may want to enable GRUB support, and run a TTY), and ensure that syslog and the kernel write to console. A screen session capturing this to a file on another server can be useful.
The usual log suspects: /var/log/messages, /var/log/syslog, /var/log/kern.log, /var/log/debug. Note that /var/log/dmesg only captures the kernel ring buffer shortly after boot, so it's probably not going to have too much information.
One of the most useful things to do is to enable netconsole, an in-kernel logging service. You'll need a syslog server somewhere (preferably nearby) to capture the output. kdump can also be useful assuming data may be safely recorded to disk.
If you're still stumped, start looking at hardware components (memory, CPU, and motherboard components are prime suspects), swapping out / disabling some or all of these, etc. Kernel drivers can cause faults, so loading/unloading modules may reveal things.
BIOS configurations can produce issues. E.g.: Dell had a "CSTATE" settings issue a couple of years ago, Intel had a northgate/southgate configuration issue a ways back. Check with your BIOS/system vendor for any suggested settings or firmware updates.
Over the years, I've had random lockups/crashes due to bad capacitors, bad RAM, BIOS, samba drivers, hardware encryption accelerators, power supply/distribution elements, motherboard wiring, mains power disruptions, and various forms of operator error / intervention. Generally, start with easy/loggable stuff, and start eliminating components (hardware or software) from possible candidates.
answered Jul 8, 2012 at 2:11
-
thx. I've put content of those files into gist.github.com/3074912. Can you help to see if there are anything abnormal? BTW, I am talking about a VM Jul 9, 2012 at 8:13
-
When did the crash(es) happen? Without some time cue, it's hard to say. Best thing to do is grab logs for an appropriate period of time prior to the crash. That said: nothing leaps based on what you posted. Jul 9, 2012 at 18:23
It's often very difficult.
Some places to look:
- The output of the
dmesgcommand (May contain information about hardware failures) - The contents of
/var/log/syslog(Look for a line saying that syslog is starting, then look immediately preceding it) - The contents of
/var/log/messages(Same as syslog) - The contents of
/var/log/auth.log(Look for any indication of intentional reboot) - Possibly anything else in
/var/log/
