Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Fedora 13 virtual machine running on top of VMWare's Hypervisor. Around 6am, it mysteriously rebooted, screwing up some long-running data import processes I was running. I've been checking, but I'm having a hard time finding out why it rebooted. What's the best approach to find out the cause of a reboot?

The machine is in a locked server rack, so it's unlikely someone manually rebooted the hardware. I'm the only one with SSH access to the machine, so it's unlikely someone rebooted the VM remotely. And there are other VMs running on Hypervisor that weren't rebooted, so it's unlikely to be caused by a power failure or reboot of the entire hardware.

Running who -b tells me when the reboot occurred:

~$ who -b
system boot  2011-12-22 06:02

Running crontab -l shows no cron jobs that would trigger a reboot.

Reviewing the historical resource usage graphs in Hypervisor's vSphere client shows that the machine had at most 5% CPU usage for several hours before the reboot, so it wasn't under any unusual load.

Unfortunately, checking /var/log/messages around the time of the reboot only shows:

Dec 22 03:50:01 myserver pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 22 03:50:01 myserver pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 22 03:50:01 myserver pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 22 03:50:01 myserver pcscd: winscard.c:309:SCardConnect() Reader E-Gate 0 0 Not Found
Dec 22 06:02:38 myserver kernel: imklog 4.4.2, log source = /proc/kmsg started.
Dec 22 06:02:38 myserver rsyslogd: [origin software="rsyslogd" swVersion="4.4.2" x-pid="1138" x-info="http://www.rsyslog.com"] (re)start
Dec 22 06:02:38 myserver kernel: Initializing cgroup subsys cpuset
Dec 22 06:02:38 myserver kernel: Initializing cgroup subsys cpu
Dec 22 06:02:38 myserver kernel: Linux version 2.6.34.7-56.fc13.x86_64 (mockbuild@x86-03.phx2.fedoraproject.org) (gcc version 4.4.4 20100630 (Red Hat 4.4.4-10) (GCC) ) #1 SMP Wed Sep 15 03:36:55 UTC 2010
Dec 22 06:02:38 myserver kernel: Command line: ro root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root rd_LVM_LV=VolGroup/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us rhgb quiet
Dec 22 06:02:38 myserver kernel: BIOS-provided physical RAM map:

So basically, nothing happened for about 2 hours, and then the machine suddenly rebooted.

Does this mean the kernel crashed? How would I confirm this? Are there any other logs I should be looking at?

share|improve this question

migrated from stackoverflow.com Dec 22 '11 at 17:52

This question came from our site for professional and enthusiast programmers.

1  
And what tell the vmWare logs? Maybe a cpu or disk io threshold caused the vmWare manager to cycle the vm? –  rene Dec 22 '11 at 17:46
    
It's very difficult to tell why the machine rebooted; there can be several reasons; e.g. does your Fedora Box or your Hypervisor have a watchdog function? Could you have exhausted RAM causing an internal exception; Your APIC emulation may be interfering... can't know without more info. –  Ahmed Masud Dec 22 '11 at 17:47
    
@rene, How would I access VMWare logs on Hypervisor? I can only currently connect via vSphere, a crappy Windows GUI, and I can't find any logging info aside from a list of events since the last reboot. I don't know how to SSH into Hypervisor itself (if that's even possible), and finding docs on Hypervisor/VMWare is a nightmare. –  Cerin Dec 22 '11 at 17:54
    
If SSH is enabled on your VM Hypervisor (is it ESX?) then you can actually SSH into the hypervisor itself. –  Ahmed Masud Dec 22 '11 at 17:58
    
Any indications of hard or soft reboot? Does the system whine about unclean shutdowns when booting up? –  Tim Dec 22 '11 at 18:13

1 Answer 1

up vote 0 down vote accepted

If there were a kernel issue and you have kernel dumps setup than there should be a dump file somewhere. Of couse, you this needed to be setup before the crash! You're probably aware by now that ESX/i is structurally similar to Linux so log files will be in roughly the same locations. A good overview is here, http://www.vmwarewolf.com/which-esx-log-file/ There are also various different methods of parsing/viewing ESX/i log files. http://www.simonlong.co.uk/blog/2010/06/03/vmware-esxi-4-log-files/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.