CentOS 5.3 Random Shutdown?

Question

Our CentOS 5.3 box is randomly shutting down as if the power cable was pulled. June 5th is when it first happened, here is a paste from /var/log/messages

Jun  5 12:06:22 mail named[23248]: unexpected RCODE (SERVFAIL) resolving 'loverspillscalm.com/NS/IN': 208.67.222.222#53
Jun  5 12:06:24 mail named[23248]: unexpected RCODE (SERVFAIL) resolving 'loverspillscalm.com/NS/IN': 12.127.17.71#53
Jun  5 15:38:57 mail named[23248]: unexpected RCODE (SERVFAIL) resolving 'yagwozat.cn/NS/IN': 12.127.17.72#53
Jun  5 15:38:58 mail named[23248]: unexpected RCODE (SERVFAIL) resolving 'yagwozat.cn/NS/IN': 208.67.222.222#53
Jun  5 16:49:25 mail shutdown[354]: shutting down for system halt
Jun  5 16:49:46 mail shutdown[550]: shutting down for system halt
Jun  5 16:49:46 mail shutdown[557]: shutting down for system halt
Jun  5 16:49:47 mail shutdown[590]: shutting down for system halt
Jun  5 22:29:56 mail syslogd 1.4.1: restart.
Jun  5 22:29:56 mail kernel: klogd 1.4.1, log source = /proc/kmsg started.
Jun  5 22:29:56 mail kernel: Linux version 2.6.18-128.1.10.el5PAE ([email protected]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Thu$
Jun  5 22:29:56 mail kernel: BIOS-provided physical RAM map:

Then it happened again on June 10th, Here is a paste from the /var/log/messages june 10th shutdown:

Jun 10 19:23:21 mail smartd[2625]: Device /dev/sda: using '-d sat' for ATA disk behind SAT layer.
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sda, opened
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sda, not found in smartd database.
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sda, is SMART capable. Adding to "monitor" list.
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sdb, opened
Jun 10 19:23:21 mail smartd[2625]: Device /dev/sdb: using '-d sat' for ATA disk behind SAT layer.
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sdb, opened
Jun 10 19:23:21 mail smartd[2625]: Device: /dev/sdb, not found in smartd database.
Jun 10 19:23:22 mail smartd[2625]: Device: /dev/sdb, is SMART capable. Adding to "monitor" list.
Jun 10 19:23:22 mail smartd[2625]: Device: /dev/sdc, opened
Jun 10 19:23:22 mail smartd[2625]: Device /dev/sdc: using '-d sat' for ATA disk behind SAT layer.
Jun 10 19:23:22 mail smartd[2625]: Device: /dev/sdc, opened
Jun 10 19:23:22 mail smartd[2625]: Device: /dev/sdc, not found in smartd database.
Jun 10 19:23:22 mail smartd[2625]: Device: /dev/sdc, is SMART capable. Adding to "monitor" list.
Jun 10 19:23:22 mail smartd[2625]: Monitoring 0 ATA and 3 SCSI devices
Jun 10 19:23:22 mail smartd[2627]: smartd has fork()ed into background mode. New PID=2627.
Jun 10 19:25:08 mail saslauthd: auth_zimbra_init: zimbra_cert_check is off!
Jun 10 19:25:08 mail saslauthd: auth_zimbra_init: 1 auth urls initialized for round-robin
Jun 10 19:27:39 mail shutdown[7064]: shutting down for system halt
Jun 10 19:27:39 mail init: Switching to runlevel: 0
Jun 10 19:27:40 mail shutdown[7100]: shutting down for system halt
Jun 10 19:27:41 mail smartd[2627]: smartd received signal 15: Terminated
Jun 10 19:27:41 mail smartd[2627]: smartd is exiting (exit status 0)
Jun 10 19:28:15 mail avahi-daemon[2564]: Got SIGTERM, quitting.
Jun 10 19:28:15 mail avahi-daemon[2564]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::2e0:81ff:feb5:f036.
Jun 10 19:28:15 mail avahi-daemon[2564]: Leaving mDNS multicast group on interface eth0.IPv4 with address xxx.xxx.xxx.xxx.
Jun 10 19:28:16 mail xinetd[2478]: Exiting...
Jun 10 19:28:20 mail rpc.statd[2287]: Caught signal 15, un-registering and exiting.
Jun 10 19:28:21 mail named[2234]: shutting down: flushing changes
Jun 10 19:28:21 mail named[2234]: stopping command channel on 127.0.0.1#953
Jun 10 19:28:21 mail named[2234]: stopping command channel on ::1#953
Jun 10 19:28:21 mail named[2234]: no longer listening on 127.0.0.1#53
Jun 10 19:28:21 mail named[2234]: no longer listening on xxx.xxx.xxx.xxx#53
Jun 10 19:28:21 mail named[2234]: exiting
Jun 10 19:28:21 mail auditd[2165]: The audit daemon is exiting.
Jun 10 19:28:21 mail kernel: audit(1244687301.413:125): audit_pid=0 old=2165 by auid=4294967295
Jun 10 19:28:21 mail pcscd: pcscdaemon.c:572:signal_trap() Preparing for suicide
Jun 10 19:28:22 mail pcscd: hotplug_libusb.c:376:HPRescanUsbBus() Hotplug stopped
Jun 10 19:28:22 mail pcscd: readerfactory.c:1379:RFCleanupReaders() entering cleaning function
Jun 10 19:28:22 mail pcscd: pcscdaemon.c:532:at_exit() cleaning /var/run
Jun 10 19:28:22 mail kernel: Kernel logging (proc) stopped.
Jun 10 19:28:22 mail kernel: Kernel log daemon terminating.
Jun 10 19:28:23 mail exiting on signal 15
Jun 10 21:48:11 mail syslogd 1.4.1: restart.
Jun 10 21:48:11 mail kernel: klogd 1.4.1, log source = /proc/kmsg started.
Jun 10 21:48:11 mail kernel: Linux version 2.6.18-128.1.10.el5PAE ([email protected]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Thu May 7 11:14:31 EDT 2009
Jun 10 21:48:11 mail kernel: BIOS-provided physical RAM map:

My logs from /var/log/acpi:

[Mon May 11 15:06:26 2009] starting up
[Mon May 11 15:06:26 2009] 1 rule loaded
[Mon May 11 15:06:33 2009] client connected from 3076[68:68]
[Mon May 11 15:06:33 2009] 1 client rule loaded
[Mon May 11 15:29:34 2009] exiting
[Mon May 11 15:31:26 2009] starting up
[Mon May 11 15:31:26 2009] 1 rule loaded
[Mon May 11 15:31:29 2009] client connected from 3015[68:68]
[Mon May 11 15:31:29 2009] 1 client rule loaded
[Mon May 11 15:35:17 2009] exiting
[Mon May 11 15:37:02 2009] starting up
[Mon May 11 15:37:02 2009] 1 rule loaded
[Mon May 11 15:37:04 2009] client connected from 2705[68:68]
[Mon May 11 15:37:04 2009] 1 client rule loaded
[Mon May 11 15:43:36 2009] exiting
[Mon May 11 15:45:19 2009] starting up
[Mon May 11 15:45:19 2009] 1 rule loaded
[Mon May 11 15:45:21 2009] client connected from 2662[68:68]
[Mon May 11 15:45:21 2009] 1 client rule loaded
[Mon May 11 16:47:53 2009] exiting
[Mon May 11 16:49:42 2009] starting up
[Mon May 11 16:49:42 2009] 1 rule loaded
[Mon May 11 16:49:44 2009] client connected from 2668[68:68]
[Mon May 11 16:49:44 2009] 1 client rule loaded
[Mon May 11 17:02:52 2009] exiting
[Mon May 11 17:04:38 2009] starting up
[Mon May 11 17:04:38 2009] 1 rule loaded
[Mon May 11 17:04:40 2009] client connected from 2662[68:68]
[Mon May 11 17:04:40 2009] 1 client rule loaded
[Tue May 12 08:41:24 2009] exiting
[Tue May 12 08:43:10 2009] starting up
[Tue May 12 08:43:10 2009] 1 rule loaded
[Tue May 12 08:43:12 2009] client connected from 2664[68:68]
[Tue May 12 08:43:12 2009] 1 client rule loaded
[Tue May 12 09:54:10 2009] exiting
[Tue May 12 09:55:52 2009] starting up
[Tue May 12 09:55:52 2009] 1 rule loaded
[Tue May 12 09:55:54 2009] client connected from 2650[68:68]
[Tue May 12 09:55:54 2009] 1 client rule loaded
[Tue May 12 14:09:24 2009] exiting
[Tue May 12 14:11:10 2009] starting up
[Tue May 12 14:11:10 2009] 1 rule loaded
[Tue May 12 14:11:12 2009] client connected from 2564[68:68]
[Tue May 12 14:11:12 2009] 1 client rule loaded
[Tue May 12 14:50:57 2009] exiting
[Tue May 12 14:52:42 2009] starting up
[Tue May 12 14:52:42 2009] 1 rule loaded
[Tue May 12 14:52:43 2009] client connected from 2566[68:68]
[Tue May 12 14:52:43 2009] 1 client rule loaded
[Tue May 19 14:12:45 2009] starting up
[Tue May 19 14:12:45 2009] 1 rule loaded
[Tue May 19 14:12:47 2009] client connected from 2589[68:68]
[Tue May 19 14:12:47 2009] 1 client rule loaded
[Fri May 22 17:35:54 2009] starting up
[Fri May 22 17:35:54 2009] 1 rule loaded
[Fri May 22 17:35:56 2009] client connected from 2586[68:68]
[Fri May 22 17:35:56 2009] 1 client rule loaded
[Mon May 25 18:28:22 2009] received event "button/power PWRF 00000080 00000001"
[Mon May 25 18:28:22 2009] notifying client 2586[68:68]
[Mon May 25 18:28:22 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Mon May 25 18:28:22 2009] BEGIN HANDLER MESSAGES
[Mon May 25 18:28:22 2009] END HANDLER MESSAGES
[Mon May 25 18:28:22 2009] action exited with status 0
[Mon May 25 18:28:22 2009] completed event "button/power PWRF 00000080 00000001"
[Mon May 25 19:16:30 2009] received event "button/power PWRF 00000080 00000002"
[Mon May 25 19:16:30 2009] notifying client 2586[68:68]
[Mon May 25 19:16:30 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Mon May 25 19:16:30 2009] BEGIN HANDLER MESSAGES
[Mon May 25 19:16:30 2009] END HANDLER MESSAGES
[Mon May 25 19:16:30 2009] action exited with status 0
[Mon May 25 19:16:30 2009] completed event "button/power PWRF 00000080 00000002"
[Mon May 25 19:16:31 2009] received event "button/power PWRF 00000080 00000003"
[Mon May 25 19:16:31 2009] notifying client 2586[68:68]
[Mon May 25 19:16:31 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Mon May 25 19:16:31 2009] BEGIN HANDLER MESSAGES
[Mon May 25 19:16:31 2009] END HANDLER MESSAGES
[Mon May 25 19:16:31 2009] action exited with status 0
[Mon May 25 19:16:31 2009] completed event "button/power PWRF 00000080 00000003"
[Mon May 25 19:16:31 2009] received event "button/power PWRF 00000080 00000004"
[Mon May 25 19:16:31 2009] notifying client 2586[68:68]
[Mon May 25 19:16:31 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Mon May 25 19:16:31 2009] BEGIN HANDLER MESSAGES
[Mon May 25 19:16:31 2009] END HANDLER MESSAGES
[Mon May 25 19:16:31 2009] action exited with status 0
[Mon May 25 19:16:31 2009] completed event "button/power PWRF 00000080 00000004"
[Wed May 27 15:16:04 2009] received event "button/power PWRF 00000080 00000005"
[Wed May 27 15:16:04 2009] notifying client 2586[68:68]
[Wed May 27 15:16:04 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed May 27 15:16:04 2009] BEGIN HANDLER MESSAGES
[Wed May 27 15:16:04 2009] END HANDLER MESSAGES
[Wed May 27 15:16:04 2009] action exited with status 0
[Wed May 27 15:16:04 2009] completed event "button/power PWRF 00000080 00000005"
[Wed May 27 15:16:06 2009] received event "button/power PWRF 00000080 00000006"
[Wed May 27 15:16:06 2009] notifying client 2586[68:68]
[Wed May 27 15:16:06 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed May 27 15:16:06 2009] BEGIN HANDLER MESSAGES
[Wed May 27 15:16:06 2009] END HANDLER MESSAGES
[Wed May 27 15:16:06 2009] action exited with status 0
[Wed May 27 15:16:06 2009] completed event "button/power PWRF 00000080 00000006"
[Thu May 28 05:40:38 2009] received event "button/power PWRF 00000080 00000007"
[Thu May 28 05:40:38 2009] notifying client 2586[68:68]
[Thu May 28 05:40:38 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Thu May 28 05:40:38 2009] BEGIN HANDLER MESSAGES
[Thu May 28 05:40:38 2009] END HANDLER MESSAGES
[Thu May 28 05:40:38 2009] action exited with status 0
[Thu May 28 05:40:38 2009] completed event "button/power PWRF 00000080 00000007"
[Thu May 28 05:40:39 2009] received event "button/power PWRF 00000080 00000008"
[Thu May 28 05:40:39 2009] notifying client 2586[68:68]
[Thu May 28 05:40:39 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Thu May 28 05:40:39 2009] BEGIN HANDLER MESSAGES
[Thu May 28 05:40:39 2009] END HANDLER MESSAGES
[Thu May 28 05:40:39 2009] action exited with status 0
[Thu May 28 05:40:39 2009] completed event "button/power PWRF 00000080 00000008"
[Fri Jun  5 16:49:25 2009] received event "button/power PWRF 00000080 00000009"
[Fri Jun  5 16:49:25 2009] notifying client 2586[68:68]
[Fri Jun  5 16:49:25 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Fri Jun  5 16:49:25 2009] BEGIN HANDLER MESSAGES
[Fri Jun  5 16:49:25 2009] END HANDLER MESSAGES
[Fri Jun  5 16:49:25 2009] action exited with status 0
[Fri Jun  5 16:49:25 2009] completed event "button/power PWRF 00000080 00000009"
[Fri Jun  5 16:49:45 2009] received event "button/power PWRF 00000080 0000000a"
[Fri Jun  5 16:49:45 2009] notifying client 2586[68:68]
[Fri Jun  5 16:49:45 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Fri Jun  5 16:49:45 2009] BEGIN HANDLER MESSAGES
[Fri Jun  5 16:49:46 2009] END HANDLER MESSAGES
[Fri Jun  5 16:49:46 2009] action exited with status 0
[Fri Jun  5 16:49:46 2009] completed event "button/power PWRF 00000080 0000000a"
[Fri Jun  5 16:49:46 2009] received event "button/power PWRF 00000080 0000000b"
[Fri Jun  5 16:49:46 2009] notifying client 2586[68:68]
[Fri Jun  5 16:49:46 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Fri Jun  5 16:49:46 2009] BEGIN HANDLER MESSAGES
[Fri Jun  5 16:49:46 2009] END HANDLER MESSAGES
[Fri Jun  5 16:49:46 2009] action exited with status 0
[Fri Jun  5 16:49:46 2009] completed event "button/power PWRF 00000080 0000000b"
[Fri Jun  5 16:49:47 2009] received event "button/power PWRF 00000080 0000000c"
[Fri Jun  5 16:49:47 2009] notifying client 2586[68:68]
[Fri Jun  5 16:49:47 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Fri Jun  5 16:49:47 2009] BEGIN HANDLER MESSAGES
[Fri Jun  5 22:29:59 2009] starting up
[Fri Jun  5 22:29:59 2009] 1 rule loaded
[Fri Jun  5 22:30:00 2009] client connected from 2579[68:68]
[Fri Jun  5 22:30:00 2009] 1 client rule loaded
[Wed Jun 10 19:21:04 2009] received event "button/power PWRF 00000080 00000001"
[Wed Jun 10 19:21:04 2009] notifying client 2579[68:68]
[Wed Jun 10 19:21:04 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed Jun 10 19:21:04 2009] BEGIN HANDLER MESSAGES
[Wed Jun 10 19:21:04 2009] END HANDLER MESSAGES
[Wed Jun 10 19:21:04 2009] action exited with status 0
[Wed Jun 10 19:21:04 2009] completed event "button/power PWRF 00000080 00000001"
[Wed Jun 10 19:21:07 2009] received event "button/power PWRF 00000080 00000002"
[Wed Jun 10 19:21:07 2009] notifying client 2579[68:68]
[Wed Jun 10 19:21:07 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed Jun 10 19:21:07 2009] BEGIN HANDLER MESSAGES
[Wed Jun 10 19:21:08 2009] END HANDLER MESSAGES
[Wed Jun 10 19:21:08 2009] action exited with status 0
[Wed Jun 10 19:21:08 2009] completed event "button/power PWRF 00000080 00000002"
[Wed Jun 10 19:23:16 2009] starting up
[Wed Jun 10 19:23:16 2009] 1 rule loaded
[Wed Jun 10 19:23:18 2009] client connected from 2583[68:68]
[Wed Jun 10 19:23:18 2009] 1 client rule loaded
[Wed Jun 10 19:27:39 2009] received event "button/power PWRF 00000080 00000001"
[Wed Jun 10 19:27:39 2009] notifying client 2583[68:68]
[Wed Jun 10 19:27:39 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed Jun 10 19:27:39 2009] BEGIN HANDLER MESSAGES
[Wed Jun 10 19:27:39 2009] END HANDLER MESSAGES
[Wed Jun 10 19:27:39 2009] action exited with status 0
[Wed Jun 10 19:27:39 2009] completed event "button/power PWRF 00000080 00000001"
[Wed Jun 10 19:27:40 2009] received event "button/power PWRF 00000080 00000002"
[Wed Jun 10 19:27:40 2009] notifying client 2583[68:68]
[Wed Jun 10 19:27:40 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Wed Jun 10 19:27:40 2009] BEGIN HANDLER MESSAGES
[Wed Jun 10 19:27:40 2009] END HANDLER MESSAGES
[Wed Jun 10 19:27:40 2009] action exited with status 0
[Wed Jun 10 19:27:40 2009] completed event "button/power PWRF 00000080 00000002"
[Wed Jun 10 19:28:17 2009] exiting
[Wed Jun 10 21:48:13 2009] starting up
[Wed Jun 10 21:48:13 2009] 1 rule loaded
[Wed Jun 10 21:48:15 2009] client connected from 2559[68:68]
[Wed Jun 10 21:48:15 2009] 1 client rule loaded
[Thu Jun 11 08:35:04 2009] received event "button/power PWRF 00000080 00000001"
[Thu Jun 11 08:35:04 2009] notifying client 2559[68:68]
[Thu Jun 11 08:35:04 2009] executing action "/bin/ps awwux | /bin/grep gnome-power-manager | /bin/grep -qv grep || /sbin/shutdown -h now"
[Thu Jun 11 08:35:04 2009] BEGIN HANDLER MESSAGES
[Thu Jun 11 08:35:04 2009] END HANDLER MESSAGES
[Thu Jun 11 08:35:04 2009] action exited with status 0
[Thu Jun 11 08:35:04 2009] completed event "button/power PWRF 00000080 00000001"
[Thu Jun 11 08:35:39 2009] exiting
[Thu Jun 11 08:38:48 2009] starting up
[Thu Jun 11 08:38:48 2009] 1 rule loaded
[Thu Jun 11 08:38:50 2009] client connected from 2568[68:68]
[Thu Jun 11 08:38:50 2009] 1 client rule loaded
[Thu Jun 11 09:11:14 2009] exiting
[Thu Jun 11 09:11:14 2009] starting up
[Thu Jun 11 09:11:14 2009] 0 rules loaded
[Thu Jun 11 09:11:19 2009] client connected from 2568[68:68]
[Thu Jun 11 09:11:19 2009] 1 client rule loaded
[Thu Jun 11 17:53:57 2009] received event "button/power PWRF 00000080 00000001"
[Thu Jun 11 17:53:57 2009] notifying client 2568[68:68]
[Thu Jun 11 17:53:57 2009] completed event "button/power PWRF 00000080 00000001"
[Thu Jun 11 17:53:57 2009] received event "button/power PWRF 00000080 00000002"
[Thu Jun 11 17:53:57 2009] notifying client 2568[68:68]
[Thu Jun 11 17:53:57 2009] completed event "button/power PWRF 00000080 00000002"
[Thu Jun 11 17:53:57 2009] received event "button/power PWRF 00000080 00000003"
[Thu Jun 11 17:53:57 2009] notifying client 2568[68:68]
[Thu Jun 11 17:53:57 2009] completed event "button/power PWRF 00000080 00000003"
[Thu Jun 11 17:53:57 2009] received event "button/power PWRF 00000080 00000004"
[Thu Jun 11 17:53:57 2009] notifying client 2568[68:68]
[Thu Jun 11 17:53:57 2009] completed event "button/power PWRF 00000080 00000004"
[Wed Jun 17 07:41:06 2009] received event "button/power PWRF 00000080 00000005"
[Wed Jun 17 07:41:06 2009] notifying client 2568[68:68]
[Wed Jun 17 07:41:06 2009] completed event "button/power PWRF 00000080 00000005"
[Wed Jun 17 16:40:06 2009] received event "button/power PWRF 00000080 00000006"
[Wed Jun 17 16:40:06 2009] notifying client 2568[68:68]
[Wed Jun 17 16:40:06 2009] completed event "button/power PWRF 00000080 00000006"
[Wed Jun 17 16:40:12 2009] received event "button/power PWRF 00000080 00000007"
[Wed Jun 17 16:40:12 2009] notifying client 2568[68:68]
[Wed Jun 17 16:40:12 2009] completed event "button/power PWRF 00000080 00000007"
[Wed Jun 17 16:40:16 2009] received event "button/power PWRF 00000080 00000008"
[Wed Jun 17 16:40:16 2009] notifying client 2568[68:68]
[Wed Jun 17 16:40:16 2009] completed event "button/power PWRF 00000080 00000008"
[Wed Jun 17 16:40:16 2009] received event "button/power PWRF 00000080 00000009"
[Wed Jun 17 16:40:16 2009] notifying client 2568[68:68]
[Wed Jun 17 16:40:16 2009] completed event "button/power PWRF 00000080 00000009"
[Wed Jun 17 16:40:44 2009] received event "button/power PWRF 00000080 0000000a"
[Wed Jun 17 16:40:44 2009] notifying client 2568[68:68]
[Wed Jun 17 16:40:44 2009] completed event "button/power PWRF 00000080 0000000a"
[Wed Jun 17 17:20:03 2009] received event "button/power PWRF 00000080 0000000b"
[Wed Jun 17 17:20:03 2009] notifying client 2568[68:68]
[Wed Jun 17 17:20:03 2009] completed event "button/power PWRF 00000080 0000000b"
[Wed Jun 17 17:49:54 2009] received event "button/power PWRF 00000080 0000000c"
[Wed Jun 17 17:49:54 2009] notifying client 2568[68:68]
[Wed Jun 17 17:49:54 2009] completed event "button/power PWRF 00000080 0000000c"
[Wed Jun 17 17:49:54 2009] received event "button/power PWRF 00000080 0000000d"
[Wed Jun 17 17:49:54 2009] notifying client 2568[68:68]
[Wed Jun 17 17:49:54 2009] completed event "button/power PWRF 00000080 0000000d"
[Wed Jun 17 17:50:01 2009] received event "button/power PWRF 00000080 0000000e"
[Wed Jun 17 17:50:01 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:01 2009] completed event "button/power PWRF 00000080 0000000e"
[Wed Jun 17 17:50:01 2009] received event "button/power PWRF 00000080 0000000f"
[Wed Jun 17 17:50:01 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:01 2009] completed event "button/power PWRF 00000080 0000000f"
[Wed Jun 17 17:50:16 2009] received event "button/power PWRF 00000080 00000010"
[Wed Jun 17 17:50:16 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:16 2009] completed event "button/power PWRF 00000080 00000010"
[Wed Jun 17 17:50:24 2009] received event "button/power PWRF 00000080 00000011"
[Wed Jun 17 17:50:24 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:24 2009] completed event "button/power PWRF 00000080 00000011"
[Wed Jun 17 17:50:33 2009] received event "button/power PWRF 00000080 00000012"
[Wed Jun 17 17:50:33 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:33 2009] completed event "button/power PWRF 00000080 00000012"
[Wed Jun 17 17:50:33 2009] received event "button/power PWRF 00000080 00000013"
[Wed Jun 17 17:50:33 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:33 2009] completed event "button/power PWRF 00000080 00000013"
[Wed Jun 17 17:50:35 2009] received event "button/power PWRF 00000080 00000014"
[Wed Jun 17 17:50:35 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:35 2009] completed event "button/power PWRF 00000080 00000014"
[Wed Jun 17 17:50:35 2009] received event "button/power PWRF 00000080 00000015"
[Wed Jun 17 17:50:35 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:35 2009] completed event "button/power PWRF 00000080 00000015"
[Wed Jun 17 17:50:35 2009] received event "button/power PWRF 00000080 00000016"
[Wed Jun 17 17:50:35 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:35 2009] completed event "button/power PWRF 00000080 00000016"
[Wed Jun 17 17:50:47 2009] received event "button/power PWRF 00000080 00000017"
[Wed Jun 17 17:50:47 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:47 2009] completed event "button/power PWRF 00000080 00000017"
[Wed Jun 17 17:50:47 2009] received event "button/power PWRF 00000080 00000018"
[Wed Jun 17 17:50:47 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:47 2009] completed event "button/power PWRF 00000080 00000018"
[Wed Jun 17 17:50:51 2009] received event "button/power PWRF 00000080 00000019"
[Wed Jun 17 17:50:51 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:51 2009] completed event "button/power PWRF 00000080 00000019"
[Wed Jun 17 17:50:54 2009] received event "button/power PWRF 00000080 0000001a"
[Wed Jun 17 17:50:54 2009] notifying client 2568[68:68]
[Wed Jun 17 17:50:54 2009] completed event "button/power PWRF 00000080 0000001a"
[Wed Jun 17 18:37:48 2009] received event "button/power PWRF 00000080 0000001b"
[Wed Jun 17 18:37:48 2009] notifying client 2568[68:68]
[Wed Jun 17 18:37:48 2009] completed event "button/power PWRF 00000080 0000001b"
[Wed Jun 17 18:37:50 2009] received event "button/power PWRF 00000080 0000001c"
[Wed Jun 17 18:37:50 2009] notifying client 2568[68:68]
[Wed Jun 17 18:37:50 2009] completed event "button/power PWRF 00000080 0000001c"
[Wed Jun 17 18:55:13 2009] received event "button/power PWRF 00000080 0000001d"
[Wed Jun 17 18:55:13 2009] notifying client 2568[68:68]
[Wed Jun 17 18:55:13 2009] completed event "button/power PWRF 00000080 0000001d"
[Wed Jun 17 18:57:59 2009] received event "button/power PWRF 00000080 0000001e"
[Wed Jun 17 18:57:59 2009] notifying client 2568[68:68]
[Wed Jun 17 18:57:59 2009] completed event "button/power PWRF 00000080 0000001e"
[Thu Jun 18 00:20:47 2009] received event "button/power PWRF 00000080 0000001f"
[Thu Jun 18 00:20:47 2009] notifying client 2568[68:68]
[Thu Jun 18 00:20:47 2009] completed event "button/power PWRF 00000080 0000001f"
[Thu Jun 18 00:20:47 2009] received event "button/power PWRF 00000080 00000020"
[Thu Jun 18 00:20:47 2009] notifying client 2568[68:68]
[Thu Jun 18 00:20:47 2009] completed event "button/power PWRF 00000080 00000020"
[Thu Jun 18 13:43:16 2009] received event "button/power PWRF 00000080 00000021"
[Thu Jun 18 13:43:16 2009] notifying client 2568[68:68]
[Thu Jun 18 13:43:16 2009] completed event "button/power PWRF 00000080 00000021"
[Thu Jun 18 13:43:19 2009] received event "button/power PWRF 00000080 00000022"
[Thu Jun 18 13:43:19 2009] notifying client 2568[68:68]
[Thu Jun 18 13:43:19 2009] completed event "button/power PWRF 00000080 00000022"
[Fri Jun 19 14:04:15 2009] received event "button/power PWRF 00000080 00000023"
[Fri Jun 19 14:04:15 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:15 2009] completed event "button/power PWRF 00000080 00000023"
[Fri Jun 19 14:04:18 2009] received event "button/power PWRF 00000080 00000024"
[Fri Jun 19 14:04:18 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:18 2009] completed event "button/power PWRF 00000080 00000024"
[Fri Jun 19 14:04:18 2009] received event "button/power PWRF 00000080 00000025"
[Fri Jun 19 14:04:18 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:18 2009] completed event "button/power PWRF 00000080 00000025"
[Fri Jun 19 14:04:18 2009] received event "button/power PWRF 00000080 00000026"
[Fri Jun 19 14:04:18 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:18 2009] completed event "button/power PWRF 00000080 00000026"
[Fri Jun 19 14:04:18 2009] received event "button/power PWRF 00000080 00000027"
[Fri Jun 19 14:04:18 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:18 2009] completed event "button/power PWRF 00000080 00000027"
[Fri Jun 19 14:04:19 2009] received event "button/power PWRF 00000080 00000028"
[Fri Jun 19 14:04:19 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:19 2009] completed event "button/power PWRF 00000080 00000028"
[Fri Jun 19 14:04:52 2009] received event "button/power PWRF 00000080 00000029"
[Fri Jun 19 14:04:52 2009] notifying client 2568[68:68]
[Fri Jun 19 14:04:52 2009] completed event "button/power PWRF 00000080 00000029"
[Fri Jun 19 14:07:46 2009] received event "button/power PWRF 00000080 0000002a"
[Fri Jun 19 14:07:46 2009] notifying client 2568[68:68]
[Fri Jun 19 14:07:46 2009] completed event "button/power PWRF 00000080 0000002a"
[Fri Jun 19 14:07:49 2009] received event "button/power PWRF 00000080 0000002b"
[Fri Jun 19 14:07:49 2009] notifying client 2568[68:68]
[Fri Jun 19 14:07:49 2009] completed event "button/power PWRF 00000080 0000002b"
[Fri Jun 19 14:07:49 2009] received event "button/power PWRF 00000080 0000002c"
[Fri Jun 19 14:07:49 2009] notifying client 2568[68:68]
[Fri Jun 19 14:07:49 2009] completed event "button/power PWRF 00000080 0000002c"
[Fri Jun 19 14:15:56 2009] received event "button/power PWRF 00000080 0000002d"
[Fri Jun 19 14:15:56 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:56 2009] completed event "button/power PWRF 00000080 0000002d"
[Fri Jun 19 14:15:57 2009] received event "button/power PWRF 00000080 0000002e"
[Fri Jun 19 14:15:57 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:57 2009] completed event "button/power PWRF 00000080 0000002e"
[Fri Jun 19 14:15:57 2009] received event "button/power PWRF 00000080 0000002f"
[Fri Jun 19 14:15:57 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:57 2009] completed event "button/power PWRF 00000080 0000002f"
[Fri Jun 19 14:15:58 2009] received event "button/power PWRF 00000080 00000030"
[Fri Jun 19 14:15:58 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:58 2009] completed event "button/power PWRF 00000080 00000030"
[Fri Jun 19 14:15:59 2009] received event "button/power PWRF 00000080 00000031"
[Fri Jun 19 14:15:59 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:59 2009] completed event "button/power PWRF 00000080 00000031"
[Fri Jun 19 14:15:59 2009] received event "button/power PWRF 00000080 00000032"
[Fri Jun 19 14:15:59 2009] notifying client 2568[68:68]
[Fri Jun 19 14:15:59 2009] completed event "button/power PWRF 00000080 00000032"
[Fri Jun 19 14:1

So it did it again, on a Friday night at around 16:00hrs. Just the same as when it first did it. Except this time no shutting down errors. — Physikal, Jun 22, 2009 at 15:39

David Mackintosh · Accepted Answer · 2009-06-13 00:24:33Z

2

I've seen exactly this.

In our case, it was the motherboard watchdog which had a bad temperature sensor. The sensor kept insisting that the motherboard was overheating, so it shut the OS, and therefore the computer, down. The shutdown logs looked similar to yours, like someone had come along and run the 'shutdown' command somewhere. When first powered on, the computer would run for 20 minutes or so before shutting down, but if you were persistant it would start the shutdown process almost before the OS boot was finished.

Go into your motherboard management (BMC, IPMI, iLO, whatever) and check to see if there are any "events" in the logs.

answered Jun 13, 2009 at 0:24

David Mackintosh

14.4k7 gold badges49 silver badges79 bronze badges

The next time I am able to shut it down to check that I will definitely do so. The odd thing about it is it has only happened after business hours. Which is sort of odd to me.
– Physikal
Jun 15, 2009 at 19:43

Add a comment |

oneodd1 · Accepted Answer · 2009-06-12 22:09:24Z

1

Another thing you might consider is if you have a remote access card that allows shutdowns. The chances are slim you would have another admin shutting down your server but it never hurts to check.

answered Jun 12, 2009 at 22:09

oneodd1

6081 gold badge6 silver badges11 bronze badges

No remote access card. Thank you for the response though.
– Physikal
Jun 15, 2009 at 19:42

Add a comment |

Matt Simmons · Accepted Answer · 2009-06-13 00:44:00Z

1

I would guess power button as well, but it's possible that something else is causing it. Is this machine connected to a UPS? Are there any other people who have root access on the machine? Can you check lastlog to see if anyone else was connected when it was shutdown? What services is this machine running, and are there any published vulnerabilities on these services?

answered Jun 13, 2009 at 0:44

Matt Simmons

20.5k10 gold badges68 silver badges116 bronze badges

It is connected to a UPS. I am the only one that has access to the machine.
– Physikal
Jun 15, 2009 at 19:42
1

Is it a smart UPS with a separate control (USB or serial) connection to the machine?
– Dan Carley
Jun 22, 2009 at 15:52

Add a comment |

Brent · Accepted Answer · 2009-06-12 21:43:40Z

0

If it IS the power button, you might be able to unplug it from the motherboard to test it.

I don't see anything in your logs prior to switching to runlevel 0, so it sounds to me like the OS thinks it's being told to shut down correctly (ie. by a power button press)

answered Jun 12, 2009 at 21:43

Brent

23.3k19 gold badges70 silver badges103 bronze badges

Add a comment |

David Pashley · Accepted Answer · 2009-06-12 22:04:11Z

0

I'd expect some sort of error message to explain why it was shutdown. You could see who was logged on to it at the time it was shutdown with "last". It could be that the server is overheating and shutting down to protect itself. I'm not sure what process would do that on CentOS. As far as the power button is concerned, there should be a log about that, but you could try stopping acpid and see if it still shuts down. That would rule out any ACPI events.

answered Jun 12, 2009 at 22:04

David Pashley

23.7k2 gold badges46 silver badges73 bronze badges

With "last" it shows that nobody was logged on since 5 days prior to reboot.
– Physikal
Jun 22, 2009 at 16:04

Add a comment |

Kevin M · Accepted Answer · 2009-06-12 22:04:47Z

0

Have you checked with any users that have access to this host? See if one of them is shutting down the computer?

Check /root/.bash_history, /home/*/.bash_history and other similar files to see if one of them shows a shutdown message.

Also check /var/log/secure to see if someone used sudo to shut it down.

answered Jun 12, 2009 at 22:04

Kevin M

2,3121 gold badge16 silver badges21 bronze badges

No shutdown commands. I am the only one that really has access to the box.
– Physikal
Jun 22, 2009 at 16:03

Add a comment |

Jason Tan · Accepted Answer · 2009-06-22 16:27:15Z

0

You need to go further back in your logs, to where you see the shutdown process start and look a bit back from there.

The last log may also help. You may be able to see who's on the box if anyone, at the time it's going down. In fact all your logs should be examined.

Its interesting that it's all being logged as mail.

More log examination required.

answered Jun 22, 2009 at 16:27

Jason Tan

2,7522 gold badges17 silver badges24 bronze badges

When I use "last" it shows that nobody was logged in at the time. mail is the hostname of the box. For the logs, that is the only time it shows any shutdown events. Atleast that include the word shutdown.
– Physikal
Jun 22, 2009 at 17:06

Add a comment |

Kamil Kisiel · Accepted Answer · 2009-06-22 18:13:21Z

0

Based on the acpi logs it looks like the system is receiving input from the power button to shut down. I would suspect that either the button is being pushed accidentally or there is a hardware problem involved.

Also: You should be able to modify the behavior in response to the power button event by editing /etc/acpi/events/power_button. Note that the path may be slightly different on CentOS 5, but that's the common location.

edited Jun 22, 2009 at 18:13

answered Jun 22, 2009 at 16:56

Kamil Kisiel

12.3k8 gold badges49 silver badges70 bronze badges

I totally agree Kamil. The only thing I don't get is, I've disabled response to the power button. So now when I tap it, it does nothing. Of course if I were to hold the button it would shut down. The only other thing that I find extremely odd. Is it NEVER does this durring hours of operation. Another thing that supports hardware failure is the fact that it is the only box that goes down in a rack w/ 8 servers.
– Physikal
Jun 22, 2009 at 17:07
I added some instructions on disabling the shutdown in response to the power button. That would be something to try.
– Kamil Kisiel
Jun 22, 2009 at 18:13

Add a comment |

Stack Exchange Network

CentOS 5.3 Random Shutdown?

8 Answers 8

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
linux
.

Linked

Hot Network Questions

CentOS 5.3 Random Shutdown?

8 Answers 8

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged linux.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
linux
.