Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a VPS that almost daily performs a reboot. I finally managed today to get a clue from my logs

May 12 03:14:49   sshd[19379]: Did not receive identification string from 173.212.118.167
May 12 03:25:03   sshd[19489]: Did not receive identification string from 82.248.74.71
May 12 03:25:15   sshd[19493]: Did not receive identification string from 98.249.28.85
May 12 03:30:25   shutdown[19545]: shutting down for system halt
May 12 03:30:29   init: Switching to runlevel: 0
May 12 03:30:34   saslauthd[1654]: server_exit     : master exited: 1654
May 12 03:30:34   sshd[1656]: Received signal 15; terminating.
May 12 03:30:34   sshd[1655]: Received signal 15; terminating.
May 12 03:30:35   kernel: Kernel logging (proc) stopped.
May 12 03:30:35   kernel: Kernel log daemon terminating.
May 12 03:30:35   syslog-ng[364]: Termination requested via signal, terminating;
May 12 03:30:35   syslog-ng[364]: syslog-ng shutting down; version='3.2.2'
May 12 03:34:23   syslog-ng[388]: syslog-ng starting up; version='3.2.2'
May 12 03:34:24   ifup:     lo        
May 12 03:34:24   ifup:     lo        
May 12 03:34:24   ifup: IP address: 127.0.0.1/8  

Looks like a process volountarily performed a reboot. This seems to happen around the same time ever day. Fortunately, downtime is quite short.

I have checked cron jobs to see if there was any relevant entry, but there wasn't. The strange thing is that a reboot is usually performed with runlevel 6, not 0. Runlevel 0 is supposed to shut down the server, but instead it comes alive again in a few seconds.

What could I do to find more about this?

share|improve this question

2 Answers 2

up vote 3 down vote accepted

According to your log, something is calling the shutdown utility to perform the shutdown. I would (temporarily) replace /sbin/shutdown with a script that logs a bunch of info, something like the following:

#!/bin/bash
set > /shutdown.env
ps auxfwww > /shutdown.ps

You can then look for PPID= in /shutdown.env, and look through /shutdown.ps to see what that pid was.

(Dont forget to chmod +x the script after you put in in place. I would run it once just to make sure it works).

share|improve this answer
    
I've got the information: pastebin.com/GQHQDn4k (shutdown.ps) and pastebin.com/hBXdaKyc (shutdown.env). Seems PPID==0 –  djechelon May 14 '12 at 11:50
    
@djechelon ok, that makes things difficult. Something is calling shutdown but the parent is gone by the time the script runs (basically the script is being backgrounded). Also there are no traces of anything fishy in that ps output. And lastly the environment has been sanitized, so theres no traces left by whatever is calling it (eg, cron). I'll keep pondering, but short of just scouring the system (especially crontab -l, /etc/cron*, /var/spool/cron/crontabs/*), I'm out of ideas. –  Patrick May 14 '12 at 15:32
    
Could the shutdown signal have been initiated from the hardware node? I remind you that it's a VPS. If the HN is rebooting perhaps it shuts down all VMs. I'm starting to think so –  djechelon May 15 '12 at 17:25
    
@djechelon If you have the VM client software installed, its possible. But I dont see any such software running in your ps output you provided. –  Patrick May 15 '12 at 19:16
    
I'm also able to reboot the server from a control panel. I suppose they can issue root commands without a client software... I don't know anything more, I have to speak with my provider –  djechelon May 15 '12 at 22:55

This line in your log init: Switching to runlevel: 0 does not mean that runlevel 6 was not used: probably it was, as your server does reboot.

Check last reboot to see if it is a regular event. If so, recheck all cron jobs; last will also show you who was logged in when the server was rebooted.

Check sudo logs (you may need to enable sudo logging).

You could also try using the audit mechanism auditctl -a entry,always -S reboot described here

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.