I have Citrix XenServer 4.1 running. On this I have Debian Etch which is used as our web development server. I have Virtualmin set-up to manage the domains (MySQL, Apache, etc.). The server is only accessible from outside via HTTP, no SSH or FTP.
Suddenly two days ago we were unable to run any commands on the server. I tried to reboot but that failed. While were able to get to the login screen were unable to login. There were boot up errors of the form:
init: cannot execute "/etc/init.d/rc"
I'v never seen these before. Searching suggested it couldn't find the
/bin/sh referenced at the top of the script. This lead to further searches that returned a lot of results about disk failing, etc. This didn't really make much sense to me since the Xen host was fine and another another Debian install running on it was also fine.
Eventually I mounted the disk that held the corrupted install and found the
/bin directory was missing. This explained everything that was happening so far. I gave the logs a cursory glance to see if anything popped out at me. None did. I copied the
/bin directory from another Debian Etch install to see if I could get the server back up. No luck, things like aptitude did not work, etc. I decided to do a fresh install since that would not take too much time and I have now restored everything.
I've been looking at the logs but I can't find anything that indicates what happened so far.
Can anyone suggest or point me in a direction that will help explain how a critical directory, such as
/bin, suddenly disappeared? I would hate for this happen again.
Thanks in advance.