OK, our data center did a RAID rebuild (and possibly some other stuff) on our server overnight. Something failed at some point, I checked on things this morning, and it appears as if the /var partition has been entirely wiped and brought back to a default install state.
i.e., everything we had in /var/logs is gone and everything we had in /var/db/mysql is gone, it's as if that stuff has been wiped out and replaced by a default install of FreeBSD (i.e. minimal logs, no stored .gzip logs, no more databases, etc.) BAD, REALLY BAD!
Actually, looking at it more, before if I did a "df -h" I'd see a /var partition. Now, I don't see it anymore. But I can get to /var... it just doesn't show up in "df -h". I do see an entry in /etc/fstab for /var.
I have pretty much no idea what they did. I'm waiting to hear from them. So I realize that this is a somewhat unanswerable question. I'm just looking for ideas on what I can check/what might have happened, etc. until I get more details from the data center about what they actually did.
In the meantime, can anyone propose a theory for where the data in /var might have gone... and how I can check if it still exists somewhere?
EDIT We're on the right track, thanks guys!
[kp@afekan /var/log]$ sudo tail -n 10 /var/log/dmesg.today
SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! Trying to mount root from ufs:/dev/mfid0s1a WARNING: /tmp was not properly dismounted /tmp: mount pending error: blocks 24 files 0 WARNING: /usr was not properly dismounted WARNING: R/W mount of /var denied. Filesystem is not clean - run fsck WARNING: R/W mount of /var denied. Filesystem is not clean - run fsck bce0: link state changed to UP bce1: link state changed to UP
They're working on it!
FIXED! Problem solved, /var never got mounted, needed to be fsck'd. Thanks guys!