Last night one of our Debian 7 Servers unexpectedly stopped working.
I've never seen such a "crash" before, so i hope someone here can give me a clue what happened here.
- Debian 7.11 @ 3.2.82-1 x86_64
- The Server is a Supermicro Server with a Intel Xeon CPU E5-2630 v4 @ 2.20GHz and 128 GB of Ram.
- Disk Controller is a AVAGO MegaRAID SAS 9361-4i with 4 SAMSUNG MZ7KM960HAHP Disks in RAID10.
Now to the problem: The server was becoming unreachable at one point. After ssh and ping are not responded i restarted the server via IPMI roundabout 10 minutes later.
And all i found in the logs was something like this:
Oct 30 03:51:53 box postfix/smtp: 6FE632E015F1: to=<XXXX@XXX.XX>, relay=mailload.mailmOct 30 04:05:03 box kernel: imklog 5.8.11, log source = /proc/kmsg started. Oct 30 04:05:03 box rsyslogd: [origin software="rsyslogd" swVersion="5.8.11" x-pid="2804" x-info="http://www.rsyslog.com"] start
As you can see, the server stopped writing to disk in the middle of the line: ...mailmOct 30 ...
SMART and RAID controller status are all fine.