We have a server on a production site, it runs 24/7 though most of its traffic is normal business hours. Its function is to run a telephony call-centre using Dialogic-based hardware.
The local users of the machine noticed the machine was unresponsive to their client applications around 8.30 this morning, when we tried to gain access to it remotely we could ping it but could not gain RDP remote access to it.
Around 9.15 we asked them to pull the power cables from the machine and restart it, when it came back up we were able to get on.
We found the RAID was doing a verify and rebuild (I presume this because of the unceremonious shutdown).
Once we were able to review the server after ensuring the live services were back operational again (no issues there), we reviewed the event log.
The last "normal" event entry I can see is some automated process having an authentication failure (LsaSrv, SPNEGO(Negotiator) Event ID 40960, at 1:19:26 and then again at 2:49:27, the next event in the log is at 9:15 when we cold-rebooted the machine. That event log entry says:
Event ID 6008 The previous system shutdown at 2:49:40 on 10/05/2011 was unexpected.
Following on from that entry there are the normal startup entries as the various services come up and the machine has been fine since.
We have run Blue Screen Viewer and it has confirmed there were no blue screens that might have caused it. The machine does not have access to a KVM in its rack unfortunately so nobody was able to see what, if anything, was on the screen before it was rebooted.
Questions: 1. There are quite a few of those authentication failures, i've asked the local admins to get that issue resolved (stop it or fix the authentication) - could that have built up and caused this problem some how?
Any ideas what has actually happened?
What steps can I take to try and identify it? Could it be hardware? It's reasonably new, a couple of years old at most, decent quality kit and this is the first issue we've had in those two years.
How does windows determine the date/time of the last unexpected shutdown? Is it based on the last log entry? Or does it keep a running watch of time somehow and then if that is set when it reboots it knows when it failed?
Could it be the machine's higher level functions simply froze, leaving only basic things like low level ping still working? And if that is the case, what does that tell me?
Bottom line is i'm being asked simple questions by management, what happened and how do we make sure it doesn't happen again, as I'm sure you can imagine :)
Many thanks, let me know if I can provide any more background or check anything on the server.