Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I recently read this paper that @codinghorror twitted about, and I wonder how do I know that my server failed due to a memory error? Particularly, how do I know if it was a correctable or an uncorrectable error and on which DIMM it happened?

share|improve this question
up vote 2 down vote accepted

SNMP traps / messages are your best bet for having a pro-active notification about a memory/DIMM error. Products like HP Systems Insight Manager, HP OpenView, and Dell OpenManage offer several configurable rules to forward SNMP messages to emails/sms/pagers to let you know exactly when a memory error or degradation occurs.

share|improve this answer

If your server is any good it has a BIOS and BMC combo that track these errors and log them in IPMI so you can see them. Normally your server will halt on an uncorrectable ECC error, the BIOS takes over in an SMI interrupt and log it in the BMC. It then resumes operation to the OS which has nothing better to do than reboot normally (sometimes it is possible to kill a process and go on). The IPMI SEL log should be the sign of an ECC error.

If your server doesn't have a good BMC/BIOS you can resort to using crash kernel loaded to which the host kernel will jump and it can log a full stack trace and dmesg log to be reviewed later to grab this info from. It will be logged in the dmesg of the crashed kernel with bold letters "HARDWARE ERROR".

share|improve this answer

just use memtest! It will tell you exactly which DIMM is having problems.

share|improve this answer
Unfortunately, that required rebooting the server and having it offline while this runs. With ECC FBDIMMs, you can usually keep the server running normally even if there are faults, just with reduced performance. as mctsonic said, SNMP or other vendor-specific monitoring tools are probably the way to go here. – MDMarra Sep 6 '09 at 0:53

Check the server's own diagnostics. As you've told us absolutely nothing about the server that's as detailed an answer as I can give.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.