How do I get notified, when a Linux machine equipped with ECC memory recognizes a memory failure? I'm interested in both correctable and uncorrectable errors.

  • if a message is written to dmesg/the syslog, this is already fine, but I'd love to know what to look for
  • installing additional daemons (like smartmontools for hard drives) is acceptable
  • Nagios/Icinga monitoring would be another way to go
  • not all machines to be monitored have IPMI

Systems of interest have Supermicro boards (X9SCM-F), regarding an HP N54L Microserver I'm just curios, but don't care too much. All systems run Debian or Ubuntu Linux.

  • 2
    I didn't even know it was reported... – Halfgaar Nov 11 '14 at 15:08
  • Running mcelog while monitoring the syslog seems like the way to go. – Jens Erat Nov 16 '14 at 21:13

The Linux kernel supports the error detection and correction (EDAC) features of some chipsets. On a supported system with ECC the status of your memory controller is accessible via sysfs:


The directory tree under that locations should correspond to your hardware, e.g.:


Depending on your hardware, you might have to explicitly load the right edac driver, cf.:

find /lib/modules/$(uname -r) -name '*edac*'

The edac-utils package provides a command line frontend and a library for accessing that data, e.g.:

edac-util -rfull          

You can setup some kind of cron-job that periodically calls eac-util and feeds the results into your monitoring system, where you can then configure some notifications.

In addition to that, running mcelog is generally a good idea. Depends on the system, but uncorrectable/correctable ECC errors are likely reported as machine check exception (MCE), as well. I mean, even brief periods of CPU throttling due to higher temperature are reported as MCE.


mcelog will monitor the memory controller and report memory error events to syslog, and in some configurations can offline bad memory pages. This is, of course, in addition to its usual use to monitor machine check exceptions and a variety of other hardware errors.

Most Linux distributions have a service set up to run it as a daemon, e.g. for EL 6:

chkconfig mcelog on
service mcelog start
  • 2
    Not supported by Ubuntu anymore. Installation gives errors. – DimiDak Feb 19 '19 at 12:09
  • Yeah, thanx I've seen that, if I'm not mistaken it talks about ubuntu18 but it doesn't work on Ubuntu 14 either. – DimiDak Feb 19 '19 at 14:02
  • 2
    The guy who posted the question says "All systems run Debian or Ubuntu" and your answer doesn't work for it. So it's pretty relevant... – DimiDak Feb 19 '19 at 14:06
  • 1
    @DimiDak He added that after this answer was written. – Michael Hampton Feb 19 '19 at 14:08

This depends on your server hardware. A whitebox or a Supermicro system will handle this differently than a Dell, HP or IBM...

One of the value-add features of high-end servers is that there's a level of hardware/OS integration. Nicer servers will report what you're looking for as part of the management agents and/or out-of-band management solution (ILO, DRAC, IPMI).

You should use the tools native to your hardware platform.

Excerpt from an HP ProLiant servers running Linux and the HP Management agents:

ECC Memory Correctable Errors  detected.


Advanced ECC Memory  Engaged

or a more severe

A correctable memory log entry indicates a memory module needs to be

or the worst... Ignoring an error for 6 days until the server crashes because of bad RAM

0004 Repaired       22:21  12/01/2008 22:21  12/01/2008 0001
LOG: Corrected Memory Error threshold exceeded (Slot 1, Memory Module 1)

0007 Repaired       02:58  12/07/2008 02:58  12/07/2008 0001
LOG: POST Error: 201-Memory Error Single-bit error occured during 
memory initialization, 
Board 1, DIMM 1. Bank containing DIMM(s) has been disabled.

0008 Repaired       19:31  12/08/2009 19:31  12/08/2009 0001
LOG: ASR Detected by System ROM

These were logged, plus SNMP traps and emails were sent.

Generically, you'll see Machine Check Exceptions in the kernel ring buffer, so you can check dmesg or run mcelog. In my experiences with Supermicro gear without IPMI, that didn't catch everything, and I still had RAM errors slip through the cracks and cause outages. Unfortunately, this led to archaic RAM burn-in policies before system deployments.


The rasdaemon package was created as a replacement for edac-tools, and newer kernels don't even support edac-tools or mcelog.

An update to the EDAC linux kernel drivers changed how the memory error counters were managed in userspace, so edac-tools and mcelog are effectively deprecated.

  • 1
    The rasdaemon writeup put together by @gabriele is comprehensive. – spaceman spiff Mar 14 '20 at 0:12

As mentioned by another poster mcelog is deprecated and effectively replaced by rasdaemon. I made a writeup on how to install and configure it on many Linux distributions, including instructions to properly setup DIMM labels.

  • The excellent writeup by @gabriele really helped me set up the rasdaemon when I started digging in to my ECC memory errors today. One thing I learned about my board (Supermicro H8DM8-2) is that it would boot fine (and I could find the controller / RAS Column / Channel) with a DIMM in an "A" module. The "B" slots are used for 128 bit wide memory access and would not boot with a DIMM only in "B" (BIOS memory error). After getting data from an A slot, I then rebooted with memory in A and B slots and it was apparent what additional EDAC data was attributable to the B slot. – user2070305 Mar 8 at 1:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.