Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Not really familiar with Intel's "Mirrored Channel Mode" for a Blade Server setup (your typical moderately-heavy MySQL OLTP database running on the bare metal blade; no virtualization right now).

From the Intel docs I was able to find:

The Intel Xeon Processor 5500 series and Intel Xeon Processor 5600 series support channel mirroring to configure available channels of DDR3 DIMMs in the mirrored configuration. The mirrored configuration is a redundant image of the memory, and can continue to operate despite the presence of sporadic uncorrectable errors. Channel mirroring is a RAS feature in which two identical images of memory data are maintained, thus providing maximum redundancy.

On the Intel Xeon Processor 5500 series and Intel Xeon Processor 5600 series processors based Intel server boards, mirroring is achieved across channels. Active channels hold the primary image and the other channels hold the secondary image of the system memory. The integrated memory controller in the Intel Xeon Processor 5500 series and Intel Xeon Processor 5600 series processors alternates between both channels for read transactions. Write transactions are issued to both channels under normal circumstances.

However, I'm not really pickin' up what they're layin' down here. I lose half my storage capacity, but I gain "redundancy" of memory and possible gain read/write performance benefits? Like RAID 1 for RAM? Anybody have any practical experience with this configuration?

share|improve this question
up vote 6 down vote accepted

Personally I would sooner use some form of clustering rather than that level of hardware resilience. It makes sense for cheapo components like disks to double up on them, but mirroring memory is a nice-to-have but not that useful. I mean what's more likely to fail; a CPU, your OS, your software, your mobo, your PSU/s. I'd sooner put the money towards clustering.

share|improve this answer
My thoughts exactly: while certainly useful for a very specific failure point, I could tick off many more HA "boxes" by going the clustering route with another blade or two (and likely another chassis in another data center). – gravyface Mar 5 '11 at 3:33
clustering won't help you when calculations have to be done on schedule or within a tineframe. In some situations failover takes longer than the operation required to be performed – Jim B Mar 6 '11 at 1:33

"RAID 1 for RAM" is an accurate description. In my experience, there isn't much performance benefit, but depending on the bus speed vs the speed of the modules, your mileage may vary.

As far as redundancy goes.. well, it's not terribly often that a module goes bad.

Personally, I turn off mirroring whenever I see it enabled.

share|improve this answer
thanks Shane. Ever do any benchmarking before/after? – gravyface Mar 4 '11 at 19:32
@gravyface Can't say that I have, unfortunately; just haven't noticed a discernible performance different on vs off (on database servers and vm hosts). Some hard numbers would definitely be nice. – Shane Madden Mar 4 '11 at 19:38
I'm going to do some benchmarks then. See if it makes a difference. Can't say the error-correction benefit sounds like a tangible benefit, but I'm curious to see how it performs. I'm going to wait a couple days for some additional answers and then mark this correct. – gravyface Mar 4 '11 at 20:15
I see dimms go bad failrly regualarly, however given the size of the environment I should see 1 dimm every 2 weeks (statistically speaking) – Jim B Mar 4 '11 at 20:22

I have read that this kind of thing (you can do it with CPUs as well) is very useful in the huge supercomputer clusters.

Some of these clusters are running so many machines that there will be a machine failure every couple of hours. Faster than the jobs can complete. That really messes up the computation. Adding redundancy like this to each node can more than double the time between failure.

share|improve this answer
so this is high-end stuff now trickling down to the mainstream, I take it. Not really seeing much value for my needs. Thanks though. – gravyface Mar 5 '11 at 3:29
Yes, it is high end. Wait until you get hot plug CPU's AND CPU MIRRORING (!) in a pc ;) Mainframes can switch over to another cpu when one fails. – TomTom Mar 5 '11 at 11:45

This memory mode was really designed for situations where you need high availability You shouldn't see much of a performance difference (since the loss of one channel probably isn't noticeable under normal operations) however you actually lose alot of ram. With mirroring enabled, only one-third of total memory is available for use because two DIMM slots are the primary channel, two DIMM slots are the backup channel, and two DIMM slots are not used. (at least that's how it is on IBMs)

I typically recommend that it be turned off (if you have an app or OS that likes ram - and let's face it: is there one that doesn't?) or save up to upgrade to the ex5 chipset from IBM (hp and others soon to follow with similar offerings) that adds a boatload more QPI.

There are the occasional "this server has to be up regardless of the number of shots fired at it" and this type of redundancy helps. Additionaly of you've purchased less than stellar quality ram this might save you from a blue screen or 2.

share|improve this answer
Yeah, don't see much need for this right now. Obviously nobody likes downtime, but when we address HA, we'll be looking at clustering most certainly. – gravyface Mar 5 '11 at 3:31
this in response to your "see one bad DIMM every two weeks" -- how often are you seeing bad DIMMs past the grace period? I can't recall ever having bad RAM once in production; I usually notice it the first few hours/days under typical workload. – gravyface Mar 5 '11 at 3:36
Our results mirror the rate google sees We have similarly configured servers- fully populated boards with lots of dimms, and applications that are memory intensive. In looking at 1 of my vmware environments I see 3 bad dimms across 18 fully populated IBM hs22vs (324 dimms) These servers have been up for about a year now. – Jim B Mar 6 '11 at 1:31

"Is it worth it ?"

Perhaps "What does it do, and why do I want that" is a better question.

First, it is up to the CPU Mfg. to create a CPU with the capability (since, unlike RAID or ECC Memory internal CPU Hardware support is required), but the actual implementation in SPARC , POWER, x86, etc., can vary (and be improved over the years; thus the need to update this answer occasionally).

Next the Motherboard must support the Feature (a good means to accomplish this, in this case, would be to ensure that the maximum number of Memory Slots are present).

Lastly, upon discovering the Fault, the OS must "do the right thing" - that MIGHT be important. If you are driving a Car or flying a Plane do you want to see a BSOD and be busy rebooting your Computer while you are out of control or would a warning be preferable ?

You need one Computer "up" IF it is the job of that Computer to be the 'brains' of the Operation, otherwise you need to sit by the Phone and work mighty fast (impossible).

When it is "do or die", then "do you must, or die you will".

Here are a few different implementations from different Mfgs. along with similar technologies to "Memory Mirroring).

By having two (or more) of everything in ONE Box each part (that is still functioning after a failure) can recover with the LOWEST latency possibly; that MIGHT be critical to you or you might want to spend you money elsewhere (and gain a larger Memory capacity in that Box).

When your Company's motto is "Goal = 100% Uptime" then this is one piece of the puzzle, so is having two (or more) of these Boxes in two (or more) Countries (etc.). Every little Bit (pun) helps a Bit (seriously: save a bit, save a byte).

Most people need not worry about MM, much like most people don"t need RAID, ECC or even to back up their Data: do not let them Host your Company's Software unless you can afford downtime.

If your Server has this implemented be sure you can charge extra for it or make use of the Feature to automatically manage your Servers, otherwise you don't need it.

Here are some example implementations from various Mfgs:

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.