Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've found myself needing redundant storage at a block level. File-level replication (Gluster, GFS, etc) does not work for my use case.

It looks like DRBD is the go-to solution for block replication. There don't seem to be too many other sane options. Have I failed in my research, or is DRBD the only game in town?

share|improve this question
Hi Charles, why can't you use file level replication?\ – nsn Oct 4 '15 at 17:51
The use case was replication of VM disk devices across machines, where the devices were backed by LVM volumes and served via iSCSI - block devices, not files. The end goal, as noted in comments below, was basically DIY iSCSI failover. – Charles Oct 6 '15 at 21:56
up vote 7 down vote accepted

Yes, DRBD is the only replicated block device out there that can handle concurrent writes. If you plan to put a filesystem on top, it obviously needs to handle multiple writers as well, like GFS(2) and OCFS(2) do.

Please note that if you can afford higher levels of abstraction for redundancy, you'll likely be much, much happier with file-level semantics, so you should really think twice before going with block-level semantics. If you can't use higher levels of abstractions, but have money to throw at the problem, you can achieve significantly better performance with a good SAN.

But you probably know that already.

share|improve this answer
In this case, the thing I'm actually trying to achieve is some variety of warm failover for iSCSI targets to create a simplistic SAN. This is mostly a learning exercise. My distro of choice doesn't come with native DRBD support due to silly political decisions and running a slightly-too-old kernel. – Charles Jul 4 '12 at 20:06
DRBD will allow you to do active/standby and more recently active/active mirroring. If you chose active/active you'd have to ensure the filesystem supported it (hence GFS etc in the above). You'll probably want to use something like Heartbeat to trigger a failover (or just rely on an administrator doing 'drbdadm $resource up|down' as necessary). – TheGingerDog Jan 2 '14 at 10:26

You could set up a RAID set using iSCSI devices but I'd be wary of doing this with asymmetric storage devices (which in the case of remote storage includes the network) - OTOH DRBD is explicitly designed to support such usage.

Is there a reason you don't like DRBD?

Have I failed in my research

If you think that GFS is a replicating cluster filesystem, then I'm afraid so.

share|improve this answer
It isn't? Okay, yeah, I suppose that on it's own it isn't. – Charles May 30 '12 at 17:02

I have heard about a variant of the network block device (NBD) which supports replication: ENBD. However, I don't know about the status of that project. The website doesn't look as if it was still supported, though.

share|improve this answer
Yikes, no updates since the 2.4 kernels? Still, good find. – Charles May 30 '12 at 8:20
Another solution would have been to export a file which is replicated using a cluster file system with NBD, but I don't think you would want to do that. No, DRBD is really the way to go! Using it for some years, never lost any data. – Oliver May 30 '12 at 9:33

Well, there's also MARS (Light). According to the documentation this widely used at German provider 1&1

share|improve this answer
Isn't it async only? "Synchronous or near-synchronous operating modes are planned for the future, but are expected to work reliably only over short distances (less than 50km), due to fundamental properties of distributed systems." <-- from MARS docs – BaronSamedi1958 Jun 13 at 16:10

There is one alternative: You could use SAN devices with native replication where the disk arrays's controllers do all the replication work themselves. It's quite expensive though, but doesn't need configuration on the hosts.

share|improve this answer
It happens that I'm basically trying to build a simple SAN. – Charles May 30 '12 at 8:19
@Charles: I guessed something like that, but it is an alternative :) – Sven May 30 '12 at 8:29

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.