18

I've found myself needing redundant storage at a block level. File-level replication (Gluster, GFS, etc) does not work for my use case.

It looks like DRBD is the go-to solution for block replication. There don't seem to be too many other sane options. Have I failed in my research, or is DRBD the only game in town?

2
  • Hi Charles, why can't you use file level replication?\
    – nsn
    Oct 4, 2015 at 17:51
  • The use case was replication of VM disk devices across machines, where the devices were backed by LVM volumes and served via iSCSI - block devices, not files. The end goal, as noted in comments below, was basically DIY iSCSI failover.
    – Charles
    Oct 6, 2015 at 21:56

7 Answers 7

12

Yes, DRBD is the only replicated block device out there that can handle concurrent writes. If you plan to put a filesystem on top, it obviously needs to handle multiple writers as well, like GFS(2) and OCFS(2) do.

Please note that if you can afford higher levels of abstraction for redundancy, you'll likely be much, much happier with file-level semantics, so you should really think twice before going with block-level semantics. If you can't use higher levels of abstractions, but have money to throw at the problem, you can achieve significantly better performance with a good SAN.

But you probably know that already.

3
  • In this case, the thing I'm actually trying to achieve is some variety of warm failover for iSCSI targets to create a simplistic SAN. This is mostly a learning exercise. My distro of choice doesn't come with native DRBD support due to silly political decisions and running a slightly-too-old kernel.
    – Charles
    Jul 4, 2012 at 20:06
  • DRBD will allow you to do active/standby and more recently active/active mirroring. If you chose active/active you'd have to ensure the filesystem supported it (hence GFS etc in the above). You'll probably want to use something like Heartbeat to trigger a failover (or just rely on an administrator doing 'drbdadm $resource up|down' as necessary). Jan 2, 2014 at 10:26
  • Is there any software for replication with file-level semantics? I can't find any
    – brauliobo
    Mar 16 at 17:35
8

Well, there's also MARS (Light). According to the documentation this widely used at German provider 1&1

1
  • Isn't it async only? "Synchronous or near-synchronous operating modes are planned for the future, but are expected to work reliably only over short distances (less than 50km), due to fundamental properties of distributed systems." <-- from MARS docs Jun 13, 2016 at 16:10
2

You could set up a RAID set using iSCSI devices but I'd be wary of doing this with asymmetric storage devices (which in the case of remote storage includes the network) - OTOH DRBD is explicitly designed to support such usage.

Is there a reason you don't like DRBD?

Have I failed in my research

If you think that GFS is a replicating cluster filesystem, then I'm afraid so.

1
  • It isn't? Okay, yeah, I suppose that on it's own it isn't.
    – Charles
    May 30, 2012 at 17:02
1

I have heard about a variant of the network block device (NBD) which supports replication: ENBD. However, I don't know about the status of that project. The website doesn't look as if it was still supported, though.

2
  • 1
    Yikes, no updates since the 2.4 kernels? Still, good find.
    – Charles
    May 30, 2012 at 8:20
  • 1
    Another solution would have been to export a file which is replicated using a cluster file system with NBD, but I don't think you would want to do that. No, DRBD is really the way to go! Using it for some years, never lost any data.
    – Oliver
    May 30, 2012 at 9:33
1

There is one alternative: You could use SAN devices with native replication where the disk arrays's controllers do all the replication work themselves. It's quite expensive though, but doesn't need configuration on the hosts.

2
  • 1
    It happens that I'm basically trying to build a simple SAN.
    – Charles
    May 30, 2012 at 8:19
  • @Charles: I guessed something like that, but it is an alternative :)
    – Sven
    May 30, 2012 at 8:29
1

The question is wrong:

Is DRBD the only viable block replication solution for Linux? I've found myself needing redundant storage at a block level.

No, it is not. You have e.g. Linux MD (software RAID), LVM RAID. They provide redundancy for block devices.

So you probably wanted to ask:

Is DRBD the only viable NETWORK block replication solution for Linux?

And then again you have other options.

If the client is one node only you can build software RAID on the client which replicates over several network storages.

If you have multiple clients you can use clustered LVM block devices.

File-level replication (Gluster, GFS, etc) does not work for my use case.

GFS( Redhat's GFS2) is a shared device cluster file system. It does not provide redundancy. Other local filesystems like BTRFS and ZFS can do it though. As well as other distributed file systems.

-1

That's an inaccurate statement. SIOS Technology www.us.sios.com has DataKeeper (https://us.sios.com/resources/product-brief-sios-datakeeper/) that does block level replication in Linux and Windows.

2
  • 1
    This is why product recommendations are off-topic for ServerFault. You're answering an 8 year old question. Maybe the statement was accurate back then. But ServerFault isn't the right place to keep up with product developments. Apr 16, 2021 at 18:52
  • "Maybe the statement was accurate back then." -- It was, but it has been a very long time since I asked about this problem. While it's not the type of block level replication that I wanted, I've since settled on (Proxmox-provided) ZFS to solve my VM disk image replication needs.
    – Charles
    Apr 18, 2021 at 16:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .