2

I would like to share EBS backed file system of size up to 20GB in 200K files across multiple EC2 instances. Master-master would be ideal, but I'd be happy with master-slave solution too.

I know that generic Linux solution would be to use DRBD, but is there some tool specific to AWS for that?

BTW. I've found similar question that was asked over a year ago: Sharing / replicating EBS across AWS nodes

Improve this question
2
  • Please post more when you find an actual answer to the problem. The accepted answer below is a start, but functional examples of "varous distributed, replicating or cluster file systems" is lacking.
    – Shef
    Jun 6, 2011 at 19:07
  • @Shef: actual solution to the problem was redesign to shared-nothing architecture.
    – vartec
    Oct 3, 2012 at 9:53

1 Answer 1

Reset to default
3
+100

Block devices as a computing model is not designed for access from more than one device. The EBS abstraction of this hardware model is not capable of changing the rules of the game because the file system drivers in the overlying computing instances are still the same. They can only do magic like snapshotting because the read access in the underlying layer doesn't interfere with what the software on the system expects to be going on on a block devices. If some system other than the one EC2 instance the volume is connected to tried to change the bits on your disks directly, the file system would self destruct.

The only thing you can do that is special with EBS is use snapshots to clone drives and attach each device clone to a separate instance. This would be useful if you had a relatively static data set with only periodic updates and needed fast local access to eat from many instances.

You will need to move your sharing architecture up one level to the filesystem. There are lots of tools to do this, including shared file systems such as nfs. Of course your performance will be limited to that of the EC2 instance you connect the device to and use as the file server.

If you require something more subtle than those two approaches, you will need to look into on of the varous distributed, replicating or cluster file systems. Also as you mentioned there are lower level synchronization tools like DRDB, but I suggest you are more likely to need a filesystem level than block level synchronization system.

For extra credit: check out some of the talk going on these days about whether block devices are ever the right model to use such as this article: Magical Block Store: When Abstractions Fail Us.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .