Can someone share their experience in building a distributed mirrored filesystem between multiple FreeBSD machines? I. e. we have two (three, four...) servers and special partition "part1" mounted on each of them. We make some changes on it on the machine1 and theese changes instantly take effect on "part1" at all other machines. There are not often write operations on our "cluster", but very often read operations (like static web data of high loaded internet project). We want to have a symmetric access to all machines at the same time (witout "blocking" access to one of them). Our goal is providing high availability, fault tolerance and redudance (and probably hot-swap adding and removing members of this "cluster"). Are there any native technologies like Ceph for Linux?
There are a couple of options for FreeBSD.
HAST is built into FreeBSD base and it's rather like DRBD. It's not bad to set up, but it is limited to two nodes, a primary and a secondary.
There is also work being done to port Ceph to FreeBSD but it doesn't sound production ready.
Hopefully this quick rundown is helpful. =)