Anyone have any experience using MooseFS? I want an easy distributed storage platform to store static data archive of about 10 TB and serve it to 20-40 nodes. Also I want to be able to add storage as the archive grows without having to rebuild the filesystem. I don't care if it's a bit slow. I just want it to be simple and stable. Basically from what I can see for OS X it's between MooseFS and Gluster. Any other suggestions?
closed as not constructive by Magellan, Ward♦, Scott Pack, mdpc, HopelessN00b Nov 22 '12 at 1:15
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.
I just installed moosefs on 8 servers(8 core, 6 2tb drives), and it worked very well. Its a little slow(50-70MBs). one server died and moosefs rebalanced the data across the remaining systems without loosing any data.
I've played around with both systems, not production though. Gluster is nice, installation incredible easy, but MooseFS felt much more industrial and robust, and web monitoring definitely helps. I'd give MooseFS a try. Just remember to install a metalogger for backup.
I've only had good experiences with MooseFS. After testing Ceph on a 6-node cluster for a few weeks and experiencing data loss, I switched over to MooseFS and it worked great, both in terms of speed and stability, and still does. Having a system that is fully distributed without a master node could be nice, though.
My experiance with MooseFS is rock stable. I use Ucarp to HA the MooseFS metadataserver with meta data loggers. Set Moosefs replication goal to 2-3 in MooseFS. Use MooseFS CGI to monitor the cluster replication state. Experience are MooseFS writes are a little bit slow, reads are super fast.
My experience from before of GlusterFS is not stable. I would provoke it with test cases things like a three node cluster. For example I tried a test with a sample Linux iso file, shutdown a node, write 0 bytes to the sample iso file. Bring up the shutdown node again. See file size glitch on the clients between the shutdown node file size and the live nodes file size. It just short flipped size, before it resumed the final file size. Depending on your use cases this may or may not be acceptable.
I would either use MooseFS or Ceph with XFS more leaning to MooseFS since its been in over seven years in production use by its authors.
I haven't seen MooseFS in a production environment, but GlusterFS is a mature product that I've seen work well in large and small installs.
If you haven't read the MooseFS Wikipedia article you may want to check it out:
- No clustering for the metadata server: SINGLE POINT OF FAILURE!
- Striping ... for better or worse.
OpenStack object based storage is a newcomer brought to you by Rackspace and NASA. It might be worth looking into.