Similar to Is there a Distributed SAN/Storage System out there? or Distributed redundant storage solution?, except with a couple key differences (and two years newer).

Context: I have a small Rocks cluster (a few dozen machines of varying capabilities), and would like to have all of my nodes pool their excess storage space to allow for a distributed filesystem. I maybe can spare a machine that would act as a server for it, but would prefer not to.

Unlike the above, all of these machines share user lists, and no user can gain root--security and encryption concerns are nearly negligible. (FS mounted by root; contains one directory per user with their perms)

I feel like it should be pretty easy to do, I just can't find any solutions that aren't WAY more complicated than what I'm looking for.

EDIT: My motivation is that every now and then a big IO load hits, taking down the NFS. I am well aware that "go buy a nice SAN with a bunch of SSDs" is a quite nice solution; it's just not available at the moment--I'd like to know what I can do with software and the hardware I have.

  • If you haven't, take a look at GlusterFS – uSlackr Jun 13 '12 at 1:29
  • I read over its description before, but hadn't previously considered making all machines simultaneously clients and servers... does that even work? (without insane overhead) – zebediah49 Jun 13 '12 at 1:32
  • I don't think you have to use clients as servers, but it is a versatile file virtualization platform. – uSlackr Jun 13 '12 at 1:35
  • It doesn't appear designed for that-- I was wondering if I could choose to do so (so I don't have to dedicate any servers). I believe the answer might be yes. – zebediah49 Jun 13 '12 at 1:42
  • What about Dat and IPFS? – CMCDragonkai Aug 3 '18 at 12:37

This isn't a very good answer, but I'd like to make the question closed.

uSlackr's suggestion of GlustreFS seems to the best option. There are a number of examples of it being used on clusters, and its elastic hash metadata model means that there is no centralized metadata server.

There is nothing that says a single machine cannot run both client and server.

That said, it fulfills the requirements as outlined.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.