Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to know what the best (read: highly performant and fault tolerant) filesystem solution based on Amazon's S3 is. I have looked a s3fs, but I notice that it has not been worked on for a while now, and that got me thinking about stability and features.

What other filesystem solutions have you used and what has your experience been with it?

share|improve this question

closed as primarily opinion-based by kasperd, Sven Mar 29 at 21:23

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

up vote 3 down vote accepted

You're not going to get anything that's "highly performant" (at least when compared to modern disk storage systems) when dealing with the latencies inherent with using remote storage. Performance is further hampered by the fact that the target "device" isn't a block device, which necessitates an abstraction layer that even further inhibits performance. S3 is at its best when used for what it was intended for, storing and retrieving individual files using http(s).

Regarding fault tolerance: here's an area where S3 excels. Standard S3 buckets will survive the failure of two of Amazon's facilities, which is quite good for what you're paying. If your needs aren't that stringent, they also recently released the slightly less redundant (but ~30% cheaper) RRS buckets, which are configured to survive the failure of one of Amazon's facilities.

share|improve this answer
Erik, thank you for sharing your opinion and thoughts. I should have qualified my "performant" requirement with the caveat that I understand a remote, abstracted "filesystem" will not meet the performance of a local FS. That being said, with the multitude of s3f2 solutions out there, have you used one, and if so, why did you choose that one over another. Bottom line, I will be implementing one, and I would like to know before I do, which one has the best track record. – QWade Sep 26 '10 at 14:01
Understood. Yes, I've used this s3fs version: . It worked fairly well. One nice thing is that it supports disk cache, which can help speed up IO a fair amount. – EEAA Sep 26 '10 at 17:21

Maybe S3QL will be interesting to you. I am not going to praise it here (because I'm the developer), but there is a comparison of S3QL to other S3 file systems at

share|improve this answer
s3fs's file limit size is currently "only" 64GB, while with S3QL it is S3's own limit (2TB). – mr-euro May 8 '12 at 21:50

Nasuni is a startup with a relatively low-cost cloud storage appliance. Worth looking into if you need to protect TBs of data on S3 simply.

p.s. I am not affiliated with the company, just a fan of the technology.

share|improve this answer

You can also try Amazon EFS, ObjectiveFS or SoftNAS.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.