I have an application that only runs on Linux, which will generate ~500TB of data over the next year, which all must be kept and backed up each day.

It would be temping to go with ZFSonLinux or btrfs, but they are not really stable.

One option could be having an OmniOS host with ZFS and mount it over NFS, but then there are the file locking issues.

Reading about OpenAFS, they write in the docs

AFS hides its distributed nature, so working with AFS files looks and feels like working with files stored on your local machine.


Does that mean, I can use OpenAFS instead of NFS without the file locking issues NFS have?

  • Never heard of OpenAFS. Have a look at CephFS also, we're using it and it's been incredibly stable and reliable. Although we're not using snapshots. – Matt Apr 15 '15 at 21:28
  • Does CephFS make the mounted fs look like it is local to the application? – Jasmine Lognnes Apr 15 '15 at 21:30
  • 2
    ZFS on Linux is perfectly stable. – ewwhite Apr 15 '15 at 21:40
  • @JasmineLognnes - yes. See: ceph.com/docs/master/cephfs . It just mounts as another filesystem and is posix compliant (if that matters). It depends what your requirements are really. Since you were looking at OpenAFS I thought I'd throw cephfs into the mix. If your app is going to run for 5 years then that's a lot of storage you need. ceph will store replicas of your data automatically. Not the same as a backup though. – Matt Apr 15 '15 at 22:16
  • @Matt ZFS snapshots and failed disk handling are the main reasons for using ZFS. How do you backup a large ceph fs on a daily basis? – Jasmine Lognnes Apr 16 '15 at 6:29

I'm not familiar enough with NFS to know what specific locking issues you may be referencing, but I generally hear that OpenAFS works better with whole-file locks, yes.

However, OpenAFS does not work well with byte-range locks across different machines (that is, locking certain byte ranges in a file, as opposed to locking entire files). If you are only accessing locked files from a single Linux client, then there should be no issues, but if you are trying to coordinate locks across multiple OpenAFS clients, then that will not work.

It's also not completely clear to me why you're using any networked filesystem at all, or why you're not considering other traditional local filesystems like XFS, or even ext4 (these may not satisfy your requirements, but it's not clear what your requirements are beyond storing 500TB of data...). To be clear about it, OpenAFS doesn't export a local filesystem like NFS does. Data stored in an OpenAFS fileserver is stored in its own format, so you cannot access the data outside of using an OpenAFS client. That is, even if you are accessing the files on the same machine as the server hosting them, you must go through the OpenAFS client over the AFS protocol, etc.

Also note that people typically find setting up OpenAFS to be more complex than NFS (at least, for non-Kerberized NFS).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.