1

I have a locally attached 5 TB RAID6 system (IBM DS3512). It will be used as a storage for large data files that are written in sequential writes, and then read back for processing. Eventually the data is deleted.

Directory traversal is not important, as we have our own indexing service.

Since this is an online system, availability and resilience from corruption is important, as is fast rebuild time.

Does XFS have particular advantages over EXT4 in this context?

Furthermore, how would I go about tuning the filesystem?

The target system runs RHEL 6.3.

2
  • What is "your own indexing service"? If it gives a filename, the directory traversal certainly matters, more so if there are lots of small(ish) files in few directories...
    – vonbrand
    Feb 20, 2013 at 20:12
  • Yes, it is based on file names (full path). Files are typically 30MB+ in size. We employ directory hashing to spread files across directories to keep number of files per directory reasonable. Feb 21, 2013 at 14:11

2 Answers 2

4

I'd go with the default ext4, if only because it has shown in practice that it can take quite a beating, and in case of trouble there will probably be much more expertise at hand.

Oh, and before wishing you good luck, don't believe what colored squares with missing pieces tell you on random Internet sites. They might be spouting nonsense, no, they are spouting nonsense unless they know your problem and setup intimately. Set up an experiment with realistic data and workload, and measure. See if the difference really matters, look for other inputs. Check what your operating system vedor recommends.

Oh, and good luck! Please do add an answer (or a comment) with your analysis and conclusions here or contribute it to your distribution's documentation. Might even think of writing an article for LWN...

2

I think XFS can be tuned well for this purpose. It caches aggressively, handles large files well, works with large file and directory counts and is resilient.

On a RHEL 6.x system, you'll want to employ the tuned-adm framework and bias it towards the intended performance characteristics of your application.

Based on your description, it makes sense to consider the throughput-performance tuned profile.

Note:
There is a small load-related bug that impacts XFS on November 2012 and newer EL6 kernels... There is also a unique optimization feature that is now default in the in-kernel XFS version.

While a big proponent of XFS for the past decade, I've been moving many installations to ZFS on Linux as a replacement, especially for large filesystems.

7
  • How would you reckon XFS fares in terms of recovery in case of a server crash? The raid controller is battery backed, so it's really only kernel crashes / uncontrolled shutdowns we need to care about. Feb 21, 2013 at 14:14
  • We have used ZFS before, as the systems ran Solaris before. Is ZFS on Linux compatible with a pre-existing ZFS from a Solaris host? Feb 21, 2013 at 14:15
  • @AsgeirS.Nilsen Absolutely. You can zpool export/import existing pools into the Linux version.
    – ewwhite
    Feb 21, 2013 at 14:25
  • 1
    XFS still has some pretty serious memory needs. I recently set up a 10TB XFS file system with a few million files (log archive storage) and was unable to run xfs_check on the system with 24GB of RAM. I've read that this has been greatly improved in the recent past but this was on CentOS 6.3, roughly the equivalent of RHEL 6.3.
    – user143703
    Feb 23, 2013 at 16:14
  • @yoonix that doesn't sound right.
    – ewwhite
    Feb 23, 2013 at 16:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .