up vote 2 down vote favorite
share [g+] share [fb]

I have a mail server using maildir format, with about 100GB of data, and several directories have over 20000 files each. I'm currently using ext3, and I noticed that it's a bit slow sometimes. So I'd like to try a faster file system, but it needs to be stable and reliable.

Which one would you recommend?

link|improve this question
1  
What testing have you done (if any) to verify that the filesystem itself is your bottleneck? – ErikA Oct 13 '10 at 18:45
I have to admit.. I haven't done any proper testing. Do you think it's likely not the filesystem? – aditsu Oct 13 '10 at 19:19
What makes you think that the filesystem is your bottleneck? "it's a bit slow sometimes" what exactly is slow? – Ben Quick Oct 14 '10 at 7:09
Opening a big folder in the email client takes a while, and rsync (for backing up maildirs to another machine) also takes its time checking which files are changed. It seems that I'm having some openvpn speed issues too. What do you recommend for testing the filesystem performance? – aditsu Oct 14 '10 at 7:22
Opening mail folders could be due to indexes for your mail(box|dir|client). As for performance testing, from google.co.uk/… points to iozone cyberciti.biz/tips/… – Ben Quick Oct 14 '10 at 7:51
feedback

6 Answers

up vote 3 down vote

I used XFS for a mail server setup (has been ext3 before) and definitely noticed an improvement with all the small files but it has been a while back and IIRC there are/were some "issues" with XFS and power-cuts (or any abrupt disruption to disk access for that matter) leaving all opened files filled with "0" and similar.

But as mention by others: testing is in order, the same FS can behave quite differently on different hardware and as I said in the beginning it has been a while since I setup that server.... (I guess about 4 years) and the available FS and also their performance might have changed quite a bit.

all in all this is just my 2ยข

link|improve this answer
+1 for mentioning testing. I also use XFS everywhere (many small files and some large files), and it comes out best on I/O testing every time. – pauska Oct 13 '10 at 21:11
+1 I've always seen performance benefits from XFS for mail servers too, and it's my general preference for them. – Twirrim Oct 14 '10 at 0:48
I've seen lots of contradicting opinions about XFS, ranging from "you WILL DEFINITELY LOSE DATA, and it's only good for large files" to "it's rock solid and optimized for many small files". I don't really know what to believe. Testing is desirable indeed, but it can take a lot of time to do it well. – aditsu Oct 21 '10 at 10:22
feedback
up vote 2 down vote

I agree with the above recommendations. Testing is very important.

Do also consider ReiserFS. One of its design goals was for efficient handling of large numbers of small (< 4KiB) files, which is what a typical maildir implementation has to deal with. Wikipedia has an informative article about it, highlighting both its pros and cons.

link|improve this answer
This sounds like a job that would be best served with ReiserFS. There is always many ways to do something, some may yield better performance than others. XFS is a great general system FS, this just doesn't look like a general situation. – TechZilla Sep 21 '11 at 16:06
feedback
up vote 1 down vote

I would go with XFS. You may want to read this thread (It first appeared in comp.mail.imap and comp.os.linux.misc but cannot find it in Google Groups)

link|improve this answer
feedback
up vote 0 down vote

How about ext4?

link|improve this answer
A valid choice, but it would be nice to find out more about the benefits in this scenario. – aditsu Oct 13 '10 at 19:22
That's what the link is for. :) – Cypher Oct 14 '10 at 2:22
feedback
up vote 0 down vote

Compared to EXT3, EXT4's 'big directory' performance is much better. Several improvements were made to handle the case of "umpty thousand files in one directory". The biggest improvements are made in the 100K+/per-dir range, so you might not be see any significant improvement in switching with your loads (testing would be a good idea) .

link|improve this answer
feedback
up vote 0 down vote

ext3 can be tuned to better handle large directories

run this: tune2fs -O dir_index /dev/hdXY
then this: e2fsck -D -f /dev/hdXY

The first one enables it for all future directories. The second one crawls through all your existing directories and builds the hash trees. That may take a good long while, so plan for some downtime.

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.