I'm running a storage server that has to mount home-dirs of different users at several (5) centos servers. The NFS storage server has direct Fibrechannel storage (with high speed, verified).

When the NFS is mounted on a client, big file writes go fast (30MB/s). However lots of small files have a serious problem. If I write to the NFS, or even worse, copy files from one folder on the NFS to another NFS folder, this is very slow (~100kB/s). A first search revealed that NFS has the problem of doing slow file creations (somewhere 30 files /s). I understand that this will be a cause of slow speed with lots of small files.

Can this be solved with another NFS configuration or by using another protocol? It would not be feasable that users have to copy their files to a local disk each time.

my NFS configuration: NFS4

/NFSFOLDER IP/MASK(rw,sync,no_root_squash_subtree_check)

at client side:
IP:/NFSFOLDER /mountpoint nfs4

NFS settings are set to defaults.

Thanks in advance

  • iSCSI, NBD, GlusterFS, SSHFS (Encrypted, probably not faster). Quite a lot of alternatives, but mostly exporting block devices not a filesystem. – Devon Feb 11 '15 at 18:47
  • I need a File system. Block exports are done at a lower level but do not sypport (I think) the multi-access folder I need. – Jeffrey Feb 11 '15 at 20:02
  • Or if you're really game, I've had quite a bit of success with cephfs. It's quite a different architecture to what your running though. – hookenz Feb 11 '15 at 20:04
  • We've had success using CIFS (i.e., Samba) with Unix extensions, although I haven't tested performance. – Josh Kelley Feb 11 '15 at 21:05

This slow behavior is caused by the "sync" directive on you NFS export. You can change it to "async" for better performance, but you need to understand what it means in regard to data safety.

I suggest you to read the exports (5) man page: http://linux.die.net/man/5/exports

  • Async will only be possible for non-critical data because there is no UPS – Jeffrey Feb 11 '15 at 20:01
  • Sure, but a test with it will be useful to understand if and how the "sync" behavior affect your setup/performance. Anyway, what is the latency between NFS clients and server? – shodanshok Feb 12 '15 at 9:10
  • Does async also give better performance for reading. So are read-only shares better placed in async mode? – Jeffrey Feb 12 '15 at 21:17
  • I should not: the async behavior is a write-only affair. From the man page: "Reply to requests only after the changes have been committed to stable storage" – shodanshok Feb 12 '15 at 21:34
  • Thanks, works fine now. Non critical temporary data is stored in async mode and is much faster! – Jeffrey Feb 13 '15 at 7:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.