Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a home network with a 1 GBit/sec switch as the central point, and all computers with 1 GBit/sec NIC's. However, I get at most 200 MBit/sec copy speeds over NFS shares. It seems I should be able to get much more. What can I realistically expect and how do I tune my systems and NFS server to get higher speeds ?

All systems involved are Linux, by the way (mix of CentOS and Fedora).

share|improve this question
1  
How fast are your source disks ? Does hdparm -t show they are capable of doing more than 20mb/sec sustained ? –  Dave Cheney May 3 '09 at 6:21
1  
It would be helpful to see a relevant line from /etc/fstab so we can see the options you're using to mount the NFS share. –  Philip Durbin May 4 '09 at 17:21
add comment

closed as off topic by Sam Jan 19 '12 at 12:02

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

7 Answers

You're getting beaten by the overhead. You want to look into jumbo frames and also expand your nfs read/write data size. There's lots of guides to NFS performance tuning on the 'net, like this one that doesn't look too out of date.

Also realize that you're never going to be able to do better than disk speed, so make sure that's not a bottleneck.

share|improve this answer
    
Most modern NFS stacks automatically optimize the read/write data size and will likely do a better job than you will. 'cat /proc/mounts' for the current values. –  Martijn Heemels Sep 12 '12 at 9:56
add comment

You should defiantly look into jumbo frames. Even cheap 5 port GbE hubs can do it these days, however you have to enable jumbo frames on both ends of the connection. To enable jumbo frames, do this

ifconfig eth0 mtu 9000

Here are the mount options that I use when automounting home directories from our filer

rw,intr,soft,nfsvers=3.tcp,nolock,noatime,rsize=32768,wsize=32768

Please update the question and let us know if this improves the situation

share|improve this answer
    
Note: not just the endpoints of the connection, but all devices along the path (and preferably in the entire LAN segment) must have jumbo-frames enabled, switches, servers, clients, etc., for them to work. Also, jumbo-frames are no magical solution, but they do help. –  Martijn Heemels Sep 12 '12 at 9:50
add comment

Jumbo frames can certainly help, but don't expect it to magically give you 100% increase in throughput. Check out my post here in another question that discusses a way to test network throughput without bringing disk i/o into the picture. This will give you a baseline to see if there are perhaps other factors not related to the network.

If you do plan on enabling Jumbo Frames, as some here have suggested, please be aware of some limitations and requirements.

  • All devices along the path must support and have jumbo frames enabled. This includes:
    • Both the sending and receiving nic
    • Both switch ports the devices are connected to
    • All L2 switch ports along the path from sender to receiver (switch-to-switch connections)
    • All L3 interfaces along the path, if applicable
  • Make absolutely sure that all devices in the same vlan support and are configured for jumbo frames.

Reason for the second bullet: Consider two PCs setup in the same vlan, one configured for 9000 byte frames, and the other for 1500 byte frames. The PC1, configured for standard frames, will be able to send packets successfully to PC2. However, the reply that PC2 send will all be 9000 byte frames. Since PC1 is configured for standard frames, it will be unable to process the jumbo frames, and they will be dropped. To fix this issue, you'll need to insert an L3 device (router) between the two PCs so they are able to communicate.

share|improve this answer
add comment

Bottleneck can be your network or your hard drives. First of all check which are, then optimize.

share|improve this answer
add comment

Have you got the ability to switch on NIC Interrupt Coalescence, Receive-Side Scaling or any form of TCP-offload (checksum or whole-packed)? They will make a huge difference if you can enable them, which depends on your NICs and IP stack.

share|improve this answer
add comment

For NFS specific information, be aware that newer implementation of NFS support not only UDP, but also TCP. Default is still usually UDP. Oddly enough, on very fast networks, TCP can be faster due to windowing. Try it both ways and compare.

For other tips, just search the site. The general form of this topic has already been beat to death eleventy-billion times. :-)

share|improve this answer
add comment

Don't forget to ensure that you have jumbo frames enabled, and make sure that any network switches you use also support jumbo frames and have then enabled. You may want to run RAID striping for speed on the NFS share.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.