Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm using a FreeBSD NAS with RAID-Z.

I can read ~300MB/s from the ZFS disks to /dev/null on the box, but only get about 50MB/s over GBit-Ethernet with SMB to Windows 7 (Samba 3.5.6). Both systems have Intel-PCIe-NICs and are connected directly.

Samba is configured to use AIO and I already tried to tune TCP/IP:

kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendspace=1048576
net.inet.tcp.recvspace=1048576
net.inet.tcp.sendbuf_max=8388608
net.inet.tcp.recvbuf_max=8388608
net.inet.tcp.delayed_ack=0

Any ideas what's causing the bottleneck? I think the link should handle 100 MB/s easily.

share|improve this question
    
How old are the systems? I was just working with someone yesterday with the exact same issue and they were using an 8 year old server that simply couldn't push more than ~48MB/s over a NIC. Try using netperf (in the ports tree)? –  Chris S Jan 7 '11 at 14:30
    
@Chris S: PCIe slots aren't so old –  Javier Jan 7 '11 at 14:37
    
Are you using jumbo frames? –  bot403 Jan 7 '11 at 14:49
    
The system is new (Athlon II X2 2,8Ghz). No I'm not using jumbo frames. But I don't think they would improve the throughput that much? –  Axel Gneiting Jan 7 '11 at 14:49
    
FreeNAS allows exporting the same files through different means. Try transferring the same files using a different protocol. If the speed changes, it's the protocol, if the speed doesn't change, then it's hardware or configuration. –  Marcin Jan 7 '11 at 16:04
show 2 more comments

2 Answers

up vote 1 down vote accepted

Okay, there actually were two problems:

  1. Jumbo frames weren't so irrelevant after all. That gave me a huge performance boost.
  2. My SSD disk can easily write 100 MB/s, but I recently turned on NTFS compression which created problems (second long stalls while writing). A TRIM optimization resolved that.
share|improve this answer
add comment

To test the cable and bandwidth of your drivers, you should take out hard disk (and anti-virus for the windows box) write limits.

So you can either:

  • create a RAM disk on the windows box, download/push files onto it

  • create and redirect a samba share to /dev/null and push files into it from the windows box

  • run a cross-platform bandwidth utility (probably part of the Intel NIC utilities on the windows platform).

If anyone knows of a cross-platform windows-unix benchmark tool, please let me/us know :-)

And yes, you will need to make sure you have matching jumbo frames (2048 Bytes seems to be the 'norm') on your nics and switch. The standard TCP/IP packet size is a bit small for Gigabit streams.

share|improve this answer
    
netperf works on Windows and Unix –  Axel Gneiting Jan 7 '11 at 17:51
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.