up vote 3 down vote favorite
2
share [g+] share [fb]

I'm using a FreeBSD NAS with RAID-Z.

I can read ~300MB/s from the ZFS disks to /dev/null on the box, but only get about 50MB/s over GBit-Ethernet with SMB to Windows 7 (Samba 3.5.6). Both systems have Intel-PCIe-NICs and are connected directly.

Samba is configured to use AIO and I already tried to tune TCP/IP:

kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendspace=1048576
net.inet.tcp.recvspace=1048576
net.inet.tcp.sendbuf_max=8388608
net.inet.tcp.recvbuf_max=8388608
net.inet.tcp.delayed_ack=0

Any ideas what's causing the bottleneck? I think the link should handle 100 MB/s easily.

link|improve this question
How old are the systems? I was just working with someone yesterday with the exact same issue and they were using an 8 year old server that simply couldn't push more than ~48MB/s over a NIC. Try using netperf (in the ports tree)? – Chris S♦ Jan 7 '11 at 14:30
@Chris S: PCIe slots aren't so old – Javier Jan 7 '11 at 14:37
Are you using jumbo frames? – bot403 Jan 7 '11 at 14:49
The system is new (Athlon II X2 2,8Ghz). No I'm not using jumbo frames. But I don't think they would improve the throughput that much? – Axel Gneiting Jan 7 '11 at 14:49
FreeNAS allows exporting the same files through different means. Try transferring the same files using a different protocol. If the speed changes, it's the protocol, if the speed doesn't change, then it's hardware or configuration. – Marcin Jan 7 '11 at 16:04
show 2 more comments
feedback

2 Answers

up vote 1 down vote accepted

Okay, there actually were two problems:

  1. Jumbo frames weren't so irrelevant after all. That gave me a huge performance boost.
  2. My SSD disk can easily write 100 MB/s, but I recently turned on NTFS compression which created problems (second long stalls while writing). A TRIM optimization resolved that.
link|improve this answer
feedback
up vote 0 down vote

To test the cable and bandwidth of your drivers, you should take out hard disk (and anti-virus for the windows box) write limits.

So you can either:

  • create a RAM disk on the windows box, download/push files onto it

  • create and redirect a samba share to /dev/null and push files into it from the windows box

  • run a cross-platform bandwidth utility (probably part of the Intel NIC utilities on the windows platform).

If anyone knows of a cross-platform windows-unix benchmark tool, please let me/us know :-)

And yes, you will need to make sure you have matching jumbo frames (2048 Bytes seems to be the 'norm') on your nics and switch. The standard TCP/IP packet size is a bit small for Gigabit streams.

link|improve this answer
netperf works on Windows and Unix – Axel Gneiting Jan 7 '11 at 17:51
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.