Intel PRO/1000 Server NIC on Windows latency. How to measure?

Question

Please note - this question is about latency of the card itself. Not latency of internet or anything else. Thank you for your attention on this matter.

It is often said that good server NIC does make difference compared to low grade cards and on-board Ethernet chips.

I understand that on-board chip will likely consume CPU time and stand-alone card would use its own chip.

Also server card would be faster than low-grade card.

Are these 2 facts explaining most of the server cards advantages over on-board and low-grade cards ethernet?

How would you measure latency on the card itself? I saw some reports stating some microseconds figures of NIC latency but I dont see how to measure that... Any advise will be appreciated.

EDIT: Since everyone is talking about LOM chipset. My particular chip is Broadcom BCM5716C Nextreme II GigE. What Intel PRO card can beat it?

EDIT2 : Please note that the whole exercise is about improving latency nothing else. If I can win 50us for $100 price I'd be happy.

Answer 1

you are correct that, in general, a server grade NIC should have a smaller impact on system performance and have lower latency than a low-cost NIC.

however, it is not true that an add-in NIC always performs faster than an on-board NIC. nearly every server nowadays comes with multiple network ports onboard using the same components as their server grade add-in NICs. the deciding factor is not onboard vs. add-in, but again the quality of the design of the NIC, and in the case of onboard NICs the quality of the integration.

with the exception of vendor-provided apps that provide hardware testing tools (such as the HP Network Config Utility), i can't think of a way to test NIC latency that doesn't involve something external such as a switch. and then you'd have too many variables to get measurements accurate in the microsecond range.

Answer 2

FYI, there is a bug in tcpip.sys that results in lower performance for adapters high-throughput, low-latency networks. This issue occurs because TCP window scaling does not work when network latency range is from 1 to 8 milliseconds. Resolved in tcpip.sys version 6.1.7601.21921.

Slow data transfer speed in Windows 7 or in Windows Server 2008 R2
http://support.microsoft.com/kb/2675785

Answer 3

On board LOM (LAN on motherboard) chipsets may not provide as many offload/optimization features as a separate purpose-built NIC but that doesn't imply that it would itself consume host CPU resources. Indeed, some of the LOM chipsets (or ones very close) are used on standalone NIC's.

Measuring latency in a meaningful way is a lot harder than it might seem. You would likely need a custom low level driver, very specialized measurement software and a very high resolution external time source. It isn't a trivial issue.

While there are likely some wins in terms of latency (probably in the hundreds of nanoseconds at most) the bigger win is usually in terms of throughput and host CPU (those optimizations and offloads mentioned earlier). There are also often management features and troubleshooting tools that make server NIC's desirable as well.