I have a lot of experience with PCIe, some with USB, and little with Ethernet, so the following has got me confused:

I'm evaluating the performance of a USB-to-Ethernet controller, vs a PCIE-to-Ethernet controller (nic), in terms of latency. The precision is important in this experiment, down to the microsecond (I know that in real life it could have no real impact).

To get a baseline of comparison, I hooked up 2 identical PC motherboards directly with a 1 meter Ethernet cable, to their onboard Ethernet ports. Both PCs are running latest version of Windows 10 (build 1709), with no firewall, no virus protection, and no other network connections or WIFI enabled.

After a few seconds each host got an ip, and I tried from the command line:

ping <host ip> -n 100

The Windows ping.exe only shows "<1ms" latency, so I looked for a more accurate tool. I found hrping online, which has accuracy of microseconds: https://www.cfos.de/en/ping/ping.htm#options

Using hrping, I tried:

hrping <host ip> -n 100

and this time I got an average result of 0.426ms for the 100 packets.

After than I ran some tests - instead of using the on board Ethernet controller, on one of the hosts I used once my nic controller, and the next time I used my USB-to-Ethernet controller. Each time I used the same hrping command to send 100 pings.

After a couple of hours I again hooked up the 2 PCs directly with the 1m Ethernet cable, to their onboard Ethernet ports, and ran hrping again. I expected roughly the same 0.426ms as before, but this time I got 0.845ms. So I ran the tool again, and suddenly all the numbers were higher. I tested my nic and USB-ethernet controller again, and their results were higher too by roughly a 0.5ms, compared to the previous results.

My question is this - why do the ping values vary so much? In the case of the 2 PCs connected directly, what is interfering with the ping results? There are no switches in between them, and the PCs were not doing anything, so why so many different results? Is it just that hrping is not returning reliable numbers?

Is there a more accurate method of measuring the network latency between the 2 PC's, in order to compare it with the latency through my nic and USB Ethernet adapter?

  • Very odd question to ask here to be honest but whatever way you cut it you're always going to see lower latency between PCIe-based NICs unless they're terrible/cheap NICs simply because using USB introduces do many additional components, both software and hardware, to add their own little latencies to the picture. No idea why you're seeing this issue, see if a reboot of both machines helps but ultimately it's likely to be a bad/inappropriate test as your end-design is unlikely to be two machine connected by a single cable - try testing in a more realistic setting - and don't use USB :) – Chopper3 Feb 28 '18 at 15:15
  • As you say, I expected the nic to perform better than the USB ethernet controller, but the results were so close it is not conclusive. Restarting the PCs didn't change the results in my second round of testing. – Ori Rosen Feb 28 '18 at 15:20
  • This can be power-management related. Try setting the power profile for maximum performance and disabling the windows firewall. Also, if you NICs support interrupt coalescing, disable it. That said, this kind of test is better done on Linux than Windows (which is quite slow in processing deferred procedure calls/IRQs). Moreover, it require profound understanding of the IP stack/kernel drivers. – shodanshok Feb 28 '18 at 16:16
  • Ping isn’t really representative of real world use. I don’t know why it changed between tests. But, I would try something like iPerf to push real, heavy duty TCP or UDP traffic through the connection. That should give a more realistic result. – Appleoddity Mar 1 '18 at 5:46
  • I agree that perhaps using Windows was not the best idea. I will accept a solution to test the latency using Linux on the 2 PCs – Ori Rosen Mar 1 '18 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.