I have a cluster of machines with two 10Gbps intel network cards on each machine. 64 port Router, with each port of capacity 10Gbps is being used. when i try to measure maximum network performance between machines connected to same router, I see that i get close to 99% bandwidth for a pair of network cards between only 2 specific machines, but i get only around 40-60% bandwidth between rest of pairs of machines.

The driver settings are the same in all machines.

The network test tools i used were iperf and NTttcp.

All machines are identical with respect to CPU, memory, network cards.

I tried swapping the cables of the high network performance bandwidth machines with cables of other machines. That dint seem to effect the results.

Can someone please suggest what might be wrong?

  • 64 port router or switch? What settings are in the switch? is there some port filtering switched on some ports and not others? – Matt Sep 12 '11 at 22:43

Your switching hardware backplane will have some bandwidth limitation which limits the total overall transfer rate of data going through the switch and which is lower than the total sum of your port's wirespeed (64 * 10 G * 2 (duplex operation) = 1,3 Tbps) or have a maximum processing capacity in frames per second which is hit even before backplane saturation for small-frame-workloads. If you are measuring while the switch is otherwise heavily loaded, you might hit one of its limits - check with the switch datasheet to rule this out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy