I created a bond out of two interfaces of 10G with the following config in two servers:
Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Then I ran iperf3 server in one of the servers and iperf3 client in the other with a very simple config:
iperf3 -c $IP_SERVER -t 25 -O 5 -P 2
-P 2
makes iperf3 create two clients with different TCP source ports and thus the hash segregates the traffic between both physical links of the bond. However, in the 25 seconds I run the test, I see the throughput fluctuating between 11Gbits/s and 19Gbits/s. I repeatedly see this behavior in all the runs. I don't see this when running the iperf3 test in the single physical links that make the bond. Why could this be happening? Any hint?