0

We have a RedHat 7.4 Linux server with 2 10Gbps bonded, but for some reason, it doesn't send files at speeds faster than 1Gbps to another server with a 10Gbps NIC, receiving works just fine, but not sending.

Can anybody tell me where I can check what's causing this problem and how to fix it?

As a reference we ran iPerf on both ends and receiving reaches up to 9.53Gbps, but sending never gets higher than 955Mbps.

Thanks.

  • What software do you use to send/receive files? What mode of iperf (tcp/udp) have you used? – Anton Danilov May 2 '19 at 18:01
  • I think we are getting closer to an answer, it turns out the bond is set to mode=0, this mode is not supported by the switch(es) for transmission, but the switches allow for receiving, so we are going to set this mode to5 or 6 to favor transmission (5) or to balance all communication to see if it fixes it, I'll post the results of this change when it's done. – Jorge Hernandez May 3 '19 at 19:06
0

There are various reasons of your issue. General ways to troubleshoot it:

  1. So iperf gives the normal numbers, guess the issue isn't in network itself, but somewhere else. But check the counters of both bond member interfaces with ethtool -S and ip -s -s link show commands. Maybe iperf traffic passes trought normal leg, but file exchange traffic passes through ill leg of bond. Check the size of rings of NICs. Check the statistics of the pause frames - play with flowcontrol.
  2. First check point - the storage subsystem. Slow disks can be bottleneck. Use tools like blktrace, iotop, iostat.
  3. Capture the traffic with tcpdump/wireshark and analyse it. Wireshark has great built-in tools for it. Check the graph of window size - it shows existence of network issues like loss or congestions.
  4. Next check point - the size of read/write socket buffers. Increase it by your app config or change the default values with sysctl (variables *_rmem and *_wmem). Check the statistic counters with netstat or nstat.
  5. Look at the cpu load. Maybe one cpu core is overloaded, but other are idle.
  6. Check the GRO/LRO and other offloadings. Play with it.
  7. Check the congestion control algorithms.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.