3

While doing some benchmarking on a network app I discovered strange behavior that involves 100MBit and 1GBit ethernet clients connected through a switch (at-gs900/8e). I have one server (Gbit) and multiple clients, the network app uses TCP/IPv4. The clients receive big data streams over a single connection each.

As long as only clients with GBit speed receive data from the server all is fine, the uplink of the server is saturated even by just one client; when multiple clients receive data, the bandwidth is split up evenly among them. But as soon as one 100MBit client also receives data from the server, the output drops to pretty much exactly 100MBit/s per client, even for the GBit ones. So with only the 100MBit client receiving data I get 100MBit/s output, with an additional GBit client I get 200MBit/s and so on... My first idea was like "maybe the server's NIC switches back to 100MBit/s FD", but that clearly doesn't make any sense as I get 100MBit/s output per client, not in total.

As a side note, all clients were continuously connected to the server during the test (idle TCP connections). Can anyone enlighten me what is going on here, and if this is expected behavior? I really hope not, as it would make switches quite stupid devices. I hope to get some different models to test during the next days...

Update: I found out that this is caused by ethernet flow control, namely so called pause frames. I did not figure out who is creating these frames, the 100MBit client or the switch itself, but it is still weird to me that it won't throttle the server to 100MBit/s in total, instead of 100MBit/s per client... So the "solution" on the server is to disable flow control, which at least in my scenario is fine, as TCP deals with congestion on its own: ethtool -A eth0 autoneg off rx off tx off will result in 100MBit/s for the 100Mbit/s client and 900MBit/s for a second, GBit-capable client. :-)

Improve this question
1
  • The ports on the switch are set to auto negotiate for duplex, how are the nics configured? From what I can tell they have to be auto negotiate or half duplex.
    – dbasnett
    Jul 16, 2013 at 12:44

1 Answer 1

Reset to default
0

I once had a buffering switch that behaved somewhat similarly: if one port in the flow was at a lower speed, the other ports were dragged down to the lower speed whenever communicating with it.

Am I correct in saying that you're seeing all your ports being dragged down, even when they do not have a flow involving the slow one? If so, it sounds like a bug, one you should discuss with your switch salesperson.

And note I said salesperson: the usual customer support organizations are tasked with making you happy with what you have, not fixing stuff and so are unlikely to get you a replacement switch (:-))

--dave

Improve this answer
1
  • Just tested that. The other ports are not affected, I still get 1GBit/s between two clients while the server is serving the 100MBit client. But I found out how to "fix" this, I'll update my question in a second.
    – Simon
    Jul 19, 2013 at 11:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .