Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

While doing some benchmarking on a network app I discovered strange behavior that involves 100MBit and 1GBit ethernet clients connected through a switch (at-gs900/8e). I have one server (Gbit) and multiple clients, the network app uses TCP/IPv4. The clients receive big data streams over a single connection each.

As long as only clients with GBit speed receive data from the server all is fine, the uplink of the server is saturated even by just one client; when multiple clients receive data, the bandwidth is split up evenly among them. But as soon as one 100MBit client also receives data from the server, the output drops to pretty much exactly 100MBit/s per client, even for the GBit ones. So with only the 100MBit client receiving data I get 100MBit/s output, with an additional GBit client I get 200MBit/s and so on... My first idea was like "maybe the server's NIC switches back to 100MBit/s FD", but that clearly doesn't make any sense as I get 100MBit/s output per client, not in total.

As a side note, all clients were continuously connected to the server during the test (idle TCP connections). Can anyone enlighten me what is going on here, and if this is expected behavior? I really hope not, as it would make switches quite stupid devices. I hope to get some different models to test during the next days...

Update: I found out that this is caused by ethernet flow control, namely so called pause frames. I did not figure out who is creating these frames, the 100MBit client or the switch itself, but it is still weird to me that it won't throttle the server to 100MBit/s in total, instead of 100MBit/s per client... So the "solution" on the server is to disable flow control, which at least in my scenario is fine, as TCP deals with congestion on its own: ethtool -A eth0 autoneg off rx off tx off will result in 100MBit/s for the 100Mbit/s client and 900MBit/s for a second, GBit-capable client. :-)

share|improve this question
    
The ports on the switch are set to auto negotiate for duplex, how are the nics configured? From what I can tell they have to be auto negotiate or half duplex. –  dbasnett Jul 16 '13 at 12:44

1 Answer 1

I once had a buffering switch that behaved somewhat similarly: if one port in the flow was at a lower speed, the other ports were dragged down to the lower speed whenever communicating with it.

Am I correct in saying that you're seeing all your ports being dragged down, even when they do not have a flow involving the slow one? If so, it sounds like a bug, one you should discuss with your switch salesperson.

And note I said salesperson: the usual customer support organizations are tasked with making you happy with what you have, not fixing stuff and so are unlikely to get you a replacement switch (:-))

--dave

share|improve this answer
    
Just tested that. The other ports are not affected, I still get 1GBit/s between two clients while the server is serving the 100MBit client. But I found out how to "fix" this, I'll update my question in a second. –  Simon Jul 19 '13 at 11:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.