Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My Ubuntu servers ethernet interface that connectes to ISP's multiplexer shows errors. Here is the snapshot:

          RX packets:204564288 errors:3193970 dropped:0 overruns:0 frame:3138402
          TX packets:29305799 errors:38752 dropped:0 overruns:0 carrier:38762
          collisions:2205053 txqueuelen:1000

Ubuntu interface is capable of full duplex but it negotiates only half duplex connection. When I connected a different device (a router) to MUX, it also showed such errors. The bandwidth assigned is 50 mbps, but I get only 20 mbps. The ISP is reluctant to change their device (looks like a ethernet switch or hub) in the MUX. The ISP engineers blame that its fault at my side. But I checked with more than 3 devices, all showed errors. So, are there any tools for Linux which I can use to probe deep into the causes of those errors, or is there anything I can do to reconfigure my server's interface so as to get rid of those errors?

share|improve this question

1 Answer 1

up vote 8 down vote accepted

You very likely have a duplex mismatch on account of the ISP hard-coding their side to 100-Full essentially disabling auto-negotiation on the ISP Ethernet PHY.

With the ISP set to 100-Full and your side remaining at auto/auto (a hunch, but a common one), the auto-negotiation on your side will configure the interface to 100-Half -- a duplex mismatch as the ISP side will remain 100-Full.

Fix

You can fix the problem by hard-coding your Ethernet PHY to 100-Full -- or specifically whatever the ISP is set to. Most ISP's use 100-Full.

Additional Detail

With the duplex mismatch of 100-Full to 100-Half, the 100-Full side disables CSMA/CD while CSMA/CD remains in effect on the 100-Half side. The 100-Full side transmits without regard to whether or not the medium is free. The 100-Half side perform CSMA/CD checks and backoff as defined by CSMA/CD. This is why you can only achieve 20 Mb/s on what should be a 50 Mb/s Internet circuit. The CSMA/CD backoff on account of the 100-Half side detecting collisions is limiting throughput.

By hard-coding the interface to 100-Full to match the ISP, both sides will have CSMA/CD disabled, therefore backoff and collision detection will be disabled and you should achieve numbers much closer to your 50 Mb/s Internet circuit data rate.

History

Many ISP's hard-code their Ethernet PHY handoffs as there was a time when it was much more reliable to do so. When the original 802.3u 100 Mb/s Fast Ethernet standard was released auto-negotiation of speed and duplex was present, but not required. It wasn't until 802.3z 1 Gb/s Gigabit Ethernet standard when auto-negotiation was required by the standard.

Many network engineers have misconceptions about auto-negotiation. The biggest misconception is that auto-negotiation can properly negotiate speed and duplex if only one side implements auto-negotiation. This is false -- as you have seen.

The reason for this likely stems from the following -- if one side is hard-coded at 100-Full, the other side running auto-negotiation always seems to figure out the 100 Mb/s part. Same if one side is hard-coded to 10-Full -- the other side running auto-negotiation can figure out the 10 Mb/s part. The ability to determine the link speed is from a feature called parallel detection which tries the received physical layer signal on all locally supported link speeds until a match is found. However, parallel detection only works for the speed, not for duplex matching. This is why duplex mismatches can occur -- as an interface will always fall back to half-duplex when it cannot determine the other side through auto-negotiation.

Soapbox

At one time there was spotty support for auto-negotiation and it caused just as many problems as it was intended to solve. That time, in this network engineer's opinion -- has passed. While auto-negotiation problems still exist, the number of problems I have seen due to auto-negotiation being configured in the last 5 years dwarfs the number of problems I have seen on account of auto-negotiation being disabled.

I have never had an ISP unwilling to change their Ethernet handoff to auto/auto upon being asked. With most cable and DSL modems and gateways this is not a problem. It is the NxT1 and fiber managed CPE routers with Ethernet handoff where this problem usually resides. The problem is that a network administrator has to ask in the first place.

With an ISP hard-coding to 100-Full they have given an obligation. An obligation that must be documented and continued. Auto-negotiation is technology that is now stable, has been for years, and takes care of this problem for us. As mentioned earlier, the number of problems caused by auto-negotiation is far outweighed by the number of issues that arise on account of it being disabled in 2011. Technology exists to solve this problem, use it. Perhaps we should be manually setting the initial TCP SYN's, MSS, and managing the Receive Window for every TCP virtual circuit as well? I kid.

Rant off.

share|improve this answer
    
I had tried this command to force the interface to go to a full duplex mode: sudo ethtool -s eth0 duplex full speed 100 autoneg off. But the link went down. But your answer has given me some hope. I will try and test again. Also I will ask the ISP if they can enable auto negotiation in the MUX. –  nixnotwin Nov 6 '11 at 1:08
    
@nixnotwin Verify that the interface is settling in at 100-Half and not 10-Half with auto-negotiation turned on -- hard-code the specific speed and full-duplex. If the link went down after hard-coding and disabling auto-negotiation it is a possible that you have a MDI/MDI-X problem -- as the auto-MDI/MDI-X in the PHY could be disabled as well. If you are using a straight through patch cable, try a cross-over. If you are using a cross-over, try a straight through patch cable. –  Weaver Nov 6 '11 at 16:05
    
Somehow we convinced the ISP to enable auto-negotiation. After that every problem we had--interface errors, ICMP packet loss, streaming jitter, router freezing--and host of other issues suddenly vanished. Now the bandwidth reaches 50 mbits, and not a single error gets shown in the ethernet interface. –  nixnotwin Nov 25 '11 at 13:49
2  
@nixnotwin That is great news. In the future if you have to deal with hyper-hesitant admins (be they net, system, Windows, etc.) I find the phrase "humor me and let's try this just for a minute -- maybe we will both learn something" to be very effective. –  Weaver Nov 28 '11 at 5:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.