Tell me more ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
up vote 1 down vote favorite
2

I have a Dell r710 (running Debian GNU/Linux 6.0) which is currently in a datacenter hundreds of miles away, and the ethernet card is stuck at 10baseT half duplex. 10baseT is OK, but I want it to be full duplex.

The card in question is a broadcom BCM5709:

areion:~# lspci |grep Ethernet
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

using the bnx2 driver/firmware:

areion:~# apt-cache policy firmware-bnx2
firmware-bnx2:
  Installed: 0.28
  Candidate: 0.28
  Version table:
 *** 0.28 0
        500 http://ftp.us.debian.org/debian/ squeeze/non-free amd64 Packages
        100 /var/lib/dpkg/status

And running the stock kernel for squeeze:

areion:~# uname -a
Linux areion 2.6.32-5-amd64 #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux

I've tried changing the ethernet settings:

ethtool -s eth0 speed 10 duplex full autoneg off

But then I lose all connectivity, and have to reboot the server in order to connect to it again. To avoid having to constantly reboot the server, and to find out if the ethtool command was hanging, I switched to using this script:

#!/bin/bash

ethtool -s eth0 speed 10 duplex full autoneg off 2>&1 > ethtool.log &

sleep 10
if [ $(jobs -r |wc -l) -gt 0 ]; then
   kill %1 || kill -9 %1
else
  echo '-----------' >> ethtool.log
  ethtool eth0 >> ethtool.log
fi
dmesg |tail > dmesg-recent.log

ethtool -s eth0 autoneg on

This allows me to see the status of the interface after running the command:

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supports auto-negotiation: Yes
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        MDI-X: Unknown
        Supports Wake-on: g
        Wake-on: d
        Link detected: no

From dmesg:

[  442.865157] bnx2: eth0 NIC Copper Link is Down

It seems that either autonegotation is failing, or the switch that I am plugged into is advertising 10Mbit/half duplex. I am told that the switch is running at 10M full duplex.

Might this be a driver issue, hardware issue, cable issue, switch issue? Is there anything I can do remotely to identify the source of the problem, or at least eliminate possibilities?

Update:

Turning autonegotiation off, and setting the options to 10/HD, 10/FD, 100/HD, 100/FD, etc (anything other than autonegotiated 10/HD) results in NO-CARRIER, as reported by ip link show dev eth0. The same is true for leaving autonegotiation on, and advertising anything other than 10/HD. Is there any way this could be something other than an issue with the switch?

share|improve this question
IME, when a link auto-negotiated to 10base half-duplex, there a serious cable issue, or the other end plain doesn't support anything faster. – Chris S♦ Mar 29 '11 at 12:48

3 Answers

up vote 1 down vote

It is better to have auto-negotiation enabled on both ends. This is the current recommendation of the networking vendors (including Cisco). See: http://www.cisco.com/en/US/tech/tk389/tk214/technologies_tech_note09186a0080094781.shtml

For your problem, you have to find out how it is configured at the remote end. If it is auto-negotiation, then you should use auto-negotiation on your server too. If it is set at a specific speed and duplex you should match the remote end.

Please keep in mind that is possible to keep the negotiation on and allow only negotiation for a certain speed and duplex. Also the advertising of capabilities can be configurable.

If the network card has negotiated at 10/HD, this usually means that the remote end has negotiation disabled and it is set at 10/FD. Your bad performance is caused by the duplex mismatch.

You should also try with a different network card. Changing the driver will not help because the detection of link (to detect the carrier and the speed) is done by PHY part of the NIC.

share|improve this answer
This is all true, but I don't have physical access to the server, and thus cannot switch out network cards. I also cannot dictate how the host's switch is configured. As mentioned in my question, specifically disabling autonegotation and setting 10/FD fails. – vezult Mar 29 '11 at 13:06
I deal with big BGP-driven networks and our mantra is disable all auto-neg where possible (and flow control and TSO). – Jonathan Ross Mar 29 '11 at 13:35
Can you also try to keep the negotiation and set to 10/FD? ethtool -s eth0 speed 100 duplex full autoneg on advertise 0x002 – Mircea Vutcovici Mar 29 '11 at 13:40
@serverfault.com/users/75985/jonathan-ross Back in the days it was like this. But now it is not recommended anymore. For gigabit it is mandatory to have autoneg. – Mircea Vutcovici Mar 29 '11 at 13:43
Speed is auto-configured whether or not negotiation is enabled. If the other end is hard-coded, advertising gets set to 10HD. There was a time when hardware didn't negotiate properly, but that is no longer the case. – BillThor Mar 29 '11 at 14:38
show 1 more comment
up vote 0 down vote

Can you put another device like a switch between you and the Server that you can change the settings on physically ? That might help.

share|improve this answer
That's not really an option. Again, the server is not within my physical control, and swapping in hardware, adding in switches, cables, etc are not something I can do from here. – vezult Mar 29 '11 at 15:24
up vote 0 down vote

It's not recommended to disable auto-negotiation, but if you still decide to, you must disable it on both ends and set both ends to use the exact same speed and duplex. If you only disable auto-negotiation on one end, the other end is forced to assume 10MB/HD. If you don't have access to the switch on the other end of the Ethernet cable or that switch does not support such customization, then disabling auto-negotiation is not an option. In some cases, you can leave auto-negotiation enabled, but tell the card or switch to advertise a lower speed or half-duplex.

share|improve this answer
Thanks, but this is ground covered by previous answers, comments, and my question itself (specifically the "Update" portion). I don't control the switch, thus I can't dictate what they require, recommended or not. Neither can I verify how they actually have it configured. They claim it is set to 10/FD, no auto-negotiation. The only thing at all that allows any sort of connectivity is setting auto-negotation on (on my end), or autoneg on and specifically advertising 10/HD. The result in every case is either no connectivity, or 10/HD. – vezult Mar 30 '11 at 19:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.