1

Recently we bought 3 Emulex OC10 cards to try in our proxmox test setup. They are the IBM type 49Y4201. The goal is to setup a normal 10G ethernet connection(so no iSCSI or FCoE) in Linux. The cards seem to work in the system, you can get into the firmware bios and setup and flash the leds on both ports etc.

In the OS be2net driver was loaded and complained about old firmware and loss of irqs so after a lot of hassle I flashed the newer 4.1 firmware. After a verify and a reboot, it is again able to go into the bios.

Setup both NICs with proxmox or with ifconfig and then ifup gives a kernel message as follows:

[80221.424544] be2net 0000:21:00.0 enp33s0f0: Link is Down
[80292.286008] be2net 0000:21:00.1 enp33s0f1: Link is Down

and on the other host

[79712.635565] be2net 0000:21:00.0 enp33s0f0: Link is Down
[79716.421629] be2net 0000:21:00.1 enp33s0f1: Link is Down

Cables we have are Cisco branded SFP-H10GB-CU3M and they are listed as supported on IBM's site, but no link LEDs are burning on the card as the cables are plugged in.

The output of ethtool on both hosts:

Settings for enp33s0f0:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full 
                                10000baseT/Full 
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: FIBRE
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00002000 (8192)
                               hw
        Link detected: no

be2net driver info:

[    1.077098] be2net 0000:21:00.0: be2net version is 12.0.0.0
[    1.077106] be2net 0000:21:00.0: enabling device (0000 -> 0002)
[    1.077422] be2net 0000:21:00.0: PCIe error reporting enabled
[    3.252360] be2net 0000:21:00.0: FW config: function_mode=0x20003, function_caps=0x7
[    3.372359] be2net 0000:21:00.0: Max: txqs 1, rxqs 5, rss 4, eqs 16, vfs 0
[    3.372361] be2net 0000:21:00.0: Max: uc-macs 30, mc-macs 64, vlans 15
[    3.372446] be2net 0000:21:00.0: enabled 1 MSI-x vector(s) for NIC
[    3.480586] be2net 0000:21:00.0: created 1 TX queue(s)
[    3.504350] be2net 0000:21:00.0: created 1 RX queue(s)
[    3.603279] be2net 0000:21:00.0: FW version is 4.1.334.36
[    3.609848] be2net 0000:21:00.0: HW Flow control - TX:1 RX:1
[    3.617493] be2net 0000:21:00.0: Adapter does not support HW error recovery
[    3.617678] be2net 0000:21:00.0: Emulex OneConnect: PF vNIC-2 port 0
[    3.617705] be2net 0000:21:00.1: be2net version is 12.0.0.0
[    3.617712] be2net 0000:21:00.1: enabling device (0000 -> 0002)
[    3.617924] be2net 0000:21:00.1: PCIe error reporting enabled
[    3.868352] be2net 0000:21:00.1: FW config: function_mode=0x20003, function_caps=0x7
[    3.988362] be2net 0000:21:00.1: Max: txqs 1, rxqs 5, rss 4, eqs 16, vfs 0
[    3.988364] be2net 0000:21:00.1: Max: uc-macs 30, mc-macs 64, vlans 15
[    3.988449] be2net 0000:21:00.1: enabled 1 MSI-x vector(s) for NIC
[    4.096359] be2net 0000:21:00.1: created 1 TX queue(s)
[    4.120348] be2net 0000:21:00.1: created 1 RX queue(s)
[    4.242124] be2net 0000:21:00.1: FW version is 4.1.334.36
[    4.248700] be2net 0000:21:00.1: HW Flow control - TX:1 RX:1
[    4.256697] be2net 0000:21:00.1: Adapter does not support HW error recovery
[    4.256884] be2net 0000:21:00.1: Emulex OneConnect: PF vNIC-2 port 1
[    4.257756] be2net 0000:21:00.1 enp33s0f1: renamed from eth1
[    4.272475] be2net 0000:21:00.0 enp33s0f0: renamed from eth0
[80221.424544] be2net 0000:21:00.0 enp33s0f0: Link is Down
[80292.286008] be2net 0000:21:00.1 enp33s0f1: Link is Down

Does anyone know how to troubleshoot this? Am I missing something?

0

Those adapters need reset by Emulex (now Broadcom) tools:

toolname SetPortEnabled MAC-ADDRESS 0
toolname SetPortEnabled MAC-ADDRESS 1

toolname - depends on OS and tools' version

MAC-ADDRESS - MAC address of adapter

Example from my system:

brcmhbacmd SetPortEnabled '00-90-fa-22-66-5c' 0
brcmhbacmd SetPortEnabled '00-90-fa-22-66-60' 0

brcmhbacmd SetPortEnabled '00-90-fa-22-66-5c' 1
brcmhbacmd SetPortEnabled '00-90-fa-22-66-60' 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.