4

The Problem

I've got four identical blade servers.

I've got Windows Server 2012 R2 on one blade and Xen on all the others.

On the Windows-hosted server (only) the 10GbE NICs cannot negotiate 10Gbit. They settle at 1Gbit only.

10GbE on Xen hosts works properly.

The Environment

I've got DELL PowerEdge M1000e chassis with:

  • 2x M8024-k 10GbE switch (A1, A2)
  • 4x M620 identical blades with following NIC:
    • slot A: Broadcom NetXtreme 10GbE 2P 57810s (the 10GbE card this issue is about)
    • slot B: Broadcom NetXtreme GbE 4P 5719S (not related to issue, I hope)

All the available firmwares are identical and up-to-date.

Additional info & diagnostic

  • The machine has previously been used under some *nix system (probably also Xen) and previous sysadmin had left no notice of any such problems (which would otherwise be expected). I therefore assume it was working properly.
  • This happens on both ports. Both ports are on single card though.
  • When I try to set speed manually at 10Gbit, I get Windowsy "network cable not attached" status.
  • Windows has all the latest drivers.
  • As stated above, the problem does not appear on Xen-hosted servers. It's Windows only.
  • The configuration on both switches (A1 and A2) has been reviewed, ports are set to auto-negotiation and I found no other differences in the port configurations between Windows and Xen-assigned ports.
  • This was the only machine with NIC partitioning enabled (NPAR+SR-IOV), but I have disabled it and it made no change.

The Question

Since I still believe it might be some weird Windows issue, I have posted this question to exclude any possible software sources of the problem, especially those which might be Windows-specific.

Do you have any hints where to look and what may be wrong?

  • Did you check the driver versions? Broadcom is - ah - reputed for - ah - interesting definitions of high quality, at least in the past. – TomTom Feb 24 '14 at 20:17
  • 1
    What driver do you have installed? Last time I remember setting up a 10 Gb link, I needed a particular driver to get it working at 10Gb... so that's where I'd look first. – HopelessN00b Feb 24 '14 at 20:18
  • It's "Broadcom 18.2.0 / DELL 17.8.4.2". – Michał Sacharewicz Feb 24 '14 at 20:38
2

The problem was hardware, not software. And apparently very simple one.

Once the blade has been shut down, physically removed from the chassis and then reattached (at the same slot), we have finally attained a proper 10G link.

Just that :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.