3

I bought a 10 Gbps NIC and installed it in my Dell PowerEdge 540 that's running ESXi 6.7. I would like to use a port on the new NIC for my management network.

I swapped the ethernet cable to the new port. Then, in ESXi, I selected Configure Management Network > Network Adapters. Then I deselected the old port and selected the new one (the one that ESXi said was connected). I Hit enter and accepted the changes and allowed ESXi to restart the management network.

I figured this was all I needed to do, but this makes the server inaccessible and unable to ping the rest of the network. I figured my switch might need a moment to handle the changes, but even after several minutes I still couldn't ping the network from the ESXi host.

Is there something else I need to do that I'm forgetting?

Edit: I checked the VMWare compatibility guide and my new NIC (BCM57416) is supported. This was the NIC that Dell sold me for this server.

1 Answer 1

2

Legal obligatory note: I work for Dell

The Problem

While there are myriad reasons this can happen (see alternatives at the bottom of this post), by far the most common I see is VLAN mismatch.

If you have the VLAN set on the vmkernel NIC here:

enter image description here

then you need to make sure it matches the VLAN for the management portgroup:

enter image description here enter image description here

and that in turn needs to match what is on your switch (which is usually a trunk). If all those things don't line up, then the traffic just gets dropped. The setting in the UI should automatically update the setting you see on the terminal (pretty sure it didn't used to do that).

Concepts

When you enable the VLAN option on the vmkernel NIC what you are actually doing is configuring a single-VLAN trunk that is going to tag any outbound traffic with the specified VLAN. Subsequently you have to make sure your upstream switch is configured properly as a trunk which allows that VLAN. Alternatively, you could do no tagging from ESXi and configure your switch side as an access interface.

You can see this in action here where I added VLAN 9 to my vmkernel NIC in the webgui and then dumped all outbound traffic on the interface. Notice it has the 802.1Q tag on it with VLAN 9.

enter image description here

You can see this also automatically updates the setting on the terminal.

enter image description here

So the flow of traffic here is vmkernel NIC -> portgroup (tagging happens here) -> vSwitch -> outbound uplink interface

Alternatives

The above is the most common, but you may also just have a regular network connectivity issue in which case check the usual suspects:

  • IPs
  • Subnet masks
  • ID10T errors like is it plugged in
  • DNS? Not sure how it could be DNS in your scenario, but the answer is it's always DNS. But if it's not always DNS then it's NTP.
  • Blame the distant end.

I usually just keep a constant ping running and dump the traffic with wireshark. Start at L2 of the OSI model and work your way up. Do I see ARP working? If ARP is working where is my ICMP ping packet dying? Which direction? Etc

2
  • These are all good ideas. I am using a VLAN, but I am using the same cable and port in my switch (I just swapped the ports the cable is going into on the on the ESXi host) and I didn't change any VLAN or IP settings in ESXi (I just changed which port ESXi is configured to use), so ESXi is still set to the correct VLAN, IP address, and subnet. ESXi tells me that the cable is connected to the port in question, so it seems like that's not an issue either.
    – Dave
    Sep 21 at 13:14
  • 1
    I would do a port mirror on your switch, get the inbound traffic coming from ESXi, and then you can start diagnosing what's going on. First thing to check, if you do a ping from the management interface to your external gateway and verify that ARP is working correctly. If ARP is working and the ARP cache on both sides is correct, you're good on OSI L2, next go to L3. Can you ping? If not, where is it dying? Sep 21 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.