We are currently trying to rearrange our network for performance. Having already begun to solve the issue that our network diagram looked like a straight line, I am now looking at the 4 nics for the virtual host.

As they currently are, 3 of the nics go to one switch, and one to another - but I get the feeling this was not so much a choice for design so much as a choice for what ports were free when it was installed.

Given an extremely simplified version of what our network looks like without the virtual host:

Simple mudmap (no vHost)

I am not sure whether I should be hooking all 4 of the nics to the central switch like so:

Simple mudmap (central vHost)

or distributing them so that any client has a short route to the host (but only to one of it's NICs), like so:

Simple mudmap (distributed vHost)

for the record, we are using vSphere eSXI vSphere 5.1, and the virtual machines being run provide entirely in-network services - none of them are internet-facing.

The layout of the nics <-> the vSwitch environment is:

vSwitch layout

  • 1
    Are all 4 NICs configured to be used by the same vSwitch within ESXi? – Shane Madden May 26 '13 at 22:58
  • 1
    Just plugging multiple ports into your ESXi host without granular failover or teaming rules won't do anything. Chances are that most of your traffic is pinned to the first NIC port. Can you please give more detail about what you're trying to do and perhaps network addressing, vSwitch config, etc.? – ewwhite May 26 '13 at 22:58
  • I think so, I will post a screencap of how it looks in vSphere – Chris O'Kelly May 26 '13 at 22:58
  • @ewwhite I can give more info, but I'm not sure what info to give. Perhaps you can clarify? – Chris O'Kelly May 26 '13 at 22:59
  • @ShaneMadden yeah so there's just the one vSwitch 0 – Chris O'Kelly May 26 '13 at 23:03

There are reasons that you'd want to spread your uplinks from a vSwitch out between multiple physical switches -- but mostly for redundancy purposes, not for decreasing the length of the path that the traffic will need to take for communication between a client and a virtual machine.

How exactly this behaves depends on the configuration of your vSwitch -- in the NIC teaming tab, the settings that dictate how the NICs will be used are the Load Balancing drop-down, as well as the active/standby state of the adapters.

The Port ID and MAC hash configurations will both effectively assign each VM's traffic to a specific physical NIC - so traffic outbound from that VM (and inbound to the VM, due to the switches' MAC address table) will be stuck to the same physical NIC until something changes (like the NIC topology changing due to one going down, for instance).

The other load balancing option is IP hash, but you don't want that when uplinking to multiple switches; you'll get all kinds of MAC address flapping.

In general, layer 2 switching is not capable of taking the shortest physical path to something; there's just one working path, at least until TRILL starts taking off.

You can potentially manually control which switch a specific VM tends to be pointed at; would that be helpful?

  • It's not particularly helpful for our situation to point one VM at one switch, but it is helpful to know that (if I read correctly) we will not be missing out on an easy optimization by just plugging all 4 NICs into the central switch. Thanks very much – Chris O'Kelly May 26 '13 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.