I've got a couple of vSphere 5.0 hosts in a small lab environment here and I've noticed a strange behaviour. When on of the hosts gets rebooted, it is unresponsive to the network until I log into the ESX console, Press F2 to customize and select Restart management network. Once this is done, the networking works perfectly as expected.

Each host has two NICs which are trunked together using Etherchannel to a Cisco 3750. The link is also a .1q VLAN trunk and the management network is configured on VLAN121 with the VM traffic configured on VLAN118.

Why would the host be completely dead to the world until I physically kick it?


Sample switch config for trunk:

interface Port-channel2
 description Blade 1 EtherChannel Trunk
 switchport trunk encapsulation dot1q
 switchport mode trunk
interface GigabitEthernet4/0/1
 description Bladecenter1 CPM 1A
 switchport trunk encapsulation dot1q
 switchport mode trunk
 speed 1000
 duplex full
 channel-group 2 mode on

Vswitch teaming settings:

Management port group settings:

  • Can you post us your vswitch/port-group settings please? – Chopper3 Dec 12 '11 at 11:43
  • I've just noticed that the Mgmt network port group has an MTU of 1500, whereas the Port-Channel interface has an MTU of 9000. Is it possible this is related? – growse Dec 12 '11 at 11:59
  • Possibly, be interesting to see the status of the ports/trunk from the cisco side before and after the reset. BTW I've never seen this kind of problem before and we have many, many host - hence my interest – Chopper3 Dec 12 '11 at 12:07
  • Give me a few hours and I'll schedule a reboot to replicate to see what the switch does. I guess that the switchports come back up as 'OK', but will have to test to confirm. – growse Dec 12 '11 at 12:20
  • @growse MTU 9000 is the default for jumbo frames. Port-Chans presumably has Jumbo Frames enabled hence why you are seeing this. Personally, I'd set both at the same frame size. Also, the switches must be configured to use Jumbo Frames. This can be one of the reasons why it 'crashes' until a manual restart. – Cold T Dec 12 '11 at 17:32

What kind of NICs are you using? Have you taken a look if these NICs are listed in the VMWare Compatibility Guide?

http://www.vmware.com/resources/compatibility/search.php (In "What are you looking for" click "IO Devices")

Also, in your network configuration in vSphere the load balancing property must be set to Route based on ip hash

  • NICs are Broadcom NetXtreme II BCM5708 1000Base-SX - they were the default in the IBM HS21 blades that I'm using and I believe they're listed on the guide. Nic Teaming Load balancing setting is also set to Route based on IP hash for all hosts. – growse Dec 12 '11 at 11:01
  • What happens if you reboot with only one cable from the trunk attached to the switch? – duenni Dec 12 '11 at 11:08
  • I don't know Cisco but just to be sure: ESXi (or VMWare in general) only supports static link aggregation. Is the aggregation static or dynamic on your Cisco 3750? – duenni Dec 12 '11 at 11:14
  • I've added switch configuration to the question. Trunks are configured statically (I believe) with Etherchannel, but not LACP. – growse Dec 12 '11 at 11:34

Check that nothing else is sitting on the IP Address allocated to the management network. On boot if it detects something else on that IP address it will not fire up.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.