We have an existing VMware ESX 3.5 cluster (6 hosts, VI 2.5) and need to move the service consoles to a new subnet. We would like to do this without downtime for the VMs.

In my previous experiments I've found that the cluster network constraint checks are blocking my attempts to reconfigure hosts to have a different Service Console subnet.

I've tried adding a secondary service console (with a different name) in the new subnet on all the hosts and setting das.AllowNetwork0 to restrict to that new service console, but if I configure a host to only have that service console and not the old one it fails to join the cluster with errors about non-matching network configurations. It fails whether the new service console has an alternate name or a matching name.

We have trunked connections for both subnets. We can certainly temporarily set up a service console on a subnet.

Current configuration on all hosts is (simplified and redacted):

# Current service console, on port in VLAN 5
Switch Name    Uplinks   
vSwitch0       vmnic0    

  PortGroup Name      VLAN ID  Uplinks   
  Service Console     0        vmnic0    

# connected to dedicated VMotion switch
Switch Name    Uplinks   
vSwitch1       vmnic1    

  PortGroup Name      VLAN ID  Uplinks   
  VMotion             0        vmnic1    

# Old subnets
Switch Name    Uplinks   
vSwitch4       vmnic8,vmnic4

  PortGroup Name      VLAN ID  Uplinks   
  XXX.YYY.9.0_24      9        vmnic4,vmnic8
  XXX.YYY.5.0_24      5        vmnic4,vmnic8 # 

# New subnets:
Switch Name    Uplinks   
vSwitch7       vmnic11,vmnic7

  PortGroup Name      VLAN ID  Uplinks   
  XXX.YYY.27.0_25     27       vmnic7,vmnic11
  XXX.YYY.30.0_24     30       vmnic7,vmnic11
  Service Console SF  30       vmnic7,vmnic11

Basically we want to move vmnic0 from VLAN5 to VLAN30 (switching to a new cable to do it). We have enough capacity to have 2 hosts in maintenance mode. We have a couple spare ethernet ports on the hosts, and as you can see above we also have trunk connections that can give us an interface in every subnet.

I prefer the dedicated port for the permanent service console because I've had bad experiences reconfiguring a port/vSwitch that I'm using to reach the interface for reconfiguring the host. We may want to keep a backup service console with another IP in the same subnet on the trunked connections in that subnet (vSwitch7/VLAN30).

I've considered creating a new cluster in VI, copying all our Resource Pools and other configuration over, putting 2 hosts with the new network configuration into the cluster, migrating VMs to the new cluster, and then moving the other 4 hosts to the new cluster one at a time (move host, move enough VMs to free up a host in old cluster, move next host).


The configuration that you've mentioned with all hosts having access to both service console networks, using das.AllowNetwork0 to force use of the new network, then adding a host to the cluster with only access to the new network, will work. One caveat, though - you'll need to disable and re-enable HA for it to change over to the new interface (and admit a new host that's trying to talk HA on that interface).

Oh, also keep in mind that it'll not use networks that have vMotion enabled by default - that needs das.allowvMotionNetworks = true set on the cluster. And if you're managing from outside the subnet, make sure to switch out the default gateway before cutting the old service console.

Your plan to move to a whole new cluster will also work well!

  • So the secret is disabling and re-enabling HA. I'll be trying that shortly. – freiheit Sep 15 '11 at 15:07
  • The technique that seems to be working best is simply disabling HA on the cluster and leaving it disabled until all the hosts are reconfigured. Otherwise the HA checks still get in the way. – freiheit Sep 15 '11 at 22:56
  • @freiheit Huh. I'd say it's worth troubleshooting why it's failing the checks.. but it's really probably not worth the effort. – Shane Madden Sep 16 '11 at 0:54
  • Turns out that editing the IP info in /etc/hosts, /etc/vmware/esx.conf is also necessary before HA can be re-enabled on the cluster. That was probably the original issue causing it to fail the checks, but I only fixed it after changing the network config for all cluster members. – freiheit Sep 16 '11 at 22:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.