We have an existing VMware ESX 3.5 cluster (6 hosts, VI 2.5) and need to move the service consoles to a new subnet. We would like to do this without downtime for the VMs.
In my previous experiments I've found that the cluster network constraint checks are blocking my attempts to reconfigure hosts to have a different Service Console subnet.
I've tried adding a secondary service console (with a different name) in the new subnet on all the hosts and setting das.AllowNetwork0 to restrict to that new service console, but if I configure a host to only have that service console and not the old one it fails to join the cluster with errors about non-matching network configurations. It fails whether the new service console has an alternate name or a matching name.
We have trunked connections for both subnets. We can certainly temporarily set up a service console on a subnet.
Current configuration on all hosts is (simplified and redacted):
# Current service console, on port in VLAN 5
Switch Name Uplinks
vSwitch0 vmnic0
PortGroup Name VLAN ID Uplinks
Service Console 0 vmnic0
# connected to dedicated VMotion switch
Switch Name Uplinks
vSwitch1 vmnic1
PortGroup Name VLAN ID Uplinks
VMotion 0 vmnic1
# Old subnets
Switch Name Uplinks
vSwitch4 vmnic8,vmnic4
PortGroup Name VLAN ID Uplinks
XXX.YYY.9.0_24 9 vmnic4,vmnic8
XXX.YYY.5.0_24 5 vmnic4,vmnic8 #
# New subnets:
Switch Name Uplinks
vSwitch7 vmnic11,vmnic7
PortGroup Name VLAN ID Uplinks
XXX.YYY.27.0_25 27 vmnic7,vmnic11
XXX.YYY.30.0_24 30 vmnic7,vmnic11
Service Console SF 30 vmnic7,vmnic11
Basically we want to move vmnic0 from VLAN5 to VLAN30 (switching to a new cable to do it). We have enough capacity to have 2 hosts in maintenance mode. We have a couple spare ethernet ports on the hosts, and as you can see above we also have trunk connections that can give us an interface in every subnet.
I prefer the dedicated port for the permanent service console because I've had bad experiences reconfiguring a port/vSwitch that I'm using to reach the interface for reconfiguring the host. We may want to keep a backup service console with another IP in the same subnet on the trunked connections in that subnet (vSwitch7/VLAN30).
I've considered creating a new cluster in VI, copying all our Resource Pools and other configuration over, putting 2 hosts with the new network configuration into the cluster, migrating VMs to the new cluster, and then moving the other 4 hosts to the new cluster one at a time (move host, move enough VMs to free up a host in old cluster, move next host).