Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is there an easy way to allow a single vSphere 5.0 host communicate on multiple subnets and allow VMs hosted there to communicate on those subnets?

I've got a small farm of vSphere 5.0 that are managed by a vCenter instance. For this farm, I've been allocated an internal /26 subnet (lets say it's 10.13.111.0/26).

Each vSphere host has two NICs in that are connected to a cisco 3750 switch as a "Route based on IP hash" trunk to give 2GBit bandwidth between each host and the switch. This link is also a .11q trunk. On the hosts themselves, I've got two virtual machine port groups, one on VLAN 118 called "Public" and the other on vlan 999 called "Private". The private VLAN is addressed in the 192.168.100.0/24 space and is used for inter-VM traffic. I have two VMKernel Ports both attached to the same 118 VLAN and addressed on the 10.13.111.0/26 network.

Here's what the configuration of one of the hosts looks like:

vm config

The problem is that I've run out of IP addresses in this subnet, and have been allocated another subnet - 10.13.110.0/26. However, if I add another VMKernel Port on VLAN 118 to this host and assign it 10.13.110.1, that IP address remains unreachable. I'm not sure if this is an issue on the switch or on the VMware vSphere host itself.

How can I configure this subnet to be usable by the host?

EDIT

Just to add, the 3750 has VLAN118 defined as:

interface Vlan118
 ip address 10.13.111.62 255.255.255.192
end

Can I just expand the definition to include the new subnet and have it all magically work?

EDIT 2

This is looking like it's a switch config issue.

If I reconfigure the VLAN118 on the 3750 as follows:

interface Vlan118
 ip address 10.13.111.62 255.255.255.192
 ip address 10.13.110.5 255.255.255.192 secondary
end

then the switch can now ping the VMKerenel device on the new .110 network. However, everything past the switch in the wider network can't ping the device.

The gateway for the .111 network is 10.13.111.3 and for the .110 network it's .110.3 - both of these IP addresses are hosted by a router that is connected to the core switch (or maybe even the core switch). I've got an uplink to that core switch from my 3750.

I guess that any traffic from .110 to .111 needs to flow from vSphere through my 3750 out to the core to .110.3, hop over to .111.3 and then back into my 3750 and into vSphere. Does that sound about right?

share|improve this question
    
Do you really need direct communication with the vsphere host on every VLAN/subnet? The normal (and best practice) setup is to use a single dedicated VLAN for management. –  pauska Oct 25 '11 at 12:57
    
I agree that it'd be better to have a separate management VLAN, but I'm constrained in what VLANs I've got available to me for management and services. –  growse Oct 25 '11 at 13:06
    
That still doesnt answer the question.. why do you need a vmkernel on the new subnet? –  pauska Oct 25 '11 at 13:31
    
I'm migrating the management network onto a new subnet. To do this, I need to add it while connected to the old one, then remove the old one. –  growse Oct 25 '11 at 13:40

2 Answers 2

up vote 1 down vote accepted

You stated that you cannot communicate to the newly assigned address - where are you trying to communicate from?

For non-routed traffic (within the same subnet), your newly assigned VMkernel addresses should already be working just fine. You should be able to verify by communicating with them from another device on the new 10.13.110.0/26 network.

Routed traffic is another story. Two issues with routed traffic:

  • Something needs to handle inter-vlan routing - if it's that 3750 (I'm guessing not - I'd expect the gateway to be .1 or .63, not .62) then it's not got an address on the new 10.13.110.0/26 network.
  • Only one gateway is permitted for all VMkernel traffic - setting up the gateway on the new subnet will drop the config for the old subnet's first hop.

Since your subnets are sliced into very small chunks (what's up with that, anyway? Does someone in your organization need reminded that 10.0.0.0/8 contains 16.8 million addresses?), I bet most of these hosts' communication is routed.

The tricky step of this migration for you will be the change to the hosts' gateways, as everything that's connecting to them from outside the subnet will need to be switched over to the new address.

share|improve this answer
    
Thanks for the comments. I'm going to update the question addressing some of your questions and points. As for the minute slicing, the 10.0.0.0/8 has to cope with the whole organisation globally, and as a lowly dev management infrastructure manager I get a tiny chunk of that space, because they're trying to preserve addresses. Don't ask me why. I'd love a /24 and be done with it. –  growse Oct 25 '11 at 15:39
    
@growse (re: your edit) Please review my answer again. Only one VMkernel gateway is configured throughout the entire host. Routed communication will need to be to the VMkernel interface that can see that gateway. –  Shane Madden Oct 25 '11 at 20:54
    
Ah! I see what you mean. It looks like I'm going to have to argue with the network guys some more about changing this setup to accommodate my needs then. Might just have to get a larger IP block and put everything on that, rather than trying to split between two subnets on the same switch. –  growse Oct 25 '11 at 21:29

You need to properly differentiate between VMware Host networking and VMware Guest networking.

Typically, the only network traffic that directly involves VMware host machines is (Vcenter) management and cluster-related communication.

Traffic to and from individual VMware guests is completely unrelated to this; in fact, they should usually not be connected to the same network(s).

That said, Virtual switch ports can be trunked, or tagged, or whatever you desire for your guest VMs.

share|improve this answer
    
I'm going to try and migrate the management traffic onto the 10.13.110.0/26 subnet and keep the VM traffic on 10.13.111.0/26. My issue is just getting the rest of the network to even see the VMKernel on 10.13.110.0/26 network. –  growse Oct 25 '11 at 13:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.