KVM network bridge with two NICs

Question

Greetings,

I'm trying to set up bridged networking with KVM and am getting nowhere. There are docs and tutorials on the subject, but they all seem to conflict or don't provide enough info. I was wondering if someone can give me a high-level overview of how to get this working. I can probably work out the details myself (configuring the interfaces, adding routes, etc), I just need help on the big picture: how everything is interconnected.

I have a RHEL5 server with KVM installed and running. It has two physical NICs, eth0 and eth1 in the same VLAN. I would like to use eth1 for all traffic between the guests and the rest of the network and reserve eth0 for host management, guest migrations, etc if possible. I'm not picky about which one gets the default route, although it would be nice if we could make it eth0. All of the guests will have static IPs. I would prefer that when a new guest is added, the networking configuration only needs to be set from within the guest itself. Basically, I want this:

eth0: all host traffic
eth1: all guest traffic

Open to any other suggestions if this isn't possible or will be kludgy/difficult. Pointers to existing documentation might not be helpful since I've already been though just about everything out there.

Thanks for any help.

Update

Okay, here's what I have so far:

• eth0 is configured with a static IP address as per normal. No surprises there.
• br0 is brought up as a bridge with no IP address
• eth1 is brought up with no IP address. It is added to the br0 bridge.
• the VM instance is configured to use br0 and has a static IP address configured

Using this configuration, I can connect to outside machines from inside the guest, but packets destined to the guest never make it. Here's what my config looks like so far:

# cat /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=YES

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# host-only interface
DEVICE=eth0
TYPE=Ethernet
HWADDR=00:23:7D:E1:1F:22
IPADDR=10.8.91.18
NETMASK=255.255.252.0
GATEWAY=10.8.91.254
BOOTPROTO=none
ONBOOT=yes

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
# guest-only interface
DEVICE=eth1
TYPE=Ethernet
BRIDGE=br0
HWADDR=00:23:7d:e1:1f:26
BOOTPROTO=NONE
ONBOOT=YES

(virbr0 is the default kvm/libvirt NAT bridge, it's not being used here)
# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.00237de11f26       no              vnet0
                                                        eth1
virbr0          8000.000000000000       yes

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.122.0   *               255.255.255.0   U     0      0        0 virbr0
10.8.88.0       *               255.255.252.0   U     0      0        0 eth0
169.254.0.0     *               255.255.0.0     U     0      0        0 eth0
default         10.8.91.254     0.0.0.0         UG    0      0        0 eth0

Here's the relevant section of the guest's libvirt XML definition:

<interface type='bridge'>
  <mac address='54:52:00:76:94:73'/>
  <source bridge='br0'/>
</interface>

Update 2

Okay I think I'm all set. I was tripped up by the fact that my guest OS (SLES 11, which I'm not familiar with) has a fairly restrictive set of firewall rules by default. Once the firewall was emptied, I can make connections in and out.

Answer 1

I suppose you know how to set up a bridge on top of a NIC, so basically all you need is to set up two bridges over the two NICs, and bring up the VMs to connect to the right bridge. Come to think of it, one of the NICs can be left without a bridge, since the guests will have no need to use it.

That was a general description of the issue the way I see it. Can you please show what you have done by now, so we could try and find the mistake you made, if there is one of course

Answer 2

I have a similar scenario with ubuntu 12.04.02 as server OS.
My currently single guest is running CentOS.
The server has 4 NICs and I want to run eth0 solely for the server, use eth1 for iSCSI to the storage and eth2 and eth3 for the guests.
Eth0, eth2 and eth3 connect to the same lan.

I am not sure if I correctly figured out how to setup the interfaces though.
The descriptions I have found so far only deal with 1 NIC in the server.

Currently I have configured eth0 with dhcp IP as I use a fixed reservation on the dhcp server.
My bridge for eth2 is named br2

I have configured eth2 and br2 as follows

auto  eth2  
iface eth2 inet manual  

auto  br2  
iface br2 inet manual  
  bridge_ports   eth2  
  netmask   255.255.255.0  
  network   192.168.168.0  
  gateway   192.168.168.1  
  broadcast 192.168.168.255  
  bridge_stp     off  
  bridge_fd      0  
  bridge_maxwait 0  

and

 brctl addif br2 eth2  

Interfaces eth2 and br2 are brought up without an IP as expected.
The guests then also request their address using dhcp.