Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am running a Debian Linux server on Lenny. Within it, I am running another Lenny instance using KVM. Both servers are externally available, with public IPs, as well as a second interface with private IPs for the LAN. Everything works fine, except the VM sees all network traffic as originating from the host server. I suspect this might have something to do with the iptables-based firewall I'm running on the host.

What I'd like to figure out is: how to I properly configure the host's networking such that all of these requirements are met?

  1. Both host and VMs have 2 network interfaces (public and private).
  2. Both host and VMs can be independently firewalled.
  3. Ideally, VM traffic does not have to traverse the host firewall.
  4. VMs see real remote IP addresses, not the host's.

Currently, the host's network interfaces are configured as bridges. eth0 and eth1 do not have IP addresses assigned to them, but br0 and br1 do.

/etc/network/interfaces on the host:

# The primary network interface
auto br1
iface br1 inet static
    address 24.123.138.34
    netmask 255.255.255.248
    network 24.123.138.32
    broadcast 24.123.138.39
    gateway 24.123.138.33
    bridge_ports eth1
    bridge_stp off

auto br1:0
iface br1:0 inet static
    address 24.123.138.36
    netmask 255.255.255.248
    network 24.123.138.32
    broadcast 24.123.138.39

# Internal network
auto br0
iface br0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    bridge_ports eth0
    bridge_stp off

This is the libvirt/qemu configuration file for the VM:

<domain type='kvm'>
  <name>apps</name>
  <uuid>636b6620-0949-bc88-3197-37153b88772e</uuid>
  <memory>393216</memory>
  <currentMemory>393216</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='i686' machine='pc'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <disk type='file' device='disk'>
      <source file='/raid/kvm-images/apps.qcow2'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='54:52:00:27:5e:02'/>
      <source bridge='br0'/>
      <model type='virtio'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:52:00:40:cc:7f'/>
      <source bridge='br1'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>

Along with the rest of my firewall rules, the firewalling script includes this command to pass packets destined for a KVM guest:

# Allow bridged packets to pass (for KVM guests).
iptables -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT

(Not applicable to this question, but a side-effect of my bridging configuration appears to be that I can't ever shut down cleanly. The kernel eventually tells me "unregister_netdevice: waiting for br1 to become free" and I have to hard reset the system. Maybe a sign I've done something dumb?)

share|improve this question

3 Answers 3

You bridged your VMs to the wrong interface. They should be bridged to the network interface that connects to the outside world (br1 in your case).

Keep in mind that each VM should also have its IP address set in the guest, not on the host.

share|improve this answer

why do you need an alias on br1:0? this might be in the way there
besides the alias, the idea is to use the following scheme:
eth0->br0 <--VM's tap device
the host should be able to use br0 as it's IF and the VMs will be using the tap devices as virtual NICs plugged into a virtual switch (which br0 effectively becomes here)

the same goes for every network of course, so for eth1, you'll have to set up an br1, and bring up the VMs to be plugged into br1

share|improve this answer
    
br1:0 is there to bind a second IP address to the host's public interface, for a second SSL-secure website. I also added the snippet of XML configuration for the KVM guest. I believe the network is configured as you illustrate, but if memory serves, without the iptables rule, the guest can't reach the Internet. –  Steve Madsen Oct 8 '10 at 17:59
    
how is the VM started exactly? libvirt xml file would be nice, or at least ps -ef |grep qemu –  dyasny Oct 8 '10 at 18:03
    
It is started with libvirt, using the standard Debian init.d script at host boot time. I added the entire XML configuration above. –  Steve Madsen Oct 12 '10 at 19:09

I have the same setup with Lenny/Squeeze VM inside a Lenny host using lib-virt and virtio bridging to br0 (no br1 nor br0:0). It works correctly without any special iptables configuration on the host. I do all my firewall configuration inside each guest VM.

I think that you might benefit from checking the network routes to see if your VM are using the host as a gateway. Mine is configured to use an external router as a gateway. Of course, my VM and host are on different IP addresses within the same range.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.