Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm having an issue with performing virtualisation via libvirt/qemu-kvm, in which my set up for bridged networking doesn't appear to be working correctly. I've followed every tutorial I can find and spent hours and hours going through forums, but currently to no avail.

Both the host and guest are using Centos6.

Here's the set up for the host:

[root@node2 ~]# ifconfig -a
bridge0   Link encap:Ethernet  HWaddr BC:30:5B:E8:C0:4F  
          inet addr:10.59.190.253  Bcast:10.59.190.255  Mask:255.255.255.0
          inet6 addr: fe80::be30:5bff:fee8:c04f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5315 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3350 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:316961 (309.5 KiB)  TX bytes:3943630 (3.7 MiB)

eth0      Link encap:Ethernet  HWaddr BC:30:5B:E8:C0:4F  
          inet6 addr: fe80::be30:5bff:fee8:c04f/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:5286 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5325 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:409939 (400.3 KiB)  TX bytes:4095006 (3.9 MiB)
          Interrupt:16 Memory:c0000000-c0012800 

eth1      Link encap:Ethernet  HWaddr BC:30:5B:E8:C0:50  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:17 Memory:c2000000-c2012800 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:28 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2832 (2.7 KiB)  TX bytes:2832 (2.7 KiB)

sit0      Link encap:IPv6-in-IPv4  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vnet0     Link encap:Ethernet  HWaddr FE:54:00:3A:C5:C9  
          inet6 addr: fe80::fc54:ff:fe3a:c5c9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:492 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 b)  TX bytes:29516 (28.8 KiB)

I have two physical adapters, eth0 and eth1. eth1 is currently unplugged and not being used in this set up. eth0 is bridged to bridge0. bridge0 is using DHCP to get its network configuration. The adapters use the following configurations:

[root@node2 network-scripts]# cat ifcfg-eth0
DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT=yes
HWADDR=BC:30:5B:E8:C0:4F
BRIDGE=bridge0
[root@node2 network-scripts]# cat ifcfg-bridge0
DEVICE=bridge0
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=dhcp
DELAY=0
NM_CONTROLLED=no

...and iptables set up as follows:

[root@node2 network-scripts]# iptables --list-rules
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A INPUT -p icmp -j ACCEPT 
-A INPUT -i lo -j ACCEPT 
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT 
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5900 -j ACCEPT 
-A INPUT -j REJECT --reject-with icmp-host-prohibited 
-A FORWARD -m physdev --physdev-is-bridged -j ACCEPT 
-A FORWARD -m physdev --physdev-is-bridged -j ACCEPT 
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

So, that's the set up. I then used the following command to create and install a Centos6 guest image:

virt-install --name=centos-bridge-test --disk path=/var/lib/libvirt/images/centos6-minimal,size=5 --vnc --ram=512 --cdrom=/var/lib/libvirt/images/CentOS-6.0-x86_64-minimal.iso --vnclisten=0.0.0.0 -b bridge0  

My understanding is that this set-up should have been sufficient for the bridged networking to "just work," however, when I start the guest using the virsh command (virsh start centos-bridge-test) it has no network connectivity.

Here's an xml dump of the libvirt domain:

[root@node2 network-scripts]# virsh dumpxml centos-bridge-test
<domain type='kvm' id='7'>
  <name>centos-bridge-test</name>
  <uuid>5d611267-2feb-c6f7-7a48-29f9695a4a75</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.0.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/libvirt/images/centos6-minimal'/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <disk type='block' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-0'/>
      <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:3a:c5:c9'/>
      <source bridge='bridge0'/>
      <target dev='vnet0'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-gb'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux'>
    <label>system_u:system_r:svirt_t:s0:c384,c608</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c384,c608</imagelabel>
  </seclabel>
</domain>

The interface appears to be set up correctly (from looking at the libvirt documentation).

So, whenever I start the guest, there is no network connectivity.

[root@localhost ~]# ifconfig -a
eth0       Link encap:Ehternet  HWaddr 52:54:00:3A:C5:C9
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000 
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
           Interrupt:10 Base address:0x6000

lo         Link encap:Local Loopback  
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0 
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Edit: Note that eth0 is not up here, so I've been running ifup eth0 to get the adapter to come up, which then gives the following output:

[root@localhost ~]# ifconfig -a
eth0       Link encap:Ehternet  HWaddr 52:54:00:3A:C5:C9
           inet6 addr: fe80::5054:ff:fe3a:c5c9/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets65 errors:0 dropped:0 overruns:0 frame:0
           TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000 
           RX bytes:3896 (3.8 KiB)  TX bytes:398 (398.0 b)

lo         Link encap:Local Loopback  
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0 
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

...and when I attempt to access the network for either an internal or external address, I get this:

[root@localhost ~]# ping 10.59.190.253
connect: Network is unreachable
[root@localhost ~]# ping www.google.com
ping: unknown host www.google.com
[root@localhost ~]# ping 173.194.66.99
connect: Network is unreachable

Additional Info: The following is some additional information requested form the comments for this question. These commands were run from the host with the guest currently running:

[root@node2 network-scripts]# brctl show
bridge name         bridge id               STP enabled     interfaces
bridge0             8000.bc305be8c04f       no              eth0
                                                            vnet0
[root@node2 network-scripts]# ps -ef | grep qemu
qemu      1597     1  0 11:28 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name centos-bridge-test -uuid 5d611267-2feb-c6f7-7a48-29f9695a4a75 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/centos-bridge-test.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -boot c -drive file=/var/lib/libvirt/images/centos6-minimal,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=20,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:3a:c5:c9,bus=pci.0,addr=0x3 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 0.0.0.0:0 -k en-gb -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
root      2815 32057  0 14:31 pts/0    00:00:00 grep qemu

I'd really appreciate any help possible at the moment! If you need me to provide any further configurations, etc, please do let me know.

Edit: I'm using a minimal install of Centos6, so I don't have any window manager installed, so I'm trying to do everything via virsh and virsh-install, and would like to try to avoid using virsh-manager. Thanks.

share|improve this question
    
good job posting all this data, but there's another thing required: start the VM up, and get brctl show and ps -ef |grep qemu –  dyasny Nov 21 '11 at 12:23
    
@dyasny Thanks for letting me know. I've added that information to the question. –  a.pollock Nov 21 '11 at 14:27
    
@dyasny I've edited the question to add the output from ifconfig on the guest after running ifup eth0. This should provide some more information. –  a.pollock Nov 21 '11 at 15:21

2 Answers 2

up vote 2 down vote accepted

The answer to this issue was that because I'd done a non-graphic install of Centos 6 minimal on the guests, the network interfaces had not been automatically set up to work. My lack of knowledge of virtualisation and Centos meant that I assumed that this would be taken care of for me, but I was incorrect in this assumption.

Fix: I needed to go into ifcfg-eth0 on the guests and turn on ONBOOT=yes and BOOTPROTO=dhcp, then restart the network adapter via ifdown eth0 then ifup eth0. After doing this, the guests received IP addresses and everything worked as expected.

share|improve this answer
1  
oh wow, it's like checking the power cable for a server that just wouldn't start up :) –  dyasny Nov 22 '11 at 10:52
    
I have same issue. I have ubuntu installed and i want to install centos inside ubuntu. I have successfully installed but when i am going to reboot it gets no bootable device error. can i write ONBOOT=yes in ubuntu eth0. –  Suresh Jan 23 '13 at 6:07

OK, the settings seem ok. This leaves
1. try to disable iptables completely 2. switch the network device from RTL to virtio

Here's how I have IPTables set up normally: http://fpaste.org/OKG5/

share|improve this answer
    
Thanks very much for your feedback. Here's how it went: 1. I've disabled iptables by running service iptables stop and then restarted the network with service network restart and confirmed that iptables was disabled with iptables -L, which is showing ACCEPT across the board. I restarted the guest and tried to access the network and received the same problems. Also note that I had to run ifup eth0 to actually get the adapter to come up. –  a.pollock Nov 21 '11 at 15:11
    
2. I wasn't entirely sure how to do this. A quick look at the libvirt website led me to add <model type='virtio'> to the domain XML under the interfaces element. After that I reran the guest, but still I'm having the same issue. If I've mistaken what you meant by switching the network device from RTL to virtio, do please let me know. I really appreciate the help you've given so far! –  a.pollock Nov 21 '11 at 15:13
    
does the VM actually get an IP from DHCP? –  dyasny Nov 21 '11 at 15:41
    
It doesn't appear to, no. eth0 doesn't seem to be up initially. I have to run ifup eth0 to get it up, and even then it doesn't appear to have an IPv4 address (it does seem to have an IPv6 address though). I've updated the question with the printout from ifconfig -a before and after bringing up eth0. –  a.pollock Nov 21 '11 at 15:45
    
I checked the logs on the DHCP server and it is not receiving any DHCP requests from the guest OS. –  a.pollock Nov 21 '11 at 15:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.