Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a dedicated server running Debian 6, with Libvirt (0.9.11.3) and Qemu-KVM (qemu-kvm-1.0+dfsg-11, Debian).
I am having a problem getting bridged/routed networking to work in KVM guests with newer kernels (>2.6.38). NATted networking works fine though.
Older kernels work perfectly fine as well.
The host kernel is at version 3.2.0-2-amd64, the problem was also there on an older host kernel.
The contents of the host's /etc/network/interfaces (ip removed):

# Loopback device:
auto lo
iface lo inet loopback

# bridge
auto br0
iface br0 inet static
  address   176.9.xx.xx
  broadcast 176.9.xx.xx
  netmask   255.255.255.224
  gateway   176.9.xx.xx
  pointopoint   176.9.xx.xx
  bridge_ports eth0
  bridge_stp off
  bridge_maxwait 0
  bridge_fd 0
  up route add -host 176.9.xx.xx dev br0 # VM IP
  post-up mii-tool -F 100baseTx-FD br0

# default route to access subnet
up route add -net 176.9.xx.xx netmask 255.255.255.224 gw 176.9.xx.xx br0

The output of ifconfig -a on the host:

br0       Link encap:Ethernet  HWaddr 54:04:a6:8a:66:13  
          inet addr:176.9.xx.xx  Bcast:176.9.xx.xx  Mask:255.255.255.224
          inet6 addr: fe80::5604:a6ff:fe8a:6613/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20216729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19962220 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:14144528601 (13.1 GiB)  TX bytes:7990702656 (7.4 GiB)

eth0      Link encap:Ethernet  HWaddr 54:04:a6:8a:66:13  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:26991788 errors:0 dropped:12066 overruns:0 frame:0
          TX packets:19737261 errors:270082 dropped:0 overruns:0 carrier:270082
          collisions:1686317 txqueuelen:1000 
          RX bytes:15459970915 (14.3 GiB)  TX bytes:6661808415 (6.2 GiB)
          Interrupt:17 Memory:fe500000-fe520000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:6240133 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6240133 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6081956230 (5.6 GiB)  TX bytes:6081956230 (5.6 GiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:79:e4:5a                                                                                                                                                                                                                        
          inet addr:192.168.100.1  Bcast:192.168.100.255  Mask:255.255.255.0                                                                                                                                                                                                   
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                                                                                                                                                                                                                   
          RX packets:225016 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                                                                              
          TX packets:412958 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                                                                            
          collisions:0 txqueuelen:0                                                                                                                                                                                                                                            
          RX bytes:16284276 (15.5 MiB)  TX bytes:687827984 (655.9 MiB)                                                                                                                                                                                                         

virbr0-nic Link encap:Ethernet  HWaddr 52:54:00:79:e4:5a                                                                                                                                                                                                                       
          BROADCAST MULTICAST  MTU:1500  Metric:1                                                                                                                                                                                                                              
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                                                                                   
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                                                                                 
          collisions:0 txqueuelen:500                                                                                                                                                                                                                                          
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)                                                                                                                                                                                                                               

vnet0     Link encap:Ethernet  HWaddr fe:54:00:93:4e:68                                                                                                                                                                                                                        
          inet6 addr: fe80::fc54:ff:fe93:4e68/64 Scope:Link                                                                                                                                                                                                                    
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                                                                                                                                                                                                                   
          RX packets:607670 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                                                                              
          TX packets:5932089 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                                                                           
          collisions:0 txqueuelen:500                                                                                                                                                                                                                                          
          RX bytes:83574773 (79.7 MiB)  TX bytes:1092482370 (1.0 GiB)                                                                                                                                                                                                          

vnet1     Link encap:Ethernet  HWaddr fe:54:00:ed:6a:43                                                                                                                                                                                                                        
          inet6 addr: fe80::fc54:ff:feed:6a43/64 Scope:Link                                                                                                                                                                                                                    
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                                                                                                                                                                                                                   
          RX packets:922132 errors:0 dropped:0 overruns:0 frame:0                                                                                                                                                                                                              
          TX packets:6342375 errors:0 dropped:0 overruns:0 carrier:0                                                                                                                                                                                                           
          collisions:0 txqueuelen:500 
          RX bytes:251091242 (239.4 MiB)  TX bytes:1629079567 (1.5 GiB)

vnet2     Link encap:Ethernet  HWaddr fe:54:00:0d:cb:3d  
          inet6 addr: fe80::fc54:ff:fe0d:cb3d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9461 errors:0 dropped:0 overruns:0 frame:0
          TX packets:665189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:4990275 (4.7 MiB)  TX bytes:49229647 (46.9 MiB)

vnet3     Link encap:Ethernet  HWaddr fe:54:cd:83:eb:aa  
          inet6 addr: fe80::fc54:cdff:fe83:ebaa/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1649 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12177 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:77233 (75.4 KiB)  TX bytes:2127934 (2.0 MiB)

The guest's /etc/network/interfaces, in this case running Ubuntu 12.04 (ip removed):

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 176.9.xx.xx
        netmask 255.255.255.248
        gateway 176.9.xx.xx                # Host IP
        pointopoint 176.9.xx.xx            # Host IP
        dns-nameservers 8.8.8.8 8.8.4.4

The output of ifconfig -a on the guest:

eth0      Link encap:Ethernet  HWaddr 52:54:cd:83:eb:aa  
          inet addr:176.9.xx.xx  Bcast:0.0.0.0  Mask:255.255.255.255
          inet6 addr: fe80::5054:cdff:fe83:ebaa/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14190 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1768 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2614642 (2.6 MB)  TX bytes:82700 (82.7 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:954 errors:0 dropped:0 overruns:0 frame:0
          TX packets:954 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:176679 (176.6 KB)  TX bytes:176679 (176.6 KB)

Output of ping -c4 on the guest:

PING google.nl (173.194.35.151) 56(84) bytes of data.
64 bytes from muc03s01-in-f23.1e100.net (173.194.35.151): icmp_req=1 ttl=55 time=14.7 ms
From static.174.82.xx.xx.clients.your-server.de (176.9.xx.xx): icmp_seq=2 Redirect Host(New nexthop: static.161.82.9.176.clients.your-server.de (176.9.82.161))
64 bytes from muc03s01-in-f23.1e100.net (173.194.35.151): icmp_req=2 ttl=55 time=15.1 ms
From static.198.170.9.176.clients.your-server.de (176.9.170.198) icmp_seq=3 Destination Host Unreachable
From static.198.170.9.176.clients.your-server.de (176.9.170.198) icmp_seq=4 Destination Host Unreachable

--- google.nl ping statistics ---
4 packets transmitted, 2 received, +2 errors, 50% packet loss, time 3002ms
rtt min/avg/max/mdev = 14.797/14.983/15.170/0.223 ms, pipe 2

The static.174.82.xx.xx.clients.your-server.de (176.9.xx.xx) is the host's IP.
I have encountered this problem with every guest OS I've tried, that being Fedora, Ubuntu (server/desktop) and Debian with an upgraded kernel. I've also tried compiling the guest kernel myself, to no avail.
I have no problem with recompiling a kernel, though the host cannot afford any downtime.
Any ideas on this problem are very welcome.

EDIT: I can ping the host from inside the guest.

share|improve this question

2 Answers 2

Ok, I have found the solution myself. Apparently, the article on the Hetzner wiki wasn't complete. Redirects were still being sent, resulting in Hetzner (where we have our server) blocking the traffic. I am not sure how it had to do with what kernel version the guests were running, if someone can explain that I would very much like to hear it.

You can find out if you suffer from the same problem by executing the following command:

for file in `find /proc/ -iname send_redirects`; do echo $file; cat $file; done

which outputs something like:

/proc/sys/net/ipv4/conf/all/send_redirects
0
/proc/sys/net/ipv4/conf/default/send_redirects
1
/proc/sys/net/ipv4/conf/lo/send_redirects
1
/proc/sys/net/ipv4/conf/eth0/send_redirects
0
/proc/sys/net/ipv4/conf/br0/send_redirects
1
/proc/sys/net/ipv4/conf/vnet1/send_redirects
1
/proc/sys/net/ipv4/conf/vnet0/send_redirects
1
/proc/sys/net/ipv4/conf/virbr0/send_redirects
1
/proc/sys/net/ipv4/conf/virbr0-nic/send_redirects
1
/proc/sys/net/ipv4/conf/vnet2/send_redirects
1
/proc/sys/net/ipv4/conf/vnet3/send_redirects
1

I have to admit I do not know which ones should or should not be 0, but I have set them all to 0, and now it works fine, so I'll leave it at that.

for file in `find /proc/ -iname send_redirects`; do echo 0 > $file; done
share|improve this answer

I had the same issue, thanks for sharing your solution. I think the setting is needed only on physical and bridge interface (eth0 and br0). You can make the change persistent by setting it into /etc/sysctl.d/

#/etc/sysctl.d/10-no-icmp-redirects.conf
# Hetzner says:
# Because of our network setup, the Host machine could send ICMP
# "redirect" messages to all guests, telling them to find the Hetzner
# gateway directly.  That is impossible: Hetzner would throw away the
# traffic from the virtual interfaces because of their non registered
# MAC addresses (ie different from the main interface).
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.eth0.send_redirects=0
net.ipv4.conf.br0.send_redirects=0
net.ipv4.conf.default.send_redirects=0
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.