Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a KVM instance running inside of an Ubuntu host. I'm not able to ping/connect from the VM to the private interface of a physical node on the same private network of the host, although I can arping to it, and I can ping the physical node's public interface.

Set-up

  • host-a: physical node I'm trying to reach
  • host-b: physical node that hosts the VM
  • vm: KVM instance running inside of host-b

host-a

host-a has two interfaces (eth0, eth1) configured for bonding (bond0), and has two vlans attached to bond0:

  • bond0.133: public IP (xxx.xxx.xxx.195/27)
  • bond0.168: private IP (192.168.168.10/24)

host-b

host-b has a similar configuration:

  • bond0.133: public IP (xxx.xxx.xxx.197/27)
  • bond0.168: private IP (192.168.168.12/24)

host-b also has a bridge (br100) for connecting to the KVM instances, with an IP of 172.16.0.6/24.

 $ brctl show
bridge name bridge id       STP enabled interfaces
br100       8000.02163e77ca37   no      vlan100
                            vnet0
                            vnet1
virbr0      8000.000000000000   yes     

Routing table looks like this:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         xxx.xxx.xxx.193 0.0.0.0         UG    100    0        0 bond0.133
172.16.0.0      0.0.0.0         255.255.255.0   U     0      0        0 br100
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
192.168.168.0   0.0.0.0         255.255.255.0   U     0      0        0 bond0.168
192.168.169.0   192.168.168.1   255.255.255.0   UG    0      0        0 bond0.168
xxx.xxx.xxx.192 0.0.0.0         255.255.255.224 U     0      0        0 bond0.133

host-b also has some iptables configurations set up so that the VM has a public IP address (xxx.xxx.xxx.202)

vm

vm has a single interface (eth0) with an IP address of 172.16.0.13

Routing table looks like this:

 $ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.16.0.6      0.0.0.0         UG    100    0        0 eth0
172.16.0.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0

Issue

I'm trying to connect/ping from vm (172.16.0.13) to host-a on the private interface (192.168.168.12), but it times out. Traceroute looks like this:

$ traceroute -n 192.168.168.10
traceroute to 192.168.168.10 (192.168.168.10), 30 hops max, 60 byte packets
 1  172.16.0.6  0.767 ms  0.452 ms  0.309 ms
 2  * * *
 3  * * *
 4  * * *

From vm, I can:

  • ping to the public interface of host-a (xxx.xxx.xxx.195)
  • ping to the private interface of host-b (192.168.168.10)
  • arping to the private interface of host-b

arping looks like this:

$ sudo arping 192.168.168.10
ARPING 192.168.168.10 from 172.16.0.13 eth0
Unicast reply from 192.168.168.10 [XX:XX:XX:XX:XX:XX]  0.977ms
Unicast reply from 192.168.168.10 [XX:XX:XX:XX:XX:XX]  0.863ms
Unicast reply from 192.168.168.10 [XX:XX:XX:XX:XX:XX]  0.991ms
Unicast reply from 192.168.168.10 [XX:XX:XX:XX:XX:XX]  0.863ms
^CSent 4 probes (1 broadcast(s))
Received 4 response(s)

Is the problem here a routing configuration issue, or an iptables configuration issue, or something else?

host-b iptables-save output:

(public IP changed to xxx.xxx.xx)

# Generated by iptables-save v1.4.10 on Wed Feb 29 09:06:59 2012

*mangle
:PREROUTING ACCEPT [11670067:61782546800]
:INPUT ACCEPT [6465406:57204740255]
:FORWARD ACCEPT [5203256:4577980317]
:OUTPUT ACCEPT [6505299:7929051270]
:POSTROUTING ACCEPT [11708537:12507030679]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill 
COMMIT
# Completed on Wed Feb 29 09:06:59 2012
# Generated by iptables-save v1.4.10 on Wed Feb 29 09:06:59 2012
*nat
:PREROUTING ACCEPT [3744:690380]
:INPUT ACCEPT [2966:617201]
:OUTPUT ACCEPT [4763:295965]
:POSTROUTING ACCEPT [4905:304425]
:nova-compute-OUTPUT - [0:0]
:nova-compute-POSTROUTING - [0:0]
:nova-compute-PREROUTING - [0:0]
:nova-compute-float-snat - [0:0]
:nova-compute-snat - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-POSTROUTING - [0:0]
:nova-network-PREROUTING - [0:0]
:nova-network-float-snat - [0:0]
:nova-network-snat - [0:0]
:nova-postrouting-bottom - [0:0]
-A PREROUTING -j nova-network-PREROUTING 
-A PREROUTING -j nova-compute-PREROUTING 
-A OUTPUT -j nova-network-OUTPUT 
-A OUTPUT -j nova-compute-OUTPUT 
-A POSTROUTING -j nova-network-POSTROUTING 
-A POSTROUTING -j nova-compute-POSTROUTING 
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE 
-A POSTROUTING -j nova-postrouting-bottom 
-A nova-compute-snat -j nova-compute-float-snat 
-A nova-network-OUTPUT -d xxx.xxx.xxx.202/32 -j DNAT --to-destination 172.16.0.13 
-A nova-network-POSTROUTING -s 172.16.0.0/12 -d 10.128.0.0/24 -j ACCEPT 
-A nova-network-POSTROUTING -s 172.16.0.0/12 -d 172.16.0.0/12 -j ACCEPT 
-A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination xxx.xxx.xxx.197:8773 
-A nova-network-PREROUTING -d xxx.xxx.xxx.202/32 -j DNAT --to-destination 172.16.0.13 
-A nova-network-float-snat -s 172.16.0.13/32 -j SNAT --to-source xxx.xxx.xxx.202 
-A nova-network-snat -j nova-network-float-snat 
-A nova-network-snat -s 172.16.0.0/12 -j SNAT --to-source xxx.xxx.xxx.197 
-A nova-postrouting-bottom -j nova-network-snat 
-A nova-postrouting-bottom -j nova-compute-snat 
COMMIT
# Completed on Wed Feb 29 09:06:59 2012
# Generated by iptables-save v1.4.10 on Wed Feb 29 09:06:59 2012
*filter
:INPUT ACCEPT [233402:138096824]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [259241:112092547]
:nova-compute-FORWARD - [0:0]
:nova-compute-INPUT - [0:0]
:nova-compute-OUTPUT - [0:0]
:nova-compute-inst-17 - [0:0]
:nova-compute-local - [0:0]
:nova-compute-provider - [0:0]
:nova-compute-sg-fallback - [0:0]
:nova-filter-top - [0:0]
:nova-network-FORWARD - [0:0]
:nova-network-INPUT - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-local - [0:0]
-A INPUT -j nova-network-INPUT 
-A INPUT -j nova-compute-INPUT 
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT 
-A FORWARD -j nova-filter-top 
-A FORWARD -j nova-network-FORWARD 
-A FORWARD -j nova-compute-FORWARD 
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT 
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
-A OUTPUT -j nova-filter-top 
-A OUTPUT -j nova-network-OUTPUT 
-A OUTPUT -j nova-compute-OUTPUT 
-A nova-compute-inst-17 -m state --state INVALID -j DROP 
-A nova-compute-inst-17 -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A nova-compute-inst-17 -j nova-compute-provider 
-A nova-compute-inst-17 -s 172.16.0.6/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT 
-A nova-compute-inst-17 -s 172.16.0.0/24 -j ACCEPT 
-A nova-compute-inst-17 -p icmp -j ACCEPT 
-A nova-compute-inst-17 -p tcp -m tcp --dport 22 -j ACCEPT 
-A nova-compute-inst-17 -j nova-compute-sg-fallback 
-A nova-compute-local -d 172.16.0.13/32 -j nova-compute-inst-17 
-A nova-compute-sg-fallback -j DROP 
-A nova-filter-top -j nova-network-local 
-A nova-filter-top -j nova-compute-local 
-A nova-network-FORWARD -i br100 -j ACCEPT 
-A nova-network-FORWARD -o br100 -j ACCEPT 
COMMIT
# Completed on Wed Feb 29 09:06:59 2012
share|improve this question
    
Could be iptables. What does iptables-save say? –  dbanck Feb 29 '12 at 15:06
    
@dbanck I added the iptables-save output –  Lorin Hochstein Feb 29 '12 at 15:23
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.