0

I am currently setting up a virtualized environment with KVM that spans across two physical host machines (Host1 and Host2). Both are connected to the same VLAN. The network interfaces on both machines are configured like this:

auto lo
iface lo inet loopback

auto enp4s0
iface enp4s0 inet static
    address <<public-IP>>
    netmask <<netmask>>
    gateway <<gateway>>

auto enp4s0.4000
iface enp4s0.4000 inet manual
    vlan-raw-device enp4s0
    mtu 1400

auto br0
iface br0 inet static
    address 192.168.100.<<1 or 2, depending on the host>>/24
    bridge_ports enp4s0.4000

The VMs are connected to the network like this:

<interface type='bridge'>
  <mac address='52:54:00:91:64:64'/>
  <source bridge='br0'/>
  <model type='virtio'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>

and:

auto ens3
iface ens3 inet static
    address 192.168.100.<<101 or 102, depending on the VM>>/24
    gateway 192.168.100.<<1 or 2, depending on the host>>

So far, everything works pretty well. From the VMs, I can ping both hosts as well as all other VMs across both machines. I can also access the internet, now that I have enabled NAT by adding the following iptables-rule:

iptables -t nat -A POSTROUTING -o enp4s0 -j MASQUERADE

But now comes the problem: even though I can ping all hosts on the VLAN from any machine on the VLAN, I cannot seem to connect to TCP services running inside the VMs properly. Here are my observations:

  • From Host1, I can access the web server in VM1 (running on Host1).
  • From Host2, I can access the web server in VM2 (running on Host2).
  • From VM1 (running on Host1), I CANNOT access the web server in VM2 (running on Host2) and vice versa (the connection seems to time out, I am not getting connection refused).
  • From Host1, I CANNOT access the web server in VM2 (running on Host2) and vice versa (again, the connection seems to time out).
  • Also, using netcat, I have figured out that I CANNOT access TCP services on Host1 from Host2 (again, the connection seems to time out).

What really surprises me is that ping works, but curl does not. To me (being not very experienced in networking), it seems like this is probably some sort of configuration problem with iptables. This is how iptables is set up currently on both hosts (running and iptables -t nat -L -nv:

Output of iptables -L -nv:

Chain INPUT (policy ACCEPT 31 packets, 3143 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 5 packets, 520 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 23 packets, 4089 bytes)
 pkts bytes target     prot opt in     out     source               destination

Output of iptables -t nat -L -nv:

Chain PREROUTING (policy ACCEPT 19 packets, 1313 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 6 packets, 352 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   13   961 MASQUERADE  all  --  *      enp4s0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

I have even tried using tcpdump while attempting to establish a netcat connection between Host1 (nc -l 4444) and Host2 (nc 192.168.100.2 4444). There seems to be no traffic getting through.

This is how iptables is set up on the VMs:

Output of iptables -L -nv:

Chain INPUT (policy ACCEPT 31 packets, 3143 bytes)
 pkts bytes target     prot opt in     out     source               destination    
    6   399 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   tcp dpt:80 ctstate NEW,ESTABLISHED   

Chain FORWARD (policy ACCEPT 5 packets, 520 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 23 packets, 4089 bytes)
 pkts bytes target     prot opt in     out     source               destination
    4  1066 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   tcp spt:80 ctstate ESTABLISHED  

Output of iptables -t nat -L -nv:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination              

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

I have been working on this for two days now and I can't seem to figure it out. Any help would be greatly appreciated.

3
  • whats your target With the vlan? what Benefiz you think to gain if you use it?
    – djdomi
    Aug 12, 2019 at 20:14
  • I'm renting both host servers from an online provider that gives them one free public IP. If you want them together in an internal network (which i obviously need, because I'm running distributed internal services on the machines), they give you the option of putting them together in a VLAN for you. Both servers now have their public IPs, but they are also part of the virtual network 192.168.100.0/24and have private IP addresses in that network (192.168.100.1 and 192.168.100.2). Aug 13, 2019 at 5:49
  • ok, now we get one point closer, however we still need that you update your question with the /etc/network/interfaces by seperated from the both servers used iptables rules because in my point of thinking both should use the same gateway but in my point of view, would it not be easier to create a pptp or openvpn s2s vpn?
    – djdomi
    Aug 13, 2019 at 6:05

1 Answer 1

0

I have just figured out the problem myself. While I was writing a support ticket to my hosting provider, I thought about the additional firewall they offer. I assumed it only covered the public IP and not the private subnet on the VLAN. However, after turning that firewall off, everything now works as expected!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .