0

I have quite complicated network setup, but I am also newbie in routing and similar topics.

My ISP is providing several static IPs via PPPoE, let's call them IP0, IP1, IP2, IP3 and so on. IP0 is assigned to the router, and provided to the entire LAN, and also to my host by the interface INT1.

On different interface on that host INT2, there is separate connection to the modem. Via INT2 I can get separate PPP connections with IP1, IP2, IP3, but I also get new virtual interfaces PPP1, PPP2, PPP3.

I successfully managed to create necessary routing tables and routing rules, so I can use all 4 ext IPs on that host, and also assign ext IP to VMs running on that host. For example my host is seen on the internet as IP0, VM1 as IP2, VM2 as IP2, and VM3 as IP3. I can connect to all 4 of them from external hosts via correct IPs.

There is one thing that is not working. I cannot connect from VM1 to the services on VM2 and VM3 via WAN. From VM1, IP2 and IP3 are not pingable, and there is no connection to any services. Same thing for VM2 and VM3. On the host I also cannot ping IP2 and IP3 via PPP1, or in any other combination.

I can workaround the problem by replacing ext IPs of services by VM's internal IP in LAN, but I am looking for more transparent solution.

Any ideas?

My OS is Debian 10, and I am using nftables as firewall.

network diagram

My network config:

ip rule show
0:  from all lookup local
32672:  from IP1 lookup PPP1-RT
32673:  from IP2 lookup PPP2-RT
32674:  from IP3 lookup PPP3-RT
32676:  from VM1_VLAN_IP lookup PPP1-RT
32678:  from VM2_VLAN_IP lookup PPP2-RT
32679:  from VM3_VLAN_IP lookup PPP3-RT
32766:  from all lookup main
32767:  from all lookup default
ip route show table PPP1-RT
default dev PPP1 scope link metric 1
PPP_GATEWAY_IP dev PPP1 scope link src IP1 metric 1
ip route show table main
default via HOST_LAN_IP dev INT1 proto dhcp metric 1
PPP_GATEWAY_IP dev PPP1 proto kernel scope link src IP1
PPP_GATEWAY_IP dev PPP2 proto kernel scope link src IP2
PPP_GATEWAY_IP dev PPP3 proto kernel scope link src IP3
VM_VLAN_SUBNET dev VLAN proto kernel scope link src VLAN_HOST_IP metric 425
LAN_SUBNET dev INT1 proto kernel scope link src HOST_LAN_IP metric 1
/etc/nftables.conf

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
        chain input {
                type filter hook input priority 0; policy drop;
                ct state established,related accept
                iif "lo" accept
                ct state invalid log prefix "Invalid packet: " drop
                ip protocol icmp accept
                ip protocol igmp accept
                ip6 nexthdr ipv6-icmp accept
                udp dport 0-65535 ct state new jump UDP
                tcp dport 0-65535 tcp flags & (fin | syn | rst | ack) == syn ct state new jump TCP
                udp dport 0-65535 log prefix "UDP reject: " reject
                tcp dport 0-65535 log prefix "TCP reject: " reject with tcp reset
                ip protocol != { tcp, udp, icmp } log prefix "Other IPv4 reject: " reject with icmp type prot-unreachable
                log prefix "Other reject: " reject with icmpx type admin-prohibited
        }

        chain forward {
                type filter hook forward priority 0; policy drop;
        meta l4proto tcp tcp flags & (syn|rst) == syn counter packets 53 bytes 3072 tcp option maxseg size set rt mtu
                ct state established,related accept
                ct state invalid log prefix "Invalid forward packet: " drop
        #vlan
        iifname "VLAN" ip saddr VM_VLAN_SUBNET accept
        #VM1
            oifname "VLAN" ip daddr VM1_VLAN_IP tcp dport { SOME_OPEN_PORTS } accept
    #VM2
          oifname "VLAN" ip daddr VM2_VLAN_IP tcp dport { SOME_OPEN_PORTS } accept
    #VM3
          oifname "VLAN" ip daddr VM3_VLAN_IP tcp dport { SOME_OPEN_PORTS } accept
          log prefix "Other forward reject: " reject with icmpx type host-unreachable
        }

        chain output {
                type filter hook output priority 0; policy accept;
        }

        chain UDP {
                        #some services

        }

        chain TCP {
                        #some services
        }
}

# NAT
table ip nat {
        chain prerouting {
                type nat hook prerouting priority 0; policy accept;
                #VM1
        ip daddr { IP1 } tcp dport { SOME_OPEN_PORTS } dnat VM1_VLAN_IP
                #VM2
        ip daddr { IP2 } tcp dport { SOME_OPEN_PORTS } dnat VM2_VLAN_IP
                #VM3
        ip daddr { IP3 } tcp dport { SOME_OPEN_PORTS } dnat VM3_VLAN_IP
        }

        chain postrouting {
                type nat hook postrouting priority 100; policy accept;
                #VM1
        ip saddr VM1_VLAN_IP oifname "PPP1" snat IP1
                #VM2
        ip saddr VM2_VLAN_IP oifname "PPP2" snat IP2
                #VM3
        ip saddr VM3_VLAN_IP oifname "PPP3" snat IP3
        }
}
2
  • It would be helpful if you can attach a diagram. Jan 8, 2020 at 12:58
  • Thanks for suggestion. Added.
    – user554684
    Jan 8, 2020 at 13:48

0

You must log in to answer this question.