0

I am having an issue getting ports to forward to the VM's via MASQUERADE iptable rules. I had this working on the same servers with Proxmox. But now with XCP-NG I cannot get this to work. I did once in the past on one of these servers. But I have not been able to replicate it again. Network connection for outbound and getting updates and http/https request on the VM's is working just fine. I just cant get the ports to forward to the VoyOs/Pfsense VM's so I can pass them on to the internal VM's.

In short I can get network to the VM's but I cannot get the ports to forward.

I have two server both running XCP-NG but one is running VoyOS and the other OPNsense:

  • XCP-NG : 8.2
  • OPNsense : 21.7.1
  • VoyOS : Latest

I followed these guides:

xenserver-single-ip-howto

setting-up-gateway-using-iptables-and-route-on-linux

create-private-network-bridge-proxmox-with-nat

xen-host-and-guest-sharing-same-ip

Network Map

               - Management Interface
NIC 0  --> xenbr0                                                 
               - MASQUERADE to xenbr1 (10.0.0.254) --> Wan- VoyOs/Pfsense (10.0.0.2) --> Lan Internal (10.0.1.1) --> VM's (10.0.1.x)
                                                             

Tried:

iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o xenbr0 -j MASQUERADE

iptables -A PREROUTING -t nat -p tcp -i xenbr0 -d XXX.XXX.XXX.XX --dport 2727 -j DNAT --to 10.0.0.2:2727

and removing -d XXX.XXX.XXX.XX external IPaddress

iptables -A PREROUTING -t nat -p tcp -i xenbr0 --dport 2727 -j DNAT --to 10.0.0.2:2727 

Then I removed those rules and tried:

iptables -I FORWARD 1 -i xenbr1 -j ACCEPT

iptables -t nat -A POSTROUTING -o xenbr0 -j MASQUERADE

And lastly tried:

iptables --append FORWARD --in-interface xenbr1 -j ACCEPT
iptables -D FORWARD --in-interface xenbr1 -j ACCEPT

IP Tables Rules

# iptables --list-rules
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N RH-Firewall-1-INPUT
-N xapi_nbd_input_chain
-N xapi_nbd_output_chain
-A INPUT -p tcp -m tcp --dport 10809 -j xapi_nbd_input_chain
-A INPUT -p gre -j ACCEPT
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A OUTPUT -p tcp -m tcp --sport 10809 -j xapi_nbd_output_chain
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -i xenapi -p udp -m udp --dport 67 -j ACCEPT
-A RH-Firewall-1-INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m conntrack --ctstate NEW -m udp --dport 694 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 21064 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m multiport --dports 5404,5405 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
-A xapi_nbd_input_chain -j REJECT --reject-with icmp-port-unreachable
-A xapi_nbd_output_chain -j REJECT --reject-with icmp-port-unreachable

IP tables

# iptables -L && iptables -t nat -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd
Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DNAT       tcp  --  anywhere             anywhere             tcp dpt:mgcp-callagent to:10.0.0.2:2727
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  10.0.0.0/24          anywhere

/etc/sysconfig/iptables

# cat /etc/sysconfig/iptables
# Generated by iptables-save v1.4.21 on Mon Aug 16 20:14:40 2021
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [6179:9587238]
:RH-Firewall-1-INPUT - [0:0]
:xapi_nbd_input_chain - [0:0]
:xapi_nbd_output_chain - [0:0]
-A INPUT -p tcp -m tcp --dport 10809 -j xapi_nbd_input_chain
-A INPUT -p gre -j ACCEPT
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A OUTPUT -p tcp -m tcp --sport 10809 -j xapi_nbd_output_chain
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -i xenapi -p udp -m udp --dport 67 -j ACCEPT
-A RH-Firewall-1-INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m conntrack --ctstate NEW -m udp --dport 694 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m conntrack --ctstate NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 21064 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m multiport --dports 5404,5405 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
-A xapi_nbd_input_chain -j REJECT --reject-with icmp-port-unreachable
-A xapi_nbd_output_chain -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Mon Aug 16 20:14:40 2021
# Generated by iptables-save v1.4.21 on Mon Aug 16 20:14:40 2021
*nat
:PREROUTING ACCEPT [5:322]
:INPUT ACCEPT [3:164]
:OUTPUT ACCEPT [1:73]
:POSTROUTING ACCEPT [1:73]
-A PREROUTING -i xenbr0 -p tcp -m tcp --dport 2727 -j DNAT --to-destination 10.0.0.2:2727
-A POSTROUTING -s 10.0.0.0/24 -o xenbr0 -j MASQUERADE
COMMIT
# Completed on Mon Aug 16 20:14:40 2021

Ifconfig

# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 78:2b:cb:3c:81:65  txqueuelen 1000  (Ethernet)
        RX packets 24705  bytes 3641727 (3.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 47612  bytes 37970921 (36.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 78:2b:cb:3c:81:66  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 5951  bytes 33860750 (32.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5951  bytes 33860750 (32.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vif1.0: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST>  mtu 1500
        ether fe:ff:ff:ff:ff:ff  txqueuelen 32  (Ethernet)
        RX packets 6  bytes 444 (444.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vif1.1: flags=4291<UP,BROADCAST,RUNNING,NOARP,MULTICAST>  mtu 1500
        ether fe:ff:ff:ff:ff:ff  txqueuelen 32  (Ethernet)
        RX packets 1442  bytes 95984 (93.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1336  bytes 143058 (139.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

xapi0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether be:16:43:65:a2:4f  txqueuelen 1000  (Ethernet)
        RX packets 6  bytes 444 (444.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

xenbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet XXX.XXX.XXX.XX  netmask 255.255.255.252  broadcast XXX.XXX.XXX.XX
        ether 78:2b:cb:3c:81:65  txqueuelen 1000  (Ethernet)
        RX packets 23841  bytes 3101836 (2.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 33197  bytes 36975647 (35.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

xenbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.254  netmask 255.255.255.0  broadcast 10.0.0.255
        ether 78:2b:cb:3c:81:66  txqueuelen 1000  (Ethernet)
        RX packets 1442  bytes 95984 (93.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1336  bytes 143058 (139.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

/etc/sysctl.conf

# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.