I've got a Linux based Firewall/Router running iptables and SNAT for traffic passing through it to the Internet via a single uplink - This is working fine. There is also a small number of applications running on this server which also work fine.
I've also got an OpenVPN client running on this router, and I'm wanting to route a specific local user's traffic down this tunnel.
I've got the tunnel up and running which has created an interface tun_v.
I've created a new routing table (/etc/iproute2/rt_tables) v.tunnel and I'm adding a single, default route in this table down the tunnel.
I've created an IP rule to use this table for traffic marked with 1
I mark my user's traffic with 1.
ip route add default via 172.25.26.61 dev tun_v table v.tunnel
ip rule add fwmark 1 table v.tunnel
iptables -t mangle -A OUTPUT -m owner --uid-owner 1002 -j MARK --set-xmark 0x1/0xffffffff
I'm also masquerading traffic on the router:
iptables -t nat -A POSTROUTING -o wan -j MASQUERADE #this is normal NAT to the Internet
iptables -t nat -A POSTROUTING -o tun_v -j MASQUERADE #For the tunnel
The firewall also default drops input(and forward) chains so I allow related:
iptables -A INPUT -i wan -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i tun_v -m state --state RELATED,ESTABLISHED -j ACCEPT (there are some other firewall rules but not related to this)
If I generate traffic from this user (with uid 1002) and tcpdump the tun_v interface, I can see my packet go out, and the reply come back. However I never see the packet reach my application. ping doesn't show any reply, trying to establish a TCP connection I see the SYN go out, the SYNACK come back and then a TCP retransmission of the SYN go out.
If I add a 'normal' route that uses this tunnel e.g.
ip route add a.b.c.d via 172.25.26.61
then I can (from any user) ping the specified server.
I'm sure there is something else I'm missing, but I can't see what. As I understand it the route working for all users to a.b.c.d (table main) means the tunnel and all iptables rules are correct. The marking of the traffic is working and applying to the correct rule is also working, because I can see the packet leave the correct interface, and a reply come back.
It feels like there is a firewall rule somewhere causing the problem, but as mentioned above, the test rule down the tunnel for a.b.c.d works.
If I delete the MARK from iptables, then all traffic from UID 1002 works (via the normal wan interface).
Any help or suggestions would be gratefully received!