13

I have a small network with a router, which maintains a connection to Internet, a server and some workstations in a local network.

Network map

Server is meant to be accessed from the Internet, and there are several DNAT entries set in the router iptables, like this:

-A PREROUTING -i ppp0 -p tcp -m multiport --dports 22,25,80,443 -j DNAT --to-destination 192.168.2.10

External packets come to router via ppp0 interface, and internal ones go from br-lan, which actually includes the switch and WLAN adapter. The problem is, while external access works fine, trying to access the server from inside the LAN by a DNS-resolved external IP (assigned to ppp0) fails.

The only solution I was able to invent is to add static entries to router's /etc/hosts pointing to the internal IP, but as there are no wildcards (and I have at least three top-level domains assigned to that system, not counting tens of subdomains), that's rather crunchy and failure-prone. Can you suggest something better?

I've only found this question, which was not very helpful.

If that's relevant, the router runs OpenWRT 10.03 Kamikaze with dnsmasq.

2
  • What version of OpenWRT?
    – Corey S.
    Nov 23 '10 at 12:44
  • @Corey 10.03 Stable, but that has nothing to do with openwrt itself, isn't it?
    – whitequark
    Nov 23 '10 at 16:44
6

I am surprised that after almost 8 years, nobody has explained how to do this the correct way using the UCI configuration system used by default in OpenWRT.

Steven Monday's answer is correct, yet it is using iptables commands directly, which is a lower layer than the UCI configuration system, and is best left untouched by most OpenWRT users if possible.

The correct way to access internal servers through their public IP/port combos from another internal host in UCI is by enabling the configuration option reflection under each specific DNAT target in the file /etc/config/firewall. This behavior is documented here.

For example:

config redirect option target 'DNAT' option src 'wan' option dest 'lan' option proto 'tcp' option src_dport '44322' option dest_ip '192.168.5.22' option dest_port '443' option name 'apache HTTPS server' option reflection '1'

Note: According to the indicated OpenWRT documentation, reflection is enabled by default. In my testing, this was not the case.

18

I deleted my original answer, because I wasn't fully confident that it was correct. I have since had some time to set up a little virtual network of VMs to simulate the network in question. Here is the set of firewall rules that worked for me (in iptables-save format, for the nat table only):

-A PREROUTING -d 89.179.245.232/32 -p tcp -m multiport --dports 22,25,80,443 -j DNAT --to-destination 192.168.2.10
-A POSTROUTING -s 192.168.2.0/24 -o ppp0 -j MASQUERADE
-A POSTROUTING -s 192.168.2.0/24 -d 192.168.2.10/32 -p tcp -m multiport --dports 22,25,80,443 -j MASQUERADE

The first POSTROUTING rule is a straightforward way of sharing the internet connection with the LAN. I left it there for completeness.

The PREROUTING rule and the second POSTROUTING rule together establish the appropriate NATs, so that connections to the server via the external IP address can happen, regardless of whether the connections originate from outside or from inside the LAN. When clients on the LAN connect to the server via the external IP address, the server sees the connections as coming from the router's internal IP address (192.168.2.1).

Interestingly, it turns out that there are a couple of variations of the second POSTROUTING rule that also work. If the target is changed to -j SNAT --to-source 192.168.2.1, the effect is (not surprisingly) the same as the MASQUERADE: the server sees connections from local LAN clients as originating from the router's internal IP address. On the other hand, if the target is changed to -j SNAT --to-source 89.179.245.232, then the NATs still work, but this time the server sees connections from local LAN clients as originating from the router's external IP address (89.179.245.232).

Finally, note that your original PREROUTING/DNAT rule with -i ppp0 does not work, because the rule never matches packets coming from the LAN clients (since those don't enter the router via the ppp0 interface). It would be possible to make it work by adding a second PREROUTING rule just for the internal LAN clients, but it would be inelegant (IMO) and would still need to refer explicitly to the external IP address.

Now, even after having laid out a "hairpin NAT" (or "NAT loopback", or "NAT reflection", or whatever one prefers to call it) solution in full detail, I still believe that a split-horizon DNS solution---with external clients resolving to the external IP and internal clients resolving to the internal IP---would be the more advisable route to take. Why? Because more people understand how DNS works than understand how NAT works, and a big part of building good systems is choosing to use parts that are maintainable. A DNS setup is more likely to be understood, and thus correctly maintained, than an arcane NAT setup (IMO, of course).

4
  • This works perfectly, thank you a lot! I agree that DNS setup is better, but you cannot forward different ports on same external IP to different machines on LAN with it. Anyway, I'm the only one who will ever maintain this setup, so it's fine.
    – whitequark
    Nov 24 '10 at 6:25
  • I'm glad to hear this worked for you! Nov 24 '10 at 6:35
  • What is "IMO"? International Meteor Organization www.imo.net? Apr 2 '17 at 20:39
  • @macmadness86 IMO == In My Opinion Apr 15 '17 at 0:04
3

A common solution is to point your internal hosts at a local DNS server that returns the correct "internal" address for these hostnames.

Another solution -- and we're using this where I work on our Cisco firewalls -- is to rewrite DNS responses on the firewall that correspond to these addresses. I don't think there are tools for Linux that do this right now.

You should be able to configure the routing on your gateway to do the right thing. You may need to configure the servers to be aware of their externally mappped ip address (e.g., by assigning it to a dummy interface). With this configuration, communication from one internal system to another internal system -- using it's "external" address -- would go through the router.

3
  • Hmm. So are you suggesting adding the external IP to servers' interfaces and then configuring router so it will forward all packets to the external IP coming from inside the LAN to that server? Interesting, I'll test it soon.
    – whitequark
    Nov 23 '10 at 17:30
  • Can you suggest the configuration? I tried this: ip rule add to 89.179.245.232 dev br-lan table 10; ip route add 89.179.245.232 via 192.168.2.10 dev br-lan table 10, and it isn't working.
    – whitequark
    Nov 24 '10 at 5:40
  • What's in routing table 10? On the internal servers, you probably want them to have both a local 192.168.x.x address (for communicating locally) and the public address (as an alias) on their primary interface.
    – larsks
    Nov 24 '10 at 14:06
2

What you are asking to do is called NAT Loopback and it requires that you add a SNAT rule so that packets originating from your LAN to your Server will go back through the router:

-A POSTROUTING -p tcp -s 192.168.2.0/24 -d 192.168.2.10 -m multiport --dports 22,25,80,443 -j SNAT --to-source 89.179.245.232
3
  • Sadly, that does not work. I've originally missed the -i ppp0 option in my rule in question, as that was handled by other chain; this rule would prevent routing of packets coming from LAN (and if I'd enable it, packets will go from wrong source and will be rejected).
    – whitequark
    Nov 23 '10 at 18:11
  • Have you tried it? It will only affect packets from your LAN going to your server IP on those very specific ports.
    – SiegeX
    Nov 23 '10 at 18:43
  • Yes, I did. (And I tried changing the first rule, too). E.g. dig sends a packet to 192.168.2.1#53, and then gets an unexpected reply from 192.168.2.10#53, with or without your rule.
    – whitequark
    Nov 24 '10 at 4:34
0

larsks comment about hosting an internal version of the namespace\domain is generally the way I've handled this issue in the past. Of course, you need a DNS server internally in order to do this.

3
  • Yeah, I've wrote that I am using dnsmasq. Any ideas on setting up automatic substitution?
    – whitequark
    Nov 23 '10 at 17:47
  • I know nothing about OpenWRT and Kamikaze, but based on what I'm reading - what if you added the following to your /etc/dnsmasq.conf "cname=ext-hostname.domain.com,int-hostname.domain.com"
    – CurtM
    Nov 23 '10 at 17:58
  • Well, as far as I was able to determine, dnsmasq's cname does not support masks, and thus is not applicable for me due to subdomain count.
    – whitequark
    Nov 24 '10 at 5:08
0

I came up with the following solution to allow my guest network to connect to ports that were forwarded from my wan to lan network. This script is placed in my "Network -> Firewall -> Custom Rules" section:

# lookup the public IP (DDNS resolves to this)
wanip=$(ip route get 8.8.8.8 | awk -F"src " 'NR==1{split($2,a," ");print a[1]}')

# Guest network to LAN
# srcname is the guest network name, this needs to match for iptables
srcname=guest
# CIDR notation of guest and lan networks
srcnet=192.168.15.0/24
tgtnet=192.168.10.0/24
# router (openwrt) IP on lan network
tgtrouter=192.168.10.1
# host on lan network where ports are normally forwarded
tgthost=192.168.10.5
# ports to forward as a list or range
tcpports=8080,9090
udpports=12345

prechain=prerouting_${srcname}_rule
postchain=postrouting_${srcname}_rule

# reset the tables to prevent duplicate rules
iptables -t nat -F ${prechain}
iptables -t nat -F ${postchain}

iptables -t nat -A ${prechain} -s ${srcnet} -d ${wanip}/32 -p tcp -m tcp -m multiport --dports ${tcpports} -m comment --comment "${srcname} NAT reflection TCP DNAT" -j DNAT --to-destination ${tgthost}
iptables -t nat -A ${postchain} -s ${srcnet} -d ${tgthost}/32 -p tcp -m tcp -m multiport --dports ${tcpports} -m comment --comment "${srcname} NAT reflection TCP SNAT" -j SNAT --to-source ${tgtrouter}
iptables -t nat -A ${prechain} -s ${srcnet} -d ${wanip}/32 -p udp -m udp -m multiport --dports ${udpports} -m comment --comment "${srcname} NAT reflection UDP DNAT" -j DNAT --to-destination ${tgthost}
iptables -t nat -A ${postchain} -s ${srcnet} -d ${tgthost}/32 -p udp -m udp -m multiport --dports ${udpports} -m comment --comment "${srcname} NAT reflection UDP SNAT" -j SNAT --to-source ${tgtrouter}

To support reboots, I needed to run the following from the ssh command line on openwrt (otherwise, I believe there is a race condition where some rules were added and then flushed during the reboot):

uci set firewall.@include[0].reload="1"
uci commit firewall

NAT reflection is setup for connections within the LAN network to itself, but not to other networks if you have created multiple interfaces to isolate traffic. I tried configuring a forwarding rule from the web interface, but that sends all traffic to a port from the guest network to that LAN host. The above only intercepts requests to the WAN IP instead of all network traffic.

An internal DNS could be used instead of this, but only if all port forwards only go to a single host. If you have multiple hosts where you forward different ports, you can repeat the rules for different ports to different tgthost IP's and ports.

2
  • In current kernels there is conntrack match module. And all you need to solve the issue is use the single rule like that: iptables -t nat -A POSTROUTING --dst <lan-net> ! --src <lan-gw-ip> -m conntrack --ctstate DNAT --ctorigdst <wan-gw-ip> -j MASQUERADE May 13 '19 at 20:51
  • @AntonDanilov nice, I like that. The rules I used were based off of reflection NAT rules already in OpenWRT for connections from the same subnet. Not sure if they had any other reasons for that other than possibly being written before conntrack was available.
    – BMitch
    May 13 '19 at 21:31
0

For anyone looking to do this using nftables (the official replacement for iptables), here's what I came up with:

define dnat_targets = {
    http : 10.0.10.1 . http, 
    https : 10.0.10.1 . https,
    32400 : 10.0.10.4 . 32400, 
    25565 : 10.0.10.8 . 25565, 
}

define dnat_allowed = {
    10.0.10.1 . http,
    10.0.10.1 . https,
    10.0.10.4 . 32400, 
}

table inet nat {
    map dnat_destinations {
        type inet_service : ipv4_addr . inet_service
        elements = $dnat_targets
    }

    set dnat_masq {
        type ipv4_addr . inet_service
        elements = $dnat_allowed
    }

    chain prerouting {
        ip daddr != 10.0.0.0/8 fib daddr type local dnat ip addr . port to tcp dport map @dnat_destinations
    }

    chain postrouting {
        ip saddr 10.0.0.0/8 ip daddr . tcp dport @dnat_masq masquerade
    }
}

table inet filter {
    set dnat_allowed {
        type ipv4_addr . inet_service
        elements = $dnat_allowed
    }

    chain forward {
        ip daddr . tcp dport @dnat_allowed accept
    }
}

Note that this is a snippet from my firewall definition, you need to have the chains created before using this. It also doesn't match the original post situation exactly, the ip address and RFC1918 ranges need to be adapted.

To add DNAT targets, you only need to add them to the dnat_targets definition and then allow the destination in the dnat_allowed set. As far as I can tell you can't do it with only the dnat_targets set, but one advantage of doing it this way is that you can toggle the dnat rules on and off by removing them from the dnat_allowed set while still keeping the mapping around in case you want it later.

One really nice thing about this is that the maps are first class constructs in nftables, in other words, no matter how many DNAT targets you have (1,10,100,1000) the packets only need to traverse these three rules which use the maps and sets (similar to how ipset works).

Another thing I added was to make this working using an unknown public IP address, so you can safely use this when the wan interface is using DHCP, and you can still run a LAN accessible website, or anything else for that matter, on the router box.

Finally, this does not show the masquerade rule for the wan connection which is required to be fully functional.

Going through the rules one at a time:

ip daddr != 10.0.0.0/8 fib daddr type local dnat ip addr . port to tcp dport map @dnat_destinations

This is the main DNAT rule, if the destination address is not in 10.0.0.0/8 and it is a local address (defined as any address assigned to an interface on the local machine) then we're going to do DNAT on it. This is what makes sure that you match only packets intended for an unknown WAN address. Note that if you skip daddr != 10.0.0.0/8 then the dnat rule will match request to any interface on the router, so an admin web UI would be dnated instead of shown when access via 10.0.0.1:80 for example, and if you skip fib daddr type local then any request for anything using any of the ports in the map (any website for example) will be dnated (e.g. if you try to go to google.com on one of your clients you'd end up at the site you're trying to host instead).

My router has several lan addresses, if yours only has one you can simplify the rule further. The DNAT is defined using the map lookup on the destination port.

ip saddr 10.0.0.0/8 ip daddr . tcp dport @dnat_masq masquerade

This is analogous to the masquerade rule in the iptables answer, if the source is a lan address and the destination address and port are in the dnat_allowed set, then masquerade. The reason this makes it work, by the way, is that if you don't alter the source address, then when the server goes to respond to the web request, it will see a LAN address, then arp "who has" that LAN address, your machine will respond to that arp request, and then it will directly send the packet to your local machine. Your local machine, however, made the request to the routers public IP, so when it gets a response from the server directly it doesn't know what to do with it since it wasnt expecting a packet from that address, so it flags it as invalid and drops it. By masquerading here, you force the web server to reply to the router and the router sends it back to your machine using conntrack.

ip daddr . tcp dport @dnat_allowed accept

Finally, the forward filter rule, if the destination address and port are in the dnat_allowed set, then accept the packet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.