I have troubles accessing a host private interface (ip) from a docker container. I'm fairly certain that it's related to my Iptables rules (or perhaps routing). When I add the
--net=host flag to
docker run, everything works as expected. Similarly when I specify that the INPUT policy is following a liberal
-P INPUT ACCEPT, things also work as I would expect. However these are undesirable and unsafe options i'd like to avoid.
Since it's not specific to my services (DNS) I've excluded that from the problem, since searching for that in combination with docker yields in a different (popular) problem area, adding noise to the search results.
Also linking of Docker containers is not a viable option, because certain containers need to be run with the --net=host option, preventing linking and I want to create a consistent situation where possible.
I have the following Iptables rules. A combination of CoreOS, Digital Ocean and Docker I assume.
-P INPUT DROP -P FORWARD ACCEPT -P OUTPUT ACCEPT -N DOCKER -A INPUT -i lo -j ACCEPT -A INPUT -i eth1 -j ACCEPT -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT -A FORWARD -o docker0 -j DOCKER -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i docker0 ! -o docker0 -j ACCEPT -A FORWARD -i docker0 -o docker0 -j ACCEPT
My (relevant) host interfaces:
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 inet 10.129.112.210/16 brd 10.129.255.255 scope global eth1 valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default inet 172.17.42.1/16 scope global docker0 valid_lft forever preferred_lft forever
And I run a docker container:
$ docker run --rm -it --dns=10.129.112.210 debian:jessie # Specifying the DNS is so that the public DNS servers aren't used.
At this point I want to be able to use a local service, bound on 10.129.112.210:53. So that the following should yield a reply:
$ ping google.com ^C $ ping user.skydns.local ^C
When I run the same command from my host:
$ ping photo.skydns.localPING photo.skydns.local (10.129.112.206) 56(84) bytes of data. 64 bytes from 10.129.112.206: icmp_seq=1 ttl=64 time=0.790 ms ^C
$ cat /etc/resolv.conf nameserver 10.129.112.210 nameserver 127.0.0.1 nameserver 184.108.40.206 nameserver 220.127.116.11
The point here is not to access public hosts, but rather internal ones, using the local DNS service available on the host (via another docker instance).
To illustrate it even further (My ascii art design skills surpass my iptables fu, so that should say enough at this point):
______________________________________________ | __________________________ Host | | | Docker DNS container | | | ``````````````````````|``` | | | | | ,----------,---( private n. interface ) | | | | | | | | ( public n. interface )--- | | | | | | | ( loopbck n. interface ) | | | | | | | | | | | __|_______________________ | | | | Docker service container | | | | `````````````````````````` | | | | | | | | [ Local host service using DNS. ] | | | |______________________________________________| private (host) network interface: eth1 (10.129.0.0/16) Docker network interface: docker0 (172.17.0.0/16)
I've searched, read and applied different example Iptables configurations, but I know too little of the more "advanced" Iptables rules to understand whats going on and thus to get the desired result.
iptables -t nat -nL:
Chain PREROUTING (policy ACCEPT) target prot opt source destination DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 Chain DOCKER (2 references) target prot opt source destination