0

How can the docker-host reach an exported port without hitting the iptables DNAT-rule?

Bonus: Why are the iptables log messages only shown with curl 127.0.0.1:8082 but not with curl localhost:8082?

UPDATE: Similar to https://stackoverflow.com/questions/29629620, connecting to the container works with empty iptables rules too (# systemctl stop iptables clears all tables, sets policies to accept).

Setup

docker-compose.yml

version: '3.2'

services:
    webapp:
        image: httpd:latest
        networks:
            - webapp
        ports:
            - "8082:80"


networks:
    webapp:
        driver: bridge
#port mapping does not work for internal networks
#        internal: true

Observed Behaviour

  1. Packet starts in OUTPUT chain
  2. Packet leaves (lo), arrives again (lo)
  3. Accepted by conntrack target in *filterINPUT chain

Expected Behaviour

$ curl 127.0.0.1:8082

Although the nat-table [is traversed only once per connection][1], the counters for the DNAT should not be zero; Traversal for first packet:

  1. Packet starts in OUTPUT chain
  2. Packet leaves unchanged (-> lo), arrives again (lo)
  3. Packet still unchanged: *natPREROUTING -jump-> *natDOCKER
  4. DNAT --to-destination 172.18.0.2:80
  5. Routing decision --> *filterFORWARD (destination is foreign host)
  6. *filterFORWARD -jump-> *filterDOCKER ang get accepted, further packets already accepted by conntrack target

Counters

*nat (UPDATE: full output instead of DOCKER-chain)
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 44 packets, 4248 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 44 packets, 4248 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all  --  *      !br-cac172d73716  172.18.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.2           172.18.0.2           tcp dpt:80

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  br-cac172d73716 *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       tcp  --  !br-cac172d73716 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8082 to:172.18.0.2:80


*filter
Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-cac172d73716 br-cac172d73716  0.0.0.0/0            172.18.0.2           tcp dpt:80

with: docker network ls

NETWORK ID          NAME                DRIVER              SCOPE
cac172d73716        docker_webapp       bridge              local
3
  • 1
    iptables -t nat -L -n -v please – Andreas Rogge Feb 17 '19 at 18:38
  • the difference between localhost and 127.0.0.1 nowadays is that localhost also means ::1 (i.e. IPv6) and that is preferred. – Andreas Rogge Feb 17 '19 at 18:39
  • updated the counters section; according to localhost: shouldn't curl try 127.0.0.1 anyway as the container is only connected via ip4? – michi099 Feb 17 '19 at 20:28
0

TLDR: Due to compatibility reasons docker runs a TCP forwarding proxy by default, --userland-proxy=false switches port exposure entirely to iptables


Docker also starts a proxy-server by default. This service is a simple TCP forwarding proxy, accepting connections on the *filterINPUT-chain and forwards it into the container.

According to this article, these are the main reasons for this proxy to exist:

  • To *filterFORWARD rules, the device must be configured for routing, eg. net.ipv4.ip_forward must be set to 1. This is not enabled by default.

  • Linux doesn't allow routing on the loopback interface (lo) by default, this can be enabled by setting net.ipv4.route_localnet to 1

  • The people at docker wanted to support RHEL 6.5 which runs a 2.6 kernel. As discussed here, there exist cases where a packet has to be returned to the bridge it came from. This is only possible with bridge hairpinning which was not available on this system.

ss -lnp | grep 8082 shows:

Netid  State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port
tcp    LISTEN  0       128     *:8082              *:*                users:(("docker-proxy",pid=3066,fd=4))

If the kernel supports bridge hairpinning the switch --userland-proxy=false can be used. The docker manual states:

The --userland-proxy parameter, true by default, provides a userland implementation for inter-container and outside-to-container communication. When disabled, Docker uses both an additional MASQUERADE iptable rule and the net.ipv4.route_localnet kernel parameter which allow the host machine to connect to a local container exposed port through the commonly used loopback address: this alternative is preferred for performance reasons.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.