2

I have a game server running inside a Docker container that listens on UDP port 1111 (EXPOSE 1111/udp). When I publish this server on the same port on the host (-p 1111:1111/udp) everything works as expected.

Symptoms

When I publish it on a different host port (-p 2222:1111/udp) it starts behaving erratically:

  • The server responds to queries as expected (this is done over the exact same port used to join the server);
  • The server does not respond when trying to join it. None of the packets actually reach the server program (they do appear to be routed to the container correctly, see traffic logs below). This is confirmed by the fact that there are no connection attempts logged on the server.

NAT

Here are the POSTROUTING and DOCKER chains from the iptables NAT table as set by Docker:

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  172.17.0.0/16        0.0.0.0/0           
POSTROUTING_direct  all  --  0.0.0.0/0            0.0.0.0/0           
POSTROUTING_ZONES_SOURCE  all  --  0.0.0.0/0            0.0.0.0/0           
POSTROUTING_ZONES  all  --  0.0.0.0/0            0.0.0.0/0           
MASQUERADE  udp  --  172.17.0.2           172.17.0.2           udp dpt:1111

Chain DOCKER (2 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           
DNAT       udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:2222 to:172.17.0.2:1111

Traffic Analysis

Analyzing traffic flow by adding a LOG rule to the FORWARD chain in iptables shows expected results when querying the server (request comes in, response goes out):

Mar 29 00:24:08 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=39 TOS=0x08 PREC=0x40 TTL=116 ID=24686 PROTO=UDP SPT=56333 DPT=1111 LEN=19
Mar 29 00:24:08 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=43 TOS=0x08 PREC=0x40 TTL=116 ID=24687 PROTO=UDP SPT=56333 DPT=1111 LEN=23
Mar 29 00:24:08 hostname kernel: udp connection: IN=docker0 OUT=ens3 PHYSIN=veth3d1a8aa MAC=removed SRC=172.17.0.2 DST=my_ip LEN=111 TOS=0x00 PREC=0x00 TTL=63 ID=9115 DF PROTO=UDP SPT=1111 DPT=56333 LEN=91
Mar 29 00:24:08 hostname kernel: udp connection: IN=docker0 OUT=ens3 PHYSIN=veth3d1a8aa MAC=removed SRC=172.17.0.2 DST=my_ip LEN=43 TOS=0x00 PREC=0x00 TTL=63 ID=9116 DF PROTO=UDP SPT=1111 DPT=56333 LEN=23
Mar 29 00:24:08 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=39 TOS=0x08 PREC=0x40 TTL=116 ID=24688 PROTO=UDP SPT=56333 DPT=1111 LEN=19
Mar 29 00:24:08 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=39 TOS=0x08 PREC=0x40 TTL=116 ID=24689 PROTO=UDP SPT=56333 DPT=1111 LEN=19
Mar 29 00:24:08 hostname kernel: udp connection: IN=docker0 OUT=ens3 PHYSIN=veth3d1a8aa MAC=removed SRC=172.17.0.2 DST=my_ip LEN=41 TOS=0x00 PREC=0x00 TTL=63 ID=9119 DF PROTO=UDP SPT=1111 DPT=56333 LEN=21
Mar 29 00:24:08 hostname kernel: udp connection: IN=docker0 OUT=ens3 PHYSIN=veth3d1a8aa MAC=removed SRC=172.17.0.2 DST=my_ip LEN=145 TOS=0x00 PREC=0x00 TTL=63 ID=9120 DF PROTO=UDP SPT=1111 DPT=56333 LEN=125
Mar 29 00:24:08 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=43 TOS=0x08 PREC=0x40 TTL=116 ID=24690 PROTO=UDP SPT=56333 DPT=1111 LEN=23
Mar 29 00:24:08 hostname kernel: udp connection: IN=docker0 OUT=ens3 PHYSIN=veth3d1a8aa MAC=removed SRC=172.17.0.2 DST=my_ip LEN=43 TOS=0x00 PREC=0x00 TTL=63 ID=9121 DF PROTO=UDP SPT=1111 DPT=56333 LEN=23

This is what happens when I actually try to join the server, the incoming packets appear to be routed correctly but there's no response whatsoever:

Mar 29 00:24:26 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24694 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:27 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24696 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:28 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24698 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:29 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24700 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:30 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24702 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:31 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24704 PROTO=UDP SPT=63682 DPT=1111 LEN=12
Mar 29 00:24:32 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24706 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:33 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24708 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:34 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24710 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:35 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24712 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:36 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24714 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:37 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24716 PROTO=UDP SPT=63685 DPT=1111 LEN=12
Mar 29 00:24:38 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24718 PROTO=UDP SPT=63688 DPT=1111 LEN=12
Mar 29 00:24:39 hostname kernel: udp connection: IN=ens3 OUT=docker0 MAC=removed2 SRC=my_ip DST=172.17.0.2 LEN=32 TOS=0x08 PREC=0x40 TTL=116 ID=24720 PROTO=UDP SPT=63688 DPT=1111 LEN=12

Environment

The machine is running Fedora Server 29. iptables is controlled by firewalld.

Docker (just updated from 1.13):

Client:
 Version:           18.09.4
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        d14af54
 Built:             Wed Mar 27 18:36:04 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.4
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       d14af54
  Built:            Wed Mar 27 18:04:46 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Given the above traffic logs it seems that the packets do reach the container, but not the actual server program when I try to join it. As far as I know there's nothing inside the container that can block packets. I am completely clueless at this point.

  • What type of issue could I possibly be looking at here?
  • How should I proceed to troubleshoot the issue?

Maybe a bit far-fetched, but could packet size have anything to do with it? It's the only substantial difference I can see between the two logs, and well, I have kind of run out of ideas.

migrated from unix.stackexchange.com Mar 28 at 18:37

This question came from our site for users of Linux, FreeBSD and other Un*x-like operating systems.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.