0

I have two dedicated servers: "web" (YYY.YYY.YYY.YYY) and "monitor" (XXX.XXX.XXX.XXX). Both are in the same network of a mass hoster (hetzner).

Now on "web" I have 3 prometheus metric endpoints running : docker-engine (9323) on the bare metal host and neo4j (2004), telegraf (9273) as docker containers. Both docker containers map their output ports to the host correctly, so the following calls executed on "web" work:

lynx http://YYY.YYY.YYY.YYY:9323/metrics => OK
lynx http://YYY.YYY.YYY.YYY:9273/metrics => OK
lynx http://YYY.YYY.YYY.YYY:2004/metrics => OK

But calling those endpoints from "monitor" server works only for the bear metal service docker-engine (9323)

lynx http://YYY.YYY.YYY.YYY:9323/metrics => OK
lynx http://YYY.YYY.YYY.YYY:9273/metrics => timeout
lynx http://YYY.YYY.YYY.YYY:2004/metrics => timeout

UFW status verbose delivers the following

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
[...]
9323/tcp                   ALLOW IN    XXX.XXX.XXX.XXX
9273/tcp                   ALLOW IN    XXX.XXX.XXX.XXX
2004/tcp                   ALLOW IN    XXX.XXX.XXX.XXX
[...]

There are no other rules with those IPs and no general rules which apply to subnets, interfaces etc. All other rules are for discreet ports, likie 22, 80, 443 etc.

The strange thing is, that it worked just few hours before. In the meantime I was experimenting a little with this here https://medium.com/@pitapun_44686/what-is-the-best-practice-of-docker-ufw-under-ubuntu-69e11c826b31 and appended the following block to the very end of /etc/ufw/after.rules

*filter
:ufw-user-forward - [0:0]
:DOCKER-USER - [0:0]
-A DOCKER-USER -j RETURN -s 10.0.0.0/8
-A DOCKER-USER -j RETURN -s 172.16.0.0/12
-A DOCKER-USER -j RETURN -s 192.168.0.0/16
-A DOCKER-USER -j ufw-user-forward
-A DOCKER-USER -j DROP -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 192.168.0.0/16
-A DOCKER-USER -j DROP -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 10.0.0.0/8
-A DOCKER-USER -j DROP -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 172.16.0.0/12
-A DOCKER-USER -j DROP -p udp -m udp --dport 0:32767 -d 192.168.0.0/16
-A DOCKER-USER -j DROP -p udp -m udp --dport 0:32767 -d 10.0.0.0/8
-A DOCKER-USER -j DROP -p udp -m udp --dport 0:32767 -d 172.16.0.0/12
-A DOCKER-USER -j RETURN
COMMIT

Now I commented it out and restarted ufw. Those ports 9273 and 2004 are stil not accessible, so this was not the reason.

I configured ufw log level to high, but I can not see any connection attepmtps or dropped packets from host XXX.XXX.XXX.XXX.

Trying to telnet into the one working port (telnet YYY.YYY.YYY.YYY 9323) I can see the communication in the ufw logs, but not for tho other two ports.

[UFW AUDIT] SRC=XXX.XXX.XXX.XXX DST=YYY.YYY.YYY.YYY DPT=9323 =>
[UFW AUDIT] SRC=YYY.YYY.YYY.YYY DST=XXX.XXX.XXX.XXX SPT=9323

I provisioned ufw using ansible "ufw" module.

What other reasons could there be? What is going on? :-)

Could it be the hosters network imposes some kind of filter between those servers because of "suspisious" activity (frequent communication)? It also happens, that I run some excessive artillery.io tests from XXX.XXX.XXX.XXX to port 80/443 on YYY.YYY.YYY.YYY today, but this hypothesis does not explain why only those two ports are not working anymore.

And the ultimate test, shutdown of ufw on YYY.YYY.YYY.YYY, does not hel either. Ports 9273 and 2004 are not accessible, 9323 is.

Here is the output from iptables -L -v -n https://pastebin.com/HVeJGXb9

2
  • show iptables -L -v -n
    – AlexD
    Jan 6, 2022 at 17:56
  • @AlexD I added a pastebin to the end of my question Jan 6, 2022 at 18:03

1 Answer 1

1

As the ports 9273 and 2004 are exposed by Docker, the packets to these ports are going through FORWARD chain and then through DOCKER-USER chain. The rules you added to DOCKER-USER are blocking most of the external traffic to docker containers networks. You need to allow forwarded traffic with ufw route allow or you can add rules directly to DOCKER-USER chain.

1
  • Ok. Thank you. In other words, this snippet I added inserted the rules permanently and they are active even after removal from /etc/ufw/after.rules. Now I know at least there is no "magic" involved ;-) and this snipped does what it should. Simplest in my case will be to expose those ports by my frontend proxy. Jan 7, 2022 at 8:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .