I have these rules in UFW (default deny):
80 ALLOW 10.0.0.0/8 443 ALLOW 10.0.0.0/8 80 DENY Anywhere 443 DENY Anywhere
I am connecting from a machine on 10.0.0.0/8 to Apache listening on port 80. We noticed nginx running on another machine (a load balancer in front of this apache machine) was occasionally throwing "connect timeout" errors. I am able to recreate the timeout by simply running a for loop in a bash script that hits port 80 from that nginx machine. I get perhaps 3 or 4 timeouts in a batch of 1000 tests.
In /var/log/messages I see these when the timeouts happen:
Dec 1 01:01:01 webserver.mydomain.com kernel: [UFW BLOCK] IN=eth0 OUT= MAC=00:11:22:33:44:55:66:77:88:99:aa:bb:cc:dd SRC=10.0.0.5 DST=10.0.0.2 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=16923 DF PROTO=TCP SPT=60064 DPT=80 WINDOW=5792 RES=0x00 SYN URGP=0
Why would UFW be blocking these? If I disable UFW these connection timeouts disappear completely and everything works as expected. Other ports (like 22) don't have this problem, I can run scripts that hammer those endlessly without any issues even with ufw activated.