0

On a FreeBSD 12 server, I started to notice that pf is blocking out having Flags [FP.], the logs are flooded with something like this:

00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.59.122.48894: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 141.101.98.92.16036: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.89.53.45136: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.88.60.43016: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.89.101.58320: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.179.50.21756: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.91.39.18516: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.90.202.25684: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000004 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 172.69.226.63.52316: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000003 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 162.158.90.202.25662: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP
00:00:00.000005 rule 2/0(match): block out on bge0: 213.59.241.172.80 > 198.41.242.26.29508: Flags [FP.], seq 0:187, ack 1, win 65535, length 187: HTTP

Any idea of why this could be happening?

Rule 2 is

@2 block drop log all

Server running mainly HAproxy and these settings on /etc/sysctl.conf:

debug.debugger_on_panic=0
debug.trace_on_panic=1
kern.ipc.shmmax=2147483648
kern.ipc.somaxconn=32768
kern.panic_reboot_wait_time=0
net.inet.icmp.drop_redirect=1
net.inet.icmp.icmplim=10
net.inet.icmp.log_redirect=0
net.inet.icmp.maskrepl=0
net.inet.ip.accept_sourceroute=0
net.inet.ip.random_id=1
net.inet.ip.redirect=0
net.inet.ip.sourceroute=0
net.inet.tcp.blackhole=2
net.inet.tcp.drop_synfin=1
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.finwait2_timeout=1000
net.inet.tcp.msl=2500
net.inet.tcp.recvbuf_auto=1
net.inet.tcp.recvbuf_inc=16384
net.inet.tcp.recvbuf_max=134217728
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.sendbuf_max=134217728
net.inet.udp.blackhole=1
security.bsd.see_other_gids=0
security.bsd.see_other_uids=0
security.bsd.see_jail_proc=0
security.bsd.stack_guard_page=1
security.bsd.unprivileged_proc_debug=0
security.bsd.unprivileged_read_msgbuf=0
net.inet.tcp.mssdflt=1460
net.inet.tcp.minmss=536
net.inet.tcp.syncache.rexmtlimit=0
net.inet.ip.maxfragpackets=0
net.inet.ip.maxfragsperpacket=0
net.inet.tcp.abc_l_var=44
net.inet.tcp.initcwnd_segments=44
kern.ipc.maxsockbuf=614400000
net.inet.tcp.syncookies=1
net.inet.tcp.tso=1
kern.random.fortuna.minpoolsize=256
net.inet.tcp.isn_reseed_interval=123

These are the pf rules after adding a pass before block (rule 2 now is rule 7)

scrub in all no-df max-mss 1440 fragment reassemble
block drop in log on ! bge0 inet6 from 2001:4ba0:85a3:105::/64 to any
block drop in log on bge0 inet6 from fe80::eeeb:b8ff:fe87:9514 to any
block drop in log inet6 from <__automatic_2bacaf44_0:7> to any
block drop in log on ! bge0 inet from 213.59.241.128/25 to any
block drop in log inet from 213.59.241.172 to any
pass quick from <allow:5> to any flags S/SA keep state (if-bound)
block drop log all
block drop quick from <banned:0> to any
block drop quick from <bruteforce:0> to any
block drop in log quick from no-route to any
block drop in quick on bge0 proto tcp all flags FPU/FSRPAUEW
block drop in quick on bge0 proto tcp all flags FSRPAUEW/FSRPAUEW
block drop in quick on bge0 proto tcp all flags FSRAU/FSRPAUEW
block drop in quick on bge0 proto tcp all flags /FSRPAUEW
block drop in quick on bge0 proto tcp all flags SR/SR
block drop in quick on bge0 proto tcp all flags FS/FS
pass in quick on bge0 proto tcp from any to any port = http flags S/SA keep state (if-bound)
pass in quick on bge0 proto tcp from any to any port = https flags S/SA keep state (if-bound)
pass in quick on bge0 proto tcp from any to any port = 2222 flags S/SA keep state (if-bound)
pass in quick proto ipencap all keep state (if-bound)
pass inet proto icmp all icmp-type echoreq keep state (if-bound)
pass inet proto icmp all icmp-type unreach keep state (if-bound)
pass proto ipv6-icmp all keep state (if-bound)
pass out quick proto tcp all flags S/SA keep state (if-bound)
pass out all flags S/SA keep state (if-bound)

For this line:

pass quick from <allow:5> to any flags S/SA keep state (if-bound)

Allow is a table, something like:

group1 = "10.0.0.4"
group2 = "5.19.152.31 95.116.0.173:"
table <allow> { $group1, $group2 }

Something that I notice is that even I have the pass before the block:

pass quick from <allow:5> to any flags S/SA keep state (if-bound)
block drop log all

Some "allowed" groups are beeing blocked, for example:

00:00:00.589761 rule 7/0(match): block out on bge0: 213.59.241.172.4567 > 5.19.152.31.61847: Flags [RP.], seq 2:81, ack 0, win 0, length 79

The rule 7 is the block drop log all, In this case, 5.19.152.31 is in the allow table but is in being blocked

Therefore wondering how to full allow/skip certain IP's, network ranges before getting block.

  • 1) Do you have there only one interface ? 2) does the state table have enough free entries ? – drookie Mar 9 at 23:17
  • and I don’t like this MASSIVE sysctl tampering with default oid values around net.inet. Are you sure about what you’re doing here ? Because if you’re the author of these tweaks then you have to be experienced enough to deal with pf on your own; if you’re not the author of these then it’s a cargo cult. – drookie Mar 9 at 23:20
  • the server has 4 interfaces, currently only using one bge0, regarding the sysctl settings any hint, I have being tunning net.inet.tcp.msl but don't think could justify the block out rule – nbari Mar 10 at 6:20
  • Show ys traffic on other interfaces then, if they are up. – drookie Mar 10 at 6:30
  • other interfaces are down – nbari Mar 10 at 7:01
-1

I recommend starting with this configuration (yeah, I skipped mtu tweaks until your explanation why 1440 size is chosen), since yours is a total mess:

oif = "bge0"

# because you don't want to filter localhost
set skip on lo0

# because it's a default rule
block drop log all

# because you don't want to drop ICMP
pass proto icmp all

# or ICMP6
pass proto ipv6-icmp all

# okay then
block drop quick from <banned:0> to any
block drop quick from <bruteforce:0> to any

# because you need ssh (is the 2222 new port for ssh ? Jeezus, use blacklistd)
pass in on $oif proto tcp from any to any port 22

# because flags S/SA and keep state is the default
pass in on $oif proto tcp from any to any port { 80, 443, 2222 }

# because you need to configure both ends with ipinip, so why bother about states
pass in on $oif proto ipencap all

# because you want to allow all the locally-originated sessions
pass out all
  • What is the difference between using only block in log all vs block log all and need to use pass out all? – nbari Mar 10 at 9:45
  • Well it's obvious - block in log all blocks everything inbound, while block log all blocks outbound too. pass out all passes everything out locally-originated. Is there anyone from technical personnel around and who let you to the server console ? – drookie Mar 10 at 10:17
  • All the blocks I have are related to a proxy, HTTP request are accepted then the proxy tries to connect to the remote backends (this from my understanding may not be locally-originated) by using block in solve the problem but I would like to better understand if the why of this – nbari Mar 10 at 10:31
  • Short answer - no. No, you don’t need all of these rules. Since you don’t understand how pf is working, you need to start with something fresh and simple. Yet equivalent to what you have and not interfering with your services. You can use the configuration proposed or keep searching for answers, it’s up to you. – drookie Mar 10 at 10:34
  • I am indeed asking because with your configuration I still have the problem, that's why I post the sysctl settings since I thought that they may be affecting the pf rules – nbari Mar 10 at 10:37

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.