I have this happening on a few servers with the same firewall rules, so I suspect I am missing something in my iptables configuration but not sure what is wrong. This is happening on some CentOS servers and well as my Ubuntu servers. I've used iptables for years and thought I knew what I was doing... apparently not the case.
I have SSH running on a non-standard port (2022). I have firewall rules to allow access for my personal IPs, followed by blocking specific ports including 2022, then followed by a deny all rule. In the last 3 weeks or so, my logs are showing failed login attempts on SSH from outside IPs that are not in my accept list. I have a VPN service on my laptop so I can try logging in from various countries, IPs, etc and the firewall is not blocking me anymore like it used to. I used my cellphone as a hotspot so I could ensure I had random IP that should be blocked, yet I could still log in to ssh. I used nmap from random IPs and it shows port 2022 as OPEN.
I'm not sure what could be going on, the firewall used to block SSH on not-allowed IP addresses properly and I don't think I've made any changes before this started and I don't have anything like fail2ban complicating things. I've even checked for rootkits as well and nothing shows up. I've Googled a round a lot, but the search results are so noisy with not-quite relevant responses that I've given up and decided to post the question here and hopefully get some better guidance.
This server is running ubuntu 14.04.6 LTS
My allowed IP's are on the 209.xxx and 216.xxx ranges Here are my iptables rules (iptables -L -n):
ACCEPT all -- 127.0.0.1 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.1 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.2 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.3 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.4 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.5 0.0.0.0/0 ACCEPT all -- 209.xxx.xxx.6 0.0.0.0/0 ACCEPT all -- 216.xxx.xxx.1 0.0.0.0/0 ACCEPT all -- 216.xxx.xxx.2 0.0.0.0/0 ACCEPT all -- 74.xxx.xxx.2 0.0.0.0/0 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x3F/0x00 RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x17/0x02 limit: avg 1/sec burst 2 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp flags:0x3F/0x3F ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:110 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:143 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:465 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:993 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:995 DROP tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:2022 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination --------------------------------
Typical Server logs are showing failures like:
Jul 26 05:29:38 SERVERNAME sshd: Invalid user postgres from 126.96.36.199 Jul 26 05:29:39 SERVERNAME sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.8.131.52 Jul 26 05:29:40 SERVERNAME sshd: Failed password for mysql from 184.108.40.206 port 56352 ssh2 Jul 26 05:29:40 SERVERNAME sshd: Received disconnect from 220.127.116.11: 11: Normal Shutdown, Thank you for playing [preauth] Jul 26 05:29:40 SERVERNAME sshd: reverse mapping checking getaddrinfo for usa1.getlark.com [18.104.22.168] failed - POSSIBLE BREAK-IN ATTEMPT!
Here is the script I created to implement the rules with comments:
APPEND="sudo /sbin/iptables -A INPUT" INSERT="sudo /sbin/iptables -I INPUT" OUTPUT="sudo /sbin/iptables -A OUTPUT" # drop old rules and start from scratch sudo /sbin/iptables -F sudo /sbin/iptables -X # allow local host $INSERT -s 127.0.0.1 -j ACCEPT # Allow full access to our approved IPs first: $APPEND -s 209.xxx.xxx.1 -j ACCEPT $APPEND -s 209.xxx.xxx.2 -j ACCEPT $APPEND -s 209.xxx.xxx.3 -j ACCEPT $APPEND -s 209.xxx.xxx.4 -j ACCEPT $APPEND -s 209.xxx.xxx.5 -j ACCEPT $APPEND -s 209.xxx.xxx.6 -j ACCEPT $APPEND -s 216.xxx.xxx.1 -j ACCEPT $APPEND -s 216.xxx.xxx.2 -j ACCEPT $APPEND -s 74.xxx.xxx.2 -j ACCEPT # drop Null packets $APPEND -p tcp --tcp-flags ALL NONE -j DROP # block syn flood attack $APPEND -p tcp --syn -m limit --limit 1/s --limit-burst 2 -j RETURN # block recon/Xmas Packets $APPEND -p tcp --tcp-flags ALL ALL -j DROP # don’t lock me out if I screwed up: $APPEND -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT # Allow/Block our legit services $APPEND -p tcp --dport 80 -j ACCEPT $APPEND -p tcp --dport 110 -j DROP $APPEND -p tcp --dport 143 -j DROP $APPEND -p tcp --dport 443 -j ACCEPT $APPEND -p tcp --dport 465 -j DROP $APPEND -p tcp --dport 993 -j DROP $APPEND -p tcp --dport 995 -j DROP $APPEND -p tcp --dport 2022 -j DROP # LAst Rule - Block everything else $APPEND -j REJECT --reject-with icmp-host-prohibited