2

so I had a problem with DDoS, I think I solved it, but the solution is causing me problems.

After months of suffering and trying almost everything I finally solved the problem by blocking IPs one by one and denying them with the command:

deny ip;

I included all the denied IPs in a file, which I am including to my configuration. Now the file contains more then 1.9 million IPs, I cannot block IP ranges, just specific IPs.

Nginx is now taking too long to restart. If I split the included file into two files of 900 000 or four files of 500 000 will this make a difference?

Is there any way I can make Nginx check the file / files faster?

3
  • Just set up fail2ban and block only the IPs that are currently causing problems. I doubt that you need to block so many IPs all of the time. Feb 19 '19 at 10:57
  • Sadly, I do need to ban all of them, the attack is constantly changing IPs. I tried fail2ban several times, but did not do the job.. The load was still over the top. Feb 19 '19 at 14:55
  • Then you put them in an ipset and ban them from the host firewall. Feb 20 '19 at 16:13
2

With a file of that size I would not do it in nginx, but in iptables instead:

#!/bin/bash
# here's your list of IPS
CURRENT_BL=/path/to/my/ip_black_list.txt
# create/flush recreate the tables
iptables -F BLACKHOLE
iptables -N BLACKHOLE 
for BAD_IP in $(cat $CURRENT_BL)
do
        ipset add ipset-blacklist $BAD_IP 2>/dev/null || \
                echo "Failed to add ${BAD_IP}"
done
# REJECT the matching target
iptables -A BLACKHOLE -p all -m set --match-set ipset-blacklist src -j REJECT 
iptables -A BLACKHOLE -j RETURN
# assume your nginx is on 80 and 443
iptables -A INPUT -p tcp -m multiport --destination-ports 80,443 -j BLACKHOLE
iptables -A INPUT -p tcp -m multiport --destination-ports 80,443 -j ACCEPT

This allows you to append new IPs with:

ipset add ipset-blacklist X.X.X.X

Without reloading your nginx or the entire list. It's quite performant

Edit:

To get all the above working you'll need to install ipset and iptables

apt-get install iptables ipset

To make these rules permanent, you would use iptables-save and iptables-restore to save the currently-defined iptables rules to a file and (re)load them (e.g., upon reboot).

So, for instance, you would run

sudo iptables-save | sudo tee /etc/iptables.conf

to save your current iptables rules to /etc/iptables.conf and then insert these lines in /etc/rc.local:

# Load iptables rules from this file
iptables-restore < /etc/iptables.conf

don't forget to make sure /etc/rc.local is executable

Edit 2: this version splits the list into two ipsets, one for ipv4 and the other for ipv6, not checked the syntax of anything except the regex but should work.

Warning the regular expression is really really long, make sure you copy the entire line

#!/bin/bash
# here's your list of IPS
CURRENT_BL=/path/to/my/ip_black_list.txt
# create/flush recreate the tables
iptables -F BLACKHOLE
iptables -N BLACKHOLE 
# inverse regex matches everything except ipv6 addresses
for IPV4 in $(egrep -v "(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))" $CURRENT_BL)
do
        ipset add ipset-blacklist-ipv4 $IPV4 family inet 2>/dev/null > /dev/null
done
# regular expression matches only ipv6 addresses
for IPV6 in $(egrep "(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))" $CURRENT_BL)
do
        ipset add ipset-blacklist-ipv6 $IPV6 family inet6 2>/dev/null > /dev/null
done
# REJECT the matching target
iptables -A BLACKHOLE -p all -m set --match-set ipset-blacklist-ipv4 src -j REJECT 
ip6tables -A BLACKHOLE -p all -m set --match-set ipset-blacklist-ipv6 src -j REJECT 
iptables -A BLACKHOLE -j RETURN
# assume your nginx is on 80 and 443
iptables -A INPUT -p tcp -m multiport --destination-ports 80,443 -j BLACKHOLE
iptables -A INPUT -p tcp -m multiport --destination-ports 80,443 -j ACCEPT
14
  • Hmm, I was not aware of this approach. Where should I add the above configuration? And I am including all the blocked IPs automatically with awk command from the console. Can I do the same with this approach? Feb 19 '19 at 14:58
  • @Emmanuel-Ab Absolutely, two alternatives 1) have the AWK command populate the /path/to/my/ip_black_list.txt with a single IP address per list or 2) have the AWK command return a single IP per line to std out and replace the for loop line above with: awk [..your..program..] | while read BAD_IP. That done you'd run this script (it'll apply immediately) - and either call it on boot, or save and restore the resulting rules using iptables-save and iptables-restore
    – Nanzikambe
    Feb 19 '19 at 16:50
  • @Nazikambe One more question, I have to install iptables with the following command sudo apt-get install iptables-persistent and than edit this file sudo nano /etc/iptables/rules.v4 an apply the configuration that you gave me above? Thank you in advance! Feb 19 '19 at 21:52
  • And if I block the ips with iptabes, what will the blocked users see in the browser when they try to enter the website? Maybe a 403 error message? Feb 19 '19 at 21:59
  • @Emmanuel-Ab if you block an IP with iptables, its firewalled at a network level. Someone trying to access any service on your server will either get a connection that never completes (you used -j DROP) or a connection refused message (you used -j REJECT)
    – Nanzikambe
    Feb 20 '19 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.