1

so i'm trying to use .htaccess to ban a large list (50,000) of suspicious spam IP addresses that i got from this site.

the list is almost 1M in size when i add the deny from directive for each address in the list...but when i apply the loaded .htaccess file to the server it barfs and no pages load with a 403 error.

my questions...

  1. is there a limit to the size of a .htaccess file?
  2. it there a limit to the number of deny from ip addresses that can be contained in the file?
  3. is there a better way to do what im trying to do at the server level (I understand I could check the list in the webapp during submit but i'm trying to learn stuff)
  • Is it a VPS or dedicated server, i.e. do you have complete control over the configuration? If so, what is the OS? – dartonw Jun 16 '14 at 18:08
  • its a vps...ubuntu server 13.04 – menriquez Jun 16 '14 at 18:09
  • 1
    That's end of life...you should upgrade it as soon as possible. – Michael Hampton Jun 16 '14 at 19:11
1

There is no limit on the size of a .htaccess file other than the OS file size limits (usually 2GB or more). However, there are major performance implications involved in using .htaccess files because of the way that Apache processes them recursively (traversing the directory tree up) on every page load. In fact, Apache recommends against using .htaccess unless absolutely necessary, e.g. no access to higher-level config.

The standard way to handle IP blocking is with iptables, the built-in Linux firewall. You can use other applications to help manage iptables, such as Fail2ban. See this blog post for a way to do this with Fail2ban on a permanent basis. You can also block them by adding a route: route add -host 192.168.0.123 reject. Remember that you can use entire classes of IP addresses with all of these, so that rather than listing 192.168.0.1, 192.168.0.2,... 192.168.0.254 you can specify the network: 192.168.0.0/24.

Whatever method you use, remember to be very careful not to block yourself, especially from services such as SSH.

  • ahhh...right! much better solution. im marking this as answered...tyvm – menriquez Jun 16 '14 at 20:17
2

is there a better way to do what im trying to do at the server level

I don't know if its better, but I prefer this:

  1. create for every IP a new file, structured like deny/198/198.51.100.201
  2. use the following code in your .htaccess to block the IP:

    RewriteCond %{REMOTE_ADDR} ^([0-9]{1,3})\.
    RewriteCond /usr/www/{your_path}/deny/%1/%{REMOTE_ADDR} -f
    RewriteRule . - [F]
    

I never benchmarked it but I refuse to believe that it is faster to compare every request against 50000 IPs instead of taking benefit of the inode cache of a filesystem. Or builds Apache an index of the listed IPs?!

Finally it was not interesting for me to know that because I use this technique to offer my visitors a captcha to unblock themselfs if they were blocked by accident. Other methods (allow, deny, iptables, ...) did not offer such a functionality.

Note: I used a subfolder for the first ip block to avoid more than 32000 files per folder.

  • This solution is elegant and a gem! Such a simple way to block dynamically via Apache. Wonder why this has not received more votes. I also like your article where you provided a great way to self unlock. Am wondering what other ways are possible instead of reCaptcha in commercial use. – pipepiper Nov 11 '17 at 10:40
  • @pipepiper Thank you. The question itself is not really popular so it will never receive many votes. I'm using this since 2012 without problems (the captcha page includes a telephone number and nobody has ever called me regarding problems). Of course you can easily use other captcha mechanisms. e.g. let the user solve an equation like it is done at php.net. The same mechanism can be easily ported to other webservers as well. e.g. for nginx the -f flag could be used: serverfault.com/q/122315/44086 – mgutt Nov 12 '17 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.