Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

About 24 hours ago a DDOS attack started against my site. The apache logs look like this:

190.56.92.50 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3a) Gecko/20030105 Phoenix/0.5"
79.162.132.75 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 SeaMonkey/1.0.4"
178.95.250.125 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Microsoft Internet Explorer/4.0b1 (Windows 95)"
41.224.117.207 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.1 (compatible; MSIE 5.0; Symbian OS; Nokia 6600;452) Opera 6.20 [ru]"
95.133.12.150 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1)"
78.139.156.77 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.60 (Windows NT 5.1; U; en) Presto/2.1.1"
91.86.86.111 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 5.0; SunOS 5.9 sun4u; X11)"
46.73.152.227 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
178.212.247.179 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; ; Linux armv5tejl; U) Opera 8.02 [en_US] Maemo browser 0.4.31 N770/SU-18"
189.243.119.252 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Symbian OS; Nokia 6600/5.27.0; 6329) Opera 8.00 [ru]"
187.159.55.110 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/2.0 (compatible; MSIE 3.01; Windows 98)"
79.162.132.75 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.0 (Windows NT 5.1; U; en)"
46.118.8.181 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)"
213.110.96.153 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Symbian OS; Nokia 6630/4.03.38; 6937) Opera 8.50 [es]"
46.201.147.160 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50"
201.230.239.84 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1)"
201.230.239.84 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Opera/9.01 (X11; Linux i686; U; en)"
31.223.223.66 - - [10/Nov/2011:19:09:16 +0200] "GET /browse.php HTTP/1.0" 403 1207 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5"

As you can see, all the requests are going to the same url: browse.php, but, of course, from differents IPs and User-Agents. I have denied access to browse.php with .htaccess, so the attack doesn't affect other pages/sites by overloading the webserver.

Now I have the apache access log for the last 24 hours, with a size of 1.4 GB.

How can I, based on the logs, get a list of IPs sorted by the number of times they accessed browse.php and got a 403 Forbidden error?

share|improve this question
add comment

2 Answers

up vote 3 down vote accepted

Try: cat logfile | grep -P 'browse\.php\S* HTTP\/1\.\d"\s403' | cut -f1 -d" " | sort | uniq -c | sort -n

share|improve this answer
    
If you're saving it to a file, then try sort -nr at the end instead, to put the most frequently appearing ones first. –  Michael Hampton Apr 14 '13 at 19:20
add comment

try to install fail2ban and configure it for apache. Fail2ban actually blocks multiple parallel requests of bots and it can easily be configured to block special regex based requests as well.

Once i tried to manually block them, but its a Big pain, because they are millions out there, and you will get tired manually blocking them through self made scripting.

share|improve this answer
    
I think this does not answer the question presented in bold. –  mailq Nov 10 '11 at 18:07
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.