Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I will do my best to try and explain this as it's strange and confusing to me. I posted a little while ago about a sustained spike in mysql queries on a VPS I had recently setup. It turned out to be a single post on a site I was developmenting. The post had over 30,000 spam comments! Since the site was one I was slowly building I hadn't configured the anti-spam comment software yet.

I've since deleted the particular post which has given the server a break but the post's url keeps on getting hit. The frustrating thing is every hit is from a different IP. How do I even start to block/prevent this? Is this even something I need to worry about?

Here are some more specific details about my setup, just to give some context:

  • Ubuntu 8.10 server with ufw setup
  • The site I'm building is in Drupal which now has Mollom setup for spam control. It wasn't configured before.
  • The requests happen inconsistently. Sometimes it's every couple seconds and other times it's a an or so between hits. However it's been going on pretty much constantly like that for over a week.

Here is a sample of my apache access log from the last 15 minutes just for the page in question:

dev.domain-name.com:80 97.87.97.169 - - [28/Mar/2010:06:47:40 +0000] "POST http://dev.domain-name.com/comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 202.149.24.193 - - [28/Mar/2010:06:50:37 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 193.106.92.77 - - [28/Mar/2010:06:50:39 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 194.85.136.187 - - [28/Mar/2010:06:52:03 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 220.255.7.13 - - [28/Mar/2010:06:52:14 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 195.70.55.151 - - [28/Mar/2010:06:53:41 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 71.91.4.31 - - [28/Mar/2010:06:56:07 +0000] "POST http://dev.domain-name.com/comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 98.209.203.170 - - [28/Mar/2010:06:56:10 +0000] "POST http://dev.domain-name.com/comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 24.255.137.159 - - [28/Mar/2010:06:56:19 +0000] "POST http://dev.domain-name.com/comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 77.242.20.18 - - [28/Mar/2010:07:00:15 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 94.75.215.42 - - [28/Mar/2010:07:01:34 +0000] "POST /comment/reply/3 HTTP/1.0" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 89.115.2.128 - - [28/Mar/2010:07:03:20 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 75.65.230.252 - - [28/Mar/2010:07:05:05 +0000] "POST http://dev.domain-name.com/comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 206.251.255.61 - - [28/Mar/2010:07:06:46 +0000] "POST /comment/reply/3 HTTP/1.0" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
dev.domain-name.com:80 213.194.120.14 - - [28/Mar/2010:07:07:22 +0000] "POST /comment/reply/3 HTTP/1.1" 404 5895 "http://dev.domain-name.com/blog/2009/11/23/another" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

I understand this is an open ended question, but any help or insight you could give would be much appreciated.

share|improve this question

3 Answers 3

up vote 1 down vote accepted

This is tricky one. In most cases, my first priority is keeping production available. Depending on how many different source IPs are involved, you may have to baby sit this or use a combination of scripts and manual solutions.

My first step would be to stop as many as I could to buy some time to get a handle on the situation.

Grok out the IP addresses..

grep 'blog/2009/11/23/another' log | awk '{print $2}' > iplist

Generate the iptables rules..

sed 's/^/iptables -A INPUT -s /g' iplist | sed 's/$/ -j DROP/g' > drop.sh

And drop them, using what you just made..

sh drop.sh

share|improve this answer
    
Warner, goodness...this is some serious help. Thank you. I've been managing linux servers for a couple years but only from a developer perspective and never from a sysadmin/security perspective. So with that in mind, I followed you up until the iptables command. Could you break that one down for me? –  mattmcmanus Mar 30 '10 at 13:46
    
Ok, after some research and staring at the command some more it's starting to make sense. I had just never used sed and have no experience with iptables. Hooray learning. –  mattmcmanus Mar 30 '10 at 14:16
    
The first command parses the log to create an IP list. The second command uses sed to prefix and suffix the IP list with the appropriate iptables syntax for dropping the traffic with netfilter. And finally, you're adding the netfilter rules by executing as a script. You're welcome, glad it helped. These type of issues can be tricky. –  Warner Mar 30 '10 at 14:26

This does appear to be an attempt to post spam. Requests appear to be coming from a botnet as the addresses don't resolve as they would for a server. Try the host command with various ip addresses. The first address appears to be a dhcp address from charter.com. Now that the address is out there it may be a long time before these go away.

If this is a development server, lock down access to port 80 to those address blocks which need access to port 80. I use Shorewall instead of ufw, as it provides me with an easy interfaces to lock down my servers, but allow access as desired.

You can lock down access in Apache web sever with the allow command. You can allow any combination of address blocks, addresses, or password protected access you want.

I would use both myself. As you have discovered, allowing open posting is an invitation to spammers. This is a problem that you do have to worry about. You appear to have addressed the problem. However, the status of the post is not shown. I would suggest you change your log format to include the post status. The combined log format will provide better information for log analysis.

share|improve this answer

Integrating something like reCAPTCHA on your comment form will stop your database from getting hit, as the bots will fail the captcha.

Using javascript to display the "Submit" button will also stop some bots from being able to submit the comment form.

share|improve this answer
    
I've been using a module called mollom on a number of drupal sites that I run and it's worked perfectly. The site I'm having a problems with is a dev site that I simply didn't configure that module yet. Configuring it was the first thing I did. So that isn't a problem any more. –  mattmcmanus Mar 30 '10 at 13:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.