Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a problem with one of my servers. Google opens lots of http connections to the apache server and basically performs a slowloris attack.

This netstat call results in the following output

netstat -plant|grep :80|awk '{print $5}'|cut -d: -f1|sort|uniq -c|sort -n

 11 209.85.227.93
 14 209.85.227.190
 30 209.85.229.118
494 209.85.229.141

All of these ip's belong to the google network. I've already forbidden Google to access any of the content that is saved on the server by using robots.txt but it has no effect on the insane amount of connections. What should I do?

share|improve this question
2  
I'd stick the IP addresses into an iptables rule to block them - not putting this as an answer because my iptables skills aren't up to scratch - but an iptables drop will stop them using resources at your end –  EightBitTony Jul 3 '11 at 19:39
1  
Great Idea, Thank you! BTW the command is: /sbin/iptables -I INPUT -s 209.85.229.141 -j DROP –  Philip Jul 3 '11 at 19:46
    
google has how to on this link. –  Sharad Chhetri Aug 25 '13 at 20:23
add comment

1 Answer

Go to google webmaster tools, put in throttling.

Several other spiders respect the Crawl-delay directive in robots.txt, but Googlebot doesn't.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.