Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm hosting a site for a volunteer organization. I've moved the site to WordPress, but it wasn't always that way. I suspect at one point it was hacked badly.

My Apache error log file has grown to 122 kB in just the past 18 hours. The large majority of the errors logged are of this form -- it's repeated hundreds of times today alone in my log files:

[Mon Nov 12 18:29:27 2012] [error] [client xx.xxx.xx.xxx] File does not exist: /home/*******/public_html/*******.org/calendar.php
[Mon Nov 12 18:29:27 2012] [error] [client xx.xxx.xx.xxx] File does not exist: /home/*******/public_html/*******.org/404.shtml

(I verified that xx.xxx.xx.xxx was a Google server.)

I suspect there was a security hole somewhere before, likely in calendar.php, that was exploited.

The files don't exist anymore, but there may be many backlinks that exist that reference here, hence why googlebot is so interested in crawling them.

How do I fix this gracefully? I still would like Google to index the site. I just want to tell it somehow not to look for these files anymore.

share|improve this question
1  
You can edit the file robots.txt to exclude these files specifically -- Google seems to respect this file fairly. Else I'd just ignore it and let Google do its thing. Google Tools for Webmasters allows even more fine-grained configuration options. Any reason, save for the log file size, this is a concern for you? –  jscott Nov 13 '12 at 2:56
2  
While technically on topic here, this question may be a better fit at our sister site Pro Webmasters. In fact, I'm pretty sure it's been answered once or twice there, too. –  Michael Hampton Nov 13 '12 at 2:57
    
@jscott Sorry I didn't mention this but it's been doing this for a year or more. These errors dominate my error log and it looks like Google hasn't done its thing yet. Would like to give it a nudge. :) –  John Nov 13 '12 at 3:15

1 Answer 1

up vote 2 down vote accepted

This is one thing that the 410 Gone error can be used for.

Google and other search engines can use this information to determine that a URL is no longer valid and is expected to never be valid again, and thus remove it from their indexes.

share|improve this answer
    
So Redirect 410 \calendar.php for example in .htaccess? –  John Nov 13 '12 at 3:11
    
Seems reasonable enough, as long as you use a / instead, and consult the documentation. I'm an nginx guy. :) –  Michael Hampton Nov 13 '12 at 3:12
    
Shoot, you're right about the slash direction. Thanks! –  John Nov 13 '12 at 3:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.