I have a very strange issue: Google is giving me that some links in my sitemap are not accessible and are giving 404 errors. However, I have tried to open the pages in question and it works fine.
The website has been built using codeigniter and here is a sample of links
Now if you try to open the link above, it will work fine. but if you try to curl -I
:
curl -I http://widwebway.com/en/solutions HTTP/1.1 404 Not Found
I have made some routes and rewrite rules to exclude the controller name from the URL. In other words, the correct link is
curl -I http://widwebway.com/en/widwebway/view/solutions HTTP/1.1 301 Moved Permanently Date: Wed, 29 Feb 2012 18:05:01 GMT Server: Apache/2.2.14 (Ubuntu) Location: http://widwebway.com/en/solutions Vary: Accept-Encoding Content-Type: text/html; charset=iso-8859-1
Apache error log gives
File does not exist: /var/www/myweb/en, referer: http://widwebway.com/en/solutions
my .htaccess
is:
ErrorDocument 404 /index.php
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^ar/widwebway/view/([a-z_]{1,50})[/]*$ /ar/$1 [L,R=301,NC]
RewriteRule ^en/widwebway/view/([a-z_]{1,50})[/]*$ /en/$1 [L,R=301,NC]
RewriteRule ^en/widwebway[/]*$ /en/home [L,R=301,NC]
RewriteRule ^ar/widwebway[/]*$ /ar/home [L,R=301,NC]
</IfModule>
Any idea of how comes that the page is there but it gives 404 on curl or Google bot?