Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We have an IIS site running Dotnetnuke that we took over from another group. We have added a robots.txt file to the root but it returns a 404. Actually any txt file in the root seems to return 404.

I can't seem to spot where they may have blocked this. Any suggestions?

EDIT: We have a duplciate of the site files and database setup on a clean IIS server and it works fine there, so it seems to be at the IIS level or somethign else outside of the web.config or similar.

EDIT2: Turns out it was a bug within a module that the developer had been using causing redirects for txt files to be messed up. Thanks everyone that replied.

share|improve this question
Which version of IIS? – Massimo Aug 31 '09 at 15:38
Verify that server root is actually configured to the folder you think it is. – Chris Nava Aug 31 '09 at 15:39
IIS 6. Root is where we expected. We cannot browse to a txt file regardless of where we place it it would appear. Tried ones uploaded via DNN and uploads file and downlaod link works but cannot browse any txt file directly via URL in browser. At a total loss as to why. – schooner2000 Aug 31 '09 at 15:42
Which 404 is returned? 404.2 or 404.3 -- or another? – Chris_K Aug 31 '09 at 16:44

Another (small) possibility: did you check that the file is really robots.txt, not robots.txt.txt - as in, did you change your Windows Explorer settings?

share|improve this answer
They are correct, any file.txt causes the issue. – schooner2000 Aug 31 '09 at 16:04
That's a good comment of atk's. It's gotten me a few times since I manage a lot of fresh installs of servers. By default, Windows hides the known extensions, so when creating a new file I often rename to something like test.txt, not realizing that it's really test.txt.txt. (actually for me, I usually do it with default.html/default.html.txt) Make sure in the folder's advanced settings that you uncheck 'Hide extensions for known file types', or simply rename to 'test' instead of 'test.txt' and see if that proves that it's from this situation. – Scott Forsyth - MVP Sep 2 '09 at 22:51

Check your mime-type configuration on the problematic IIS. Could be that IIS is serving .txt files up as something abnormal which is causing your browsers to not recognize them as general text files. Mime types are available on Web Site and Virtual Directories property page on the HTTP Headers tab.

share|improve this answer
Was going to suggest the same, you were faster :-) – Massimo Aug 31 '09 at 15:57
Appears to be correct text/plain – schooner2000 Aug 31 '09 at 16:06

You may also want to check that .txt files have not been associated with a specific application handler for whatever reason - In Properties under Home Directory click the Configuraton button. Check to make sure that .txt is not in the list of extensions. Also check to see if there are any Wildcard application maps (these would affect files of all extensions)

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.