1

We're looking to generate a list of files that haven't been hit on our webserver for the last 3 months, 6 months and year.

IIS can give us a list of files that have been hit, but i'm wondering if there is a tool out there that will generate a list of files in our webserver directory that don't appear in the log files.

Improve this question

4 Answers 4

Reset to default
3

I'm afraid there is no simple solution. I guess you'll have to compare your logs against your list of (static I suppose) files. The best way I can think of is to use the LogParser tool.

You could import your logs into a SQL database and create a table containing all URLs (select distinct ...). Then you'll have to create the list of file names in your directories using a script and importing the resulting file to another database table. Step 3 would be to select all file name from Table2 which have no counterpart in the first table.

Improve this answer
8
  • +1 - I'd go with LogParser, too. (I'm +1-ing you? I'm just making my task that much harder, eh? >smile<)
    – Evan Anderson
    Jul 8, 2009 at 15:55
  • Thank you Evan! I'm so sure that in a few weeks you'll be at the top of the user list. And you obviously deserve it.
    – splattne
    Jul 8, 2009 at 16:07
  • >smile< I expect you to give me a run for my money (you and everybody else between me and the top right now). I'd just like a screenshot of my name at the top of that list, and then I think I can ease up on the pace a little bit. This site is such fun, though...
    – Evan Anderson
    Jul 8, 2009 at 16:55
  • If you do a manual comparison don't forget to account for default filenames, otherwise you'll still have to do some manual crosschecking after the report finishes. (+1'ing because that's what I do, and also to make the race tighter :) Both you guys rock.
    – squillman
    Jul 8, 2009 at 17:12
  • Evan, yeah - that's a good attitude! If you want, I could send you that screenshot right now - give me just the time to start Photoshop. ;-)
    – splattne
    Jul 8, 2009 at 17:13
0

Splattne has the answer I was going to post. I would just add to it that the OTHER way to go about things is to do the following:

  1. Rename any file you suspect of not being in use with an "X_" prefix. For example rename suspect.htm to "x_suspect.htm"

  2. Watch your 404 logs LIKE A HAWK. We used grep to parse out all of the 404s and look for "dead" files that weren't really dead. Put them back (or fix the link) if necessary.

This is not my FAVORITE method for doing things (ie. showing a customer a 404) but it does help confirm whether the file is in use or not.

Hope this helps.

Improve this answer
0

Xenu's Link Sleuth will do an orphaned file check if you give it FTP access to your web site.

Improve this answer
0

This would be a difficult task to do generically so I think you are unlikely to find an off-the-shelf solution. While you can get a list of files accessed from the logs and compare that against the list of files in the relevant directories, you then have to account for any files that are accessed by:

  • server.transfer
  • server.execute
  • #include
  • server-side <script> tags
  • probably a number of other things - the above list are just the obvious ones that spring to mind for "classic" ASP.

and so forth. For some apps there can be many file that are used by the scripts but not seen in the web server logs because of the way that they are referenced.

Improve this answer

You must log in to answer this question.