Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This is an odd question but my gut tells me there is an easy way to do this:

I have a project that is always in development and is in PHP and it is 14 years old. Despite every attempt to keep on top of developing it there are large numbers of files in there. The PHP bit is ok, I can do what I need via a database log in every header.

I am taking about the apache stuff - the css, the gifs, the png, the old jquery references, the old js files that I may or may not ever recruit. There are around 3,000+ of these files.

Many are image references to old images no longer ever used. Some are jQuery libs that I long since stopped using. The thing is they all look like something I remember doing way back when, and there are a lot of legacy decay routines that sometimes need these old images/css/js/{insert here} to function.

Basically this isn't a website it's an PHP engine that can throw up lots of things and it is hard to track, so I simply leave these old references in.

What I want is a way to traverse the Apache logs for installations that have been live for over a year and positively ascertain whether each individual that image or css or whatever has NEVER been referenced nor pulled up since the server was created.

Is there a way to item by item verify whether Apache ever used it? I have lots of servers that run this code it would be nice to run this against every server so it would (ideally) be a way of getting distinct file calls (and a count?) from Apache logs. Urls or unc paths would be fine.

share|improve this question

2 Answers 2

up vote 1 down vote accepted

If your filesystem isn't set to discard atimes (e.g. ext3/4 with noatime) you can just use a simple find to locate files that haven't been accessed in some time.

For instance, to find files that haven't been accessed in a year or more:

find /srv/www/ancientproject -atime +365 -print

This may not solve your problem, though, for many of the same reasons voretaq7 points out. The file might be requested 15 minutes after you delete it, for instance.

share|improve this answer
    
that's a very good idea, bear in mind I am intending to run this against multiple installations (so probably 10-20 server years of user activity) and only intend to rule out based on probability –  user26676 Jun 6 '13 at 8:38

What I want is a way to traverse the Apache logs for installations that have been live for over a year and positively ascertain whether each individual that image or css or whatever has NEVER been referenced nor pulled up since the server was created.

What I want is an infinite supply of money -- you can't always get what you want.

The closest you can come is to scan the Apache access log to see which files are being accessed by clients, but this is not a guarantee that the file is never used.

  • It could be included by PHP using require
  • It could be used by something that was never done on your server (an admin page)
  • It could be that it's used infrequently and your logs don't go back that far
    (My logs roll daily and I only keep a week on the servers - that's not uncommon if you don't need them for audit purposes).

If you want to do this right you're going to have to have your developers do a proper code audit. It's the only way to know for sure what is/isn't required.

Alternatively you can scan your available logs, remove anything that isn't referenced, and then add back items as you discover stuff is broken. If you have good version control this may be an acceptable solution.

share|improve this answer
    
I am asking a question and being specific, nothing wrong with being specific. to answer some of your points in no particular order: this server environment engine is an admin page, and specifically say I am talking only about files that would be pulled up by Apache. I specifically rule out the php part - for example I would not require() anything other than another PHP file. Also, this _is_ a code audit topic request. –  user26676 Jun 5 '13 at 22:47
    
I might look into this guy he has unique visits as a choice for if you pay him .. I will email him and chart my success on here –  user26676 Jun 5 '13 at 22:53
1  
I am giving you a specific answer: NO. You cannot do this reliably using log files for the reasons I gave above (which address the entire scope of your request, as well as additional cases). You need to do a code audit (that means looking at code, not logs) to ensure you're not removing something referenced deep in the bowels of your software. Sorry if that's not what you want to hear, but that's the way it is - Apache only logs what it's seen happen, it can't predict every possible branch in your application's execution path. –  voretaq7 Jun 5 '13 at 22:57
    
voretaq7 I know that apache can log access to files, so, with the greatest of respect even if I wrote my own software to scan the file log I could do this. You really are not right, not even remotely, since the information is all there, due to time and balance of probabilities, you can rule out 99.999999999% accurately dead references. If you insist on being rude and ignorant then go to another question where they might like that? The constant reference to a "Code Audit" as a revealed option is redundant since THIS IS BLATANTLY PART OF A code audit hence the question –  user26676 Jun 5 '13 at 23:14
2  
You are wrong. I know this from about 14 years of experience. And I'm done arguing. If you don't like my answer feel free to wait for another, or post your own if you find an alternative you like better. –  voretaq7 Jun 6 '13 at 0:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.