4

(Please move to serverfault if appropriate - it seems more of an application issue to me)

I've got a PHP server, hosting MP3 files for a podcast. I'd love to get a week by week breakdown of how many times the files were downloaded (is the long tail a reality?). AWStats doesn't seem to hit the spot.

Ideally, I'd get an Excel file with one file on each line, then a column for each week (or day) with the download figures aggregated in a single cell for each file/week. Is there any solution (either PHP-based, or a Windows app into which I load logs) which could generate this format of report?

0

3 Answers 3

2

You could use the apache logs to see when the files are accessed, and just process the log file. I know there are many tools to be able to automatically parse the logs, though I have never used any.

A google search return scratchy, with a sample log here

Another thought is to pass the request through a page that will register the request into a mysql database, from there, you could generated the desired logs

1
  • Can anyone recommend some tools which would parse the logs to produce the output specified? Nothing seems to generate a table of file x date download counts.
    – user34021
    Feb 6, 2010 at 14:33
1

@phsr's proposal to serve trough a page is the best one. But if you do not want to do that, a simple bash script should be able to get what you want out of the log files.

I'm not very good in bash, etc., but if the files are in one and the same directory, something like this (pseudocode) should work:

for each $file in $mp3dir
   echo $file + ',' + `cat apache.log | grep $file | wc -l` >>myreport.csv

Of course, the grep filter may include something even more specific, to isolate irrelevant lines which may exist in the log(s).

Also, apache.log is actually cumulative cat of a specified log files for a time period (given that you have proper logrotate to rotate the logs daily).

Another approach could be to use grep to get all ".mp3" lines from the log, and using sed to form a csv line out of each entry, so you have filename and date on one line. Then feed that into a database, and do queries from there.

If someone can put this in "real" bash, please do :)

0

You will want to parse the web server log files. The log format will depend on the web server and how logging is configured. There are modules for most languages that make parsing the logs relatively trivial. No reason to resort to writing something from scratch in bash or awk.

If you dont' want to write any code then I suggest checking out splunk. There is a free version that lets you parse up to 500MB of log data daily, that's a lot of log data so you should be fine with the free/community version.

Splunk can also export results for you.

You must log in to answer this question.