Measuring accesses to files - apache

Question

So, I run a website, that among other things serves some files (usually PDFs).

All of these are stored under a specific directory on the server: /var/www/vhosts/mysite.com/httpdocs/site/pdf_files

Due to storage issues on my VPS I am thinking of getting some S3 or other cloud storage, and mount it as a drive using S3QL/S3FS. Then I will be able to have the pdf_files folder symlinked to the cloud folder and serve those files using that, without any changes on the web app (is that a good plan?)

Now, before doing that, to estimate costs, I need to measure how many file accesses people do, how many times those pdf files are downloaded each month for example. Basically how many times those pdf files are accessed through the webserver.

I'd like to do it on the apache level. What's the best way that this can be done?

e.g.: measuring the bandwidth used by files in that specific folder would also be nice, but estimating the GET requests I'll be doing to amazon is more important.

Answer 1

Assuming that you have your access log available, the quick and easy thing is to do:

grep pdf_filename access_log | grep 200

Basically, you parse the log file for the pdf file and then get HTTP 200 code message, which means that the user have successful download the file. Here is a typical log file line with a successful request:

127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)

(pay close attention to the numeric value between the 200 code and the URL - that's the size of the file that was downloaded).

Alternatively, you can download a log analyzer like Webalizer:

Webalizer

And configure it to track how many users download the pdfs as well as the transfer rate.