I would start with mod_log_config. Define one or more LogFormat
/CustomLog
setups in httpd.conf
with just the stats you're interested in and any metadata about the requests you care to filter by and then you can quickly generate comparative stats from those log files. For example:
LogFormat "%t %v %B %D %h %r" statlog
CustomLog "|/usr/bin/cronolog logs/stat.log.%Y-%m-%d" statlog
%t
is the timestamp, %v
is the virtual server hostname, %B
is bytes sent (excluding headers), %D
is microseconds elapsed, %h
is client IP address, and %r
is the first line of the actual HTTP request sent by the client. So you can leave in or out any of that other information, depending on what you're looking for, or have one log for each stat, or whatever you want. (I like using cronolog to rotate logs on a daily basis. Tack on -%H
if you want hourly rotation.)
Then you can run the appropriate columns through whatever number-crunching you want to get totals, averages, identify particularly slow or abusive pages, or what-have-you.
Additionally, if you have mod_logio enabled, you can get exact byte counts (after encryption/compression/everything) for incoming and outgoing bandwidth for each request using %I
and %O
respectively in your LogFormat
string.