I am after a web application that will parse apache log files and record how much bandwidth the user has used. We have several virtual hosts that have custom log files and the I/O is recorded at the end of the logfile. However I can't find an application that will parse multiple log files and display a summary for each site.
I believe awstats can do this but I want to be able to see all of my clients in one list. If there is something that integrates into cacti then that would be perfect.
I would suggest dumping all the information into one log, using a parser there, and then break out the logs to the respected virtual domains' log files after you've collected the data you need from them.
You will also have better high-end scalability using a single log file over 2 (common & error) per virtual.
If the victuals are on separate IP addresses is reasonably easy with something like MRTG.
Personally I could not figure out how to get Awstats to do this. I think Xerxes answer from a related serverfault question is the easiest way to do this:
cat /var/log/apache/access.log | awk '{SUM+=$10}END{print SUM/1024/1024}'
This assumes the %b bytes output at position 10 in your log file -- so make sure your CustomLog setting includes %b and adjust $10 as necessary.
The nice thing about this is that you can run it on individual log files, or grep for specific lines in a single log file. For instance, you could do
cat <log_file> | grep <virtual_domain> | awk . . .
to see the bandwidth for a single virtual domain in a combined log.
