Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have Apache set up to serve several Virtual Hosts, and I would like to see how much bandwidth each site uses. I can see how much the entire server uses, but I would like more detailed reports.

Most of the things I have found out there are for limiting bandwidth to virtual hosts, but I don't want to do that; I just want to see which sites are using how much bandwidth.

This isn't for billing purposes, just for information.

Is there an apache module I should use? Or is there some other way to do this?

share|improve this question
add comment

8 Answers

up vote 15 down vote accepted

The information you're after is all in the logs, so you should look at a log analyzer such as AWStats. The other option is to use Google Analytics.

For analyzing the logs, here's a rough example which you can use to tell you how many MB of traffic a log file reports from the command line:

cat /var/log/apache/access.log | awk '{SUM+=$10}END{print SUM/1024/1024}'
share|improve this answer
3  
useless use of cat: awk '...' /var/log/apache/access.log works too –  marco Jan 12 '11 at 17:43
    
Will the suggested way(s) calculate the traffic sent from some web page(s) to the Internet (outbound traffic)? –  Khaled Jan 13 '11 at 7:36
add comment

Awstats is one way to do this but probably not the best

share|improve this answer
add comment

I suggest you use the wonderful apache logging mechanism and its less known %I and %O flags:

Define the format:

LogFormat "%t %a %v %U%q %I %O" IOFormat

Use it in your main httpd.conf:

CustomLog /var/log/apache2/all-bw.log IOFormat

The values are probably not accounting all headers information, but are quite accurate to have a precise idea of VirtualHost traffic.

Scan the logs with a perl script to aggregate per virtual host every n minutes (5 for example) and send this to cacti.

These flags are provided by mod_logio which is probably built in your Apache (as it for my Debian's Apache).

share|improve this answer
1  
Hum, also as quoted in Apache 2.0 documentation: Note that in httpd 2.0, unlike 1.3, the %b and %B format strings do not represent the number of bytes sent to the client, but simply the size in bytes of the HTTP response (which will differ, for instance, if the connection is aborted, or if SSL is used). The %O format provided by mod_logio will log the actual number of bytes sent over the network. –  Oct May 29 '09 at 16:24
add comment

If you decide to use awstats with Apache, out of the box it will show you aggregated bandwidth for your entire server.

To see bandwidth on a per virtual host basis, I recommend installing vlogger.

Vlogger will actually gather Apache access log information for each of your virtual hosts that you set up to do so in separate directories/files.

For example if your Apache log file is in /var/log/apache2, typical vlogger installation will create something like this for your virtual hosts (e.g. vhost1.com vhost2.com):

/var/log/apache2/vhost1.com/access.log
/var/log/apache2/vhost2.com/access.log

Vlogger gives you the option to rotate these logs for you, provides a way to change the naming template of the access log file (e.g. add a date), and claims it handles a large number of log files better than Apache.

One down side to this is that you won't have an aggregated server view anymore (you'll need to aggregate logs separately or perhaps use an additional apache setting or perhaps some other method?).

I would caution against using google analytics (or any javascript based tracking) for server bandwidth monitoring as you are relying on the client to report via the javascript. GA does not report to you people who have their javascript disabled as well as any crawlers/spiders/bots.

share|improve this answer
add comment

Here is some regex to parse the log format proposed by Xerxes.

\[([0-9]+)\/(\w+)\/([0-9]{4})[^\]]+\]\s(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\s([^\/]+)[^\s]+\s(\d+)\s(\d+)

Example log:

[12/Jan/2011:14:25:04 +0000] 157.157.12.206 files.hjaltijakobsson.com / 581 669 [12/Jan/2011:14:25:04 +0000] 157.157.12.206 files.hjaltijakobsson.com / 624 747 [12/Jan/2011:14:25:04 +0000] 157.157.12.206 files.hjaltijakobsson.com /icons/blank.gif 687 186 [12/Jan/2011:14:25:04 +0000] 157.157.12.206 files.hjaltijakobsson.com /icons/compressed.gif 693 188 [12/Jan/2011:14:25:04 +0000] 157.157.12.206 files.hjaltijakobsson.com /favicon.ico 592 512

Matches:

Subpattern 1 (day of month): 12
Subpattern 2 (abbr. month): Jan
Subpattern 3 (year): 2011
Subpattern 4 (visitor host): 157.157.12.206
Subpattern 5 (virtual host): files.hjaltijakobsson.com
Subpattern 6 (incoming bytes): 581
Subpattern 7 (outgoing bytes): 669

Cheers.

share|improve this answer
add comment

Hmm, you could get evil with IPTables and string matching to log the packets for later reporting. Will only work for non SSL connections though.

Or something protocol and session aware like Snort could be shoe horned into use ...

share|improve this answer
add comment

Correct. Filtering the log is a good idea. I also want to get the bandwidth of my Apache server when download files.

LogFormat "%h %l %t %u \"%r\" %>s %b/%D**" combined

Calculates the %b and %d output, which will give you the bandwidth of current.

share|improve this answer
add comment

Slight tweak on the accepted answer assuming there is actually multiple vhosts on the server (and therefore multiple site.com.access_log 's). This will sort and list each vhost

for f in *.access_log ; do echo `awk '{SUM+=$10}END{print SUM/1024/1024}' $f` $f ; done | sort -rn | more

and for a directory of gzipped logs

for f in *.access_log.*.gz ; do echo `zcat $f|awk '{SUM+=$10}END{print SUM/1024/1024}'` $f `stat -c%z $f`; done | sort -rn | more 
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.