Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have my Apache logs set up like this:

LogFormat "%v %t %I %O" billing

How can I use AWK to generate a report which shows me the total bandwidth (received + sent) in MB per virtual host?

Here's an example log output:  [3 JULY 2013]  903 299  [8 JULY 2013]  192 138 [12 JULY 2013]  34 123  [19 JULY 2013] 616 213 [22 JULY 2013]  23  74

I'm looking for an output that sums up the 3rd and 4th columns for and without actually specifying the domains, as I have 50+ domains and wouldn't want to maintain a list. Much easier just to have the print out consolidated.

share|improve this question
up vote 1 down vote accepted

Or this:

awk '{T[$1]+=$NF+$(NF-1)} END{for(i in T) print i,T[i]}' file

would produce 2361 254

With your sample log file..

share|improve this answer
Awesome, thank you! So the file is logged is in Bytes, how can I convert this formula to present the data in MB? – James Jun 8 '13 at 14:48
I think I got it figure out: I added '/1024/1024' before the last '}' Thanks again! – James Jun 8 '13 at 14:55

You can write a small script to do this job:


domains=`awk '{print $1}' $log_file | sort | uniq`

for domain in $domains
    sum=$(grep "$domain" $log_file | \
            awk '{ for (i = 5; i <= NF; i++) s = s+$i }; END { print s+0 }')

    echo "Total bandwidth of $domain is $sum"

share|improve this answer
This processes a potentially large file for each virtual host (i.e. many times) and stores potentially large amounts of data in a variable. Also, it uses both the archaic backticks and the preferred $() form of command substitution. The echo in the for statement is unnecessary. So is the for statement in the awk line (it's just two fields - just add them). The s+0 isn't necessary. There's no need in this context to coerce an integer. A pipe acts as a line continuation character. No need for the backslash. – Dennis Williamson Jun 8 '13 at 11:27
The for in in awk is unnecessary? I don't think so.grep "$domain" $log_file ouput full fields, not two fields. The s+0 is for reason, when all bandwith in a domain is zero. The backslash is used to make my code more clearly. Accept all others advices. – cuonglm Jun 8 '13 at 12:06
The OP's example shows only two bandwidth fields. If the bandwidth fields are zero, then a zero will be printed. Only empty fields will need to be coerced. One thing I forgot to mention in my previous comment is that it's unnecessary to pipe grep into awk since AWK can do grep's job. – Dennis Williamson Jun 8 '13 at 15:01

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.