Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have nginx log file, and I want to find out market share for each major version of browsers. I am not interested in minor versions and operating systems. I would like to get something like this:

100 IE6
 99 IE7
 20 IE8
200 FF2
300 FF3

I know how to get the list of user agents from the file, but I want to aggregate the list to see only the major versions of the browsers. Is there a tool that does it?

share|improve this question

6 Answers 6

up vote 6 down vote accepted
awk -F'"' '/GET/ {print $6}' /var/log/nginx-access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn
  • awk(1) - selecting full User-Agent string of GET requests
  • cut(1) - using first word from it
  • sort(1) - sorting
  • uniq(1) - count
  • sort(1) - sorting by count, reversed

PS. Of course it can be replaced by one awk/sed/perl/python/etc script. I just wanted to show how rich unix-way is.

share|improve this answer
    
Thanks, this looks interesting. I will try it out immediately. :) –  Željko Filipin Dec 2 '09 at 11:14
    
I am trying to get your code to work. Could you please provide more information about each command? (I will do some research too.) –  Željko Filipin Dec 3 '09 at 10:54
    
you can use man grep, man awk etc. –  SaveTheRbtz Dec 3 '09 at 15:55
    
Thanks, will do that. I was not explicit enough. The only thing confusing me so far is {print $6}, but I guess it represents the piece of data in a line. –  Željko Filipin Dec 4 '09 at 9:16
1  
Thanks a lot. I guess my log is not standard, but I just had to change $6 to $8 to get it working. –  Željko Filipin Dec 31 '09 at 14:35

This is a slight variation of the accepted answer, using fgrep and cut.

fgrep '"GET ' | cut -d'"' -f6 | cut -d' ' -f1 | sort | uniq -c | sort -rn

There is something appealing about using "weaker" commands when it is possible.

share|improve this answer

While the one liner by SaveTheRbtz does the job, it took several hours to parse my nginx access log.

Here is a faster version based on his, which takes less than 1 minute per 100MB of log file (corresponding to about 1 million lines):

sed -n 's!.* "GET.* "\([[:alnum:].]\+/*[[:digit:].]*\)[^"]*"$!\1!p' /var/log/nginx/access.log | sort | uniq -c | sort -rfg

It works with the default access log format of nginx, which is the same as the combined format of Apache's httpd and has the User-Agent as the last field, delimited by ".

share|improve this answer

I'd use shell script for that: cat, awk pipe, sort and uniq will do the job

share|improve this answer
    
Thanks. I know how to parse logs. I was looking for a tool that already knows how to do it, so I do not have to write a script. –  Željko Filipin Dec 2 '09 at 11:07

Webalizer can do it.

Example:

webalizer -o reports_folder -M 5 log_file
  • -o reports_folder specifies folder where report is generated
  • -M 5 displays only the browser name and the major version number
  • log_file specifies log file name
  • source: ftp://ftp.mrunix.net/pub/webalizer/README
share|improve this answer

Awstats should do the trick, but will supply far more information. I hope this helps...

share|improve this answer
    
Thanks, Awstats looks similar to Webalizer. I will try it out. –  Željko Filipin Dec 1 '09 at 14:23
    
Awstats was to complicated to install. –  Željko Filipin Dec 3 '09 at 10:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.