Get list of user-agents from nginx log

Question

I have nginx log file, and I want to find out market share for each major version of browsers. I am not interested in minor versions and operating systems. I would like to get something like this:

100 IE6
 99 IE7
 20 IE8
200 FF2
300 FF3

I know how to get the list of user agents from the file, but I want to aggregate the list to see only the major versions of the browsers. Is there a tool that does it?

Answer 1

cat /var/log/nginx-access.log | grep "GET" | awk -F'"' '{print $6}' | cut -d' ' -f1 | grep -E '^[[:alnum:]]' | sort | uniq -c | sort -rn

• grep(1) - we need only GET requests
• awk(1) - selecting full User-Agent string
• cut(1) - using first word from it
• grep(1) - filtering false positives (everything that starts NOT from alphanumeric chars)
• sort(1) - sorting
• uniq(1) - grouping and counting
• sort(1) - sorting by count, reversed

PS. Of course it can be replaced by one awk/perl/python/etc script. I just wanted to show how rich unix-way is.

Answer 2

Awstats should do the trick, but will supply far more information. I hope this helps...

Answer 3

Webalizer can do it.

Example:

webalizer -o reports_folder -M 5 log_file

• -o reports_folder specifies folder where report is generated
• -M 5 displays only the browser name and the major version number
• log_file specifies log file name
• source: ftp://ftp.mrunix.net/pub/webalizer/README

Answer 4

I'd use shell script for that: cat, awk pipe, sort and uniq will do the job