0

I have several servers' apache access logs (multiple servers serving the same domain). What's the simplest way to calculate unique visitors for a given period. I',m on linux, so the linux environment solution is preferable

I don't want to install AWStats just to get the numbers right now. Or is it really the easiest way?

2 Answers 2

0

You can't determine actual unique human visitors from an access log. If you'd like to make an assumption that each unique client IP address is a unique visitor, then you can just feed all the logs for the time period you want through cut -d ' ' -f 1 |sort -u |wc -l (assuming the standard log format where the client IP is the first field). This doesn't take into account monitoring, spiders, NAT, proxies, or dynamic IP addresses, but you can't do much better than that without spending (much) more time than it would take to setup awstats.

0

It's difficult to track unique visitors by using the Apache access logs, but if you want a quick crude measure that you can do with shell, I've used and tested the following.

So for 3 days in November right up to now, Analytics reckons there's been 1,586 unique visitors to my site.

This piece of shell gives me 1402 visitors, which is not far off, all things considered. Bottom line, the logs are difficult to go through, but this is crude and relatively quick

This works on Apache's combined format only, you need to modify it if that's not what you're working on

I've formatted it over multiple lines here to explain it a little better.

awk -F'"' '{print $1,$6}' < log | \
    sed -e 's/\[\([0-9]\+\/[A-Za-z]\+\/[0-9]\+\).*\]/\1/' | \
    awk '{print $4, $0}' | \
    sort | uniq | \
    awk '{print $1}' | \
    uniq -c

LINE 1: prints the IP, timestamp and User agent of the client from the access log

LINE 2: remove the "time" portion of the timestamp, so we can group the lines by date

LINE 3: move the date to the start of the line, so we can sort by date (this isn't necessary for a single log file, but you can modify the script to use multiple files and sort by date, this helps here)

LINE 4: sort | uniq basically sorts all the lines and removes duplicates (i.e. multiple hits from the same IP, on the same day with the same user agent is counted now as 1 hit - this is the magic)

LINE 5: Only print the date part

LINE 6: Use uniq counting function to group these lines by the date and tell us how many is in it

Sample output:

538 16/Nov/2009
559 17/Nov/2009
305 18/Nov/2009

It's very very crude, but it's easy and quick-ish and easy to join multiple log files.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .