Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a bunch of Apache log files that I would like to analyze. I'm looking for a tool that doesn't require much setup; something that I can run a log through the command line, without messing around on our live web servers.

Any recommendations?

share|improve this question

closed as off-topic by Chris S Aug 25 at 20:19

This question appears to be off-topic. The users who voted to close gave this specific reason:

If this question can be reworded to fit the rules in the help center, please edit the question.

    
And the OS you're going to be using is... –  Izzy May 6 '10 at 19:53
    
Linux or MacOSX... –  mmattax May 6 '10 at 20:07
1  
"Analyze" doesn't say much. What are you trying to do? Look for a string, crunch numbers? –  David May 6 '10 at 20:37

7 Answers 7

wtop is cool. There's other utilities as well. Often, I'll parse logs using bash, sed, and awk.

share|improve this answer
    
wtop, and specially their log analyzer logrep are great, once you adapt the .conf to your log format it will provide a fast way to get what you need (top url, traffic, etc..) –  Aseques Aug 26 '13 at 11:05

apachetop is pretty cool; it prints live statistics. You run it with

apachetop -f /var/log/apache2/www.mysite.com.access.log

To install it in Debian/Ubuntu:

apt-get install apachetop

or from source: https://github.com/JeremyJones/Apachetop

There is also an interesting script in http://www.intuitive.com/wicked/84-exploring-apache-access_log-shell-script.shtml

share|improve this answer

What sort of output do you want?

If you are you just looking to count things then grep something logfile.txt | wc -l works great. If you want pretty graphs... not so much.

share|improve this answer
    
For windows the find command mimics grep to some degree. –  Chris Nava May 6 '10 at 20:12

While the tools above are all cool I think I know what the questioner was asking. It often pains me that I can't pull the information out of an access-log in the way I can with other files.

It's because of the dumb access log format:

127.0.0.1 - - [16/Aug/2014:20:47:29 +0100] "GET /manual/elisp/index.html HTTP/1.1" 200 37230 "http://testlocalhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"

Why did they use [] for the date and "" for other things? did they think we wouldn't know a date was in field 4? It's incredibly frustrating.

The best tool right now for this is gawk:

gawk 'BEGIN { FPAT="([^ ]+)|(\"[^\"]+\")|(\\[[^\\]]+\\])" } { print $5 }'

on the data above this would give you:

"GET /manual/elisp/index.html HTTP/1.1"

In other words, the FPAT gives you the ability to pull out the fields of the apache-log as if they were actual fields instead of just space separated entities. This is always what I want. I can then parse that a bit more with a pipeline.

Making the FSPAT work is defined here: http://www.gnu.org/software/gawk/manual/html_node/Splitting-By-Content.html

You can therefore set up an alias to make a gawk that can parse apache logs:

alias apacheawk="gawk -vFPAT='([^ ]+)|(\"[^\"]+\")|(\\\\[[^\\\\]]+\\\\])' " apacheawk '$6 ~ /200/ { print $5 } | sort | uniq

made this for me:

"GET / HTTP/1.1"
"GET /manual/elisp/index.html HTTP/1.1"
"GET /manual/elisp/Index.html HTTP/1.1"
"GET /scripts/app.js HTTP/1.1"
"GET /style.css HTTP/1.1"

and of course almost anything else is now possible.

Enjoy!

share|improve this answer

if you have a windows workstation that you can use then logparser is the tool of choice!

share|improve this answer

analog works well out of the box and doesn't require a lot of setup. logwrangler is a package that works with analog to generate nicer output and also requires little setup.

share|improve this answer

Instead of using a command line tool I would suggeset to try Apache Logs Viewer. It's a free tool which can monitor and analyze the Apache Log File. It can generate some pretty cool graphs and reports on the fly.

More info from http://www.apacheviewer.com

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.