I'm having an issue analyzing Apache logs - because there's a ton of them. Like first 5 days of March have resulted in about 50Gb of log files ( uncompressed ).

How to analyze these? What tools to use? What I'm interested in is basically the trendline in errors and pageviews. I.e. if the number of errors increased or decreased over time as well as if pageviews increased or decreased over time.

Can you recommend a tool which would be able to process these and give some kind of a report?

Edit: As noted below, I didn't specify where the Apache is running. It's a LAMP setup.

  • Kibana ? demo.kibana.org/#/dashboard – ysdx Mar 7 '14 at 9:26
  • Tried the Kibana, filled it up with tons of logs. It does get a bit slow sometimes, and ES in the background probably needs some more tweaking but it's serving the purpose right now. I did try with Graylog2 before Kibana and that didn't go that well unfortunately. Kibana seems to be much easier to set up. – Mathew Mar 23 '14 at 12:23

You don't specify the platform where Apache resides.
A good general pourpose tool is AWStats: have a look to it's home page for the wide list of features.
It's written in Perl, and parsing 50GB of logs may require some minutes, but Perl was made for this...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.