13

I have a web app running with nginx and I'd like to analyze the logs to get information about how many hits each file is getting, and which 404s and other errors we are generating (something Google Analytics can't provide).

Normally I would just drop awstats in a server, but with nginx, due to the lack of cgi-bin, it's not trivial. Is there another trivial solution to get this information? I don't mind paying a third party to do this analysis for me.

4 Answers 4

21

You can use goaccess to analyze your web logs and get a summary of hits, unique IPs, top static files, top 404s, different status codes and more. It displays a first top-level summary of information and then lets you drill down each main chapter interactively. You feed it a logfile or a pipe and it's very fast.

I discovered it here on SF and it has since changed my way of looking at webserver logs: I tend to do opportunistic, problem-driven log analysis and was dissatisfied with analog, awstats and webalizer.

1
  • Wow, thanks for mentioning GoAccess, it's freaking amazing!
    – Richie086
    Aug 31, 2018 at 18:36
7

If you are willing to get your hand dirty I would advise you to use Logstash + ElasticSearch + Kibana.

It can do much more than what you ask, but it can do what you ask well, and you will have a true inferface to actually browse your logs to understand what is - or has been - going on.

This is Kibana demo website

This is a blog post that shows how it has been setup to analyze Apache Tomcat with this platform, you can simply adapt it to your needs : http://spredzy.wordpress.com/2013/03/02/monitor-your-cluster-of-tomcat-applications-with-logstash-and-kibana/

1
  • 1
    +1 for the right answer. Jun 20, 2013 at 9:57
1

I had to use awstats with Nginx this week and you don't have to use cgi-bin necessarily. Awstats comes with the awstats_buildstaticpages.pl script that will generate static html files which you can then serve with Nginx. With a cron job I update the html files regularly.

/usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

/usr/share/awstats/tools/awstats_buildstaticpages.pl -update -config=domain.com -dir=/path/to/save/html/files awstatsprog=/usr/lib/cgi-bin/awstats.pl (Depending on your OS the paths may differ a little but you will be able to find the scripts)

In order to prevent data loss during log rotation you can add this line to your awstats.domain.com.conf file:

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /path/to/log/access.domain.tld.log /path/to/log/access.domain.tld.log.1 |"

The script will merge the last 2 logs whenever you fetch the log for updating your data. Or you can add a pre-logrotate hook on Nginx like they do here: http://www.bytetouch.com/blog/system-administration/how-to-awstats-installation-and-configuration-on-debian/

0

Depends on the level of functionality you require, but AFAIK it is possible to make AWStats generate static files - thus no need for cgi.

The other way around - it should be possible to get AWStats running as a FastCGI script. I would say it won't be even too much hacking.

Not the answer you're looking for? Browse other questions tagged .