Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a web app running with nginx and I'd like to analyze the logs to get information about how many hits each file is getting, and which 404s and other errors we are generating (something Google Analytics can't provide).

Normally I would just drop awstats in a server, but with nginx, due to the lack of cgi-bin, it's not trivial. Is there another trivial solution to get this information? I don't mind paying a third party to do this analysis for me.

share|improve this question

4 Answers 4

You can use goaccess to analyze your web logs and get a summary of hits, unique IPs, top static files, top 404s, different status codes and more. It displays a first top-level summary of information and then lets you drill down each main chapter interactively. You feed it a logfile or a pipe and it's very fast.

I discovered it here on SF and it has since changed my way of looking at webserver logs: I tend to do opportunistic, problem-driven log analysis and was dissatisfied with analog, awstats and webalizer.

share|improve this answer

If you are willing to get your hand dirty I would advise you to use Logstash + ElasticSearch + Kibana.

It can do much more than what you ask, but it can do what you ask well, and you will have a true inferface to actually browse your logs to understand what is - or has been - going on.

This is Kibana demo website

This is a blog post that shows how it has been setup to analyze Apache Tomcat with this platform, you can simply adapt it to your needs : http://spredzy.wordpress.com/2013/03/02/monitor-your-cluster-of-tomcat-applications-with-logstash-and-kibana/

share|improve this answer
1  
+1 for the right answer. –  Tom O'Connor Jun 20 '13 at 9:57

Depends on the level of functionality you require, but AFAIK it is possible to make AWStats generate static files - thus no need for cgi.

The other way around - it should be possible to get AWStats running as a FastCGI script. I would say it won't be even too much hacking.

share|improve this answer

I had to use awstats with Nginx this week and you don't have to use cgi-bin necessarily. Awstats comes with the awstats_buildstaticpages.pl script that will generate static html files which you can then serve with Nginx. With a cron job I update the html files regularly.

/usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

/usr/share/awstats/tools/awstats_buildstaticpages.pl -update -config=domain.com -dir=/path/to/save/html/files awstatsprog=/usr/lib/cgi-bin/awstats.pl (Depending on your OS the paths may differ a little but you will be able to find the scripts)

In order to prevent data loss during log rotation you can add this line to your awstats.domain.com.conf file:

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /path/to/log/access.domain.tld.log /path/to/log/access.domain.tld.log.1 |"

The script will merge the last 2 logs whenever you fetch the log for updating your data. Or you can add a pre-logrotate hook on Nginx like they do here: http://www.bytetouch.com/blog/system-administration/how-to-awstats-installation-and-configuration-on-debian/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.