Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a small project we are working on part-time that runs on Nginx/MongoDB on Ubuntu 10.04 LTS Server. We'd like to be able to see reports on things like server load, requests/sec, response time, DB load, DB response time, etc. Is there an open source or free (as in beer) tool that can parse such logs and provide a real-time report?

I looked into Splunk briefly, but I wanted to see if there are any others that are highly recommended.

share|improve this question
    
How are you going to get server load out of a parsed log? –  womble Aug 5 '11 at 3:56
    
If all you're looking for is load, there's a lot of tools out there for that, which aren't necessarily log-related. @womble, he's probably referring to the splunk *nix module, which digests system load data into text log entries on specific intervals. –  Shane Madden Aug 5 '11 at 4:11
    
This sounds more like a job for Cacti than for log parsing per se. –  anastrophe Aug 5 '11 at 4:48
add comment

4 Answers

up vote 3 down vote accepted

Most of the metrics you would like to see are accessible through the respective applications statistics, which is much more effective than log analysis. And there are several free projects aimed to collect such metrics, build graphs, check margins, send warnings, etc. All of them are designed to be easily extensible, so that a lot of plugins for collecting metrics exist and you can write your own. The names of the main projects I remember of are Munin, Cacti and Graphite, but in fact there are more.

Munin is one of the oldest and has a lot of plugins even in the standard package. From your list:

  • server load Munin has plugins to show load average (LA), iostat, vmstat
  • Nginx Munin has "nginx_request" (req/sec) and "nginx_status" (number of connections in a specific status, i.e. active, reading, writing)
  • response time As far as I know, there is no such plugin. See this question for more information
  • MongoDB MongoDB team provides several Munin plugins at GitHub
share|improve this answer
add comment

You might want to take a look at our NGINX sFlow module along with sFlowTrend.

NGINX sFlow module

sFlowTrend adds web server monitoring

share|improve this answer
    
It, ummm, might be wise to disclose your association with sFlow, and read this section of the FAQ. –  pjmorse Oct 11 '12 at 15:28
add comment

AFAIK there's nothing available for measuring requests/sec and response times. An answer to what tools you might use to capture and report on these metrics alone would fill a large book!

For Mongo, the choice is bit simpler. Assuming you don't want to roll-your-own from scratch, then have a look at Server Desnity's (hosting company) monitoring toolkit

share|improve this answer
add comment

You can do it with a shell script and push the result to Ganglia gmetric or Nagios + PNP4Nagios.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.