1

Nagios in its standard usage monitors with point-in-time checks: either something is - or is not - true.

Other tools like SGI's PCP, HP's MeasureWare, and SEC provide monitoring over time - monitoring things like average disk access time over the last five minutes, or other similar items. Is there anything like this for Nagios? I'm already running NDOUtils, which seems like a natural source for such data.

I'd like to have something that would monitor and fire off alarms based on a time-based check using historical data. Is there anything like this for Nagios?

1
  • I've still not found an add-on which basically allows you to interface with data stored by NDOUtils for time-based alarms. However, with a combination of a standard plugin plus SEC running against nagios.log I should be able to do what I want. If only there were an add-on...
    – Mei
    Feb 10, 2011 at 3:01

5 Answers 5

1

I wrote a Nagios check plugin using historical data from sar that you might be interested in. Even if it's not useful to you out of the box, you can easily use it as a starting point for more sophisticated checks.

1
  • Good idea - though it only works for system data contained in the sar reports. I'm specifically interested in working with custom data.
    – Mei
    Feb 9, 2011 at 23:11
1

You say

Usage example: Generate Alarm if CPU Load is over 95% for 10 minutes.

but NAGIOS already does this. If, for example, you don't want to know about a problem until it's been that way for thirty minutes, try (in the service definition)

max_check_attempts              6
retry_interval                  5

This will cause the service to be checked at five-minute intervals even after it goes SOFT ERROR, but not to go hard - and notify - until the sixth successive soft error (6 * 5mins = 30 mins).

If that's not what you wanted, can you explain how it falls short?

Edit: you note that this works for you, but that it can't cope with more complex judgement issues (eg, foo is at over 80% for more than 60% of the preceding 30 minutes).

That's true, but in my experience of deploying NAGIOS, and I've done quite a lot of it, there are very few circumstances where people really need to know something like that. They may want to know it, but when pressed, they generally don't have an engineering need. In those cases, "no" is the right answer; the monitoring tool is business-critical, and overloading it with loads of stupid I-want-a-pony tests to make some VP of Executive Laundry happy is the Wrong Thing.

On the odd occasions where they really do need something baroque, it's much better to put it in the plugin. For example, I did some work for a client to let them know how old the snapshots on any given NetApp volume were, and that was fine. Then a legitimate engineering requirement arose to check that the oldest member of the set of youngest snapshots on a given set of volumes was younger than a given limit (you may want to read that a couple of times!). I could probably have tortured NAGIOS into evaluating criteria based on the results of the multiple "snapshot age" plugins, but it was much more sensible in the long run to write a plugin that tracked and evaluated this one complex criterion on its own.

So I'd say to you: be careful that there is a good engineering need for evaluating odd criteria. In the few cases where there is, write your own plugin to track it.

1
  • Interesting! This may do what I want - although it is only good for some status that stays at a particular level for a particular length of time. For more sophisticated time-based alarms, this won't work: e.g., send me an alarm if this metric is over 95% for 10 out of the last 15 minutes...
    – Mei
    Feb 9, 2011 at 21:48
0

Have you considered using opsview or groundworks?

1
  • Yes... and No. The Nagios infrastructure is already in place, and these other products are more of an unknown. It would require more time getting up to speed, albeit GroundWorks has always looked interesting...
    – Mei
    Feb 9, 2011 at 21:27
0

Yes. The thing is called http://www.pnp4nagios.org/

It allows you to collect use the "performance data" as it is called in Nagios to graph this data with RRD.

With the Nagios way, Icinga can be interesting (it is a Nagios fork).

Another interesting thingy is http://collectd.org it is no way related to nagios, but you can define tresholds and check that condition from Nagios.

UPDATE

For the CPU is over 95% you can make a check that checks for the cpu load and do a check every minute for ten times.

5
  • No, that won't work - unless you can show me how PNP4Nagios can generate alarms. I already have graphs; I want alarms.
    – Mei
    Feb 9, 2011 at 21:15
  • @David: well, what kind of tool do you use for graphs then?
    – cstamas
    Feb 9, 2011 at 21:19
  • I'm already using PNP4Nagios. What I want is alarms on time based data. Usage example: Generate Alarm if CPU Load is over 95% for 10 minutes. Collectd and PNP4Nagios don't do this as far as I know.
    – Mei
    Feb 9, 2011 at 21:19
  • The problem is: using historical data requires knowledge of what has happened previously - which means referring to previously generated data. A plugin like this requires a source of historical data - which suggests NDOUtils would be part of it.
    – Mei
    Feb 9, 2011 at 21:28
  • There's plenty of other sources of historical data, though. sar has shipped with Solaris and Linux systems for ages upon ages. Plus, if you wanted, it wouldn't be difficult to write a plugin that would alarm on the values from the RRDs that PNP4Nagios maintains. Feb 9, 2011 at 22:43
0

Its been years since I looked into it, but Cacti has a plugin for alerting on thresholds called like "thold" or something.

1
  • This question is directly related to replacing said add-on; when using things like PNP4Nagios Cacti becomes redundant - and the program's interface is only a web front-end to RRDTool. PNP4Nagios is much simpler.
    – Mei
    Feb 10, 2011 at 2:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .