Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am writing a Nagios plugin to monitor trends of a certain storage resource utilization (e.g. gradual increases are fine, but an instantaneous/sudden increase or decrease in resource usage may indicate a problem). For what it's worth, it is reviewing the last N entries in an RRD file generated by a custom cacti data source/templates.

What is the "right" way to handle Nagios notification config/implementation for this? The problem is that it the plugin would exit as warning/critical for one polling period, but in the next it would be fine (or 3 polling periods later, if I look at 3 polling periods worth of data).

I guess the question is: should I just write it in such a way that it will alert for X polling periods, or should I find a way to write it such that manual intervention is required for it to clear (such as logging into the monitoring server or hitting a URL to run a script that submits a passive result)?

Your input is appreciated, and if you have any tips for how to implement the latter I'm open to them (I can think of a few ways to possibly implement it)

edit: dunxd's "answer" below helps state what I'm looking for more concisely. I'm realistically looking for input on the best way to have/implement a "sticky" alert in Nagios. I've seen it done with other systems, but not Nagios so far.

share|improve this question
add comment

2 Answers 2

It depends on how the alerts will be handled. You want someone to notice when a threshold event occurs.

If they are generating emails that will not be ignored, or you know that someone is regularly checking the Nagios trends, then it is probably ok to just use the standard Nagios behaviour.

If you don't think that will get anyone's attention, then you would want the service status to be "sticky". I'm not aware of any Nagios checks that have this behaviour, but I'd be interested to know about them too.

share|improve this answer
    
That's a great follow-up characterization to what I'm looking for, thanks. The more I think about it, I am indeed looking for a way to make this "sticky" in Nagios, and the best way to implement that. –  FliesLikeABrick Sep 28 '12 at 15:57
add comment

I think it should remain sticky until the trending result is no longer something that matches the alert levels (which should be on the rate you calculated). Actually, thats not truly sticky. :>

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.