Random server in the nagios service

Question

I have a cluster of N nodes which contains some distributed information. The query i want to monitor with nagios make sense to execute only on one node of the cluster (any). But i don't want to overload one particular node, so i want nagios to select every time random node from the list of nodes.

So, I would expect to see something like:

define service{
    service_description    Recent document time for the cluster
    hostgroup_name         random
    ...
}

where random hostgroup somehow defines random node from a predefined list of cluster nodes.

Is this possible to do with nagios3? Any suggestions?

Answer 1

Could you set up DNS to give you a random host for a certain name?

You could also write a custom service check.

If you REALLY want this feature in nagios...hack the source

Answer 2

I don't believe this is possible to do natively via nagios. But, it is possible to do via a custom check script, implemented in your language of choice. Just have the check script choose a random (or round-robin, etc.) node each time it is run.

Answer 3

Not to my knowledge, if you're running a custom plugin that you've written, it should be pretty trivial to add an option that randomly pulls from a number of passed hosts, and only test one of them.

You do mention there are N nodes, I don't suppose these all sit behind a VIP of some kind, that only points at one node, and moves to a different box as needed?

A bit more of a description of the setup would be helpful, if we can't find anything, I'd be happy to help you come up with something even if it's custom.

Answer 4

One other solution for this is to run the check on each node server and offsetting the times via a task scheduler/cron, then have the checks output the results to a central location. You can have the Nagios check look at the results in that file accordingly.

That way you wouldn't have to change anything in Nagios when adding a new node, and it could just be part of your node setup.

It is not the cleanest solution, but could help you deal with this.

Answer 5

I have a check_nrpe wrapper that does this. I create a host object with the 'name' objective set to something like 'RANDOM-001' (also supports 'RNDRBN-001'). When the check_nrpe wrapper sees this, it uses the trailing digits to pick from a Perl list of server hostnames to send the check to. Then just add this as hostname to the Service Check description. A more recent version of the wrapper grabs the hostnames from the Nagios hostgroups.cfg file.