Monitoring tool with custom views / templates?

Question

i am currently looking for an open-source monitoring solution like zabbix and icinga. While these both seem to be very powerful for monitoring generic states of hard- and software, I am missing information -- for me -- important functionality, or i could not figure out how it could work by just reading their documentation.

I would like to integrate some job-queue in such a monitoring tool. On the one hand, I need to know summary information of the queue like generic availability etc., which would be no problem to integrate with one of these tools. On the other hand i would like to have additional detailed information about whats going on in the queues.

I would like develop a plugin, which could return an arbitrary amount of detailed data -- like information about each job stored in the queue -- which i could fill in a custom view / template, which i could nicely integreate in one of these monitoring tools.

Is this possible with zabbix, icinga or any other open-source monitoring solution?

Answer 1

Have you considered Nagios? It allows one to write plugins (in any language) and generally easily integrate them into the monitoring framework. In fact, it might be possible that with all the users running Nagios out there, a plugin might be out there that does what you want.

To store more specific information about a job or specific contents in a queue, you might consider construction of some simple Web CGI scripting using the Nagios collection agent to obtain the required data from the server.

As Nagios runs under Apache, this could be a possible approach.

Answer 2

Since you've added the nagios tag, I give you an example with it and NRPE.

Firstly, write your own plugin in favourite language. Remember the return codes:

0 - OK
1 - WARNING
2 - CRITICAL
3 - UNKNOWN

Here's shell script snippet to monitor the length of a list in Redis:

#!/bin/sh

help()
{
    echo "Usage: $0 <host> <port> <key> -w <warning> -c <critical>"
}

case "$1" in
    --help)
        help
        exit
        ;;
esac

if [ $# -eq 0 ]; then
    help
    exit 3
fi
if [ $# -ne "7" ]; then
    help
    exit 4
fi
if [ $4 !="-w" -o $6 !="-c" ]; then
    help
    exit 5
fi

REDIS_CLI="/usr/local/bin/redis-cli"
LLEN=`echo "$3" | $REDIS_CLI -h $1 -p $2 llen`
if [ $LLEN -lt $5 ]; then
    echo "$3.llen:$2 OK - $LLEN | $3.llen:$2=$LLEN;$5;$7"
    exit 0
elif [ $LLEN -ge $5 -a $LLEN -lt $7 ]; then
    echo "$3.llen:$2 WARNING - $LLEN | $3.llen:$2=$LLEN;$5;$7"
    exit 1
elif [ $LLEN -ge "$7" ]; then
    echo "$3.llen:$2 CRITICAL - $LLEN | $3.llen:$2=$LLEN;$5;$7"
    exit 2
fi

Secondly, define a command in /etc/nagios/nrpe.cfg, something like this:

command[check_queue]=/usr/lib64/nagios/plugins/check_queue.sh <host> <port> \
    <queue_name> -w <warning_threshold> -c <critical_threshold>

And thirdly, on the Nagios server, this plugin can be called with:

define service{
    use                     generic-service
    host_name               <remote_server>
    service_description     <queue_name>
    check_command           check_nrpe!check_queue
    contact_groups          admin-sms
}