How can I debug why Munin won't talk to Nagios?

I've followed the instructions to connect Munin to Nagios. NSCA is installed, echo -e "foo.example.com\ttest\t0\t0" | /usr/sbin/send_nsca -H localhost -c /etc/send_nsca.cfg works when run as the munin user, and I've added contacts nagios and the contact.nagios.command /usr/bin/send_nsca -H your.nagios-host.here -c /etc/send_nsca.cfg command to the Munin configuration.

But I don't get any NSCA messages in the logs. Not for failing to send messages on the Munin side, or receiving messages on the Nagios side. Here's my questions:

1. How can I trigger alerts for Munin to be able to test?
2. How can I determine what step is failing?
3. What should I be seeing in the logs if Munin fails to send NSCA messages?
4. What should I be seeing in the Nagios logs if Munin succeeds in sending messages?
• Is this, by any chance, using munin 2.0.9 from the EPEL repositories on CentOS 6, and munin-limits isn't sending gratuitous OKs when invoked with --force? – MadHatter Jan 28 '13 at 17:01
• How can I tell what version of munin I have installed? – Chris B. Jan 28 '13 at 17:39
• Are you running Linux, and if so, what distribution? – MadHatter Jan 28 '13 at 23:47
• This is Ubuntu 12.04.1 LTS – Chris B. Jan 29 '13 at 18:42
• My Debian-fu is weak, but if memory serves, dpkg -l|grep munin. – MadHatter Jan 29 '13 at 19:57

It is not very clear from the instructions, but NSCA is only necessary if Nagios is not running on the same machine as Munin, and I see your NSCA is sending to localhost. This makes things much easier to setup and debug.

1) How can I trigger alerts for Munin to be able to test?

No idea, but will investigate and update the answer if I discover a quick method.

2) How can I determine what step is failing?

First, make sure you have the following lines in nagios.cfg:

accept_passive_service_checks=1
check_external_commands=1
command_file=/var/lib/nagios3/rw/nagios.cmd


To further debug Nagios issues, enable the debug log in nagios.cfg:

debug_level=256
debug_verbosity=2


Restart the Nagios service and keep an eye on the following logs:

/var/log/nagios3/nagios.*
/var/log/munin/munin-*


You can manually send passive notification to a specific host and service via:

sudo -u nagios echo "[$(date +%s)] PROCESS_SERVICE_CHECK_RESULT;$host;$service;$result;$notes" >>$command_file