Going to answer my own question here in detail since this drove me nuts for awhile, and hopefully it will help somebody else down the road looking for the same. Thanks to Keith's answer, though, which started me on the right path here.
To debug this, I edited the /etc/nagiosgrapher/ngraph.ncfg file and set the log to the most verbose setting possible:
I then restarted nagiosgrapher via
/etc/init.d/nagiosgrapher restart and started to watch the log at the location above.
There are three things that happened here, all of which contributed to my problem:
For whatever reason, the Debian distribution of nagiosgrapher no longer installs the check_disk.ncfg and check_load.ncfg graph definitions. I always assume that I have done something wrong when things don't work, but in this case, they just aren't there -- BUT, they are in the distribution and on the system. You just need to copy them:
cp /usr/share/nagiosgrapher/debian/cfg/ngraph.d/standard/check_disk.ncfg /etc/nagiosgrapher/ngraph.d/standard
cp /usr/share/nagiosgrapher/debian/cfg/ngraph.d/standard/check_load.ncfg /etc/nagiosgrapher/ngraph.d/standard
It is possible this is just a problem with the package install script, but I have not verified this, and your mileage may vary. This worked for me though.
I had renamed the
service-description from "Current Load" to "CPU Load" in the
/etc/nagios3/conf.d/services_nagios2.cfg file where I defined it, which seemed innocent at the time. However, the nagiosgrapher package ships with a plugin called
check_hardware which has a block defined as "CPU", and apparently, nagiosgrapher looks at that service description, parses it on whitespace, and then checks for matching blocks, in this case checking both "CPU" and "Load". Further, any time one of those blocks fails to match, the entire write fails, resulting in the message I got in my question above.
Similar to Problem 2, I had added monitoring for MySQL partitions and set the
service-description to "Disk Space: MySQL". Again, sine nagiosgrapher was parsing on whitespace, it saw both "Disk" and "MySQL" and was triggering the MySQL graph plugin, which of course failed on the disk output. Changing the service-description fixed this.
Once the above changes were made, do the following:
This will begin generating the graph data correctly. Once this has been running for awhile and each graph has been generated at least once (you can verify in the logs or by looking directly in the
/var/lib/nagiosgrapher/rrd/ directory), then restart Nagios:
At this point, everything should be working fine.
For what it's worth, these are the packages I have installed that led me to this problem and solution:
ii nagios-images 0.7 Collection of images and icons for the nagios system
ii nagios-nrpe-plugin 2.12-4 Nagios Remote Plugin Executor Plugin
ii nagios-nrpe-server 2.12-4 Nagios Remote Plugin Executor Server
ii nagios-plugins-basic 1.4.15-3squeeze1 Plugins for the nagios network monitoring and management system
ii nagios-plugins-standard 1.4.15-3squeeze1 Plugins for the nagios network monitoring and management system
ii nagios3 3.2.1-2 A host/service/network monitoring and management system
ii nagios3-cgi 3.2.1-2 cgi files for nagios3
ii nagios3-common 3.2.1-2 support files for nagios3
ii nagios3-core 3.2.1-2 A host/service/network monitoring and management system core files
ii nagiosgrapher 1.7.1-2+deb60u1 Charting add-on for Nagios