Bizarre problem with PNP4Nagios and DS values in XML file

Question

I'm running Nagios Version 3.2.0 and I've got a very odd issue with Nagios PNP with stats collection.

We run about 30 instances of Clammy on the server being checked. I have a script which basically collects data about each instance of Clammy from a PS command, parses the data and outputs to Nagios.

I pass specific dataset values via a pipe | to Perfdata and everything works great...as long as I don't try to get data on more than 2 instances of Clammy.

For example, here's the value of Performance Data when checking only 2 Clammys:

Clam_num=1;Scan_%=68.5;Scan_num=1943;Cache_%=31.5;Cache_num=895;Scan_per_sec=9.5; Clam_num=2;Scan_%=66.7;Scan_num=2552;Cache_%=33.3;Cache_num=1273;Scan_per_sec=12.8;

That works great, and PNP graphs the values just fine. However, if I make the check for 3 Clammys, I get the following output:

Clam_num=1;Scan_%=68.5;Scan_num=1943;Cache_%=31.5;Cache_num=895;Scan_per_sec=9.5; Clam_num=2;Scan_%=66.7;Scan_num=2552;Cache_%=33.3;Cache_num=1273;Scan_per_sec=12.8; Clam_num:3 Scan_%:67.3 Scan_num:6474 Cache_%:32.7 Cache_num:3143 Scan/sec:32.1 | Clam_num=3;Scan_%=67.3;Scan_num=6474;Cache_%=32.7;Cache_num=3143;Scan_per_sec=32.1;

As you can see, the parser suddenly decides that for Clam_num 3 it's going to insert both the status and performance data into the DS of what should only be performance data.

When I look at the XML file, the DS that should contain "3" for Clam_num actually contains the entire string of "Clam_num:3 Scan_%:67.3 Scan_num:6474 Cache_%:32.7 Cache_num:3143 Scan/sec:32.1 | Clam_num=3;Scan_%=67.3;Scan_num=6474;Cache_%=32.7;Cache_num=3143;Scan_per_sec=32.1;"

Even stranger, the DS values that follow resume correctly and contain what they should. e.g. The scan_% value for Clam 3 is correct, the Scan_num value is correct, etc.

Am I missing something here or is this the twilight zone?

Here's the PHP script on the Nagios box:

<?php

$ds_name[1] = " statistics";
$opt[1] = "-M --title \"Clam stat check for $hostname\" --height=250 --width=800 ";
$def[1] = "";

$colors = array(

                'Clam_num' => '#F90303',
                'Scan_%' => '#FC4A4A',
                'Scan_num' => '#FC8A8A',
                'Cache_%' => '#FBBCBC',
                'Cache_num' => '#F906F5',
                'Scan_per_sec' => '#F87BF6',

                );

$keys = array(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18);
foreach( $keys as $key){
                $def[1] .= "DEF:var$key=$rrdfile:$DS[$key]:AVERAGE " ;
                $def[1] .= "LINE:var$key".$colors[$NAME[$key]].":\"$NAME[$key]\t\" " ;
        $def[1] .= "GPRINT:var$key:LAST:\"%1.0lf Max\" ";
}

?>

This is the check script that runs on the client and collects the data:

!/bin/bash

ECHO=/bin/echo
CUT=/usr/bin/cut
GREP=/bin/grep
SORT=/usr/bin/sort

F=/var/log/purewire.log
rm /tmp/AVS_clam_*

pid_list=`ps axu | $GREP clammy | $GREP -v grep | awk '{print $2}'`

c=0
for x in $pid_list ; do

        let c=c+1

        entry=`$GREP "clammy\[$x\]" $F | $GREP "scan stats" | $SORT | tail -1`

        ## AVSCAN data
        AVSCAN=`$ECHO "$entry" | $CUT -s -d" " -f10`
        Scan_p=`$ECHO $AVSCAN | $CUT -s -d"|" -f 2`
        Scan_num=`$ECHO $AVSCAN | $CUT -s -d"|" -f 3 | $CUT -d">" -f 1`

        ## CACHE data
        CACHE=`$ECHO "$entry" | $CUT -s -d" " -f11`
        Cache_p=`$ECHO $CACHE | $CUT -s -d"|" -f 2`
        Cache_num=`$ECHO $CACHE | $CUT -s -d"|" -f 3 | $CUT -d">" -f 1`

        ## Throughput data
        TPUT=`$ECHO "$entry" | $CUT -s -d" " -f14`
        Scan_sec=`$ECHO $TPUT | $CUT -s -d"|" -f 2`

        let Tot=Scan_num+Cache_num

    #echo "Scan %: $Scan_p  Number Scanned: $Scan_num"
    #echo "Cache %: $Cache_p  Cache hits: $Cache_num"
    #echo "Scans per Sec: $Scan_sec   Total Scanned: $Tot"

        #echo $c,$Scan_p,$Scan_num,$Cache_p,$Cache_num,$Scan_sec
        if [[ $c -le 3 ]]; then
        echo $c,$Scan_p,$Scan_num,$Cache_p,$Cache_num,$Scan_sec >> /tmp/AVS_clam_"$c"_stats
                clammy=`awk -F"," '{ print "Clam_num:" $1 " Scan_%:" $2 " Scan_num:" $3 " Cache_%:" $4 " Cache_num:" $5 " Scan/sec:" $6 " | Clam_num=" $1 ";Scan_%=" $2 ";Scan_num=" $3 ";Cache_%=" $4 ";Cache_num=" $5 ";Scan_per_sec=" $6 ";"}' /tmp/AVS_clam_"$c"_stats`
        echo $clammy
        else
        exit
        fi

done
exit 0