As Ig suggests Nagios may be part of the solution.
Out of the box it will do lots of things like measuring CPU, open ports, system responding to pings. But the important thing is that it is a scheduling and reporting tool - to which its trivial to add new programs which actually measure system metrics. There are lots of other products out there to do stuff like this, but Nagios is one of the better free products (better than many of the commercial offerings). And if you look at the commercial offerings you'll spend weeks wading through all the marketing spin and bolted on functionality before you find out if its even possible to integrate your own metrics, let alone how to do it. Certainly both BMC Patrol (new version) and Oracle's Grid Control can support this - but require a lot more effort.
If you want to test end-to-end functionality then all you'd need to do is set up a scheduled job to send an email through the system (in Nagios or with a conventional scheduler) say, every 5 minutes, then set up a check in Nagios to periodically check the most recent email coming out of the system - if there is no new email (or the most recent email is over 6 minutes old) then your script should return an error status.
This kind of script is trivial to set up on a Unix system but can be more difficult (and a lot more expensive) on MSWindows boxes - unless you've got a strong aversion to Unix I'd recommend firing up Linux or BSD on a spare PC to run Nagios and the probes - but Nagios is available for MS Windows.
Sounds like your script is doing all the hard work? Not really, Nagios will manage escalations, can be configured to send alerts to different people depending on the time of day, trigger automatic actions in the event of an alert (e.g. reboot a server) and provide lots of information about why the service might be failing (e.g. disk full on email server) and lots of other stuff...erm leverage existing skillsets to integrate problem resolution strategies?
C.