Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Our CEO wants us to check all of our 100+ hosted/partner websites. Not only does she want to check if they are up and running, but she would like a diff analysis on the websites so that we can keep a log of any changes that may have occurred from test to test. Is there an open source script or app that can handle this?

share|improve this question

closed as off topic by mailq, Ward, Scott Pack, HopelessN00b, Lucas Kauffman Sep 30 '12 at 9:45

Questions on Server Fault are expected to relate to server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

How often would the interval need to be from one test to another? Nagios, Zenoss, etc. typically poll on a 5 minute interval, but that seems to frequent for a 'website diff'. – Stefan Lasiewski Sep 26 '11 at 17:36
I believe she wants this daily... We have Nagios running at the moment so I had considered the Nagios approach... I'll see which plugins are out there. – tacos_tacos_tacos Sep 26 '11 at 19:26

The up and running part is easy, you could use a whole range of free tools for that, like Nagios or Cacti or Zenoss, but the diff thing is a lot harder.

You can quite easily write your own plugins to Nagios, so you could implement that strange function yourself and have Nagios do the testing. It would take a bit of work, but it could be done.

share|improve this answer

I think you first need to sit down with your CEO and find out why they want this (and how much they're willing to spend to get it done).

Up and running is reasonable and simple -- any decent monitoring system can do this (See Nagios, InterMapper, OpenNMS). Plan on a dedicated server for this, and parlay it into monitoring internal services too.

Diffs between each run are not really reasonable IMHO - especially with "normal" (5 minute) polling intervals to see if the site is up or down (diffing requires downloading the whole site, which takes lots of space and time -- especially if they care about things like changed images/PDFs).
You can make this happen if there's a good business need for it, but you'd be talking custom code (something like WebWhacker, but command-line/noninteractive to download the website, then a recursive diff), and disk space/network bandwidth.

share|improve this answer
There's no reason the diff test has to be run as often as the up test. It's trivial in Nagios to set different tests to run at different intervals, you could do a daily diff while poling to see if the service is live every 5 mins. – Bart B Sep 27 '11 at 13:36
@BartB - True, but this doesn't make the diff test much more reasonable: We're still downloading the whole site and doing a comparison, so we need 2*sizeof($site) disk space for each site, we're still transferring N sites of data every $interval, and there's going to be CPU time/disk IO involved to do the diff. -- Not something I'd commit to without a good business reason to back up the hardware you're eventually going to need when the inevitable scope creep appears... – voretaq7 Sep 27 '11 at 15:52

Not the answer you're looking for? Browse other questions tagged or ask your own question.