Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an internal web application that has recently gone through a major version upgrade. I would like to monitor this application over the weekend and look for 'soft' errors. I will still need to spot check things by hand, but there are some common failure patterns that I think I can automate.

Examples include data with bad formatting, blank rows in tables (indicates missing non-critical data), patterns for identifiers ("TEST" means one of my devs left a testing feed on), etc.

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

Are these goals sane? What applications/tools can help with this?

This particular application is receiving data feeds from external sources. We have done our best to do integration testing against a representative sample of this data. Despite that we know we will not be able to catch every quirky format or wonky ID scheme with our preprocessors.

share|improve this question
add comment

5 Answers 5

up vote 1 down vote accepted

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

This is exactly the use case for real browser monitoring. You create test scripts that follow a certain click path through your website, submit forms etc. If something goes wrong the monitoring service will give you nice screenshots of the errors.

If you are looking for an instant, hosted solution, try AlertFox or Browsermob, both have a decent free plan.

For a diy solution, iMacros, Selenium or Watir are good starting points.

share|improve this answer
add comment

NAGIOS, using its check_http plugin, will do exactly what you describe above.

Edit: with one caveat: when you say log in, if you mean "provide htaccess/htpassword credentials", NAGIOS can do this. If you mean "perform a multi-step procedure involving authenticating, getting an auth cookie, and passing it back when asking for the resource to monitor", NAGIOS isn't so good at that.

share|improve this answer
    
Nagios combined with Selenium though, will do this ;) –  Frands Hansen Mar 11 '11 at 19:49
    
... or even Nagios combined with wget. –  John Gardeniers Mar 12 '11 at 18:44
add comment

In this case, I would suggest building a simple test using Pyccuracy and letting it run from cron using a "headless" browser. You can call it from within a script and monitor whether your app passed all the tests cases you coded.

share|improve this answer
add comment

Examples include data with bad formatting, blank rows in tables (indicates missing non-critical data), patterns for identifiers ("TEST" means one of my devs left a testing feed on), etc.

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

For this sort of pattern I believe that best-practice is to update your integration- / regression-tests and run them regularly against your live app to catch the above; I use Webrat for my Rails apps.

If you want to catch "unknown-unknowns" then you're talking about something like Fuzz testing, I guess; I'm afraid I don't have any recommendations for that.

share|improve this answer
    
We do have integration tests on the application, but those are not able to account for all of the inputs we get in the wild. Essentially we can only write integration tests to deal with conditions that we reasonably expect to exist in the wild, I need a monitoring tool to help me find the conditions in the wild to test for. –  Freiheit Mar 11 '11 at 16:08
    
Ok, I've updated my answer to add a little more context. –  nickgrim Mar 11 '11 at 17:15
    
Again fuzz testing isn't quite right either. I would use fuzzing to see how my system reacts to total garbage. I want to monitor a system in the field for quirks. I may very well have a test that covers that my system tolerates a certain issue, but if I get that in the field I would want to flag it so I can take corrective action with a particular interface on a given installation. –  Freiheit Mar 11 '11 at 20:14
add comment

WatchMouse provides real browser monitoring, they also have a free showcase site to play around with: loads.in

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.