1

I have an internal web application that has recently gone through a major version upgrade. I would like to monitor this application over the weekend and look for 'soft' errors. I will still need to spot check things by hand, but there are some common failure patterns that I think I can automate.

Examples include data with bad formatting, blank rows in tables (indicates missing non-critical data), patterns for identifiers ("TEST" means one of my devs left a testing feed on), etc.

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

Are these goals sane? What applications/tools can help with this?

This particular application is receiving data feeds from external sources. We have done our best to do integration testing against a representative sample of this data. Despite that we know we will not be able to catch every quirky format or wonky ID scheme with our preprocessors.

5 Answers 5

1

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

This is exactly the use case for real browser monitoring. You create test scripts that follow a certain click path through your website, submit forms etc. If something goes wrong the monitoring service will give you nice screenshots of the errors.

If you are looking for an instant, hosted solution, try AlertFox or Browsermob, both have a decent free plan.

For a diy solution, iMacros, Selenium or Watir are good starting points.

4

NAGIOS, using its check_http plugin, will do exactly what you describe above.

Edit: with one caveat: when you say log in, if you mean "provide htaccess/htpassword credentials", NAGIOS can do this. If you mean "perform a multi-step procedure involving authenticating, getting an auth cookie, and passing it back when asking for the resource to monitor", NAGIOS isn't so good at that.

2
  • Nagios combined with Selenium though, will do this ;) Mar 11, 2011 at 19:49
  • ... or even Nagios combined with wget. Mar 12, 2011 at 18:44
1

In this case, I would suggest building a simple test using Pyccuracy and letting it run from cron using a "headless" browser. You can call it from within a script and monitor whether your app passed all the tests cases you coded.

1

Examples include data with bad formatting, blank rows in tables (indicates missing non-critical data), patterns for identifiers ("TEST" means one of my devs left a testing feed on), etc.

I think there are applications out there that can be scripted to do things like: 1. log in 2. Go to $URL 3. select 3rd link in $LIST or $PATTERN 4. Check HTML from that link for $PATTERNS 5. Email report

For this sort of pattern I believe that best-practice is to update your integration- / regression-tests and run them regularly against your live app to catch the above; I use Webrat for my Rails apps.

If you want to catch "unknown-unknowns" then you're talking about something like Fuzz testing, I guess; I'm afraid I don't have any recommendations for that.

3
  • We do have integration tests on the application, but those are not able to account for all of the inputs we get in the wild. Essentially we can only write integration tests to deal with conditions that we reasonably expect to exist in the wild, I need a monitoring tool to help me find the conditions in the wild to test for.
    – Freiheit
    Mar 11, 2011 at 16:08
  • Ok, I've updated my answer to add a little more context.
    – nickgrim
    Mar 11, 2011 at 17:15
  • Again fuzz testing isn't quite right either. I would use fuzzing to see how my system reacts to total garbage. I want to monitor a system in the field for quirks. I may very well have a test that covers that my system tolerates a certain issue, but if I get that in the field I would want to flag it so I can take corrective action with a particular interface on a given installation.
    – Freiheit
    Mar 11, 2011 at 20:14
0

WatchMouse provides real browser monitoring, they also have a free showcase site to play around with: loads.in

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .