We run a few commercial webapps facing customers. The release schedule for them is regular, but there's always worries about testing changes, patches and upgrades. It seems like acceptance testing is promoted for contract development rather than "Commercial Off The Shelf." Is acceptance testing worthwhile for something we've already paid for? And if so, is there anything available with an eye towards:
- Free. Everyone's budget is tight, ours is no different. Open Source is also a bonus. Being able to hand off a failing test to our support contact feels pretty damn handy, and commercial licensing might get in the way of that.
- Automated. We probably don't have a budget to do something like this "by hand".
- AJAX-y. Some of the pages rely on client side javascript, so we can't just replay HTTP get requests.
- Robust. I'd rather not have to rewrite the tests for every minor revision release.
- Cross Browser, Cross Platform. Different browsers behave differently and I'd feel more confident if the web engines behind the browsers were involved somehow. And we intend to support Macs.