My team has created a large number of chef cookbooks. I was wondering what methods and frameworks I should look into so we can start creating tests to assure that our nodes are configured properly?
cucumber-chef ruby gem and a recently released book Nelson-Smith et al - "Test-driven Infrastructure with Chef" that makes use of that gem - http://www.cucumber-chef.org/
Just remembered this from my github watchlist: https://github.com/gregretkowski/vmth/
The VMTH (Virtual Machine Test Harness) provides a mechanism to unit-test your infrastructure automation - puppet policies, chef recipes, release deployment scripts, etc. It uses features of modern VM monitors (like qemu) to snapshot system state, and then reset that state after each test, so that a series of tests can be performed on a VM instance in rapid succession
There's a number of options.
- cucumber-chef rubygem by Stephen Nelson-Smith, with a related / accompanying book.
- minitest cookbook by AJ Christensen.
- vagrant to provide test virtual machines thanks to Mitchell Hashimoto
Some of the answers here seem a bit old. For CI and local testing I would look into leveraging a test harness tool like Test Kitchen which also supports Vagrant if you've been using that already.
It will allow you to run your coobkooks against many cloud and virtualizations solutions out there today: EC2, Digital Ocean, Vagrant, Docker, etc..
It also has plugins to allow you to run one of the many infrastructure test tools that exist out there today:
- serverspec - Most popular testing framework.
- goss - YAML, simple, self-contained binary, extremely fast.
- inspec - think of it as serverspec improved (written by the chef guys).
- testinfra - Python based server testing framework.
In my experience, the only way to really do this is with a staging environment and careful development practices. Mocking out the actual work to be done is incredibly messy and fragile, and CI test suites running on commit or push are a lot of effort to write and maintain (although Chef is probably a lot better at this than Puppet, which is my main experience), and take a ridiculous amount of time to run.
Just after I wrote this, I suddenly realised there might be a way to do reasonably efficient "unit" testing of Chef/Puppet recipes with a union filesystem and a significant amount of scaffolding. However, I can't find anything out there that anyone's done already in this area. I might have just found myself a project.