5

I'm using puppet parser validate in a git pre-commit hook in order to spot problems before committing files to our Puppet configuration repository. Unfortunately, this command appears to be a very lightweight syntax check that only marks errors such unbalanced quotes and brackets.

The validate command does not appear to actually parse the configuration and look for things like invalid attributes, undefined references, and so forth. For example, the following will not result in a complaint:

file { 'somefile': requires => File['some-other-file'] }

In this example, requires should be require. Similarly, this also generates no errors:

file {'somefile': require => File['file-that-does-not-exist']}

There is no resource definition for file-that-does-not-exist.

Is there any way to catch these sorts of errors without actually applying the configuration? I was hoping for some sort of flag on the puppet apply command that would completely parse a configuration without making changes, but as far as I can tell no such option exists in Puppet 2.7.1.

UPDATE

puppet apply --noop appears to try too hard in the other direction. It will try to stat() any file referenced in the manifest, which will often cause it to fail with permission errors if it attempts to stat() a file that is not accessible to the current user.

What are other folks doing?

3 Answers 3

1

In short, this is a non trivial problem and not easily solved by parsing the manifests. Compiling the catalog can expand the scope of testing, but it's not a panacea. puppet master --compile require access to the node facts, and ideally a dummy node that fully test all classes. You still have to deal with the limitations of:

  • classes that are can't be in the same catalog (apache, apache::disable)
  • cross class dependency.
  • different OS platforms.
  • nodes with different parameters.

For example, if node one include a and b, it's fine, but node two only require b, it's only a failure you'll see with node two.

class a {
  notify { 'hi': }
}
class b {
  notify { 'bye':
    require => Notify['hi'],
  }
}

If you have the resources, you can compile catalog for all nodes and that will provide fairly comprehensive coverages.

puppet apply --noop have it's limitations as well, off the top of my head: it will fail an exec that's deployed by a package, it will fail files depending on a staging location, and it's not going to test multiple platforms unless you expand testing to a representative sample of your systems. In general it providers sufficient coverage to ensure no compilation issues, give you an idea what systems are affected, what are the changes, and you can judge by the reports whether the changes are ok or a real problem.

In most cases noop is sufficient, I've seen varying degrees of automated testing, such as jenkins where each modules tests files are simulated with --noop (limitations above applies), or using Vagrant to spawn off VMs to perform full blown testing.

3
  • The problem with --noop -- and I apologize for duplicating an earlier comment -- is that it will attempt to stat() any files referenced in the configuration, which will make it fail with "permission denied" errors if attempts to stat() something to which it has no access.
    – larsks
    Jul 29, 2011 at 17:16
  • I'm sticking with "puppet parser validate" for now, I guess. I may investigate a dedicate testing system where I can run puppet apply --noop as root for various configurations.
    – larsks
    Jul 29, 2011 at 17:18
  • noop in this case is not intended for the puppet master. You can compile a catalog then apply it in noop mode on the master, but in most cases your master can not simulate the agent config (wrong platform/package, missing packages). It's intended as an agent side operation, typically used in conjunction with a test environment to get an idea what changes will occur before pushing to prod.
    – Nan Liu
    Jul 29, 2011 at 17:53
2

You may want to consider bootstrapping a test environment, such as Cucumber-puppet.

https://github.com/nistude/cucumber-puppet

1

To get a bit more validation that the resources and attributes are sensible, you could compile a sample node catalog with puppet master --compile. This should catch the first example.

I'm not sure off the top of my head whether resource references (the second example) are verified on the master or client, but you could always execute it in no-op mode with puppet catalog apply, or puppet apply. The latter would compile it again and then apply it, while the former should be able to take the compiled catalog from the earlier validation.

2
  • Hmm, puppet master --compile sure seems to want to run as root...at least in it's default invocation it wants to fix permissions on every directory that Puppet uses (which causes it to bail out when run as a non-root user). I presume I can generate a configuration file that will take care of all this...maybe pointing everything inside a temporary (mktemp -d) directory.
    – larsks
    Jul 29, 2011 at 12:56
  • puppet apply --noop ... unfortunately still tries to stat() files, which means it fails with permission errors when run as a non-root user on files that are not accessible (e.g., file { '/root/.ssh': ... }).
    – larsks
    Jul 29, 2011 at 13:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .