"We're not programmers, we're sysadmins"
My how times have changed, for the worse: a greybeard like me was expected to be a better programmer than professional programmers, or else would have never been able to pass for a system administrator.
Now, we have "system administrators", who are basically Windows desktop users who have at some point converted to Linux and cannot program, and do not find anything whatsoever wrong with that.
The elephant in the room is why management tolerates such a destructive attitude. Destructive to whom or what? To the business and to the infrastructure.
Back to Puppet[, CFEngine, Chef] subject: as soon as one sets a solution like that in, one loses. Everyone loses. Why? Because whoever comes up with the idea is not capable of designing encapsulated configuration management in the form of nice, clean, Kickstart[, JumpStart, Automated Installer, AutoYaST, Ignite-UX, NIM] operating system packages. When you have to use an automated hacking tool like Puppet (or Chef, or CFEngine), it means you lack the wherewithal to design and to implement a process which would, by that same design, enforce completely pristine and lights out managed systems, fully automated and completely non-interactive.
Another important point is, if you have to have Puppet or some such solution to correct someone hacking system or application configuration by hand, that also goes back to not having the experience to design a process, and in that process a framework where configuration is packaged into discrete components. In effect, whoever implements Puppet and the like, has no concept of component owners, releases, configuration management, Capability Maturity Model. This is rapidly developing into a very serious problem in the industry.
Working with Puppet also helped me learn Ruby, which has come to replace Bash as my default system tools language."
Why is Ruby needed, when a comprehensive, end-to-end configuration management can be encapsulated in preinstall, postinstall, preremove and postremove sections of operating system packages, just by using Bourne shell programs, AWK, and sed? That someone would go to the length of learning an esoteric language of Ruby, and a dialect thereof in the context of Puppet, is completely unnecessary. The problem of configuration management is easily solvable (and to wit, has been solved) with shell programs and AWK, and a little sed(1) here and there as a glue.
It's a cool feeling to see your Puppet manifest configure an entire machine or a new service from scratch.
It is an even cooler thing seeing it done by Kickstart, AutoYaST, or JumpStart, without a single line of code, and being able to query the operating system by using built in tools, without needing any esoteric or extra software, no client-server architecture required (SSH is more than fine, way more than fine), and seeing your operating system being aware of each and every change made to it.
5.Separate code from data. This is one of the harder concepts to learn. Hardcoding values like Monitoring Hosts into your module code is bad. Putting them in a data store (db, yaml (Hiera uses this be default), csv, whatever) that your modules can consume is good. An example is a webapp that uses Mysql. What this allows is the ability to push code and data separately. This makes your development process simpler.
...Or you could just template your configuration files with shell variables, even backquotes (for example
ls -1 ...) and write a shell script which uses AWK to call eval(1) and expand all the variables in the template, thereby leveraging the exact same powerful parser which shells have built-in. Why make it complex, when it can be really, really simple? Where will you store the configuration values? Why, anywhere you please, such as for example pkginfo(4) files, or a database like Oracle, or pretty much anywhere. No need for ultracomplex solutions. The library I mention above could simply be sourced from the preinstall or postinstall sections in the operating system packages, thereby removing duplication and leveraging a central piece of code...
But above all, I find that the above quote is an example of the next generation of system administrators needing tutoring not by system administrators, but by system engineers. Find yourself a greybeard and sign on as an apprentice.