In general (as far as I can see) the main infrastructure/configuration management systems (Puppet, Chef, Ansible, and SaltStack) are based on the philosophy that your servers are "cattle" and not "pets", and it seems that they can be antagonistic to the idea that you would ever make a configuration change directly to a server.

Although I've worked with Chef and Puppet and Salt, it has always been from the point of view of a developer working with Vagrant to get individual boxes set up for development, so my experience doesn't help answer this question.

The question is: do any of these systems support the use case where you make a change directly to a server, and leave it there for a time without worrying that a local daemon is going to overwrite it with the official configuration? You should be able to make the change (and ideally set the time window during which it is protected from overwriting) without any significant changes to (e.g.) Chef recipes or Salt states. The whole purpose of this use case is to avoid letting a five-minute configuration change be bogged down by corporate friction and the complications of finding the right (e.g.) salt state and making sure there are no side effects and going through the release/deployment cycle and testing to make sure you didn't bork anything in the process.

If you want a more concrete example, suppose you want to adjust your logrotate settings for a number of logs, to balance saving drive space and access to historical data. You have an idea of how much data you'll save at any given setting, and how much data you think you need to keep, but you're not 100% sure. It would be nice to start out just making the change, and seeing (1) anyone complains that they need more historical data for some debugging task, or (2) you're not saving as much drive space as you expected and need to save.

A five-minute change can actually be a five minute change, if your configuration mangagement system allows it to be one, and then once you're positive that you want those changes on all of your boxes that fulfill a particular role, you can letting Chef/Puppet/Salt/Ansible manage that for you.

Note: I'm not asking about a use case where the config management system is not yet managing a certain type of config file, but a case where it is already managing a certain type of config file, but you want to make local changes and have them take precedence on that given machine, and not get overwritten.

Do any of these systems support my use case? (Without requiring me to fight with the system or do backflips to make it work.)

  • 1
    Ansible wouldn't overwrite your change until you re-run the playbook that manages that config file (unless, of course, you have a cron or something to do that). – ceejayoz Dec 16 '16 at 15:49
  • 8
    You think you have a five minute change. But we've all seen these five minute changes become permanent undocumented sources of man-years of pain and suffering. We have configuration management systems to prevent this sort of thing. If you need a real five minute change to test something, do it locally, or on a new Vagrant box, or do it within the configuration management system. They're all flexible enough to support "pets". Indeed, we'll always have pets; we need them to drive the cattle! – Michael Hampton Dec 16 '16 at 15:50
  • @MichaelHampton if you read my example, you'll notice that it's the sort of thing that can only be meaningfully tested under real world conditions. The test is not whether it technically works, but whether the change actually meets your company's real needs. – iconoclast Dec 16 '16 at 20:20
  • 1
    So what you're saying is that you want a configuration management and enforcement system that supports not managing and enforcing the configuration on a system? Isn't that called 'not bothering in the first place'? – Rob Moir Dec 17 '16 at 19:20
  • 2
    I am sarcastic and I won't waste anyone's time pretending otherwise, but you are specifically asking if management tools whose entire philosophy revolves around the management of boxes in a certain way, "as cattle, instead of pets" as you say exactly correctly, can be persuaded to treat some of your cattle as if they were pets. I'm sarcastic, yes, but I'm also entirely serious when I suggest that the best way to manage pets with a system designed to treat them wholly as cattle is to not do so. I truly think this is likely to yield the best results. – Rob Moir Dec 18 '16 at 14:52

Config management systems support this use-case by disabling them on the target host for the duration of the change. Using chef as an example, you would disable chef-client on the node. This would cause the node to show up as 'stale' in the Chef server, but that is a helpful reminder that the node is out of compliance and that the change should either be rolled forward or rolled back.

For individual files you could do a chattr +i, which no tool I've seen knows how to bypass.

One extra thought for chef -- for file templates, where the file did not exist prior to the original chef run, you can tell chef to :create_if_missing such that it will only touch the file once.

  • ok, that's useful information, but entirely disabling Chef for a given server is not exactly what I described as my use case. I simply want it to not overwrite a given config file, or a handful of them. – iconoclast Dec 16 '16 at 20:16
  • You could write the recipes to have a 'maintenance flag' you could set that causes it to skip particular recipes. Fundmentally config-mgmt tools are allergic to out-of-control changes so they make it hard to do this. Another approach is to do a chmod +i to the file, which no tool I've seen knows how to bypass. – Jason Martin Dec 16 '16 at 23:34
  • 1
    I think you mean chattr +i. – Michael Hampton Dec 17 '16 at 17:38
  • @MichaelHampton updated! – Jason Martin Dec 18 '16 at 17:52

You didn't mention PowerShell DSC, but I'll throw a couple methods out for that - particularly as PowerShell is coming to other systems than Windows. Some of these methods may have equivalents in the other options.

I can think of four methods of handling your case with DSC:

  1. If you change something not tested by any of the resources, it'll simply never get caught. I expect this applies to other recipe based choices as well.
  2. Set the LCM to Apply and Monitor, then make your change by any means you want. You can test whether your system is still in compliance, but it won't attempt to fix it.

The following choices have an advantage of making your change "part of the recipe" and recreatable.

  1. Deploy your system with an explicit set up of partial configurations where your "extra" settings aren't initially published but can be added later. This assumes that any setting you plan to "temporarily" set isn't in conflict with a setting in another configuration. This actually gives you the opportunity to make your change "part of the recipe" without needing to completely regenerate the configuration.
  2. Publish an entirely new configuration incorporating your change, and start the configuration. Like the previous option, the configuration will idempotently apply (only new changes are made without affecting anything previously set.)

Regardless of whether you use DSC push or pull models, you can make these changes for a single machine or multiple.


Disregarding for the moment the possible incongruity of the concept of both managing the configuration of a system from a CM system and not doing it at the same time that commenters have already noted...

It should be mentioned that for this use case it's usually possible to make use of what has become a convention with modern-day server software - .d configuration directories which allow insertion of extra config snippets in the right places.

That way you may be able to have your CM manage the config for you via e.g. /etc/logrotate.d/local-whatever, while at the same time you're still able to fiddle with a particular machine by adding /etc/logrotate.d/local-aaa-whatever or similar, which would then cause the software to process your overridden settings.

Obviously if CM has recipes that aim to control the entirety of /etc/logrotate.d/ you still have no recourse other than to do things within CM.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.