2

Assume a directory like this:

- etc/nginx/sites-available/
    - hostA
    - hostB
    - hostC

Where hostA and hostB are created by salt. hostC was created earlier by salt as well, but since than it has been removed from pillar and is no longer needed. I want the directory to contain only files that represent current state. There is option to have the directory with clean: True, but that will result in seeing a full diff of the directory every time I highstate - in our case it's 30 different files created on every highstate. That's making it very hard to check the changes that took place. Is there any way to work around this?

Update: current output (pseudo yaml)

- directory
  - deleted: 
    - file: hostA
    - file: hostB
    - file: hostC
- file
   - new file: hostA
- file
   - new file: hostB

I would expect

- directory:
  - deleted: 
    - file: hostC
  • That's odd. It should only show stuff that's actually being changed by the currently-running highstate. Bug, maybe? – Andrew Feb 19 '16 at 15:20
  • See updated question – Tomáš Fejfar Feb 19 '16 at 16:26
  • I have the same expectation of the output that you do. It should not be treating it as a deletion followed by a recreation; it should show existing unchanged files as 'clean', if it prints them at all. Can you post the state as-written rather than just the output? – Andrew Feb 19 '16 at 22:23
3

You can get a listing of the directory you want by calling the file.find module, and since you have a list of the files you want to manage, you can delete the files that are not in the list of files to manage.

Here's some code adapted from a state file I had to manage /etc/yum.repos.d. I'm going to assume your files to manage are from pillar, and they are keys to a dict, with the values being the rest of the data that goes into rendering the file.

{% set site_dir = '/etc/nginx/sites-available/' %}
{% set sites = salt['pillar.get']('nginx_sites',{}) %}
{% set site_files = salt['file.find'](site_dir,type='f',print='name',maxdepth=0) %}
{% for site_file in site_files %}
{%   if site_file not in sites %}
delete-old-{{ site_file }}:
  file.absent:
    - name: {{ site_dir }}/{{ site_file }}
{%   endif %}
{% endfor %}
{# then go through the sites and manage the files... #}
{% for site in sites %}
manage-site-file-{{ site_file }}:
  file.managed:
    - name: {{ site_dir }}/{{ site }}
    - source: salt://nginx/files/site-file.conf
    - template: jinja
{% endfor %}
  • I think this is actually the only way to do it. Although it's way too complex for everyday use. Also it does not take into consideration that files in that folder could be created by other states. (for example we add specific host for the purpose of Let's Encrypt in lets_encrypt/init.sls state file) – Tomáš Fejfar Mar 20 '16 at 14:16
1

That's "problem" with configuration management tools. You define the state of the things that exists, but if you remove the track of some object, the automation tool won't do anything. For example, there is a similar problem when you create users. If you remove the user from the tool, it won't remove it from the servers.

The solution if to create object definition for the things you want to exists and a blacklist for the things you want to remove. I'm managing users using pillars. If I want a new user, I add the user to a list in the pillar. But if I want to remove an user, I have a blacklist for users.

Another option in your case, if just have all the site in the same file. If you remove some vhost, the file will be updated and Nginx restarted. Having one file per site makes sense when you manage servers manually, but using a configuration management tool, I think it doesn't matter.

  • We use the one file approach in some cases. But specially for vhosts it's far from ideal. If we want to use reusable states, it's hard to put everything in one file. For example we have one vhost that is used as secure endpoint (for cases where we need non-SNI valid certificate). That is in special reusable state. Sometimes we use only nginx vhosts, sometimes we use the non-SNI secure enpoint. Sometimes both. – Tomáš Fejfar Feb 21 '16 at 6:20
1

There is option to have the directory with clean: True, but that will result in seeing a full diff of the directory every time I highstate - in our case it's 30 different files created on every highstate. That's making it very hard to check the changes that took place. Is there any way to work around this?

Yes, the way is to use "require: - your_file_states" or "require_in: - directory_state". file.directory will remove all files except required.


cleanup_unknown_files:
  file.directory:
    - name: /etc/nginx/sites-available
    - clean: true

{% for sitename, details in pillar['websites'].items() %}
nginx_host_{{ sitename }}:
  file.managed:
    - name: /etc/nginx/sites-available/{{ sitename }}.conf
    - source: salt://host.jinja
    - template: jinja
    - context:
        site: {{ sitename }}
        data: {{ details }}
    - require_in:
      - cleanup_unknown_files
{% endfor %}

  • 1
    this is the correct answer – minusf May 2 at 8:35
0

file.recurse actually does exactly what you want with - clean: True in 2015.8. It will only delete files, which are not in the source path and only output the unwanted files it removed.

  • Sorry, that's not what I want :( I need to supply the files from different states. Practically it's (in pseudocode): directory.recurse( clean=true ); for (hostData in pillar.hosts) { file.managed('hostfile.jinja', hostData); }. The "recurse" directory is empty - I just need to clear the directory to get rid of the outdated hostfiles. – Tomáš Fejfar Feb 23 '16 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.