How can I ensure only salt-managed files remain in directory and keep the diff clean?

Solution 1:

You can get a listing of the directory you want by calling the file.find module, and since you have a list of the files you want to manage, you can delete the files that are not in the list of files to manage.

Here's some code adapted from a state file I had to manage /etc/yum.repos.d. I'm going to assume your files to manage are from pillar, and they are keys to a dict, with the values being the rest of the data that goes into rendering the file.

{% set site_dir = '/etc/nginx/sites-available/' %}
{% set sites = salt['pillar.get']('nginx_sites',{}) %}
{% set site_files = salt['file.find'](site_dir,type='f',print='name',maxdepth=0) %}
{% for site_file in site_files %}
{%   if site_file not in sites %}
delete-old-{{ site_file }}:
  file.absent:
    - name: {{ site_dir }}/{{ site_file }}
{%   endif %}
{% endfor %}
{# then go through the sites and manage the files... #}
{% for site in sites %}
manage-site-file-{{ site_file }}:
  file.managed:
    - name: {{ site_dir }}/{{ site }}
    - source: salt://nginx/files/site-file.conf
    - template: jinja
{% endfor %}

Solution 2:

There is option to have the directory with clean: True, but that will result in seeing a full diff of the directory every time I highstate - in our case it's 30 different files created on every highstate. That's making it very hard to check the changes that took place. Is there any way to work around this?

Yes, the way is to use "require: - your_file_states" or "require_in: - directory_state". file.directory will remove all files except required.

/etc/nginx/sites-available/:
  file.directory:
    - clean: True

/etc/nginx/sites-available/hostA:
  file.managed:
    - require:
      - /etc/nginx/sites-available/

/etc/nginx/sites-available/hostB:
  file.managed:
    - require:
      - /etc/nginx/sites-available/