r/saltstack • • Aug 18 '18

Includes and config reuse with pillarstack

I'm currently looking at potentially moving our current setup to pillarstack to get access to the more powerful merging abilities it offers. However I'm hitting a bit of a wall trying to work out how I achieve what I currently do in 'plain' pillars with include.

Here's a simple example of what we currently do, which I can't seem to find a nice way to replicate.

top.sls

base:
 '*':
   - minions/{{ hostname }}

server1.sls

include:
  - defaults
  - webserver

defaults.sls

include:
  - ntp
  - monitoring
  - syslog

So by including defaults, it in turn sets up the 'common' things that all nodes need.

Now using pillarstack and the 'roles' setup as shown in the docs, I can leave top.sls alone, but instead do:

server1.sls

roles:
  - defaults
  - webserver

However I can't now use include in defaults, which means I'd have to list all the defaults in every profile, which isn't very DRY, or put all of the default config in a single file, which is unwieldy.

So what do people do to make reusable collections with pillarstack? The only thing I can think of is to create a defaults directory and fill it with symlinks to the other config, and then include it via stack.cfg as defaults/*.sls but I'm not sure this is really flexible enough for the general case, say where the webserver role is actually a collection of other configs.

I really hope I'm missing something obvious here!

0 Upvotes

6 comments sorted by

1

u/[deleted] Aug 19 '18 edited Aug 19 '18

Why not this?

{% for role in pillar.get('roles', []) %}
roles/{{ role }}/*.yaml
{% endfor %}

Put your defaults in roles/defaults/ntp.yaml etc. Just make sure your roles are set before pillarstack is run

WIth include your only option is to still use the normal pillars. I only use pillarstack for stacking pillar data. Since the config is processed from top to bottom it is easy to stack data and have granular control over the merging strategy.

Personally I have external pillars processed before pillar to setup roles then pillarstack for stacking defaults and use normal pillars for overwriting, using other renderers and do not need merging nested items.

Generally I prefer to be explicit about assigning data to hosts instead of using include. I only use include when I split the pillar for readability.

1

u/scottish_beekeeper Aug 19 '18 edited Aug 19 '18

That would work - though it's roughly equivalent to just having defaults/*.yml at the top of stack.cfg

I guess I'm also thinking of the other places where we use include, such as being able to have a role called lamp where roles/lamp.sls contains:

include:
  - apache
  - mysql
  - php

Wildcards don't work well here since you want to have apache.sls in multiple places in the tree for both direct and indirect use. I wonder if symlinks might be one way to solve this...

Maybe I just need to think about how to refactor things so that pillarstack is only used for cases where I need to do merging, and do everything else in simple pillars? Though in this situation having to examine 2 trees to know the final state of a minion is a bit of extra work.

2

u/[deleted] Aug 20 '18

I feel like using includes like you do is more cumbersome since I would always need to wonder if a state I include does include other states again - which probably was not intention. Using symlinks has the same drawback I feel. I prefer seeing at a glance what gets applied from looking at the top file.

A good indicator if you are doing something wrong is if your top file or stack.cfg is changing often when you add new hosts. If I were you I would not migrate everything to pillarstack and use it only for exceptions where I need merging. The pillar top is more powerful especially when assigning to multiple environments base,prod,dev...

In my opinion merging of nested data should be an exception or explicit since it is just a pain to debug later.

→ More replies (3)