Saltstack – Using Pillars and starting to template

In our last post about SaltStack, we introduced the concept of grains.  Grains are bits of information that the Salt minion can pull off the system it’s running on.  SaltStack also has the concept of pillars.  Pillars are sets of data that we can push to the minions and then consume in state or managed files.  When you couple this with the ability to template with Jinja, it becomes VERY powerful.  Let’s take a quick look at how we can start using pillars and templates. 

Prep the Salt Master
The first thing we need to do is to tell Salt that we want to use Pillars.  To do this, we just tell the Salt master where the pillar state files are.  Let’s edit the salt master config file…

vi /etc/salt/master

Now find the ‘Pillar Settings’ section and uncomment the line I have highlighted in red below…

image 
Then restart the salt-master service…

systemctl restart salt-master

So we just told Salt that it should use the ‘/srv/pillar/’ directory for pillar info so we need to now go and create it…

mkdir /srv/pillar/

Now we’re all set.  Pillar information is exported to the minions in the exact same way that states are executed.  We define a ‘top.sls’ file in the pillar directory and tell it what information to send where.  Let’s create an example top file now…

image
You can probably figure out that this means Salt should distribute the pillars ‘kube_master’ and ‘kube_minions’ to all (*) of the minions.  So now let’s define these files…

image
So nothing terribly fancy in these two files, just a couple of lists that define my Kubernetes minions and masters.  So now that we have this configured, let’s run our state against the minions again…

salt ‘*’ state.highstate

Once it’s run, let’s use the following command to query all of the pillars that exists on kubminion1…

salt kubminion1.interubernet.local pillar.items

Just like with grains, there are some predefined pillars.  However, notice at the top we now have the two lists we just created…

image 
Cool  huh?  But now what do we do with this data?  This is where Saltstack starts to shine.  We can use this data (or grain data) to template configuration files.  This is huge!  Let’s look at an example so you can see what I’m talking about.  Let’s take the master systemd service definition for kube_controller.  It looks like this…

[Unit]
Description=Kubernetes Controller Manager
After=etcd.service
After=apiserver.service
Wants=etcd.service
Wants=apiserver.service

[Service]
ExecStart=/opt/kubernetes/kube-controller-manager \
--master=http://127.0.0.1:8080 \
--machines=kubminion1,kubminion2,kubminion3,kubminion4
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Rather than us, pre-populating that config file, why don’t we pull that info in from the pillar?  To do that, we use the Jinja templating language which has been integrated directly into SaltStack.  Let’s change our file to this…

[Unit]
Description=Kubernetes Controller Manager
After=etcd.service
After=apiserver.service
Wants=etcd.service
Wants=apiserver.service

[Service]
ExecStart=/opt/kubernetes/kube-controller-manager \
--master=http://127.0.0.1:8080 \
--machines={{ pillar.get('kube_minions')|join(',') }}
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Since we used Jinja in the file, we need to tell Salt about it otherwise it wont apply the Jinja logic and will copy the file as is.  This is done by modifying the file.managed statement to look like this…

/usr/lib/systemd/system/kube-controller.service:
  file:
    - managed
    - source: salt://systemd/kube-controller.service
    - template: jinja
    - user: root
    - group: root
    - mode: 755

Now all we need to do is rerun our state…

salt ‘*’ state.highstate

After it completes, we can check out the config file on the host…

[Unit]
Description=Kubernetes Controller Manager
After=etcd.service
After=apiserver.service
Wants=etcd.service
Wants=apiserver.service

[Service]
ExecStart=/opt/kubernetes/kube-controller-manager \
--master=http://127.0.0.1:8080 \
--machines=kubminion2,kubminion3,kubminion1,kubminion4
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Pretty awesome huh?  We can use Jinja to pull all sorts of data from pillars or grains directly into the configuration files.  Here are a couple more examples…

Pulling in a Grain from the host

{{ salt['grains.get']('host') }}

In this case, we’re telling the minion to pull it’s ‘host’ grain into the template.  This would return the exact value of the host (not the FQDN) but could be used to pull any grain that’s defined.

Pulling in a specific attribute from a pillar

{{ salt['pillar.get']('kube_master:ipaddress') }}

This shows us how to pull a specific variable out of a pillar.  Take for example this pillar…

kube_master:
  ipaddress: 10.10.10.1
  port: 8080
    protocol: http

If I want to just pull the ip address, I can use the exact example as shown above.  If I wanted the protocol variable, my statement would look like…

{{ salt['pillar.get']('kube_master:port:protocol') }}

As you can see, this makes pulling nested data out of a pillar very straightforward.

Pulling pillar information that’s host specific

{{ salt['pillar.get']('kube_hosts:' ~ grains['host'] ~ ':ipaddress') }}

This is a really interesting and powerful combo.  Say you have information defined in a pillar about your entire cluster.  While this is handy to have on all hosts, sometimes you just want information relevant for the host you’re working on at the time.  For instance, look at this pillar…

kube_hosts:
  kubeminion1:
    ipaddress: 10.10.10.1
  kubeminion2:
    ipaddress: 10.10.10.2

When you’re deploying your config files, you want to build a generic template that can be used on each host.  The above syntax tells Salt to pull the value of ‘ipaddress’ from ‘kube_hosts:<the host you’re running on currently>’.  So we can use grain data to pull in more specific data from the pillars. 

In these examples, we’re really just using Jinja to call Salt functions.  Jinja can also be used to provide logic in state files.  For example, take a look at this example for the Kubernetes GitHub repo…

{% if grains['os_family'] == 'RedHat' %}
{% set environment_file = '/etc/sysconfig/docker' %}
{% else %}
{% set environment_file = '/etc/default/docker' %}
{% endif %}

bridge-utils:
  pkg.installed

{% if grains.os_family == 'RedHat' %}
docker-io:
  pkg:
    - installed

docker:
  service.running:
    - enable: True
    - require:
      - pkg: docker-io

{% else %}

{% if grains.cloud is defined
   and grains.cloud == 'gce' %}
# The default GCE images have ip_forwarding explicitly set to 0.
# Here we take care of commenting that out.
/etc/sysctl.d/11-gce-network-security.conf:
  file.replace:
    - pattern: '^net.ipv4.ip_forward=0'
    - repl: '# net.ipv4.ip_forward=0'
{% endif %}

Above you can see that they’re using the data from the grains to determine things like config file location and what packages need to be installed. 

So again – this was just a taste, but I hope you’re starting to see that all of these components combine to make SaltStack a very powerful tool.

1 thought on “Saltstack – Using Pillars and starting to template

  1. joel

    Thanks for the writeup on Salt. It’s been great to get me started and more involved with writing pillar data. One thing I noticed using however in my trials here. If I use this:

    {{ pillar.get(‘flusher_user_pass’) }}

    I wind up getting None for the replaced value in my template. However if I change it to:

    {{ salt[‘pillar.get’](‘db_users:flusher_user_pass’) }}

    It works perfectly. I don’t know if it’s a version mismatch or just the way salt is parsing the pillar but at any rate it’s working with the second way of writing. Thanks again, much help!

    Reply

Leave a Reply to joel Cancel reply

Your email address will not be published. Required fields are marked *