Skip to main content

Salt diaries: states (part 2 of deploying salt on a small network)

After part 1 of this series I had Salt running properly on all minions. It's now time to get some work done with it.

We will start with something simple like making sure that ntp is installed and running on all minions. In order to to do that we will use the Salt states enforcement feature.
The default salt states configuration requires that:

  1. state definitions be kept in /srv/salt
  2. the default state be named top.sls

We will probably need to create both the directory and the files, which we can do with the following command (check that you are not overwriting your own state, needs to be done on the master only!):

mkdir -p /srv/salt
cat <<EOF >/srv/salt/top.sls
base:
  '*':
    - ntp
EOF

What this state definition means is that the base state requires all nodes (as selected by '*') to apply the ntp state. Since we have not yet defined an ntp state we are going to do it right away:

cat <<EOF >/srv/salt/ntp.sls
ntp:
  pkg:
    - installed
  service:
    - name: ntpd
    - running
    - require:
      - pkg: ntp
EOF

The ntp state basically instructs salt to check that all nodes have a package called ntp installed and a service called ntpd running. Note that the service check requires the package to be installed: this is to make sure that salt does not try to start the service before the package has been installed, which, of course, is guaranteed to fail.
Also note that we had to specify the name of the service because in CentOS the service is named differently from the package. If we didn't salt would have looked for a service called ntp as the state name implies.
Before we apply the states we can ask salt to report the to-be-applied state for all nodes. (Note: in my case I had ntp installed on some nodes, but not all of them) The command to use is:

salt '*' state.show_highstate

Link to the state commands reference.

If all is well, we can now apply the state to all nodes (to apply it to just one node use the fqdn node name instead of *):

salt -t 60 '*' state.highstate

Tip: it is probably a good idea to raise the timeout when possibly long operations like these need to be performed. When I first tried without the longer timeout some nodes became unresponsive to the master and had to the restart the salt-master process.

As the states are applied the command will output the changes for each node. Running the same command again after all nodes state has been sync'ed should not report any change.

Considerations on states definitions

In this example I could have embedded the ntp state definition in the top.sls file. I preferred not to because
  1. the top.sls file will quickly grow out of control as states are embedded into it
  2. the top.sls file imho should only be used as an entry point where node selectors and states are matched to each other. To know which states go to which nodes one only has to look into this file. On the other hand, scattering selectors and definitions among many files will quickly make the whole system unmanageable.
    Note: grain selectors still can and must be used inside individual substate definitions to account for different OS, features, etc.

Use a version control system for salt states definitions

One final (important) suggestion I have is to use git or svn or any other version control system you might be familiar with to keep track of state changes, especially if you are managing the Salt server together with colleagues. In that case you will also want to make sure that each person uses their own account and not just 'root'.

Next steps

In the next posts I am going to deploy salt on SLES servers and modify the ntp state to account for the difference in configuration and to add many more states (one that I am thinking of is a configuration for a centralized syslog server).

See all my Salt-related posts

Comments

Popular posts from this blog

Indexing Apache access logs with ELK (Elasticsearch+Logstash+Kibana)

Who said that grepping Apache logs has to be boring?

The truth is that, as Enteprise applications move to the browser too, Apache access logs are a gold mine, it does not matter what your role is: developer, support or sysadmin. If you are not mining them you are most likely missing out a ton of information and, probably, making the wrong decisions.
ELK (Elasticsearch, Logstash, Kibana) is a terrific, Open Source stack for visually analyzing Apache (or nginx) logs (but also any other timestamped data).

From 0 to ZFS replication in 5m with syncoid

The ZFS filesystem has many features that once you try them you can never go back. One of the lesser known is probably the support for replicating a zfs filesystem by sending the changes over the network with zfs send/receive.
Technically the filesystem changes don't even need to be sent over a network: you could as well dump them on a removable disk, then receive  from the same removable disk.

A not so short guide to ZFS on Linux

Updated Oct 16 2013: shadow copies, memory settings and links for further learning.
Updated Nov 15 2013: shadow copies example, samba tuning.

Unless you've been living under a rock you should have by now heard many stories about how awesome ZFS is and the many ways it can help with saving your bacon.

The downside is that ZFS is not available (natively) for Linux because the CDDL license under which it is released is incompatible with the GPL. Assuming you are not interested in converting to one of the many Illumos distributions or FreeBSD this guide might serve you as a starting point if you are attracted  by ZFS features but are reluctant to try it out on production systems.

Basically in this post I note down both the tought process and the actual commands for implementing a fileserver for a small office. The fileserver will run as a virtual machine in a large ESXi host and use ZFS as the filesystem for shared data.