Skip to main content

Salt diaries: states (part 2 of deploying salt on a small network)

After part 1 of this series I had Salt running properly on all minions. It's now time to get some work done with it.

We will start with something simple like making sure that ntp is installed and running on all minions. In order to to do that we will use the Salt states enforcement feature.
The default salt states configuration requires that:

  1. state definitions be kept in /srv/salt
  2. the default state be named top.sls

We will probably need to create both the directory and the files, which we can do with the following command (check that you are not overwriting your own state, needs to be done on the master only!):

mkdir -p /srv/salt
cat <<EOF >/srv/salt/top.sls
base:
  '*':
    - ntp
EOF

What this state definition means is that the base state requires all nodes (as selected by '*') to apply the ntp state. Since we have not yet defined an ntp state we are going to do it right away:

cat <<EOF >/srv/salt/ntp.sls
ntp:
  pkg:
    - installed
  service:
    - name: ntpd
    - running
    - require:
      - pkg: ntp
EOF

The ntp state basically instructs salt to check that all nodes have a package called ntp installed and a service called ntpd running. Note that the service check requires the package to be installed: this is to make sure that salt does not try to start the service before the package has been installed, which, of course, is guaranteed to fail.
Also note that we had to specify the name of the service because in CentOS the service is named differently from the package. If we didn't salt would have looked for a service called ntp as the state name implies.
Before we apply the states we can ask salt to report the to-be-applied state for all nodes. (Note: in my case I had ntp installed on some nodes, but not all of them) The command to use is:

salt '*' state.show_highstate

Link to the state commands reference.

If all is well, we can now apply the state to all nodes (to apply it to just one node use the fqdn node name instead of *):

salt -t 60 '*' state.highstate

Tip: it is probably a good idea to raise the timeout when possibly long operations like these need to be performed. When I first tried without the longer timeout some nodes became unresponsive to the master and had to the restart the salt-master process.

As the states are applied the command will output the changes for each node. Running the same command again after all nodes state has been sync'ed should not report any change.

Considerations on states definitions

In this example I could have embedded the ntp state definition in the top.sls file. I preferred not to because
  1. the top.sls file will quickly grow out of control as states are embedded into it
  2. the top.sls file imho should only be used as an entry point where node selectors and states are matched to each other. To know which states go to which nodes one only has to look into this file. On the other hand, scattering selectors and definitions among many files will quickly make the whole system unmanageable.
    Note: grain selectors still can and must be used inside individual substate definitions to account for different OS, features, etc.

Use a version control system for salt states definitions

One final (important) suggestion I have is to use git or svn or any other version control system you might be familiar with to keep track of state changes, especially if you are managing the Salt server together with colleagues. In that case you will also want to make sure that each person uses their own account and not just 'root'.

Next steps

In the next posts I am going to deploy salt on SLES servers and modify the ntp state to account for the difference in configuration and to add many more states (one that I am thinking of is a configuration for a centralized syslog server).

See all my Salt-related posts

Comments

Popular posts from this blog

Indexing Apache access logs with ELK (Elasticsearch+Logstash+Kibana)

Who said that grepping Apache logs has to be boring?

The truth is that, as Enteprise applications move to the browser too, Apache access logs are a gold mine, it does not matter what your role is: developer, support or sysadmin. If you are not mining them you are most likely missing out a ton of information and, probably, making the wrong decisions.
ELK (Elasticsearch, Logstash, Kibana) is a terrific, Open Source stack for visually analyzing Apache (or nginx) logs (but also any other timestamped data).

From 0 to ZFS replication in 5m with syncoid

The ZFS filesystem has many features that once you try them you can never go back. One of the lesser known is probably the support for replicating a zfs filesystem by sending the changes over the network with zfs send/receive.
Technically the filesystem changes don't even need to be sent over a network: you could as well dump them on a removable disk, then receive  from the same removable disk.

A not so short guide to TDD SaltStack formulas

One of the hardest parts about Infrastructure As Code and Configuration Management is establishing a discipline on developing, testing and deploying changes.
Developers follow established practices and tools have been built and perfected over the last decade and a half. On the other hand sysadmins and ops people do not have the same tooling and culture because estensive automation has only become a trend recently.

So if Infrastructure As Code allows you to version the infrastructure your code runs on, what good is it if then there are no tools or established practices to follow?

Luckily the situation is changing and in this post I'm outlining a methodology for test driven development of SaltStack Formulas.

The idea is that with a single command you can run your formula against a matrix of platforms (operating systems) and suites (or configurations). Each cell of the matrix will be tested and the result is a build failure or success much alike to what every half-decent developer of…