Salt diaries: states (part 2 of deploying salt on a small network)
After part 1 of this series I had Salt running properly on all minions. It's now time to get some work done with it.
We will start with something simple like making sure that ntp is installed and running on all minions. In order to to do that we will use the Salt states enforcement feature.
The default salt states configuration requires that:
We will probably need to create both the directory and the files, which we can do with the following command (check that you are not overwriting your own state, needs to be done on the master only!):
What this state definition means is that the base state requires all nodes (as selected by '*') to apply the ntp state. Since we have not yet defined an ntp state we are going to do it right away:
The ntp state basically instructs salt to check that all nodes have a package called ntp installed and a service called ntpd running. Note that the service check requires the package to be installed: this is to make sure that salt does not try to start the service before the package has been installed, which, of course, is guaranteed to fail.
Also note that we had to specify the name of the service because in CentOS the service is named differently from the package. If we didn't salt would have looked for a service called ntp as the state name implies.
Before we apply the states we can ask salt to report the to-be-applied state for all nodes. (Note: in my case I had ntp installed on some nodes, but not all of them) The command to use is:
If all is well, we can now apply the state to all nodes (to apply it to just one node use the fqdn node name instead of *):
Tip: it is probably a good idea to raise the timeout when possibly long operations like these need to be performed. When I first tried without the longer timeout some nodes became unresponsive to the master and had to the restart the salt-master process.
As the states are applied the command will output the changes for each node. Running the same command again after all nodes state has been sync'ed should not report any change.
We will start with something simple like making sure that ntp is installed and running on all minions. In order to to do that we will use the Salt states enforcement feature.
The default salt states configuration requires that:
- state definitions be kept in /srv/salt
- the default state be named top.sls
We will probably need to create both the directory and the files, which we can do with the following command (check that you are not overwriting your own state, needs to be done on the master only!):
mkdir -p /srv/salt
cat <<EOF >/srv/salt/top.sls
base:
'*':
- ntp
EOF
What this state definition means is that the base state requires all nodes (as selected by '*') to apply the ntp state. Since we have not yet defined an ntp state we are going to do it right away:
cat <<EOF >/srv/salt/ntp.sls
ntp:
pkg:
- installed
service:
- name: ntpd
- running
- require:
- pkg: ntp
EOF
The ntp state basically instructs salt to check that all nodes have a package called ntp installed and a service called ntpd running. Note that the service check requires the package to be installed: this is to make sure that salt does not try to start the service before the package has been installed, which, of course, is guaranteed to fail.
Also note that we had to specify the name of the service because in CentOS the service is named differently from the package. If we didn't salt would have looked for a service called ntp as the state name implies.
Before we apply the states we can ask salt to report the to-be-applied state for all nodes. (Note: in my case I had ntp installed on some nodes, but not all of them) The command to use is:
salt '*' state.show_highstate
Link to the state commands reference.
If all is well, we can now apply the state to all nodes (to apply it to just one node use the fqdn node name instead of *):
salt -t 60 '*' state.highstate
Tip: it is probably a good idea to raise the timeout when possibly long operations like these need to be performed. When I first tried without the longer timeout some nodes became unresponsive to the master and had to the restart the salt-master process.
As the states are applied the command will output the changes for each node. Running the same command again after all nodes state has been sync'ed should not report any change.
Considerations on states definitions
In this example I could have embedded the ntp state definition in the top.sls file. I preferred not to because
- the top.sls file will quickly grow out of control as states are embedded into it
- the top.sls file imho should only be used as an entry point where node selectors and states are matched to each other. To know which states go to which nodes one only has to look into this file. On the other hand, scattering selectors and definitions among many files will quickly make the whole system unmanageable.
Note: grain selectors still can and must be used inside individual substate definitions to account for different OS, features, etc.
Use a version control system for salt states definitions
One final (important) suggestion I have is to use git or svn or any other version control system you might be familiar with to keep track of state changes, especially if you are managing the Salt server together with colleagues. In that case you will also want to make sure that each person uses their own account and not just 'root'.
Next steps
In the next posts I am going to deploy salt on SLES servers and modify the ntp state to account for the difference in configuration and to add many more states (one that I am thinking of is a configuration for a centralized syslog server).
See all my Salt-related posts