Linting Salt states without running them

Question

I am using Saltstack in a homelab, and I often find myself checking in slightly-broken rules when testing them. I would like to be able to check them for validity, and otherwise lint them, locally and on a Jenkins instance, but I can't find any documentation about how I might do so. Is there something I'm missing?

Answer 1

Syntax issues are multi-layered in Salt (i.g. Jinja -> YAML -> state function args) and there is no tool to cover them all.

The fast answer based on this related issue is to trigger the multi-layered parsing:

salt-call state.show_highstate      | tee highstate.output.yaml
salt-call state.show_sls [state_id] | tee state_id.output.yaml

The show_* functions display state data as minion sees it before execution.

Using salt-call on minion side (instead of salt on master side) often provides better debug options - this is mostly a preference.

The problems may also be in pillar or grains (check that all required data is compiled and exists as expected):

salt-call pillar.items | tee pillar.output.yaml
salt-call grains.items | tee grains.output.yaml

Just like @cyfur01 already mentioned, running states directly (with test mode or not) is the last step to troubleshoot:

salt-call state.highstate      test=True | tee highstate.output.yaml
salt-call state.sls [state_id] test=True | tee state_id.output.yaml

Answer 2

Salt states support a testing interface. For example:

salt '*' state.highstate test=True

This should run the states and tell you everything they would do without actually changing anything -- effectively it's a dry run. While it's not directly a linting tool, it does verify that Salt is able to parse and run everything.

Answer 3

The test option is heavy for linting YAML configs. Instead try creating a precheckin script that includes something like this:

salt-call state.highstate --file-root=$PWD --local --retcode-passthrough mocked=True

• --file-root allows you to specify the location of your current checkout
• --local indicates that the action should not run through the master
• --retcode-passthrough causes this command to exit non-zero if any rule cannot be constructed
• mock=True processes all of the rules, but does not initiate connections. This is a new feature in 2015.8.5. An alternate method is to run state.show_highstate

Answer 4

I've been searching a while for a good way to achieve this QA stuff on salt state, and my best answer so far is :

1. Using jenkins to launch jobs (through ssh), based on a dev git branch that:

   • Provision a lxc on our lab proxmox private cloud (in the exact same manner we do this in prod)

   • Using salt reactors the container get its config (as it would on prod)

   • Using testinfra to run unit test on the built and config'ed container

   • Finally if everything went OK destroy the container, if not keep it alive for a morning debugging session :)

2. We also run a linting jenkins jobs as :

   for state in $(sudo /usr/bin/salt-call cp.list_states | awk '{print $2}' | grep -v "^top$"); do sudo /usr/bin/salt-call --retcode-passthrough state.show_sls ${state} ; done
   

I still have some issue getting the correct return code for this last linting job (because of ssh and so on).

This process as a whole ensure :

1. Our provisioning process is OK
2. Our code base (state + pillar) is working as expected
3. We can merge dev to prod with a great confidence ratio

The real good point of testinfra is that it can use a salt connection backend which allow testinfra to connect to container without the need to deploy ssh key, or anything else (as we are using salt-cloud for the initial provisioning)

More on testinfra salt connection backend, testinfra salt module.

This not perfect but still it does quite a good job.