11

I'm starting on a new project and I'd like to use SaltStack for managing a large deployment spanning several datacenters. Everything runs on Linux. I had previous experience with Chef but I relatively new to SaltStack. My goal is to keep the entire project in a single git repository and make SaltStack to pull configurations from this repository and apply it to minions. It seems that SaltStack have no notion of versioning but serving content from different branches in the repository would work even better. So, I was thinking to create a git repository of the following structure:

salt -+- config (for salt-master config files)
      +- states
      +- pillars
      +- reactors
      +- formulas

and use different branches for dev, qa and prod environments. I also would like to use pillars for providing datacenter specific configuration and I was thinking to use GitFS to glue everything together.

However, there are those pesky top.sls files that handled in a very peculiar way, do not fit in this picture and eventually break this concept.

There also configuration files that I'd like to keep in this repository but I have no better idea for now other then manually copy or symlink them to under /etc/salt from the config directory.

I went through documentation several times and did some experimentation in a simulation environment but I couldn't come up with any sensible project layout yet, so I'm reaching to the stackoverflow community.

My questions are:

  1. Could anybody share a repository structure and configuration they successfully used for managing a similar environment?
  2. Have anybody figured out a good way for maintaining configuration files?
dtoux
  • 1,754
  • 3
  • 21
  • 38
  • It would be difficult if not impossible for "an answer" to address such a broad series of questions. That said I upvoted your question and am making a calendar entry to revisit this next week when I have more time. Question 3's answer is "yes" so you can remove it. – Dan Garthwaite Feb 27 '15 at 17:39
  • @Dan: All I'm looking for is just a working example. – dtoux Feb 27 '15 at 18:51

2 Answers2

2

It sounds like you're headed in a reasonable direction. We don't have quite as large an operation as you (e.g. we don't use multiple environments as yet, just base; we don't use reactors, etc.), but we've got a similar layout to our main states repo.

Here are a few practices we've adopted:

Self-deploying

We use a Fabric script to deploy configuration and updated states/pillars from the repository to the salt-master. The Fabric script also has tasks to bootstrap a new machine w/ salt-minion and automate the key exchange w/ the master.

We could use GitFS, I suppose, but I had some trouble getting it set up and applying updated changesets reliably. In theory, you could bootstrap salt to deploy itself, but in practice a separate, limited deployment process has been good enough.

Segregated secrets

We keep all "secrets" in separate repositories as pillar files, to keep them out of our main repository. Our deploy script checks these out and deploys them to the correct place in our /srv/pillar tree on salt-master. This allows us to keep sensitive data (passwords, key pairs, etc.) out of our main repository. We use salt to provision Vagrant VMs on developer machines, so "development secrets" are stored in the main repository.

David Eyk
  • 12,171
  • 11
  • 63
  • 103
  • 1
    Yes, it is what we figured as well that we need at least one separate repository for pillars to keep secrets away from the main repository and this is the only sensible part, the rest is a total mess. I asked same question on github, but even developers of saltstack did not have definitive answer, so it looks like spaghetti by design. – dtoux Mar 14 '15 at 04:51
0

I'm working in AWS, and am using the boto_* suite to provision my machines. Their userdata scripts set grains to aid the top file, then install, configure, and start salt minion, which then asks the master for highstate execution.

I have a similar repository structure to you, except that I've broken my state dirs into 'provision' and 'config'.

I'm on my phone at the moment, but let me know if the approach sounds at all relevant, and I can further flesh it out once I'm back on my laptop.

Eric_H
  • 35
  • 7
  • Hi Eric, I'm using AWS also (multiple account to manage different projects) with prod, stage, dev, envs (VPCc) on each. I was using Ansible but now I am leaning Salt. Can you detail your structure? Thanks in advance – include Jul 25 '16 at 11:39