With Perforce acquiring Puppet, I finally found the motivation required to really look at my configuration management system and imagine replacing it.
Saltstack came up as an option. I almost immediately started to appreciate how easily you could schedule a run, run on a minion, or trigger a minion run from a master. Being able to target specific commands or sls files is amazing.
I was a bit shocked at how few manuals are out there, and how most of them were written ~2014. The prebuilt formulas are also a bit deprecated, and in some cases they’re just completely busted.
Still, the tools work. The docs at saltstack are good – not perfect (some items are mentioned briefly but not detailed), but still good enough to serve.
As of today, I have 51 formulas – about 30 of those are community formulas I forked to either a) make the code function at all b) add support for more recent versions and config settings or c) cover my specific edge cases.
At this point saltstack manages 17 hosts (including itself). It manages configs for powerdns, zabbix, telegraf, samba, nfs .. and my entire mail suite.
The biggest challenge I faced was inertia – this kept me from converting earlier as well. Salt applies configs for a minimum of 200 items per host. This is up to 400 on some very complicated hosts .. and all of those pieces required configuration, be it SLS files, pillar, or grains.
Similarly though, once I’d reached a certain point in this journey, inertia started working with me. I wanted to trial loki and promtail — it took about 30 minutes to write a formula for promtail to call into loki. It’ll take seconds to roll that out to my hosts.
It took about 3 months of casual tinkering to get the components duplicated out of puppet and running on salt. When I cut over, I simply removed and purged puppet, and ran the salt bootstrap. A few minor errors popped up, but by this point I knew how to fix them.
A note, I actually deviated a bit from the norm. I went with Pillarstack over pillar for most of my configuration. I found the yaml syntax did what I needed, and the very few places I needed SLS, I used pillar. It works.
I’m a real fan of how lists are processed *in order*, so my list of roles for a given host in pillarstack apply in that order, every time. Puppet would do them consistently for a given host, but not identically across hosts.
I’m a fan of how you can piggyback another value or value set onto an existing setting in pillarstack (for example add host specific path to a common list of paths for backups)
I’m definitely not using Salt to it’s limits, there are whole areas which it supports that I haven’t touched. I also don’t use (don’t currently need) separate salt environments, though it would be pretty handy if this managing a product, instead of my lab.
It was a lot of effort, but it was worth it.