KingAnt.net

Background

I’ve been using Salt (for clarity and searchability it’s also sometimes referred to as Salt Project or Salt Stack) to manage the configuration of my web server since 2014. It’s similar to Puppet, Chef (I guess they call it “Progress Chef” now), and Ansible.

At Meebo we used Puppet to manage our server config. This was like maybe 2008 through 2012. It was ok. I don’t remember the specifics but I felt like it could have been better. I don’t remember if there were fundamental problems or if I just felt that it was hard to use.

Anyway, when we chose the configuration management tech at Honor in 2014 we looked for a better option. We made a list of the leading contenders. It was a toss up between Salt and Ansible. They both seemed great. I don’t remember why we chose Salt. Maybe it seemed a little easier to use?

I Like It

I started using it for my personal web server around the same time. I’ve been happy with it. The main benefit is that it’s easier to update to a newer version of Ubuntu LTS every 2 or 4 years. My process is basically:

1. Use Vagrant to run the new Ubuntu version locally. Tweak the config as needed to get things working (update package names, change service files from SysV init to systemd, etc.), and test.
2. Provision a new EC2 instance with the new Ubuntu version. Apply the Salt states to the new server.
3. Migrate data from the old server to the new server and update the Elastic IP to point to the new server.
4. Verify that everything is good then terminate the old server.

It’s an upfront time investment to write the Salt rules, but it makes the above migration fairly easy. I run it in a “masterless” configuration, so I’m not actually running the Salt server. Rather, I have my server-config repo checked out on my EC2 instance.

Weaknesses

Salt does have weaknesses. Since the beginning I’ve felt that their documentation could be more clear. It’s hard for me to be objective since I’ve been using it for so long, but here are a few minor gripes:

• Some of the terminology is unclear. For example, I think the things in this list are typically referred as “states,” but the top of that page calls them “state modules” even though there is a different set of things that are called modules. Additionally the rules that I write to dictate how my server should be configured are also referred to as states, I think? And it’s not clear what modules are or when or how you would use them. There are often modules with the same name as a state but with different functions.
• This page about Salt Formulas has a ton of advice about good conventions for writing Salt states. That’s great, but why is it on this page? Shouldn’t it be in the “Using Salt” section of the documentation instead of here, in the documentation intended for people developing Salt itself?
• Navigating the Salt documentation is hard. Additionally there’s at least one item that’s cut off at the bottom of the right hand nav menu on this page. The last item I see is “Windows” but I know there is at least a “Developing Salt” section. Possibly also Release Notes, Venafi Tools for Salt, and Glossary.
• The term “high state” feels unnatural to me. I think it has some meaning and if you understand how Salt works then maybe there’s a moment of clarity where the pieces fit together. But mostly it feels jargony.
• It’s hard to have a state key off the result of an arbitrary command that runs mid-way through applying your configuration. There’s a thing called “Slots” that tries to solve this problem but it’s hard to use.
• Also, speaking of Slots, why is it called “Slots”? And why is the syntax so awkward? Also I found the documentation hard to read. Partially because it feels jargony. Also there’s some awkward repetition (“Slots extend the state syntax and allows you to do things right before the state function is executed. So you can make a decision in the last moment right before a state is executed.”) and clunky grammar (“There are some specifics in the syntax coming from the execution functions nature and a desire to simplify the user experience. First one is that you don’t need to quote the strings passed to the slots functions. The second one is that all arguments handled as strings.”).

Summary

So I’m happy with it and intend to keep using it. I suspect other options have their own flaws. I have a vague impression that maybe Ansible is a little more widely-used, which is useful to consider.

Also the modern approach to running your own applications in the cloud is:

• Build Docker containers for your applications.
• Either deploy your containers directly to a cloud provider like Amazon Elastic Container Service, or deploy them using Kubernetes.

So there’s less of a need to do configuration management like this on hosts. But it’s probably still valuable for anything you don’t want to run in a container (mail server? DB server? VPN server?).

In Ubuntu 22.04 Firefox is installed from a Snap package and not a dpkg like most other things and the update experience is awful. I get this notification:

However, closing Firefox and reopening it does not cause it to be updated. Apparently you have to either close it and leave it closed for a few hours until the Snap service’s next periodic update, which maybe happens every 6 hours, or close it and run a command to cause the Snap to update.

This is a terrible user experience. Nothing in the messaging informs me about the above. Also it’s absolutely unreasonable to expect the user to leave their browser closed for 3 hours (on average) until an update happens, and expecting a user to run a command manually is a poor experience. See this Stack Exchange post to see other people who were confused and annoyed by this behavior. Lots of people pointed it out in the comments on this post, too. Auto-updates should just happen and all the user should need to do is restart the application (assuming the application isn’t able to dynamically reload the affected files).

I also don’t understand why the Snap isn’t updated while the application is running. Linux is generally able to modify files that are in-use. It’s something that I see as a huge advantage that Linux has over Windows (for more discussion see this Stack Exchange question, answers, and comments). It’s plausible that some applications could misbehave if their files are changed while running—maybe Firefox suffers from this? But then I wonder what the update experience would be like if the user isn’t the administrator. Does the app get killed at some point so the update can happen?

Update 2022-08-13: A few months ago the app started crashing and kept crashing so I uninstalled it. Looking at the comments on Play Store, other people have had the same problem. That was on a three and a half year old phone (a Google Pixel 2). I have since uploaded to a newer phone so maybe the app would work now, but I haven’t tried. One thought I had is that maintaining this at the state level is more work than doing it at the national level, which is an unfortunate situation. Also I wonder if exposure notifications work across states currently, e.g. if someone from New York traveled to California.

Original Post Follows

I installed North Carolina’s SlowCOVIDNC virus exposure notification app and if you live in NC I encourage you to install it, too. You can get it from Apple App Store or Google Play.

So if you’ve been hesitating and wondering “I wonder what my computer programmer and security conscious friends think about this app?” The answer is I think you should install it.

I try harder than most people to avoid installing apps on my phone. I’m wary of apps causing problems either through incompetence or malicious intent. I think the risks in this case are low (much lower than any other random app, like a game or weather app) and the potential to reduce virus spread rate exists (and I barely even go out), so it’s worth it.