CI for dotfiles, Too — Verifying a Destructive Installer's Idempotency in Both an Ubuntu and a Kali Role

The Day My Dev Environment Stops Being Reproducible

I keep my development environment in a single script. Shell configuration, prompt, terminal multiplexer, language runtimes, and cloud CLIs all get installed in one shot with ./setup_env.sh. It ended up at 1,358 lines. On a new machine, or after reinstalling the OS, the same environment comes right back. It’s convenient.

Convenient, but one worry remained. Now I have no way to notice when that script itself breaks. If I accidentally write something that only works on one of the two operating systems, I won’t find out until the next time I touch that machine. If the routine that appends to a config file corrupts an existing .zshrc, I only notice after the accident has already happened. By consolidating manual work into a script, I’ve now entrusted the entire reproducibility of my environment to that one script.

So I added CI to my dotfiles, too. What it does is no different from validating production infrastructure. “When a dangerous change lands, turn the build red on the spot” — applied to the script that builds my development environment. This article walks through excerpts of that CI (GitHub Actions). Repository-specific names and layout names are either hidden or generalized, and the code is trimmed to excerpts that show the key points.

The Setup, in Numbers

I can’t share the contents of the actual environment, so instead I’ll convey the scale of the setup. The figures below are measured from these dotfiles and their CI.

Overall Structure

                          push / PR / dispatch
                                    │
                  ┌─────────────────┴─────────────────┐
                  ▼                                   ▼
  ┌───────────────────────────────┐   ┌───────────────────────────────┐
  │ builder (Ubuntu)              │   │ breaker (Kali container)      │
  │  shellcheck / clippy          │   │  shellcheck / clippy          │
  │  setup --dry-run              │   │  setup --dry-run              │
  │  real install in isolated HOME│   │  real install in isolated HOME│
  │  rebuild with --clean         │   │  rebuild with --clean         │
  │  assert layout ×4             │   │  assert layout ×4             │
  └───────────────┬───────────────┘   └───────────────┬───────────────┘
                  │ report (artifact)                 │ report (artifact)
                  └─────────────────┬─────────────────┘
                                    ▼
                    ┌──────────────────────────────┐
                    │ aggregate                    │
                    │  merge reports from both envs│
                    │  → CI step summary           │
                    └──────────────────────────────┘

The CI is split into three jobs: builder and breaker, which run the same installer on two operating systems, and aggregate, which merges both reports into a single view.

Running the Same Installer on Two Operating Systems

This installer has two modes. A builder for my everyday development machine (Ubuntu), and a breaker for the Kali environment used for security work; it switches between them automatically by reading /etc/os-release. Because the tools installed differ, it’s common for one side to pass while the other is broken. So the CI runs both.

builder runs on a bare Ubuntu runner, and breaker runs inside a Kali container layered on top.

jobs:
  builder:
    runs-on: ubuntu-24.04
    # ... shellcheck validates the shell scripts; cargo fmt/clippy/test validates the bundled Rust crate

  breaker:
    runs-on: ubuntu-24.04
    container:
      image: kalilinux/kali-rolling      # layer the entire Kali image on top
    # ... run the same steps on top of Kali

Just by adding a container: block, every step of that job runs entirely inside Kali. Without having to provision a runner myself, I can check “does this installer pass on Kali?” on every run. Discrepancies — like a tool that builder assumes is present but breaker lacks — get surfaced the moment you adopt this dual-role setup.

Not Settling for dry-run — Actually Run It, and Look at What Comes Out

I often see dotfiles CI that stops at running shellcheck. Of course syntax checking is necessary. But it can only tell you “it isn’t broken as syntax.” Whether running it actually produces what you intended is a separate question.

So I set up an isolated HOME and run the real installer into it. I swap in a temporary working directory as HOME, run it, and confirm the files that should have been created with test -f. Here I assert that four kinds of workspace layouts get generated (the names are generalized).

- name: Run installer with an isolated HOME
  shell: bash
  run: |
    set -euo pipefail
    export HOME="/tmp/ci-home"
    mkdir -p "$HOME"

    ./setup_env.sh --skip-download --skip-ai-cli

    # check the things that should exist, one by one
    for layout in workspace-a workspace-b workspace-c workspace-d; do
      test -f "$HOME/.config/zellij/layouts/${layout}.kdl"
    done

With --skip-download (skip fetching manual binaries when a healthy copy already exists) and --skip-ai-cli (omit installing the AI CLI), I cut out the heavy work as much as possible. What I want to see in CI is the installer’s placement logic (what goes where, and how). Note that these flags are there to reduce heavy I/O; installing mise itself and some apt preparation may still run through other paths.

Can It Rebuild After Tearing Down? — The Idempotency of the Destructive --clean

This installer has a destructive option called --clean. It resets the tools and configuration managed by mise once and rewrites the configuration — a rebuild mode, so to speak. The more dangerous the flag, the more I want to confirm that “it really does rebuild.” Even if the teardown works, if the rebuild is missing, it’s just a command that destroys the environment.

So in CI, after running normally once, I run --clean on top and assert that the layouts are generated again. I check not just that they “were deleted” but that they “were rebuilt.”

    # --clean resets and rebuilds; verify the result of that rebuild
    # (check that the layouts are "regenerated," not "left deleted")
    ./setup_env.sh --clean --skip-download --skip-ai-cli
    for layout in workspace-a workspace-b workspace-c workspace-d; do
      test -f "$HOME/.config/zellij/layouts/${layout}.kdl"
    done

Running the same installer two or three times settles into the same state (idempotent). When this breaks down, configuration gets duplicated on each rerun, or one environment ends up missing pieces. Idempotency is the lifeline of an installer, so I have CI watch over it on every run.

Idempotently Swapping Out My Own Config Block

The one thing an installer must never do is corrupt the user’s ~/.zshrc or ~/.bashrc. Any routine that appends configuration touches existing content, so it’s a seed for accidents.

The countermeasure is simple: wrap what I write in markers. I bracket it with BEGIN and END markers, and on a rerun I delete my entire block (when BEGIN is found, the range from BEGIN to END) and re-insert it. The goal is that my additions don’t get duplicated on a rerun. And before rewriting, I take a timestamped backup.

Another quietly effective piece is the routine that cleans up my own leftovers from the past. If I changed the marker name at some point, an old block wrapped in the previous markers can linger and end up duplicated alongside the new block. So when I find the old markers, I remove them too before re-inserting.

backup_if_exists() {
  local file="$1"
  if [ -e "$file" ] || [ -L "$file" ]; then
    # before overwriting, always save a timestamped copy
    local bak="${file}.bak.$(date +%Y%m%d-%H%M%S-%N)"
    cp "$file" "$bak"
  fi
}

append_block_once() {
  local file="$1" begin="$2" end="$3" content="$4"
  touch "$file"

  # if an old block wrapped in the legacy markers remains, clean it up first
  local legacy_begin="# === LEGACY_BEGIN ==="
  local legacy_end="# === LEGACY_END ==="
  if grep -qF "$legacy_begin" "$file"; then
    backup_if_exists "$file"
    sed -i "/$legacy_begin/,/$legacy_end/d" "$file"
  fi

  # if our own block already exists, delete just that range and re-insert it (idempotent)
  if grep -qF "$begin" "$file"; then
    backup_if_exists "$file"
    sed -i "/$begin/,/$end/d" "$file"
  fi
  printf '%s\n%s\n%s\n' "$begin" "$content" "$end" >> "$file"
}

And on the CI side, I actually validate this “migration from an old block.” I place a block wrapped in the old markers into .zshrc in advance, and after running the installer, I look at both:

• That the new markers (INSTALLER_BEGIN / END) are properly present (grep)
• That the old marker’s block is gone (confirming its absence with ! grep)

This confirms that old leftovers wrapped in a valid pair get reliably migrated to the new scheme. To be honest about one thing: because the implementation, upon finding BEGIN, uses sed to delete the range from BEGIN to END, if END were corrupted and missing, it could delete past that point and take other content with it. So this approach is not a guarantee that it “will never corrupt the user’s configuration.” The last line of defense is the timestamped backup taken before rewriting.

    # place a stand-in block wrapped in an old version's markers
    printf '# === LEGACY_BEGIN ===\nexport LEGACY=1\n# === LEGACY_END ===\n' > "$HOME/.zshrc"

    ./setup_env.sh --skip-download --skip-ai-cli

    grep -qF   "# === INSTALLER_BEGIN ===" "$HOME/.zshrc"  # the new block is present
    ! grep -qF "# === LEGACY_BEGIN ==="    "$HOME/.zshrc"  # the old block is gone (migrated)

One Condition = One Checker

The remaining checks hang off scripts/ci/ as 14 small validate_* checkers. I don’t roll them into one giant test because, when something breaks, I want to read which condition failed straight from the file name. This split follows the same idea as the “one condition = one gate” from my earlier post on Rust CI/CD.

Take the supply chain, for example. It mechanically checks whether the actions used in GitHub Actions are pinned to a commit SHA rather than a tag (a movable reference like @v4). Because a tag’s contents can be swapped out later, it fails if there is even a single unpinned reference.

# is the uses: reference pinned to a 40-digit SHA?
USES_RE   = re.compile(r"^\s*(?:-\s*)?uses:\s*(?P<ref>\S+)", re.MULTILINE)
SHA_PIN_RE = re.compile(r"@[0-9a-fA-F]{40}(?:\s|$)")

for match in USES_RE.finditer(workflow_text):
    ref = match.group("ref")
    if ref.startswith("./"):
        continue                     # local actions are out of scope
    if not SHA_PIN_RE.search(ref):
        failures.append(f"unpinned action reference: {ref}")

Another thing: where the installer downloads from is also constrained by an allowlist. The allowlist itself lives in trusted_download_host() inside setup_env.sh, and only the hosts enumerated there are trusted. The defense is two-layered. The CI static checker looks at whether the required hosts are all present in the allowlist, and whether the hosts of URLs hard-coded in the download helper are within the allowlist. This static inspection can’t follow download sources assembled from variables, but at runtime assert_trusted_url catches those by matching the host of every URL (including redirect targets) against the allowlist. It’s deliberate friction to prevent “before I knew it, it was pulling something from an unknown host.”

# baseline required hosts (the minimum set setup_env.sh should trust)
REQUIRED_TRUSTED_HOSTS = {
    "api.github.com", "apt.fury.io", "claude.ai", "downloads.claude.ai",
    "github.com", "mise.run", "objects.githubusercontent.com",
    "release-assets.githubusercontent.com",
}

trusted  = trusted_hosts(setup_script)           # extracted from trusted_download_host()
literals = literal_download_hosts(setup_script)  # hosts of URLs hard-coded in the helper

# 1) is any required host missing from the allowlist? (check this first)
if REQUIRED_TRUSTED_HOSTS - trusted:
    raise SystemExit("required trusted host missing")

# 2) fail if even one hard-coded host is outside the allowlist
if literals - trusted:
    raise SystemExit("download helper uses untrusted host")

To add a new download source, you have to spell out that host in setup_env.sh’s trusted_download_host(). For hard-coded URLs the static checker rejects anything outside the allowlist, and even for those routed through variables the runtime assert_trusted_url does. The aim is to make adding a source an “intentional decision” rather than an “accident.”

Summary

Validating production infrastructure and validating dotfiles are, I think, the same thing at heart. When a dangerous change lands, turn the build red on the spot. The only difference is that here it’s applied to the script that builds my development environment, not to a server.

• Run on two operating systems to surface environment-dependent breakage
• Don’t stop at dry-run; actually run it and check what came out with test -f
• After the destructive --clean, confirm that it properly rebuilds
• Confirm with grep / ! grep that an old block wrapped in the previous markers is reliably migrated to the new scheme

If You Want to Copy This, Start Here

There’s no need to copy the 1,358-line installer as-is. What’s effective are two things you can adopt in a much smaller form.

• Marker block + backup: If you have a script that appends to something like .zshrc, switch to a scheme that wraps your content in BEGIN/END and replaces only your own block, and take a timestamped backup before rewriting. Your additions won’t be duplicated on a rerun, and if something fails you can restore the state from just before.
• A smoke test that actually runs into an isolated HOME: A few lines that swap HOME for a temporary directory, run the real installer, and test -f the files that should exist. A level of assurance one step above dry-run, addable starting today.

It turns a development environment that “should work” into one that “I confirm works every time.” That alone makes a real difference in how it feels to sit down at a new machine.

For help with making setup scripts idempotent, managing .zshrc/.bashrc configuration with backups, setting up GitHub Actions CI, or pinning your supply chain, please reach out via Services.