add self-healing root SSH key systemd units

Headless Pis lose /root/.ssh/authorized_keys to imaging, first-boot tooling, or config management, which locks out the only SSH account that can fix it. - ensure-root-ssh-key: idempotent POSIX sh installer. Appends managed public keys from /usr/local/share/root-ssh-keys/authorized_keys, never removes or rewrites foreign keys, skips lines whose first field is not a known key algorithm, and enforces 0700 / 0600 root:root on the key dir and file. - ensure-root-ssh-key.service: oneshot, ordered Before=ssh.service so the key lands before the daemon starts; ConditionPathExists skips it when the managed keys file is absent. - ensure-root-ssh-key.path: PathChanged= + PathModified= on authorized_keys so deletes, renames, and in-place edits are healed at runtime, not just at boot. Writes only when a key is missing, so the unit settles after one no-op run. - authorized_keys.example: template with commented placeholder keys only; the real file stays on the host. - root-ssh-key.md: runbook with deploy, verification, revocation steps, and the gotcha that systemd has no PathMissing= key. LLM-Generated: true
2026-10-16 09:05:54 -04:00
parent 3f14b8ce14
commit bdee581480
5 changed files with 212 additions and 0 deletions
@@ -0,0 +1,61 @@
+#!/bin/sh
+# ensure-root-ssh-key - idempotently install managed SSH public keys into
+# /root/.ssh/authorized_keys. Runs at boot (ensure-root-ssh-key.service) and
+# can be run by hand at any time.
+#
+# Source of truth: /usr/local/share/root-ssh-keys/authorized_keys
+# One key per line, same format as an authorized_keys file. Blank lines and
+# lines starting with '#' are ignored. Keys already present are left alone;
+# foreign keys in authorized_keys are never removed.
+
+set -eu
+
+KEY_DIR="${KEY_DIR:-/root/.ssh}"
+AUTH_KEYS="$KEY_DIR/authorized_keys"
+MANAGED_KEYS="${MANAGED_KEYS:-/usr/local/share/root-ssh-keys/authorized_keys}"
+
+if [ ! -r "$MANAGED_KEYS" ]; then
+    echo "ensure-root-ssh-key: cannot read $MANAGED_KEYS" >&2
+    exit 1
+fi
+
+umask 077
+install -d -m 0700 "$KEY_DIR"
+[ -e "$AUTH_KEYS" ] || : > "$AUTH_KEYS"
+chmod 0600 "$AUTH_KEYS"
+chown root:root "$KEY_DIR" "$AUTH_KEYS"
+
+added=0
+checked=0
+while IFS= read -r line <&3; do
+    case "$line" in
+        '' | \#*) continue ;;
+    esac
+
+    # Basic shape check: <algo> <base64> [comment]
+    algo=${line%% *}
+    case "$algo" in
+        ssh-ed25519 | ssh-rsa | ssh-dss | ecdsa-sha2-* | sk-ecdsa-sha2-* | sk-ssh-ed25519-* | ssh-ed448) ;;
+        *)
+            echo "ensure-root-ssh-key: skipping unrecognised line: $line" >&2
+            continue
+            ;;
+    esac
+
+    checked=$((checked + 1))
+    if grep -qxF -- "$line" "$AUTH_KEYS" 2>/dev/null; then
+        continue
+    fi
+    printf '%s\n' "$line" >> "$AUTH_KEYS"
+    added=$((added + 1))
+    echo "ensure-root-ssh-key: installed key (${algo}, ${#line} chars)"
+done 3< "$MANAGED_KEYS"
+
+if [ "$added" -eq 0 ]; then
+    echo "ensure-root-ssh-key: $checked managed key(s) already present in $AUTH_KEYS"
+else
+    echo "ensure-root-ssh-key: added $added key(s), $checked managed key(s) total"
+fi
+
+chmod 0600 "$AUTH_KEYS"
+exit 0
@@ -0,0 +1,16 @@
+[Unit]
+Description=Watch /root/.ssh/authorized_keys and re-install the managed root SSH key
+Documentation=man:sshd(8)
+After=local-fs.target
+
+[Path]
+# PathChanged covers deletion / rename / creation of the file,
+# PathModified covers in-place edits (key stripped or file truncated).
+# ensure-root-ssh-key.service only writes when a managed key is missing, so
+# its own writes settle after one extra no-op run instead of looping.
+PathChanged=/root/.ssh/authorized_keys
+PathModified=/root/.ssh/authorized_keys
+Unit=ensure-root-ssh-key.service
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,15 @@
+[Unit]
+Description=Ensure managed SSH public keys are installed in /root/.ssh/authorized_keys
+Documentation=man:sshd(8)
+After=local-fs.target
+Before=ssh.service sshd.service
+ConditionPathExists=/usr/local/share/root-ssh-keys/authorized_keys
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/sbin/ensure-root-ssh-key
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,109 @@
+# Runbook: self-healing root SSH key
+
+`ensure-root-ssh-key.service` re-installs the managed SSH public key(s) into
+`/root/.ssh/authorized_keys` on every boot, and `ensure-root-ssh-key.path`
+re-applies it at runtime whenever that file is edited, deleted, or replaced.
+Useful when the key gets wiped by imaging, first-boot tooling, or config
+management — or when you need the key to come back after someone else edits the
+file.
+
+## Files on the host
+
+| Path | Purpose |
+| --- | --- |
+| `/etc/systemd/system/ensure-root-ssh-key.service` | `Type=oneshot`, `WantedBy=multi-user.target`, ordered `Before=ssh.service` so the key is in place before the daemon starts |
+| `/etc/systemd/system/ensure-root-ssh-key.path` | `PathChanged=` + `PathModified=` on `/root/.ssh/authorized_keys`, triggers the service |
+| `/usr/local/sbin/ensure-root-ssh-key` | Idempotent POSIX `sh` installer |
+| `/usr/local/share/root-ssh-keys/authorized_keys` | Source of truth — one public key per line, `#` comments allowed |
+
+## Behaviour
+
+- Appends any managed key that is missing; never removes or rewrites foreign keys.
+- Creates `/root/.ssh` (`0700`) and `authorized_keys` (`0600`, root:root) if absent.
+- Lines whose first field is not a known key algorithm are skipped with a warning.
+- Skipped, not failed, if the managed keys file is deleted.
+- Runtime self-healing: deleting, truncating, or renaming `authorized_keys` puts
+  the managed key(s) back within ~1s. That also means a managed key cannot be
+  revoked from the host alone — delete it from
+  `/usr/local/share/root-ssh-keys/authorized_keys` first.
+
+> Gotcha: systemd has no `PathMissing=` key — it is silently ignored with a
+> warning in `systemd-analyze verify`. `PathChanged=` is what fires on
+> delete/rename; `PathModified=` covers in-place appends and truncations. Set both.
+
+## Install
+
+```bash
+HOST=root@<pi-address>
+D=$PWD        # run the snippets from a checkout of this directory
+PUB=$HOME/.ssh/id_ed25519.pub          # the key that must always work
+
+ssh "$HOST" 'install -d -m 0755 /usr/local/sbin /usr/local/share/root-ssh-keys'
+tar -C "$D" -cf - ensure-root-ssh-key ensure-root-ssh-key.service ensure-root-ssh-key.path | \
+ssh "$HOST" 'tar -xf - -C /root &&
+  install -m 0755 /root/ensure-root-ssh-key /usr/local/sbin/ &&
+  install -m 0644 /root/ensure-root-ssh-key.service /root/ensure-root-ssh-key.path /etc/systemd/system/ &&
+  rm -f /root/ensure-root-ssh-key /root/ensure-root-ssh-key.service /root/ensure-root-ssh-key.path &&
+  systemctl daemon-reload'
+cat "$PUB" | ssh "$HOST" 'install -m 0644 /dev/stdin /usr/local/share/root-ssh-keys/authorized_keys'
+ssh "$HOST" 'systemctl enable --now ensure-root-ssh-key.service ensure-root-ssh-key.path'
+```
+
+Use `root-ssh-keys/authorized_keys.example` instead of `$PUB` when the host
+should carry more than one key.
+
+## Verify
+
+One-shot smoke test that proves both units without touching a live session
+(existing SSH sessions are unaffected; only new logins during the window would
+fail):
+
+```bash
+ssh "$HOST" 'cp -a /root/.ssh/authorized_keys /root/.ssh/authorized_keys.testbak
+  rm -f /root/.ssh/authorized_keys
+  systemctl start ensure-root-ssh-key.service && cat /root/.ssh/authorized_keys
+  rm -f /root/.ssh/authorized_keys.testbak'
+```
+
+Status and history:
+
+```bash
+ssh "$HOST" 'systemctl is-enabled ensure-root-ssh-key.service ensure-root-ssh-key.path
+  systemctl status ensure-root-ssh-key.service --no-pager | head -8
+  journalctl -u ensure-root-ssh-key.service -b --no-pager | tail'
+```
+
+The installer honours `KEY_DIR` and `MANAGED_KEYS`, so the create-if-missing and
+preserve-foreign-keys paths can be exercised in a scratch directory instead of
+`/root/.ssh`:
+
+```bash
+ssh "$HOST" 'install -d -m 0700 /tmp/k/src && cp /usr/local/share/root-ssh-keys/authorized_keys /tmp/k/src/
+  KEY_DIR=/tmp/k/root MANAGED_KEYS=/tmp/k/src/authorized_keys /usr/local/sbin/ensure-root-ssh-key
+  ls -ld /tmp/k/root /tmp/k/root/authorized_keys; rm -rf /tmp/k'
+```
+
+Checklist that has passed on a reference host:
+
+- `systemd-analyze verify` clean for both units, both `enabled`.
+- Repeated runs add no duplicates; a wiped file and a deleted file are both restored.
+- Missing `/root/.ssh` is created `0700`, file `0600`.
+- Foreign keys survive; no trigger loop once the path unit settles.
+- Boot ordering: `Before=ssh.service` is honoured inside the boot transaction;
+  prove it after a reboot with `systemd-analyze critical-chain ssh.service`.
+
+## Manage keys
+
+```bash
+ssh "$HOST" 'echo "<public key line>" >> /usr/local/share/root-ssh-keys/authorized_keys
+  systemctl restart ensure-root-ssh-key.service'
+```
+
+## Uninstall
+
+```bash
+ssh "$HOST" 'systemctl disable --now ensure-root-ssh-key.path ensure-root-ssh-key.service
+  rm -f /etc/systemd/system/ensure-root-ssh-key.{service,path} /usr/local/sbin/ensure-root-ssh-key
+  rm -rf /usr/local/share/root-ssh-keys
+  systemctl daemon-reload'
+```
@@ -0,0 +1,11 @@
+# Template for /usr/local/share/root-ssh-keys/authorized_keys on the host.
+# Keys listed here are (re)installed into /root/.ssh/authorized_keys at boot and
+# whenever that file changes. Copy to `authorized_keys`, add your public keys,
+# and keep the real file on the host - do not commit it.
+#
+# One key per line, authorized_keys format. Blank lines and lines starting with
+# '#' are ignored; a line whose first field is not a known key algorithm is
+# skipped with a warning.
+#
+# ssh-ed25519 AAAA<base64> comment@host
+# ssh-rsa     AAAA<base64> comment@host