homelab/active/device_raspberry_pi/systemd/root-ssh-key.md

Runbook: self-healing root SSH key

ensure-root-ssh-key.service re-installs the managed SSH public key(s) into /root/.ssh/authorized_keys on every boot, and ensure-root-ssh-key.path re-applies it at runtime whenever that file is edited, deleted, or replaced. Useful when the key gets wiped by imaging, first-boot tooling, or config management — or when you need the key to come back after someone else edits the file.

Files on the host

Path	Purpose
/etc/systemd/system/ensure-root-ssh-key.service	Type=oneshot, WantedBy=multi-user.target, ordered Before=ssh.service so the key is in place before the daemon starts
/etc/systemd/system/ensure-root-ssh-key.path	PathChanged= + PathModified= on /root/.ssh/authorized_keys, triggers the service
/usr/local/sbin/ensure-root-ssh-key	Idempotent POSIX sh installer
/usr/local/share/root-ssh-keys/authorized_keys	Source of truth — one public key per line, # comments allowed

Behaviour

• Appends any managed key that is missing; never removes or rewrites foreign keys.
• Creates /root/.ssh (0700) and authorized_keys (0600, root:root) if absent.
• Lines whose first field is not a known key algorithm are skipped with a warning.
• Skipped, not failed, if the managed keys file is deleted.
• Runtime self-healing: deleting, truncating, or renaming authorized_keys puts the managed key(s) back within ~1s. That also means a managed key cannot be revoked from the host alone — delete it from /usr/local/share/root-ssh-keys/authorized_keys first.

Gotcha: systemd has no PathMissing= key — it is silently ignored with a warning in systemd-analyze verify. PathChanged= is what fires on delete/rename; PathModified= covers in-place appends and truncations. Set both.

Install

HOST=root@<pi-address>
D=$PWD        # run the snippets from a checkout of this directory
PUB=$HOME/.ssh/id_ed25519.pub          # the key that must always work

ssh "$HOST" 'install -d -m 0755 /usr/local/sbin /usr/local/share/root-ssh-keys'
tar -C "$D" -cf - ensure-root-ssh-key ensure-root-ssh-key.service ensure-root-ssh-key.path | \
ssh "$HOST" 'tar -xf - -C /root &&
  install -m 0755 /root/ensure-root-ssh-key /usr/local/sbin/ &&
  install -m 0644 /root/ensure-root-ssh-key.service /root/ensure-root-ssh-key.path /etc/systemd/system/ &&
  rm -f /root/ensure-root-ssh-key /root/ensure-root-ssh-key.service /root/ensure-root-ssh-key.path &&
  systemctl daemon-reload'
cat "$PUB" | ssh "$HOST" 'install -m 0644 /dev/stdin /usr/local/share/root-ssh-keys/authorized_keys'
ssh "$HOST" 'systemctl enable --now ensure-root-ssh-key.service ensure-root-ssh-key.path'

Use root-ssh-keys/authorized_keys.example instead of $PUB when the host should carry more than one key.

Verify

One-shot smoke test that proves both units without touching a live session (existing SSH sessions are unaffected; only new logins during the window would fail):

ssh "$HOST" 'cp -a /root/.ssh/authorized_keys /root/.ssh/authorized_keys.testbak
  rm -f /root/.ssh/authorized_keys
  systemctl start ensure-root-ssh-key.service && cat /root/.ssh/authorized_keys
  rm -f /root/.ssh/authorized_keys.testbak'

Status and history:

ssh "$HOST" 'systemctl is-enabled ensure-root-ssh-key.service ensure-root-ssh-key.path
  systemctl status ensure-root-ssh-key.service --no-pager | head -8
  journalctl -u ensure-root-ssh-key.service -b --no-pager | tail'

The installer honours KEY_DIR and MANAGED_KEYS, so the create-if-missing and preserve-foreign-keys paths can be exercised in a scratch directory instead of /root/.ssh:

ssh "$HOST" 'install -d -m 0700 /tmp/k/src && cp /usr/local/share/root-ssh-keys/authorized_keys /tmp/k/src/
  KEY_DIR=/tmp/k/root MANAGED_KEYS=/tmp/k/src/authorized_keys /usr/local/sbin/ensure-root-ssh-key
  ls -ld /tmp/k/root /tmp/k/root/authorized_keys; rm -rf /tmp/k'

Checklist that has passed on a reference host:

• systemd-analyze verify clean for both units, both enabled.
• Repeated runs add no duplicates; a wiped file and a deleted file are both restored.
• Missing /root/.ssh is created 0700, file 0600.
• Foreign keys survive; no trigger loop once the path unit settles.
• Boot ordering: Before=ssh.service is honoured inside the boot transaction; prove it after a reboot with systemd-analyze critical-chain ssh.service.

Manage keys

ssh "$HOST" 'echo "<public key line>" >> /usr/local/share/root-ssh-keys/authorized_keys
  systemctl restart ensure-root-ssh-key.service'

Uninstall

ssh "$HOST" 'systemctl disable --now ensure-root-ssh-key.path ensure-root-ssh-key.service
  rm -f /etc/systemd/system/ensure-root-ssh-key.{service,path} /usr/local/sbin/ensure-root-ssh-key
  rm -rf /usr/local/share/root-ssh-keys
  systemctl daemon-reload'