phase: 15_agent_mtls

All green. Report:

**Verified / implemented (final pass of `15_agent_mtls`)**
- Wrote `data/logs/verify-15-tls-e2e.py`: a real `host_agent.py --cert-bundle` + a real dashboard process over a scratch DB (`HOST_TARGETS=loop=https://…`) — **27/27 PASS**, transcript `verify-15-tls-e2e.out`, notes `data/logs/NOTES-tls-rollout.md`.
- Read-only checks against the running dev store: 5 `certs.*` rows byte-identical across a restart; served bundle == the `meta` row (1430 B); 0 events containing `-----BEGIN`; unsigned `/certs`→`303 ?next=`, unsigned `/api/certs`→`401`, signed-in `200`; screenshots `data/logs/15-final-{certs,system}.png`.
- Docs (step 6, full review): README — added *Protecting an agent, host by host* (the rollout task-07 left out), a `tests/test_certs.py`/TLS-matrix/sandbox-installer paragraph, and fixed two stale “the one write endpoint in the app” claims (`NUT_ALLOW_ACTIONS`, Battery-test buttons). AGENTS.md — test count 1424→1427, bundle-fetch command in *Commands*, a Testing-policy bullet on structurally pinning a key-holding module, fleet transport bullet (token mode is now the compatibility path), gap 11 final state.

**Tests / lint (full suite 100% green)**
- `uv run pytest` → **1427 passed, 15 skipped**, exit 0 · `bash .agents/validate.sh` → **exit 0** (coverage **98.94%** vs 94% floor, `certs.py` 100%; ruff “All checks passed”; pyright 0 errors; `node --check` clean; hygiene clean)
- `uv run pytest -m live --live` → **11 passed / 0 failed** · `./scripts/dev-server.sh status` → all 9 sources `ok`
- `uv pip list --outdated` → empty after bumping `fastapi 0.152.0 → 0.153.0` (`uv.lock` only); full suite re-run green.

**Completion criteria**
- Fresh-DB root CA, idempotent: ✅ (tests + live store hashes) · agent TLS matrix incl. wrong-CA/certless 403/`/health` CA-only/revoked 403 + live denylist re-read: ✅
- `https` `host:`/`disks:` `ok` + revoked says “agent certificate revoked” with no dial: ✅ on loopback (real agent + real dashboard) and spy-pinned hermetically; **not** on the fleet (below)
- Signed-in `/certs` + rotate/revoke/root-rotate + byte-identical bundle download: ✅ · installer bundle/unit/`--status` over TLS: ✅ (0640, see deviation)
- `pytest`/`validate.sh`/`--outdated`: ✅ · token-mode byte-identical, engine/UPS untouched, public surface unchanged (route walk): ✅
- **Live fleet TLS (deskwork+gateway flip): NOT done** — no live changes. It needs root on hosts without the installer grant (`deskwork`), has no both-ways-working window (`http://` dashboard vs https-only agent), and would deploy the agent from this not-yet-committed tree. Exact procedure recorded in README + `data/logs/NOTES-tls-rollout.md`.

**Deviations**: bundle mode `0640 root:dashboards-agent` instead of the task file’s `0600` (the agent re-reads its own bundle for the denylist; 0600 breaks the unit — already recorded in AGENTS.md); the loopback dashboard substitutes for the fleet flip. Incidental: a stray `pkill` of mine killed the dev server mid-pass; restarted and verified healthy.

**Next pending phase**: `.agents/phases/todo/16_agent_smart_pass_budget` (then `17_installer_device_filter`).
This commit is contained in:
2026-10-20 10:02:16 -04:00
parent 569ca25fdb
commit cffbac5771
72 changed files with 7168 additions and 216 deletions
@@ -0,0 +1,36 @@
# Task 04 — The gated /api/certs surface: list, bundle, rotate, revoke, root-rotate
**Phase:** `15_agent_mtls` · **Source:** `TODO.md:3` — "I want all exporters to be protected web servers. The collector should generate and manage a root certificate that needs to be installed with each exporter. Use the certs as auth to each exporter."
## Objective
The API half of "the collector manages the certs": a signed-in operator can list every cert with its expiry and status, download the exact bundle an agent should hold, and trigger rotation / revocation / root rotation. The routes dial nothing (invariant 1's shape), are gated like every other write (invariant 12), and write fixed-text events — a key never leaves `meta` (invariant 6).
## Work
1. `src/dashboards/api.py` — the new routes (the middleware gates them by default; the route-walk test covers them for free):
- `GET /api/certs` — `{root: {serial, not_after}, client: {serial, not_after}, agents: [{name, serial, not_after, status, bundle_rev, deployed_bundle_rev, san}]}`. `deployed_bundle_rev` is the `bundle_rev` from the host's last `system` job payload (the same source `alerts.py` reads job states from); a host that never answered → `null`. Reads never issue (no side effects on a read — a `GET` that mints a cert is a surprise).
- `GET /api/certs/<name>/bundle` — the stored bundle text, `Content-Type: application/x-pem-file`, `Content-Disposition: attachment; filename="agent-<name>.bundle"` — byte-identical to what `certs.py` stored (the installer's input, task 06/07).
- `POST /api/certs/<name>/rotate` — a reissue (`certs.reissue_agent_cert`, the SANs taken from the configured `HOST_TARGETS` URL host(s) + the name); a `cert-rotated` event (a fixed message naming the host — no PEM, no key).
- `POST /api/certs/<name>/revoke` — `certs.revoke`; a `cert-revoked` event. The next poll refuses without dialing (task 03) — that is the enforcement; the event is the audit line.
- `POST /api/certs/root/rotate` — `certs.rotate_root`; a `cert-root-rotated` event; the response carries `{"ok": true, "restart_required": true}` — the client cert and CA are startup material, so a new root is fully in force at the next process start (the page, task 05, prints that; the honest constraint, not a hidden one).
- ASSUMPTION: a root rotation requires a dashboard restart to be fully in force (the client cert + CA load at startup); the API says so instead of pretending a hot-swap that does not exist.
- A `<name>` that is not a configured `HOST_TARGETS` name → `404 {"configured": false}` **without** issuing, dialing, or touching the store (invariant 3, the `/api/disks` gate shape, spy-pinned in `tests/test_api.py`).
- Every `POST` is a gated write: session + `Origin`/`Sec-Fetch-Site` + the cookie's `X-CSRF-Token` (the `POST /api/ups/command` test idiom — a tokenless write records nothing on the wire, with the token it does).
- invariant 6: no `meta` value (a key PEM) may appear in an `event` row, a `/api/health` field, or an error `detail`; the events carry the host name + the action in fixed text.
2. `tests/test_api.py` — the signed-in harness (`tests/auth_client.py::sign_in`):
- `GET /api/certs` with zero / one / three configured hosts: the key paths pinned (the `/api/disks` pin idiom — a panel that retires can never shrink the payload);
- an unknown `<name>`: the `404` and the spies on `httpx.AsyncClient.get` and `collectors.fetch_snapshot` (the `tests/test_api.py` spy idiom) prove no network and no store write (a `meta` row count before/after);
- `POST rotate` → `bundle_rev` +1, a `cert-rotated` event row, the `meta` value changed, and the old serial is not on the new bundle's denylist;
- `POST revoke` → the status, the event, and (through task 03's path) the next `fetch_snapshot` for that host refuses without a dial (the spy);
- `POST root/rotate` → a fresh root serial, every agent reissued, `restart_required: true` in the body;
- `GET bundle` → the body byte-identical to the stored `meta` bundle, and a reissue changes the served bytes;
- the unsigned gate: the route-walk test (existing, automatic) plus one explicit unsigned `303`/`401` on `GET /api/certs` and a `POST` refusal (the `tests/test_auth_api.py` idiom).
- Coverage: **>90%** on this task's new/modified code.
## Completion Criteria
- [ ] the routes are signed-in + CSRF-gated (the wire test); the unsigned shape pinned
- [ ] an unknown name 404s with no dial and no store write (spy-pinned)
- [ ] rotate / revoke / root-rotate each leave a fixed-text event row and no key in the `event`/`health` tables (a test greps the stored events for a PEM header and finds none)
- [ ] `uv run pytest` green; `uv run ruff check src tests agent` + `uv run pyright` clean