All green. Report:
**Verified / implemented (final pass of `15_agent_mtls`)**
- Wrote `data/logs/verify-15-tls-e2e.py`: a real `host_agent.py --cert-bundle` + a real dashboard process over a scratch DB (`HOST_TARGETS=loop=https://…`) — **27/27 PASS**, transcript `verify-15-tls-e2e.out`, notes `data/logs/NOTES-tls-rollout.md`.
- Read-only checks against the running dev store: 5 `certs.*` rows byte-identical across a restart; served bundle == the `meta` row (1430 B); 0 events containing `-----BEGIN`; unsigned `/certs`→`303 ?next=`, unsigned `/api/certs`→`401`, signed-in `200`; screenshots `data/logs/15-final-{certs,system}.png`.
- Docs (step 6, full review): README — added *Protecting an agent, host by host* (the rollout task-07 left out), a `tests/test_certs.py`/TLS-matrix/sandbox-installer paragraph, and fixed two stale “the one write endpoint in the app” claims (`NUT_ALLOW_ACTIONS`, Battery-test buttons). AGENTS.md — test count 1424→1427, bundle-fetch command in *Commands*, a Testing-policy bullet on structurally pinning a key-holding module, fleet transport bullet (token mode is now the compatibility path), gap 11 final state.
**Tests / lint (full suite 100% green)**
- `uv run pytest` → **1427 passed, 15 skipped**, exit 0 · `bash .agents/validate.sh` → **exit 0** (coverage **98.94%** vs 94% floor, `certs.py` 100%; ruff “All checks passed”; pyright 0 errors; `node --check` clean; hygiene clean)
- `uv run pytest -m live --live` → **11 passed / 0 failed** · `./scripts/dev-server.sh status` → all 9 sources `ok`
- `uv pip list --outdated` → empty after bumping `fastapi 0.152.0 → 0.153.0` (`uv.lock` only); full suite re-run green.
**Completion criteria**
- Fresh-DB root CA, idempotent: ✅ (tests + live store hashes) · agent TLS matrix incl. wrong-CA/certless 403/`/health` CA-only/revoked 403 + live denylist re-read: ✅
- `https` `host:`/`disks:` `ok` + revoked says “agent certificate revoked” with no dial: ✅ on loopback (real agent + real dashboard) and spy-pinned hermetically; **not** on the fleet (below)
- Signed-in `/certs` + rotate/revoke/root-rotate + byte-identical bundle download: ✅ · installer bundle/unit/`--status` over TLS: ✅ (0640, see deviation)
- `pytest`/`validate.sh`/`--outdated`: ✅ · token-mode byte-identical, engine/UPS untouched, public surface unchanged (route walk): ✅
- **Live fleet TLS (deskwork+gateway flip): NOT done** — no live changes. It needs root on hosts without the installer grant (`deskwork`), has no both-ways-working window (`http://` dashboard vs https-only agent), and would deploy the agent from this not-yet-committed tree. Exact procedure recorded in README + `data/logs/NOTES-tls-rollout.md`.
**Deviations**: bundle mode `0640 root:dashboards-agent` instead of the task file’s `0600` (the agent re-reads its own bundle for the denylist; 0600 breaks the unit — already recorded in AGENTS.md); the loopback dashboard substitutes for the fleet flip. Incidental: a stray `pkill` of mine killed the dev server mid-pass; restarted and verified healthy.
**Next pending phase**: `.agents/phases/todo/16_agent_smart_pass_budget` (then `17_installer_device_filter`).
4.7 KiB
4.7 KiB
Task 04 — The gated /api/certs surface: list, bundle, rotate, revoke, root-rotate
Phase: 15_agent_mtls · Source: TODO.md:3 — "I want all exporters to be protected web servers. The collector should generate and manage a root certificate that needs to be installed with each exporter. Use the certs as auth to each exporter."
Objective
The API half of "the collector manages the certs": a signed-in operator can list every cert with its expiry and status, download the exact bundle an agent should hold, and trigger rotation / revocation / root rotation. The routes dial nothing (invariant 1's shape), are gated like every other write (invariant 12), and write fixed-text events — a key never leaves meta (invariant 6).
Work
src/dashboards/api.py— the new routes (the middleware gates them by default; the route-walk test covers them for free):GET /api/certs—{root: {serial, not_after}, client: {serial, not_after}, agents: [{name, serial, not_after, status, bundle_rev, deployed_bundle_rev, san}]}.deployed_bundle_revis thebundle_revfrom the host's lastsystemjob payload (the same sourcealerts.pyreads job states from); a host that never answered →null. Reads never issue (no side effects on a read — aGETthat mints a cert is a surprise).GET /api/certs/<name>/bundle— the stored bundle text,Content-Type: application/x-pem-file,Content-Disposition: attachment; filename="agent-<name>.bundle"— byte-identical to whatcerts.pystored (the installer's input, task 06/07).POST /api/certs/<name>/rotate— a reissue (certs.reissue_agent_cert, the SANs taken from the configuredHOST_TARGETSURL host(s) + the name); acert-rotatedevent (a fixed message naming the host — no PEM, no key).POST /api/certs/<name>/revoke—certs.revoke; acert-revokedevent. The next poll refuses without dialing (task 03) — that is the enforcement; the event is the audit line.POST /api/certs/root/rotate—certs.rotate_root; acert-root-rotatedevent; the response carries{"ok": true, "restart_required": true}— the client cert and CA are startup material, so a new root is fully in force at the next process start (the page, task 05, prints that; the honest constraint, not a hidden one).- ASSUMPTION: a root rotation requires a dashboard restart to be fully in force (the client cert + CA load at startup); the API says so instead of pretending a hot-swap that does not exist.
- A
<name>that is not a configuredHOST_TARGETSname →404 {"configured": false}without issuing, dialing, or touching the store (invariant 3, the/api/disksgate shape, spy-pinned intests/test_api.py). - Every
POSTis a gated write: session +Origin/Sec-Fetch-Site+ the cookie'sX-CSRF-Token(thePOST /api/ups/commandtest idiom — a tokenless write records nothing on the wire, with the token it does). - invariant 6: no
metavalue (a key PEM) may appear in aneventrow, a/api/healthfield, or an errordetail; the events carry the host name + the action in fixed text.
tests/test_api.py— the signed-in harness (tests/auth_client.py::sign_in):GET /api/certswith zero / one / three configured hosts: the key paths pinned (the/api/diskspin idiom — a panel that retires can never shrink the payload);- an unknown
<name>: the404and the spies onhttpx.AsyncClient.getandcollectors.fetch_snapshot(thetests/test_api.pyspy idiom) prove no network and no store write (ametarow count before/after); POST rotate→bundle_rev+1, acert-rotatedevent row, themetavalue changed, and the old serial is not on the new bundle's denylist;POST revoke→ the status, the event, and (through task 03's path) the nextfetch_snapshotfor that host refuses without a dial (the spy);POST root/rotate→ a fresh root serial, every agent reissued,restart_required: truein the body;GET bundle→ the body byte-identical to the storedmetabundle, and a reissue changes the served bytes;- the unsigned gate: the route-walk test (existing, automatic) plus one explicit unsigned
303/401onGET /api/certsand aPOSTrefusal (thetests/test_auth_api.pyidiom).
- Coverage: >90% on this task's new/modified code.
Completion Criteria
- the routes are signed-in + CSRF-gated (the wire test); the unsigned shape pinned
- an unknown name 404s with no dial and no store write (spy-pinned)
- rotate / revoke / root-rotate each leave a fixed-text event row and no key in the
event/healthtables (a test greps the stored events for a PEM header and finds none) uv run pytestgreen;uv run ruff check src tests agent+uv run pyrightclean