chore(agent): phase 15_agent_mtls from TODO.md (protected exporter web servers)

The single TODO item (exporters as protected web servers, certs as auth,
root certificate generated and managed by the collector) is now phase 15:
cert management in meta, agent TLS mode, https collector dials, gated
/api/certs surface, /certs admin page, installer + live rollout.

LLM-Generated: true
This commit is contained in:
2026-10-19 13:25:45 -04:00
parent 29524b3ebc
commit 04d934dd94
9 changed files with 291 additions and 2 deletions
@@ -0,0 +1,37 @@
# Task 02 — The agent serves a protected TLS web server from one installed bundle
**Phase:** `15_agent_mtls` · **Source:** `TODO.md:3` — "I want all exporters to be protected web servers. The collector should generate and manage a root certificate that needs to be installed with each exporter. Use the certs as auth to each exporter."
## Objective
`host_agent.py` gains a TLS mode that makes the web server protected by certificate: one installed bundle holds the server cert + key, the root CA, a bundle revision, and a revocation list; the agent requires a client cert signed by the root on every path except `/health` (mirroring the token-mode `/health` exemption, so an installer probe can reach `/health` with the CA only), and it reports the bundle revision it loaded so the dashboard can tell a stale bundle from a fresh one. The token mode is byte-identical to today.
## Work
1. `agent/host_agent.py` — stdlib only (an `ssl` import; no new third-party dependency):
- a new flag `--cert-bundle <file>` (env `AGENT_CERT_BUNDLE`): one PEM text containing the server cert, the server key, the root CA cert, a `# BUNDLE-REV <n>` line, and `# REVOKED <serial>` lines. `resolve_bundle(path) -> (context, bundle_rev, revoked: frozenset[str], error)` is a pure function (the `resolve_token` idiom): parse the PEM blocks by text, extract the three blocks, write them to a temp dir for `load_cert_chain` (into `/run/dashboards-agent` when present and writable — the unit's `RuntimeDirectory`, task 06 — else `tempfile.mkdtemp`), and build the context.
- The TLS context: `ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)`, `load_cert_chain`, `load_verify_locations(ca)`, `verify_mode = ssl.CERT_OPTIONAL` — the handshake is optional and the **handler** enforces it: on every path except `/health`, `self.connection.getpeercert()` must carry a verified certificate (subject present), else `403 {"ok": false, "detail": "client certificate required"}`; that cert's serial must not be in the bundle's `# REVOKED` set, else `403 {"ok": false, "detail": "client certificate revoked"}`. The denylist file is re-read on mtime change per request (a small file; a redeployed bundle takes effect without a restart — the reason the `# REVOKED` lines live in the bundle at all).
- The startup gate: off-loopback with neither a token nor a bundle → the existing exit-2 refusal (its message now names the bundle too); a bundle present → the off-loopback bind is allowed (the client cert is the protection); a token plus a bundle → the token check is skipped on the TLS path (the handshake is the auth).
- The snapshot payload gains two **additive** top-level fields in TLS mode: `tls: true` and `bundle_rev: <n>` (absent in token mode — an additive field never changes an existing shape); the `serve()` start line says `tls (client cert required, rev <n>)`.
- `AGENT_VERSION` 1.0.0 → 1.1.0; the docstring's deployment paragraph gains one line for the TLS mode (no flavour text — the AGENTS.md audit rule).
- ASSUMPTION: `CERT_OPTIONAL` + the per-path check (not `CERT_REQUIRED`) is what keeps `/health` probeable with the CA only — the installer's `--status` and the systemd probe (task 06) cannot present a client cert the installer does not hold, and a probe that cannot health-check is a dead install.
- ASSUMPTION: the agent never generates or renews certificates (stdlib cannot mint x509); the bundle is the artifact the dashboard signs and the installer places.
2. `tests/test_agent.py` — the existing in-process harness (`serve()` in a thread on `127.0.0.1`, `urllib`/`ssl` clients), with a test CA + leafs generated by `src/dashboards/certs.py` over a dict store (no second generator):
- a client with a matching client cert → `/snapshot` 200 with `tls: true` and `bundle_rev` equal to the bundle's;
- a client with no cert → the handshake fails or a 403 on a non-`/health` path; `/health` still 200 with a CA-only context (the probe path);
- a client signed by a *different* CA → refused;
- a client whose serial is in the bundle's `# REVOKED` list → 403 "revoked"; rewrite the bundle file without the line (mtime change) → the same client is accepted on the next request (the denylist re-read);
- a token-mode agent (no bundle): the existing tests unchanged and green (byte-identical behaviour — no `tls`/`bundle_rev` fields);
- `resolve_bundle` error paths: a missing file, an empty file, a bundle missing the key block, a bundle missing the `# BUNDLE-REV` line (an error — a revless bundle is not one this app signed; ASSUMPTION: the `# BUNDLE-REV` line is mandatory).
3. `deploy/` is untouched here (task 06); the non-systemd fallback (`tempfile.mkdtemp`) keeps dev and tests working.
## Testing & Quality
- the `tests/test_agent.py` matrix above; the existing token-mode tests stay green unchanged.
- Coverage: **>90%** on this task's new/modified code.
## Completion Criteria
- [ ] a TLS-mode agent on a loopback port answers the full handshake matrix (tests) and reports `bundle_rev`
- [ ] a token-mode agent is byte-identical to today (existing tests green, no new fields present)
- [ ] `uv run pytest` green; `uv run ruff check src tests agent` + `uv run pyright` clean