chore(agent): phase 15_agent_mtls from TODO.md (protected exporter web servers)
The single TODO item (exporters as protected web servers, certs as auth, root certificate generated and managed by the collector) is now phase 15: cert management in meta, agent TLS mode, https collector dials, gated /api/certs surface, /certs admin page, installer + live rollout. LLM-Generated: true
This commit is contained in:
@@ -0,0 +1,44 @@
|
||||
# Phase 15 — Protected exporter web servers, auth by certificate
|
||||
|
||||
**Source:** `TODO.md` L3 — "I want all exporters to be protected web servers. The collector should generate and manage a root certificate that needs to be installed with each exporter. Use the certs as auth to each exporter."
|
||||
**Story:** n/a
|
||||
**Context:** `agent/host_agent.py` (the stdlib-only exporter web server; today: a bearer token via `--token`/`--token-file`, refuses an off-loopback bind without one, `ThreadingHTTPServer`, the `resolve_token` idiom), `src/dashboards/config.py` (`Target`/`parse_targets` — token in URL userinfo, `https://` URLs already accepted; **no change needed**), `src/dashboards/collectors.py` (`fetch_snapshot`, the one `UpstreamError`, redacted display URLs, the one shared `httpx.AsyncClient` from the lifespan), `src/dashboards/app.py` (the `PAGES` dict with the settings-page comment, `build_hub`/`build_context`, the lifespan that builds `ctx.auth`), `src/dashboards/api.py` (`AppContext`, the gated-write + event pattern), `src/dashboards/store.py` (`get_meta`/`set_meta`/`delete_meta` — the only durable key/value state, home of `auth.*`), `deploy/install-agent.sh` (the token-file idiom `0640 root:agent-user`, the `ProtectSystem=strict` unit with no `RuntimeDirectory`, the `--status` probe), `tests/test_agent.py` (the in-process agent harness), `tests/test_collectors.py`, `tests/test_deploy_scripts.py`, `tests/test_api.py` (the route walk, the settings-page pin, the orphan-id audit), `src/dashboards/web/pages/settings.html` + `src/dashboards/web/static/page-settings.js` (the "a page, not a dashboard" pattern the new `/certs` admin page follows)
|
||||
|
||||
## Objective
|
||||
|
||||
Every exporter (host agent) web server is protected end-to-end by certificates the dashboard itself generates and manages: the dashboard holds a root CA in the `meta` table, signs its own client cert and each agent's server cert, the agent serves TLS with a per-path client-cert check (only the dashboard can read it), the dashboard dials `https://` with its client cert, and a new `/certs` admin page lists every cert with its expiry and status and triggers rotation, revocation, and bundle re-delivery. The operator's stated follow-up — firewalling upsd 3051 and talking to the agent only — is the motivation, not part of this phase. The agent's bearer-token mode stays a compatibility mode; the dashboard's own login is a separate auth domain and is untouched.
|
||||
|
||||
**Scope (operator-confirmed):** "exporters" = the host agents this repo installs. gufo / vLLM / upsd are third-party services this repo does not deploy; their TLS (e.g. upsd `STARTTLS`) is a separate operator concern.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `14_store_performance` (todo) — **shared-file, run-order note**: 14 rewrites the hot paths in `store.py`/`api.py`; 15 only *adds* meta keys and new routes/pages. Apply on top of the file state at the time it runs; never restore a line the other phase owns.
|
||||
- `11_admin_login` (complete) — the session gate (invariant 12) for the new routes, the gated-write + CSRF + event pattern for the new `POST /api/certs/*`, the `meta` helpers, and the settings-page pattern `/certs` follows.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_cert_management.md` — `src/dashboards/certs.py`: the dashboard-managed root CA, client cert, per-agent server certs, revocation and reissue, all in `meta`; the `cryptography` dependency (dashboard-side only).
|
||||
2. `02_agent_tls.md` — `host_agent.py` TLS mode: one installed `--cert-bundle`, the per-path client-cert check, the bundle revocation list, `bundle_rev` in the snapshot; token mode unchanged.
|
||||
3. `03_collector_tls.md` — the dashboard dials `https://` agent URLs with its client cert, verifies against its root, refuses a revoked agent without dialing; error sentences stay redacted.
|
||||
4. `04_cert_api.md` — the gated `/api/certs` surface: list, per-host bundle, rotate, revoke, root-rotate; fixed-text events, no keys out of `meta`.
|
||||
5. `05_certs_admin_page.md` — the `/certs` admin page (the settings pattern): every cert with its expiry and status, issued vs deployed bundle rev, rotate/revoke/root-rotate actions, bundle download.
|
||||
6. `06_deploy_cert_bundle.md` — `install-agent.sh --cert-bundle` installs `/etc/dashboards-agent.certs` (`0600 root:agent-user`), the unit gains the flag + `RuntimeDirectory`, `--status` probes over TLS; the script pins move with it.
|
||||
7. `07_live_rollout.md` — the two live agents flip to TLS, `HOST_TARGETS` goes `https://`, the live suite + health table + screenshot prove it; the `driveripper` install procedure is documented.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_certs.py` (new) — the whole lifecycle against a dict `MetaStore`, exactly how `tests/test_auth.py` does it.
|
||||
- Integration: `tests/test_agent.py` (the TLS handshake matrix against the in-process agent), `tests/test_collectors.py` (the `https` dial, the revoked refusal with a no-dial spy, the error sentences), `tests/test_api.py` (the `/api/certs` shapes, the gate/CSRF on the new writes, the served `/certs` page pins), `tests/test_deploy_scripts.py` (the new flags/files/permissions).
|
||||
- Live: task 07 (AGENTS.md invariant 10 — the dev server stays up; screenshots in `data/logs/`).
|
||||
- Coverage: **>90%** on new/modified code; the 94% gate holds. `uv pip list --outdated` empty (the `cryptography` dependency — invariant 8).
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] a fresh database boots with a `meta` root CA, generated once and idempotent across restarts (pinned in `tests/test_certs.py`)
|
||||
- [ ] an agent run with a bundle answers `/snapshot` only to a client cert signed by its root; a wrong-CA or certless client is refused; a revoked serial gets the 403; `/health` stays probeable with the CA only
|
||||
- [ ] with `https://` `HOST_TARGETS`, the dev server's `host:`/`disks:` jobs are `ok`, and a revoked host says "certificate revoked" without any dial (spy-pinned)
|
||||
- [ ] signed-in `GET /certs` lists every cert with expiry and status; rotate/revoke/root-rotate work through the gated API; the bundle download is byte-identical to the stored bundle
|
||||
- [ ] the installer places the bundle `0600 root:agent-user`, the unit comes up in TLS mode, `--status` is green over TLS
|
||||
- [ ] `uv run pytest` green, `bash .agents/validate.sh` green, `uv pip list --outdated` empty
|
||||
- [ ] live: deskwork + gateway in TLS mode, `https://` dials, live suite green, screenshots in `data/logs/`
|
||||
- [ ] no behaviour change in completed phases: the token-mode agent is byte-identical, the engine/UPS polling is untouched, the public surface is unchanged (the route walk)
|
||||
Reference in New Issue
Block a user