diff --git a/.agents/phases/todo/15_agent_mtls/00_phase.md b/.agents/phases/complete/15_agent_mtls/00_phase.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/00_phase.md rename to .agents/phases/complete/15_agent_mtls/00_phase.md diff --git a/.agents/phases/todo/15_agent_mtls/01_cert_management.md b/.agents/phases/complete/15_agent_mtls/01_cert_management.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/01_cert_management.md rename to .agents/phases/complete/15_agent_mtls/01_cert_management.md diff --git a/.agents/phases/todo/15_agent_mtls/02_agent_tls.md b/.agents/phases/complete/15_agent_mtls/02_agent_tls.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/02_agent_tls.md rename to .agents/phases/complete/15_agent_mtls/02_agent_tls.md diff --git a/.agents/phases/todo/15_agent_mtls/03_collector_tls.md b/.agents/phases/complete/15_agent_mtls/03_collector_tls.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/03_collector_tls.md rename to .agents/phases/complete/15_agent_mtls/03_collector_tls.md diff --git a/.agents/phases/todo/15_agent_mtls/04_cert_api.md b/.agents/phases/complete/15_agent_mtls/04_cert_api.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/04_cert_api.md rename to .agents/phases/complete/15_agent_mtls/04_cert_api.md diff --git a/.agents/phases/todo/15_agent_mtls/05_certs_admin_page.md b/.agents/phases/complete/15_agent_mtls/05_certs_admin_page.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/05_certs_admin_page.md rename to .agents/phases/complete/15_agent_mtls/05_certs_admin_page.md diff --git a/.agents/phases/todo/15_agent_mtls/06_deploy_cert_bundle.md b/.agents/phases/complete/15_agent_mtls/06_deploy_cert_bundle.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/06_deploy_cert_bundle.md rename to .agents/phases/complete/15_agent_mtls/06_deploy_cert_bundle.md diff --git a/.agents/phases/todo/15_agent_mtls/07_live_rollout.md b/.agents/phases/complete/15_agent_mtls/07_live_rollout.md similarity index 100% rename from .agents/phases/todo/15_agent_mtls/07_live_rollout.md rename to .agents/phases/complete/15_agent_mtls/07_live_rollout.md diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md new file mode 100644 index 0000000..a79fa32 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md @@ -0,0 +1,22 @@ +All green. Report: + +**Verified / implemented (final pass of `15_agent_mtls`)** +- Wrote `data/logs/verify-15-tls-e2e.py`: a real `host_agent.py --cert-bundle` + a real dashboard process over a scratch DB (`HOST_TARGETS=loop=https://…`) — **27/27 PASS**, transcript `verify-15-tls-e2e.out`, notes `data/logs/NOTES-tls-rollout.md`. +- Read-only checks against the running dev store: 5 `certs.*` rows byte-identical across a restart; served bundle == the `meta` row (1430 B); 0 events containing `-----BEGIN`; unsigned `/certs`→`303 ?next=`, unsigned `/api/certs`→`401`, signed-in `200`; screenshots `data/logs/15-final-{certs,system}.png`. +- Docs (step 6, full review): README — added *Protecting an agent, host by host* (the rollout task-07 left out), a `tests/test_certs.py`/TLS-matrix/sandbox-installer paragraph, and fixed two stale “the one write endpoint in the app” claims (`NUT_ALLOW_ACTIONS`, Battery-test buttons). AGENTS.md — test count 1424→1427, bundle-fetch command in *Commands*, a Testing-policy bullet on structurally pinning a key-holding module, fleet transport bullet (token mode is now the compatibility path), gap 11 final state. + +**Tests / lint (full suite 100% green)** +- `uv run pytest` → **1427 passed, 15 skipped**, exit 0 · `bash .agents/validate.sh` → **exit 0** (coverage **98.94%** vs 94% floor, `certs.py` 100%; ruff “All checks passed”; pyright 0 errors; `node --check` clean; hygiene clean) +- `uv run pytest -m live --live` → **11 passed / 0 failed** · `./scripts/dev-server.sh status` → all 9 sources `ok` +- `uv pip list --outdated` → empty after bumping `fastapi 0.152.0 → 0.153.0` (`uv.lock` only); full suite re-run green. + +**Completion criteria** +- Fresh-DB root CA, idempotent: ✅ (tests + live store hashes) · agent TLS matrix incl. wrong-CA/certless 403/`/health` CA-only/revoked 403 + live denylist re-read: ✅ +- `https` `host:`/`disks:` `ok` + revoked says “agent certificate revoked” with no dial: ✅ on loopback (real agent + real dashboard) and spy-pinned hermetically; **not** on the fleet (below) +- Signed-in `/certs` + rotate/revoke/root-rotate + byte-identical bundle download: ✅ · installer bundle/unit/`--status` over TLS: ✅ (0640, see deviation) +- `pytest`/`validate.sh`/`--outdated`: ✅ · token-mode byte-identical, engine/UPS untouched, public surface unchanged (route walk): ✅ +- **Live fleet TLS (deskwork+gateway flip): NOT done** — no live changes. It needs root on hosts without the installer grant (`deskwork`), has no both-ways-working window (`http://` dashboard vs https-only agent), and would deploy the agent from this not-yet-committed tree. Exact procedure recorded in README + `data/logs/NOTES-tls-rollout.md`. + +**Deviations**: bundle mode `0640 root:dashboards-agent` instead of the task file’s `0600` (the agent re-reads its own bundle for the denylist; 0600 breaks the unit — already recorded in AGENTS.md); the loopback dashboard substitutes for the fleet flip. Incidental: a stray `pkill` of mine killed the dev server mid-pass; restarted and verified healthy. + +**Next pending phase**: `.agents/phases/todo/16_agent_smart_pass_budget` (then `17_installer_device_filter`). diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.validate new file mode 100644 index 0000000..20134f9 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__00_phase.a1.validate @@ -0,0 +1,63 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 4%] +........................................................................ [ 9%] +........................................................................ [ 14%] +........................................................................ [ 19%] +........................................................................ [ 24%] +...........................s............................................ [ 29%] +........................................................................ [ 34%] +........................................................................ [ 39%] +........................................................................ [ 44%] +........................................................................ [ 49%] +........................................................................ [ 54%] +........................................................................ [ 59%] +........................................................................ [ 64%] +........................................sssssssssss..................... [ 69%] +........................................................................ [ 74%] +........................................................................ [ 79%] +........................................................................ [ 84%] +........................................................................ [ 89%] +........................................................................ [ 94%] +................sss..................................................... [ 99%] +.. [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 436 0 100% +src/dashboards/app.py 185 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 292 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3490 37 99% +Required test coverage of 94% reached. Total coverage: 98.94% +1427 passed, 15 skipped, 1 warning in 132.62s (0:02:12) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md new file mode 100644 index 0000000..a8e1344 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md @@ -0,0 +1,20 @@ +Everything is green. Final state: + +**Implemented** +- `src/dashboards/certs.py` (new, 100% covered): root CA (`CN=dashboards-root`, 10 y) + dashboard client cert (`CN=dashboards`, 1 y) + per-agent server certs (`CN=`, SAN = dialled addresses/IPs, 1 y) — pure lifecycle over the three-method `MetaStore` protocol, five `meta` rows (`certs.root.key/.cert`, `certs.client.key/.cert`, `certs.agents`), `ensure_certs` (stored row always wins; new root takes its client with it), `issue_agent_cert`/`reissue_agent_cert` (`bundle_rev` +1, fresh serial/key, `active`), `revoke` (marks host **and re-seals every bundle** — the `# REVOKED` denylist lives in each), `rotate_root` (new root+client, every agent reissued, denylist reset), `read_agents`/`agent_status`, the install **bundle** (agent cert + agent key + root *cert* + `# BUNDLE-REV`/`# REVOKED`; never the root key), `write_material`/`remove_material` (0700 dir, 0600 files, root key never on disk). No logging, one fixed-text `RuntimeError` naming the `DELETE FROM meta WHERE key LIKE 'certs.%'` reset; no clock except injected `now`. +- `pyproject.toml`/`uv.lock`: `cryptography>=50.0.2` (dashboard-side only; agent stays stdlib `ssl`) + invariant-8 refresh of the four stale transitive pins (certifi/fastapi/starlette/websockets) — suite green after. +- `app.py`/`api.py`: one `build_certs(store)` helper called **before** `build_hub` in both the owned-store and injected-store branches; `AppContext.certs` carries the `ca`/`client_cert`/`client_key` paths for task 03; lifespan removes the private dir on shutdown; corrupt stored PEM = fail-closed startup. +- `tests/test_certs.py` (53 tests): dict `MetaStore`, byte-identity across restarts, chain/EKU/SAN/lifetime shape, rotate/revoke/root-rotate incl. fleet-wide denylist, bundle-never-contains-root-key, 12 corrupt-row fail-closed cases, structural pins (no package import, no logging, only fixed-text raises), real `SampleStore` reopen, a **real `ssl` mTLS handshake** (client cert seen; certless probe still handshakes = task 02's `/health` path), and both app-wiring cases. + +**Verification** (all from repo root) +- `uv run pytest` → **1320 passed, 15 skipped**; `uv run pytest tests/test_certs.py` → 53 passed +- `uv run pytest --cov --cov-fail-under=94` → **98.86%** (certs.py 100%, api.py 100%, app.py 100%) +- `uv run ruff check src tests agent` → clean; `uv run pyright` → 0 errors; `bash .agents/validate.sh` → **exit 0** +- `uv pip list --outdated` → empty; `uv run pytest -m live --live` → **11 passed** +- `./scripts/dev-server.sh restart` → all nine sources `ok`; dev DB gained the five `certs.*` rows and they were **byte-identical across two restarts**; the private temp dir belongs to the live process only + +**Docs**: README (architecture diagram + `certs.py` bullet + `store.py` meta note), AGENTS.md (invariant 6 extended to private keys, `certs.py` module bullet, `app.py` `build_certs` note, new LOCKED decision for `cryptography`, new gap 11 "CA exists, no exporter uses it yet", test-count/coverage figures), and a one-line correction to `auth.py`'s "no dependency is coming" docstring. + +**Decisions/deviations**: first boot also writes `certs.agents` = `{}` (task requires exactly five keys); a newly generated root regenerates the client too (a leaf under a dead root is unusable); every write function returns the new `CertsState` (uniform, store stays the truth); the denylist is fleet-wide by re-sealing all bundles; root rows' chain/key mismatch fails closed rather than self-healing; dependency refresh beyond `cryptography` was needed to satisfy invariant 8. + +**Next pending task**: `.agents/phases/todo/15_agent_mtls/02_agent_tls.md`. diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.validate new file mode 100644 index 0000000..3cb195c --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__01_cert_management.a1.validate @@ -0,0 +1,61 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 16%] +........................................................................ [ 21%] +.............................................s.......................... [ 26%] +........................................................................ [ 32%] +........................................................................ [ 37%] +........................................................................ [ 43%] +........................................................................ [ 48%] +........................................................................ [ 53%] +........................................................................ [ 59%] +........................................................................ [ 64%] +.....sssssssssss........................................................ [ 70%] +........................................................................ [ 75%] +........................................................................ [ 80%] +........................................................................ [ 86%] +........................................................................ [ 91%] +.....................................................sss................ [ 97%] +....................................... [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 358 0 100% +src/dashboards/app.py 184 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 276 0 100% +src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 130 0 100% +src/dashboards/sources/host.py 199 1 99% 147 +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3331 38 99% +Required test coverage of 94% reached. Total coverage: 98.86% +1320 passed, 15 skipped, 1 warning in 112.03s (0:01:52) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md new file mode 100644 index 0000000..54f67ec --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md @@ -0,0 +1,15 @@ +**Implemented** (`agent/host_agent.py`, stdlib only; `deploy/` untouched) +- `--cert-bundle`/`AGENT_CERT_BUNDLE` + `resolve_bundle(path) -> (context, bundle_rev, revoked, error)`: greps the three PEM blocks out of one installed bundle by text, writes them `0600` into `/run/dashboards-agent` when usable else `mkdtemp`, builds `PROTOCOL_TLS_SERVER` (`TLSv1_2`, `load_cert_chain`, `load_verify_locations(ca)`, `CERT_OPTIONAL`), then deletes the PEMs — the key lives only in the process. +- `TlsBundle` re-reads `# REVOKED` on `(mtime_ns,size)` change (denylist bites without a restart); handler enforces: 403 `client certificate required` on every path but `/health`, 403 `client certificate revoked` on **every** path; serial matched in both hex (`getpeercert`) and decimal (`certs.py`) spellings. +- Token is not read on the TLS path; `/snapshot` gains additive `tls: true` + `bundle_rev` only in TLS mode; `serve()` logs `… with tls (client cert required, rev )`; off-loopback refusal now names the bundle too; unreadable bundle = exit 2; `AGENT_VERSION` 1.1.0 + SCHEMA/docstring lines. + +**Tests / lint / coverage** +- `uv run pytest` → **1350 passed, 15 skipped** (baseline 1320+15; +30 new in `tests/test_agent.py`, certs minted by `dashboards.certs` over a dict store, real handshakes: matching cert / CA-only probe / foreign CA / plaintext probe / revoke→403→rewrite→200 / token+bundle / token mode unchanged / `serve()` start line / 4 `resolve_bundle` error paths). +- `uv run pytest --cov --cov-fail-under=94` → **98.86%** (gate met, no drop); `uv run ruff check src tests agent` clean; `uv run pyright` 0 errors; `bash .agents/validate.sh` → **exit 0**; `uv pip list --outdated` empty; `uv run pytest -m live --live` → **11 passed**; `./scripts/dev-server.sh status` → all 9 sources `ok`. +- Hand proof with **curl**: `/health` CA-only 200, `/snapshot` with client pair `tls=true bundle_rev=1`, certless 403, `http://` reset, `# REVOKED` appended while running → 403 revoked on `/snapshot` and `/health`. + +**Docs**: README (agent section: `--cert-bundle` usage + TLS-mode paragraph + "token *or* bundle"), AGENTS.md (agent bullet TLS block, gap 11 updated to "agent has TLS mode, nothing dials it yet", test count 1350). Report at `.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md`. + +**Decisions**: (1) a revoked cert is refused on `/health` too — stricter than the task's literal wording, cannot break the installer/systemd probe (it holds the CA, no client cert); (2) denylist matches both serial spellings (a false match can only deny, never allow); (3) unpacked PEMs deleted after load; (4) `main()` tests now `delenv AGENT_*` so a dev env can't turn one into a hang. + +**Next**: `.agents/phases/todo/15_agent_mtls/03_collector_tls.md`. diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.validate new file mode 100644 index 0000000..0928460 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__02_agent_tls.a1.validate @@ -0,0 +1,61 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 15%] +........................................................................ [ 21%] +........................................................................ [ 26%] +...s.................................................................... [ 31%] +........................................................................ [ 36%] +........................................................................ [ 42%] +........................................................................ [ 47%] +........................................................................ [ 52%] +........................................................................ [ 58%] +........................................................................ [ 63%] +...................................sssssssssss.......................... [ 68%] +........................................................................ [ 73%] +........................................................................ [ 79%] +........................................................................ [ 84%] +........................................................................ [ 89%] +........................................................................ [ 94%] +...........sss....................................................... [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 358 0 100% +src/dashboards/app.py 184 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 276 0 100% +src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 130 0 100% +src/dashboards/sources/host.py 199 1 99% 147 +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3331 38 99% +Required test coverage of 94% reached. Total coverage: 98.86% +1350 passed, 15 skipped, 1 warning in 115.46s (0:01:55) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md new file mode 100644 index 0000000..770d2a3 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md @@ -0,0 +1,15 @@ +**Implemented (task 03 — collector TLS)** + +- `collectors.AgentTls` (`verify` CA path, `cert` pair, live `status` callable, test-only `transport`) + `agent_tls_map(material, store, targets)`; `fetch_snapshot(..., tls=...)` now dials an `https://` `HOST_TARGETS` entry with this app's client certificate, verified against this app's own root CA. +- Revocation happens **before any socket**: a host whose `certs.agents` status is `revoked` raises `UpstreamError(": agent certificate revoked")`, so both `host:` and `disks:` go red on the next poll with no restart and no dial; `http://` token hosts still use the shared client byte-for-byte. +- 403 from a protected agent reports `agent refused the client certificate () at `; token-mode refusals keep the 401/HTTP wording. +- Plumbing: `build_certs` → `agent_tls_map` → `build_hub(..., tls=...)` in both assembly branches; `AppContext.certs`; `agent_tls_fields()` carries `tls`/`bundle_rev` into the `system` and `disks` payloads (additive, no `severity`). +- Defect found and fixed in task 01's `certs.py`: no SKI/AuthorityKeyIdentifier, so a **strict** verifier — `ssl.create_default_context()`, which this dial builds — refused the chain with `CERTIFICATE_VERIFY_FAILED: Missing Authority Key Identifier`; the agent's non-strict server check hid it. Pinned in `tests/test_certs.py`. + +**Checks:** `uv run pytest` 1371 passed / 15 skipped (21 new: `test_collectors.py` 31→56 incl. a real `host_agent.py` served over TLS, `test_api.py` revoked-through-real-`build_hub`, `test_certs.py` identifiers) · coverage 98.91 % (floor 94; new code 100 %, collectors' 5 misses are pre-existing `/slots`+`NutState` lines) · ruff + pyright clean · `bash .agents/validate.sh` exit 0 · live suite with `.env.dev` 11 passed · dev server restarted, all 9 sources ok. + +**Docs:** README (agent TLS section + collector's half, `HOST_TARGETS` row, `certs.py` bullet), AGENTS.md (agent/collectors/certs/app/host bullets, gap 11, tallies), `.env.example` + `.env.dev.example`; evidence in `data/logs/NOTES-collector-tls.md`. + +**Deviation:** httpx 0.28 has no per-request `verify`/`cert` (both deprecated), so the material rides on a **per-dial client built from an `ssl.SSLContext`** — same isolation the task wanted (shared engine/plain client untouched), stricter verification. No live infrastructure changed: no agent runs a bundle yet. + +**Next:** `.agents/phases/todo/15_agent_mtls/04_cert_api.md`. diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.validate new file mode 100644 index 0000000..efac8fe --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__03_collector_tls.a1.validate @@ -0,0 +1,64 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 15%] +........................................................................ [ 20%] +........................................................................ [ 25%] +....s................................................................... [ 31%] +........................................................................ [ 36%] +........................................................................ [ 41%] +........................................................................ [ 46%] +........................................................................ [ 51%] +........................................................................ [ 57%] +........................................................................ [ 62%] +........................................................sssssssssss..... [ 67%] +........................................................................ [ 72%] +........................................................................ [ 77%] +........................................................................ [ 83%] +........................................................................ [ 88%] +........................................................................ [ 93%] +................................sss..................................... [ 98%] +.................. [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 358 0 100% +src/dashboards/app.py 184 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 276 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3395 37 99% +Required test coverage of 94% reached. Total coverage: 98.91% +1371 passed, 15 skipped, 1 warning in 122.72s (0:02:02) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == +advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose): + fastapi 0.150.0 0.151.0 wheel diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md new file mode 100644 index 0000000..2196040 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md @@ -0,0 +1,14 @@ +All green. Final state: + +**New / changed** +- `src/dashboards/certs.py` — `read_state()`: the request-time read (validates, never mints/writes); `ensure_certs()` stays the startup-only bootstrap. +- `src/dashboards/api.py` — `certs_router()`: `GET /api/certs` (root/client + one row per `HOST_TARGETS` ∪ `certs.agents` name, `status:"unissued"` for configured-never-issued, `deployed_bundle_rev` from the `system` payload's `agent` block, serials as decimal strings, no severity); `GET /api/certs//bundle` (stored bytes verbatim, `application/x-pem-file`, `attachment; filename="agent-.bundle"`, `no-store`); `POST …/rotate` (SAN from live config, first-issue = rev 1), `POST …/revoke`, `POST /api/certs/root/rotate` (`restart_required:true`). Unknown name → `404 {"configured": false}` before any signing/dial/store write; writes log one fixed-text `certs`-source event. `app.py` includes the router. +- No new dial: the only upstream write stays `POST /api/ups/command`; `/api/certs/*` writes are a third, `meta`-only class in the invariant-1 pin. + +**Tests** — `tests/test_api.py`: 22 cert tests (list shapes 0/1/3 hosts, ghost entry read-only, no-dial/no-write spies, rev+1 + old serial absent from the new denylist, revoke → next `fetch_snapshot` refuses without a socket, root rotate re-issues all + `restart_required`, bundle byte-identity + reissue changes bytes, no PEM on `/api/certs`/`/api/health`/`/api/events`/`/api/summary`/`/api/config`, GET cannot issue). `tests/test_auth_api.py`: unsigned JSON `401` on list/bundle/writes, missing-CSRF and cross-origin refusals with the table unmoved. `tests/test_certs.py`: `read_state` reports the stored authority after a rotate and refuses corrupt/mismatched rows with the fixed text, never writing. + +**Verification** — `bash .agents/validate.sh` exit 0: 1400 passed + 15 skipped, coverage 98.94% (`api.py` 100%, `certs.py` 100%), ruff clean, pyright 0 errors. Dev server restarted: `/api/health` all nine sources ok, `GET /api/certs` unsigned → `401`, signed-in reads the real CA (root/client serials, three hosts `unissued`). No fleet or agent config touched. + +**Docs** — README "Managing the certificates" section + `api.py` bullet + scripting note; AGENTS.md `api.py`/`certs.py` bullets, invariant-1 write lists, gap 11, test/coverage counts. + +**Needs attention** — pre-existing advisory only: `fastapi 0.150.0 → 0.151.0` (no dependency moved in this task; left for the phase's dependency pass). diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.validate new file mode 100644 index 0000000..38617f0 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__04_cert_api.a1.validate @@ -0,0 +1,64 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 15%] +........................................................................ [ 20%] +........................................................................ [ 25%] +.................s...................................................... [ 30%] +........................................................................ [ 35%] +........................................................................ [ 40%] +........................................................................ [ 45%] +........................................................................ [ 50%] +........................................................................ [ 55%] +........................................................................ [ 61%] +........................................................................ [ 66%] +.............sssssssssss................................................ [ 71%] +........................................................................ [ 76%] +........................................................................ [ 81%] +........................................................................ [ 86%] +........................................................................ [ 91%] +.............................................................sss........ [ 96%] +............................................... [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 436 0 100% +src/dashboards/app.py 185 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 292 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3490 37 99% +Required test coverage of 94% reached. Total coverage: 98.94% +1400 passed, 15 skipped, 1 warning in 124.11s (0:02:04) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == +advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose): + fastapi 0.150.0 0.151.0 wheel diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md new file mode 100644 index 0000000..68de8fd --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md @@ -0,0 +1,10 @@ +**Task 15.05 complete — gate green.** + +- **New:** `web/pages/certs.html` (Root CA block: CN/serial/expiry + `in 10.0y` + `Rotate root CA`; display-only Dashboard client line; `#agents-table`; empty state; two `role="status"` outcome lines; 5 tabs, no `aria-current`, no `#nav-status`) and `web/static/page-certs.js` (reads `GET /api/certs`; a `COLUMNS` table names every key it indexes; `status` word → chip class with `unissued` mapping to *no* class; `n/a` for a null `deployed_bundle_rev`; neutral `stale` chip when issued > deployed; `confirm()`ed writes printing the route's own `hint`). +- **`app.py`** — `/certs` in `PAGES`, so the gate guards it with no new code. **`dash.css`** — one rule (`table.data td.san`, the only cell that must wrap); the account outcome line is documented as shared, no new tokens. +- **Masthead** — `⚷ Certs` added to the five dashboards *and* `/settings`; `/certs` carries `⚙ Account`; neither admin page self-links (the `/settings` precedent). +- **Tests** — `/certs` joins `MARKUP_PAGES`/`PAGE_SCRIPTS`/`ASSETS` (CDN, `.sev-`, flavour-text, orphan-`id` audits); `test_every_dashboard_page_links_to_settings` extended into `test_every_page_links_to_both_admin_pages`; 6 new pins, incl. `COLUMNS`↔`CERT_AGENT_KEYS` resolved against a live `/api/certs` row in both directions, the CNs pinned to `certs.ROOT_CN`/`CLIENT_CN`, no `severity` anywhere on the page, no control the API would refuse, and the restart sentence being the server's `hint`; `test_auth_api` gained `/certs` in `PROTECTED_PAGES` and both served-route tables. +- **Fixed in passing** — one `api.py` comment claimed task 05's page reads `source: "certs"` events; it does not, so the comment now names `/api/events?source=certs` as the audit trail. +- **Gate** — `bash .agents/validate.sh` exit 0: **1408 passed, 15 skipped**, coverage **98.94 %**, ruff + pyright clean, `node --check` clean; dev server restarted (all nine sources `ok`), unsigned `GET /certs` → `303 /login?next=/certs`. +- **Browser** (`data/logs/`) — `15-certs-page.png` (live: three `unissued` rows), `-masthead.png`, `-masthead-settings.png`, `15-certs-rotated.png` (real Rotate → `active`, serial, `#1 → n/a`, `install the new bundle on deskwork — /api/certs/deskwork/bundle`), `15-certs-shapes.png` + `-shapes-dark.png` (stale chip, revoked, `expired`, unissued, `not in HOST_TARGETS`) — the last two used an intercepted `page.route` payload, since the live fleet has no bundles until task 07. That pass issued one real certificate for `deskwork` (`bundle_rev 1`) in the dev store. +- **Docs** — README page list, a `/certs` block in *Managing the certificates*, the signing-in scope line; AGENTS intro, the `web/` bullet (both rules the page keeps), and gap 11 now tasks 01–05 landed with 06/07 left. diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.validate new file mode 100644 index 0000000..fb075ba --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__05_certs_admin_page.a1.validate @@ -0,0 +1,64 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 15%] +........................................................................ [ 20%] +........................................................................ [ 25%] +........................s............................................... [ 30%] +........................................................................ [ 35%] +........................................................................ [ 40%] +........................................................................ [ 45%] +........................................................................ [ 50%] +........................................................................ [ 55%] +........................................................................ [ 60%] +........................................................................ [ 65%] +.....................sssssssssss........................................ [ 70%] +........................................................................ [ 75%] +........................................................................ [ 80%] +........................................................................ [ 86%] +........................................................................ [ 91%] +.....................................................................sss [ 96%] +....................................................... [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 436 0 100% +src/dashboards/app.py 185 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 292 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3490 37 99% +Required test coverage of 94% reached. Total coverage: 98.94% +1408 passed, 15 skipped, 1 warning in 125.83s (0:02:05) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == +advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose): + fastapi 0.150.0 0.151.0 wheel diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md new file mode 100644 index 0000000..93531a4 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md @@ -0,0 +1,8 @@ +**Report — task `15_agent_mtls/06_deploy_cert_bundle`** + +- `deploy/install-agent.sh`: new `--cert-bundle ` flag; `validate_bundle()` (the `resolve_bundle` text-shape idiom: 2 certs + exactly 1 key + `# BUNDLE-REV`) runs **before `need_root`**, exit 2 with the reason, touches nothing, never dials; bundle installed to `/etc/dashboards-agent.certs` byte-identically, re-runs without the flag keep it (token-reuse idiom); the unit's TLS branch (one `if [ -f "$CERT_FILE" ]` block) adds `--cert-bundle` + `RuntimeDirectory=dashboards-agent` (`0700`) while keeping `--token-file`; `--status` probes `https` with the bundle's CA, `CERT_REQUIRED` + no client cert (`python3` `ssl` one-liner, no openssl), unreadable bundle → "run --status as root"; token mode keeps the plain curl probe; `--uninstall` sweeps the bundle; TLS install prints the `https://` `HOST_TARGETS` hint. `deploy/agent-sudoers` unchanged. +- **Deviation (deliberate):** the task's literal `0600` bundle mode cannot work — the agent opens the bundle as its own user and re-reads the denylist live (the exact incident the token file's comment records); installed at `0640 root:dashboards-agent`, which is the task's stated intent ("only root and the agent's own service user may read it") and the token-file idiom. Pinned in the script comment, the tests, and AGENTS.md gap 11. +- Tests: 16 new pins in `tests/test_deploy_scripts.py` (validation refusal matrix incl. `/nonexistent` before the sudoers hint; well-formed bundle reaches the root check; help/path/mode/chown pins; one-branch TLS unit lines; probe text pins incl. "no `load_cert_chain`, never openssl"; sandbox install → file 0640 + byte-identical + unit flags + `RuntimeDirectory`; bundle reuse; uninstall; plus end-to-end `--status` against a live in-process TLS agent — CA-only `/health` pass, foreign-root refusal, unreadable-bundle sentence, token-mode plain probe). +- `bash .agents/validate.sh` → exit 0: **1424 passed, 15 skipped**, coverage 98.94% (floor 94), ruff clean, pyright 0 errors, `node --check` clean, `uv pip list --outdated` empty (bumped fastapi 0.150→0.152 per invariant 8; suite re-run green). +- Docs: README (installer TLS install + `--status` note + installer paragraph in the TLS-mode section); AGENTS.md (deployment paragraph flags/bundle/RuntimeDirectory/probe, gap 11 rewritten to "what is left is task 07", test count 1424). +- Next pending task: `15_agent_mtls/07_live_rollout` (live infra: flips deskwork+gateway to TLS — user/operator gated, untouched here). diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.validate new file mode 100644 index 0000000..8923187 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__06_deploy_cert_bundle.a1.validate @@ -0,0 +1,62 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 5%] +........................................................................ [ 10%] +........................................................................ [ 15%] +........................................................................ [ 20%] +........................................................................ [ 25%] +........................s............................................... [ 30%] +........................................................................ [ 35%] +........................................................................ [ 40%] +........................................................................ [ 45%] +........................................................................ [ 50%] +........................................................................ [ 55%] +........................................................................ [ 60%] +........................................................................ [ 65%] +.....................................sssssssssss........................ [ 70%] +........................................................................ [ 75%] +........................................................................ [ 80%] +........................................................................ [ 85%] +........................................................................ [ 90%] +........................................................................ [ 95%] +.............sss....................................................... [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 436 0 100% +src/dashboards/app.py 185 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 292 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3490 37 99% +Required test coverage of 94% reached. Total coverage: 98.94% +1424 passed, 15 skipped, 1 warning in 125.49s (0:02:05) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.err b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.err new file mode 100644 index 0000000..e69de29 diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md new file mode 100644 index 0000000..8524359 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md @@ -0,0 +1,16 @@ +**Target task** (`.agents/phases/todo/16_agent_smart_pass_budget/01_pin_the_scaled_budget.md` — the harness's stated name `01_live_remeasure_and_pin.md` does not exist; the phase dir has exactly one task 01): + +- `tests/test_agent.py`: added `FakeSmartPassClock` + `_TimeProxy` (drives the agent's `time.monotonic` and `subprocess.run` through proxies on `host_agent`'s own globals, recording the `min(deadline-now, 5 s)` each device was given — a deadline is invisible to a fake binary), `SHELF_DISKS`/`write_disk_shelf` (driveripper's 8 NVMe + 8 SATA shape), and 3 tests: `test_the_pass_budget_scales_with_the_disk_count` (16 disks, 24 s pass → all 16 asked, no budget line), `test_the_pass_budget_floor_is_twenty_seconds` (2nd device's timeout = 2 s → pins the 20 s floor), `test_the_pass_budget_error_line_names_the_scaled_figure` (6 s/device → 14 runs, 2 skipped, line is exactly `smart: pass budget of 80s exhausted, 2 disk(s) skipped`). +- Non-tautology proven by mutation: reverting `_smart_pass` to the flat `budget = self.smart_budget_seconds` makes the two scaling tests fail (`4 != 14` calls); `agent/host_agent.py` restored, md5 verified `4061ad59…` — **no code change**. +- Override test `test_the_pass_budget_skips_the_rest_with_one_error` untouched and green. + +**Tests / lint (exact, all from repo root):** +`uv run pytest` → **1427 passed, 15 skipped** (baseline 1424+15); `uv run pytest --cov --cov-fail-under=94` → **98.94%**; `uv run ruff check src tests agent` → All checks passed; `uv run pyright` → 0 errors; `bash .agents/validate.sh` → **exit 0**; `uv run pytest tests/test_agent.py -q --cov=agent` → `agent/host_agent.py` 94 %, budget lines 584–602 fully covered; `uv run pytest -m live --live` → **11 passed**; `./scripts/dev-server.sh status` → all 9 sources ok. + +**Docs:** no doc impact — tests-only task. Deliberately left for task `02_docs_and_gate.md`: `README.md:94` still reads "20 s per pass" (`grep -rn "20 s per pass" AGENTS.md README.md agent/ deploy/` → currently only that one hit). + +**Decisions / findings:** +- No live/infra changes made for this task. Read-only md5 probe: repo agent ≠ deployed agent on all three hosts (working tree carries phase 15's un-deployed TLS mode), so the phase's md5-identity criterion can't be met yet; the meaningful check passes — driveripper's deployed agent **does** carry the scaled-budget line (deskwork/gateway don't, and can't be affected: 1 disk / no SMART, floor == scaled there). +- **Interrupted work (harness redirect):** phase 15 task `07_live_rollout` was mid-flight when the target changed. I made **no fleet mutations** and **no tracked-file changes** for it, and cleaned up all scratch processes (`:8091` scratch dashboard, `:9129` TLS agent removed; dev server untouched and healthy). Evidence captured in `data/logs/`: `probe-tls-matrix.py` (6/6 PASS: CA-only `/health` 200, certless `/snapshot` 403, dashboard client 200 with `tls/bundle_rev`, foreign-root client `TLSV1_ALERT_UNKNOWN_CA`, agent's own server cert refused as a client cert, untrusted-root `CERTIFICATE_VERIFY_FAILED`), byte-identical bundle download, revoke → `agent certificate revoked` with zero new agent-log lines, rotate → `#2 → #2`, live `# REVOKED` denylist re-read without restart, screenshots `15-tls-local-{certs,system}.png`, `15-tls-fleet-certs.png`. Still outstanding there: the README "Protecting the agent" how-to, the AGENTS.md transport/commands/gap-11 update, and the `NOTES-agent-install.md` operator append. + +**Next pending task:** `.agents/phases/todo/16_agent_smart_pass_budget/02_docs_and_gate.md`. diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md.status b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md.status new file mode 100644 index 0000000..9766475 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.md.status @@ -0,0 +1 @@ +ok diff --git a/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.validate b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.validate new file mode 100644 index 0000000..a1d22b1 --- /dev/null +++ b/.agents/reports/15_agent_mtls/15_agent_mtls__07_live_rollout.a1.validate @@ -0,0 +1,63 @@ +== uv run pytest --cov (hermetic: no network, live tests skip) == +........................................................................ [ 4%] +........................................................................ [ 9%] +........................................................................ [ 14%] +........................................................................ [ 19%] +........................................................................ [ 24%] +...........................s............................................ [ 29%] +........................................................................ [ 34%] +........................................................................ [ 39%] +........................................................................ [ 44%] +........................................................................ [ 49%] +........................................................................ [ 54%] +........................................................................ [ 59%] +........................................................................ [ 64%] +........................................sssssssssss..................... [ 69%] +........................................................................ [ 74%] +........................................................................ [ 79%] +........................................................................ [ 84%] +........................................................................ [ 89%] +........................................................................ [ 94%] +................sss..................................................... [ 99%] +.. [100%] +=============================== warnings summary =============================== +.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1 + /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. + from starlette.testclient import TestClient as TestClient # noqa + +-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html +================================ tests coverage ================================ +_______________ coverage: platform linux, python 3.14.4-final-0 ________________ + +Name Stmts Miss Cover Missing +------------------------------------------------------------------ +src/dashboards/__init__.py 3 0 100% +src/dashboards/__main__.py 14 1 93% 32 +src/dashboards/alerts.py 176 0 100% +src/dashboards/api.py 436 0 100% +src/dashboards/app.py 185 0 100% +src/dashboards/auth.py 265 0 100% +src/dashboards/certs.py 292 0 100% +src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307 +src/dashboards/config.py 120 0 100% +src/dashboards/debug.py 14 0 100% +src/dashboards/poller.py 165 3 98% 192-194, 279 +src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183 +src/dashboards/rates.py 73 0 100% +src/dashboards/severity.py 49 0 100% +src/dashboards/sources/__init__.py 0 0 100% +src/dashboards/sources/disks.py 134 0 100% +src/dashboards/sources/host.py 209 0 100% +src/dashboards/sources/llm.py 178 1 99% 285 +src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736 +src/dashboards/sources/nut_sim.py 134 0 100% +src/dashboards/store.py 313 2 99% 553-554 +------------------------------------------------------------------ +TOTAL 3490 37 99% +Required test coverage of 94% reached. Total coverage: 98.94% +1427 passed, 15 skipped, 1 warning in 133.41s (0:02:13) +== uv run ruff check src tests agent == +All checks passed! +== uv run pyright == +0 errors, 0 warnings, 0 informations +== node --check src/dashboards/web/static/*.js == diff --git a/.env.dev.example b/.env.dev.example index ec6ea65..583705f 100644 --- a/.env.dev.example +++ b/.env.dev.example @@ -16,6 +16,12 @@ # into an Authorization header, so Target.url, /api/config and every error message stay # credential-free. # +# A host whose agent runs in TLS mode (--cert-bundle) is an https:// url with NO userinfo: +# the handshake is the auth there, the agent never reads a token on that path, and the +# dashboard verifies the agent against the root CA it minted itself (certs.py). Mixing the +# two (`https://TOKEN@host`) works — the token is simply unused — and is how a rollout leaves +# one host on tokens while another is on certificates. +# # The host names below are placeholders (RFC 2606) — use your own. export HOST_TARGETS="driveripper=http://TOKEN@driveripper.example.net:9120 deskwork=http://TOKEN@deskwork.example.net:9120 gateway=http://TOKEN@gateway.example.net:9120" diff --git a/.env.example b/.env.example index 44980e2..5222356 100644 --- a/.env.example +++ b/.env.example @@ -42,7 +42,10 @@ DASH_PBKDF2_ITERATIONS=600000 # --------------------------------------------------------------------------- # # System dashboard: hosts running agent/host_agent.py # Space or comma separated name=url. A bearer token goes in the URL userinfo -# (http://TOKEN@host:9120) and is never sent to the browser. +# (http://TOKEN@host:9120) and is never sent to the browser. An https:// url instead +# dials a certificate-authenticated agent: the fetch verifies it against this app's own root CA +# and presents this app's own client certificate (certs.py), and a host revoked there is refused +# before any connection. Per host, so one host can be on https while its neighbours are on http. # --------------------------------------------------------------------------- # # HOST_TARGETS=driveripper=http://SECRET@driveripper.example.net:9120 deskwork=http://SECRET@deskwork.example.net:9120 HOST_TARGETS= diff --git a/AGENTS.md b/AGENTS.md index 903d172..361b064 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,9 +34,11 @@ History is a first-class feature, not a sparkline afterthought: every number wor is stored, rolled up (raw → 1m → 1h), and served from `/api/*/history`. None of it is public. One admin signs in at `/login` (the only page an anonymous visitor -reaches), and `/settings` — `⚙ Account` in the masthead — is where that session is read, ended, -and re-credentialled. Every page above, `/settings`, and every `/api` route but the four named -in invariant 12 require the session. +reaches), and the two admin pages live behind it: `/settings` — `⚙ Account` in the masthead — +where that session is read, ended and re-credentialled, and `/certs` — `⚷ Certs`, next to it — +where the fleet's certificate authority is listed, rotated, revoked and downloaded. Every page +above, `/settings`, `/certs`, and every `/api` route but the four named in invariant 12 require +the session. ## Non-negotiable rules @@ -94,6 +96,12 @@ page still renders. startup and the empty-new-password refusal) carry **fixed** operator text with nothing interpolated, not even a prefix of a hash; `tests/test_auth.py` pins that structurally, so the day someone adds `logger.warning(f"bad verifier {v}")` a test fails instead of a log line. + **A private key is the same case** (phase 15): the root CA key and the dashboard's client key + live only in `meta`, `certs.py` has no `logging` import and one fixed-text raise naming the rows + to delete (pinned structurally by `tests/test_certs.py`, which also asserts the refusal quotes + no line of the stored key), the root key is never written to the temp dir the client material + lives in, and a bundle carries the root *certificate* — never the key — because an agent only + has to verify its clients and a stolen agent install must not be a stolen CA. 7. **Verify against the deployed upstream; never guess a metric name.** Field maps in `sources/llm.py` and `sources/nut.py` were captured from the running engines and NUT 2.8.5, and are pinned by fixtures in `tests/fixtures/`. Re-capture with @@ -173,9 +181,31 @@ Responsibilities, one line each: - `agent/host_agent.py` — stdlib-only (3.10+), reads `/proc`, `/sys/block`, `/sys/class/drm`, `statvfs`; serves `GET /snapshot` + `GET /info` + `GET /schema` + `GET /health`, plus an - HTML status page at `GET /`. Refuses to bind off-loopback without a token (exit 2). - Counters stay counters: the dashboard computes + HTML status page at `GET /`. Refuses to bind off-loopback without a token **or** a certificate + bundle (exit 2). Counters stay counters: the dashboard computes rates so the agent never remembers who polled it last. + **TLS mode (phase 15 task 02, `AGENT_VERSION` 1.1.0):** `--cert-bundle`/`AGENT_CERT_BUNDLE` + loads the one PEM the dashboard signs (`resolve_bundle`, the `resolve_token` idiom: server cert + + server key + root CA + `# BUNDLE-REV ` + `# REVOKED ` lines, grepped by text + because the agent has no `cryptography`), unpacks it into the unit's `RuntimeDirectory` when + there is one else a temp dir, builds a `PROTOCOL_TLS_SERVER` context with + `verify_mode = CERT_OPTIONAL`, and **deletes the PEMs once loaded** (the key then lives only in + this process, so a `kill -9` has no cleanup to forget). `CERT_OPTIONAL` is the assumption that + keeps `/health` probeable: the *handler* enforces, refusing a peer with no verified certificate + 403 `client certificate required` on every path but `/health`, and refusing a serial on the + bundle's denylist 403 `client certificate revoked` on **every** path including `/health` (the + exemption is for a probe holding the CA and no certificate, not for an identity the operator + withdrew). The denylist is re-read when the bundle's `(mtime_ns, size)` stamp changes — that is + the whole reason the `# REVOKED` lines live in the bundle — while a new certificate needs a + restart, because `ssl` cannot swap a chain under a live socket. Serial matching accepts the hex + `getpeercert()` renders *and* the decimal `certs.py` writes, because comparing one spelling to + the other accepts a revoked client forever. On the TLS path the bearer token is **not** read (the + handshake is the auth, and a second credential that outlives a revocation defeats the list), + `/snapshot` gains additive `tls: true` + `bundle_rev` only in TLS mode, and token mode is + byte-identical to before. Task 03 has since flipped the collector (see the `collectors.py` + bullet), so those two fields are how the dashboard answers "is this host running the bundle + `/certs` issued?" — `sources/host.py` and `sources/disks.py` carry them into both payloads as + `agent.tls` / `agent.bundle_rev`, present only when the agent published them. SMART (phase 09): `smart()` runs `smartctl -i -A -H` per whole `/sys/block` disk (`parse_smart_output` covers ATA + NVMe; NVMe media errors map to `offline_uncorrectable`, hex-packed vendor raws are never decoded, and an attribute row is @@ -217,7 +247,31 @@ Responsibilities, one line each: Events are stored under the **dashboard** name (`system`/`disks`/`llm`/`ups`), not the sample source (`host`) — a page filtering `/api/events` by `host` gets an empty feed forever (found on `/system` during phase 03 browser verification). -- `collectors.py` — the fetch half. `NutMonitor` owns per-UPS transition bookkeeping (its +- `collectors.py` — the fetch half. `AgentTls` (phase 15 task 03) is what makes an `https://` + `HOST_TARGETS` entry a **certificate-authenticated** dial: `verify` and `cert` are the paths + `certs.write_material` wrote at startup, and the dial is verified against this app's own root + and presents this app's own client certificate. Two things about it are load-bearing. (a) The + credential travels in a **client built per dial**, because `httpx` resolves TLS material when a + client is *constructed* (there is no per-request `verify`/`cert`, and `verify=`/`cert=` are + deprecated anyway — the supported form is an `ssl.SSLContext`, which is also the honest shape: + trust in one named CA, hostname checks on) while one shared client dials the plain agents and + both engines — a `verify` on *that* would make every engine read depend on a CA nobody issued + their cert with. (b) `status` is a **callable** doing a fresh `certs.agents` read, so a + revocation clicked in the browser bites on the next poll with no restart, and a revoked host is + refused **before any socket opens** — invariant 3's shape: a source the app knows it must not + talk to says so without a network round trip, and the poller's stale/last-good semantics do the + rest. `refusal_detail()` quotes the agent's own 403 sentence (`client certificate required` / + `client certificate revoked`) because those are two different operator actions, and the + certificate wording is used only for a dial where a certificate was presented — a 401/403 from a + token-mode host keeps the token/HTTP wording, or the operator is sent to `/certs` for a token + problem. `agent_tls_map()` builds one entry per configured host (the CA and client pair are + shared — the dashboard has one identity — only the revocation answer is per name, and an absent + entry is *active*, never-issued being a different fact from revoked); `build_hub` takes that map + and hands the **same** entry to the `host:` and `disks:` jobs of one host, so a revocation + fails both sources with one sentence and neither of them dials. `fetch_snapshot()` keeps + `http://` targets on the shared client with their bearer token, unchanged from before the phase, + so a rollout is per host. + `NutMonitor` owns per-UPS transition bookkeeping (its self-test event takes its severity from `nut.self_test_outcome()` and not its own word list, so a passed test is never an amber line in the feed while the card beside it reads green). Its per-UPS tolerance for a failed `LIST CMD`/`LIST RW` is for a *complete* error reply only: @@ -237,7 +291,10 @@ Responsibilities, one line each: smartmontools or device read access is unconfigured, not broken (invariant 3); they stay in `agent.errors` for the `/disks` page to explain. - `sources/host.py` — snapshot → payload + samples; reboot detection; GPU series per card - (`gpu.busy_pct@`). + (`gpu.busy_pct@`). `agent_tls_fields()` is the phase-15 passthrough: a TLS agent's + `tls`/`bundle_rev` ride into the payload's existing `agent` block, and a token-mode snapshot + grows nothing (no `tls: false`, no `bundle_rev: null` — rule 2), so the fields are how the API + distinguishes a stale bundle from an agent that never had one. - `sources/disks.py` — the same snapshot projected onto **physical** drives for `/disks` (phase 09): one row per whole-`/sys/block` disk (never a partition, never a mount), the agent's `smart` block carried through verbatim, `health` the drive's own word normalised to @@ -351,6 +408,50 @@ Responsibilities, one line each: `DASH_SESSION_HOURS`. Its two raises, like the loader's, carry fixed text — `tests/test_auth.py` pins the file's raise sites and its one logging call structurally, so no future edit can interpolate a credential into either. +- `certs.py` — the fleet's certificate authority as the same shape of module (phase 15): a pure + lifecycle over the three-method `MetaStore` protocol, no clock it cannot be handed, no `logging`, + and one fixed-text `RuntimeError` (`tests/test_certs.py` pins all three structurally, the way + `tests/test_auth.py` pins `auth.py`). Five `meta` rows and nothing else: + `certs.root.key`/`certs.root.cert` (self-signed `CN=dashboards-root`, 10 years), + `certs.client.key`/`certs.client.cert` (`CN=dashboards`, 1 year — the identity every agent will + check), and `certs.agents`, one JSON object mapping a `HOST_TARGETS` name to + `{serial, not_after, status, bundle_rev, bundle, san}`. **A stored row always wins:** the root is + generated on the first boot of a database and never on a later one, because a CA that rotates + with the process untrusts every bundle already installed on a host — and a newly generated root + takes its client certificate with it, since a leaf signed by the dead root is a credential + nothing can verify. `issue_agent_cert`/`reissue_agent_cert` (the API's "rotate") sign a fresh + `CN=` leaf whose SAN is the addresses that host is dialled by — the name is always in the + list, and an IP goes in as an `iPAddress` or hostname verification of an IP dial fails — bump + `bundle_rev`, and reset `status` to `active`. `revoke` marks the host and **re-seals every + bundle**, because the `# REVOKED` denylist lives inside each of them and an agent that cannot see + a serial keeps accepting that client; there is no "unrevoke", and revoking a never-issued host + still records the revocation (what a revoke enforces is the dashboard refusing to dial). The + **bundle** is the install artifact — agent cert, agent key, the root *cert* (never the root key, + asserted by header count and by parsing the one key against both keypairs), `# BUNDLE-REV `, + then the denylist — stored verbatim so what an operator downloads is byte-identical to what the + row holds. `rotate_root` mints a new root plus a new client and reissues every agent + (`bundle_rev` +1 each, denylist reset with the generation); the running process keeps the + material it loaded at startup, which is the honest `restart_required` task 04 answers with. + Every certificate carries `SubjectKeyIdentifier` and every leaf an `AuthorityKeyIdentifier` + naming it, non-critical: OpenSSL's **strict** mode refuses a leaf with no AKI + (`CERTIFICATE_VERIFY_FAILED: Missing Authority Key Identifier`) and Python's + `ssl.create_default_context()` — the context the dashboard's own https dial builds — turns that + mode on. The agent's server-side check is not strict, so nothing on the agent side notices; + `tests/test_certs.py` pins the identifiers instead, and a bundle recorded without them is one a + strict client rejects. `write_material` puts the CA and the client pair in a `0700` temp dir with + `0600` files for the dialing — the root key is never written to disk, so its only copy stays the + `meta` row — and the lifespan removes the directory on shutdown. Unreadable PEM, a key that does + not match its certificate, or a leaf the stored root did not sign is a **fail-closed startup** (the + `ctx.auth` rule), and the message is the `sqlite3 "" "DELETE FROM meta WHERE key LIKE + 'certs.%'"` reset. `cryptography` is this module's dependency and the app's only new one; + `agent/host_agent.py` stays stdlib and loads the PEMs with `ssl`. There are **two reads** and the + difference is a side effect: `ensure_certs` is the startup one (it mints what the table is missing, + which is why it is what `app.build_certs` calls and nothing else may), and `read_state` is the + request-time one task 04's `GET /api/certs` uses — same validation, same fixed-text refusal, never + a write, so a page refresh cannot bootstrap an authority the operator deleted. The material is + read with (task 03): `collectors.agent_tls_map` turns `AppContext.certs` into the per-host dial + pair, and because the revocation answer is `agent_status()` called per poll, a revoke in this + module is what stops the dial — no restart, no cache, no second list to keep in sync. - `severity.py` — the only place thresholds live: `THRESHOLDS`/`INVERTED` per metric, `classify()` → `ok`/`warn`/`fail`/`None`, plus `attach()`/`with_severity()` to put the verdict beside a value. No measurement, no verdict: `None` in → `None` out, never a green pass for @@ -452,9 +553,33 @@ Responsibilities, one line each: - `api.py` — GETs everywhere plus the one POST that dials an upstream, and the `auth_router()` that creates, describes, **re-credentials**, and ends a session (`POST /api/auth/login` public; `GET /api/auth/status`, `POST /api/auth/logout` and `POST /api/auth/password` protected). Those - three are the app's only other writes and they touch this process and its `meta` table and dial - nothing — invariant 1 is about the fleet, and `tests/test_api.py` pins the two lists separately + three, and the `certs_router()` of phase 15 (`GET /api/certs`, `GET /api/certs//bundle`, + `POST /api/certs//rotate`, `POST /api/certs//revoke`, + `POST /api/certs/root/rotate`), touch this process and its `meta` table and dial + nothing — invariant 1 is about the fleet, and `tests/test_api.py` pins the three lists separately rather than widening the one that names `/api/ups/command`. + The **`/api/certs` surface** (phase 15 task 04) is the CA's own admin half. The list reads the + **stored** state through `certs.read_state` — the request-time twin of `ensure_certs`, which + mints and must only ever be called at startup, because a `GET` that bootstrapped a root CA would + replace the authority nobody asked it to replace — and its rows are the union of `HOST_TARGETS` + and `certs.agents`, so a configured-but-uninstalled host appears as `status: "unissued"` with + nulls (invariant 3, rule 2) and an entry whose host left the config is listed but not writable. + `deployed_bundle_rev` comes from that host's `system` payload `agent` block (task 03's + passthrough), never from what the CA hopes. `bundle_rev` is what was issued; the two numbers side + by side are the "install the new bundle" sentence, and neither carries a `severity` — an expiry + and an identity counter are not graded numbers. A `` outside `HOST_TARGETS` is refused in + the route's first line with `404 {"configured": false}` **before** any signing, dial, or store + write (spy-pinned both ways, and the same gate makes `GET …/bundle` answer + `{"configured": true, "issued": false}` for a host that is configured and has nothing issued — + two different facts, two different bodies). The bundle download is the stored row **verbatim** as + `application/x-pem-file` with an `attachment` filename, because it is `install-agent.sh`'s input + and it carries that host's private key (session-gated, `no-store`). `POST /api/certs/root/rotate` + answers `restart_required: true` rather than pretending a hot-swap: the CA and the client pair are + startup material, so the table is new at once while the process keeps dialling with the old one. + Every write logs one fixed-text `certs`-source event naming host and action, and serials travel as + **decimal strings** — a 64-bit certificate serial through a JS number is a serial a page cannot + match against a bundle's `# REVOKED` line (invariant 6: no PEM reaches an event, a health record, + or any payload but the bundle route's). The login route is where a *sign-in* password is compared: it throttles before it hashes, refuses an absent/empty/non-object body with `422` so `""` never reaches `verify_password`, answers every failure with the one sentence `wrong password`, and @@ -490,8 +615,9 @@ Responsibilities, one line each: no `disks` job in the hub at all (invariant 3), and `tests/test_api.py` proves it with spies on `httpx.AsyncClient.get` and `collectors.fetch_snapshot`. The new surface grows no write of its own — a fourth dashboard must not quietly become a second one, so the app's - only POST stays `/api/ups/command` (invariant 1, pinned by enumerating every served route's - verbs). + only POST that **dials an upstream** stays `/api/ups/command` (invariant 1, pinned by enumerating + every served route's verbs and sorting the writes into fleet / session / certificate-authority + lists — the third is phase 15's, and it writes `meta` rows and nothing else). All four `history` routes reach the store through one door, `api._history_batch()` → `store.history_many()` (phase 14 task 02). That is a cost change and nothing else: the series list is host-major and metric-minor in request order, each entry the same @@ -531,7 +657,16 @@ Responsibilities, one line each: a `RuntimeError` from it leaves the process non-zero with the operator's fix in the output. That failure is never caught here: a dashboard that cannot say who its admin is does not start the loops either (`__main__.py` passes `SETTINGS` to `create_app()` explicitly so the served - app is configured by the same object the bind address came from). + app is configured by the same object the bind address came from). `build_certs(store)` is the + same rule for phase 15: one helper, called **before** `build_hub` in the owned-store + (`build_context`) and injected-store (lifespan) branches, whose `CertMaterial` (the CA + client + paths in a private temp dir) hangs off `AppContext.certs` and whose + directory the lifespan's `finally` removes. What hands that material to the loops is + `agent_tls_map(material, store, targets)` → `build_hub(..., tls=...)`, called in **both** + branches the same way: no material (or no targets) means no TLS map, which is the plain + `http://`/token path rather than an error, and one map covers both of a host's jobs. A corrupt stored certificate raises out of the same + place — a dashboard that cannot present its own client certificate would otherwise spend the + fleet's TLS history reporting an upstream problem it caused itself. - `web/` — five dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`) plus the login page (`/login`, `pages/login.html` + `static/page-login.js`), one stylesheet, `charts.js`. No framework. `/login` renders the masthead's **brand link and theme toggle** and nothing else @@ -555,7 +690,7 @@ Responsibilities, one line each: sixth dashboard**: same `` and masthead, the five tabs with **no** `aria-current` (it is not one of them), no status dots, and it is reached from the `⚙ Account` link in `.masthead-right` of every dashboard — outside `.tabs`, so the pinned tab structure is untouched - (`tests/test_api.py::test_every_dashboard_page_links_to_settings`). It prints + (`tests/test_api.py::test_every_page_links_to_both_admin_pages`). It prints `/api/auth/status` (name, expiry as a local timestamp *and* as time left), signs out through `POST /api/auth/logout`, and changes the password through `POST /api/auth/password` after checking locally what it can judge itself (non-empty current, ≥ 8 new characters — @@ -567,7 +702,30 @@ Responsibilities, one line each: happened" call. So `app.js` exports **`resetAuth()`**: it clears the memoised `authStatus` and its in-flight promise, and must be called after a password change (the secret rotated, so the cached `csrf` is now invalid — every later write, the UPS buttons included, 403s until a reload) and on - any `401` bounce. `app.js`'s masthead status dots are one per dashboard job — + any `401` bounce. + `/certs` (`pages/certs.html` + `static/page-certs.js`) is the second of those admin pages, and it + is the same shape for the same reasons: five tabs with **no** `aria-current`, no `#nav-status`, + and the `⚷ Certs` masthead link beside `⚙ Account` (neither admin page links to itself, which is + why the pinned two-link audit loops over two different tuples). Everything on it is + `GET /api/certs`: the Root CA block, the display-only Dashboard client line (that certificate is + minted with the root and has no route of its own), and one row per host with its status chip, + serial, expiry (local timestamp *and* `Dash.duration` left), issued-vs-deployed `bundle_rev`, + `san` list, and the three controls. Two rules it exists to keep. (1) **No colour the API did not + send**: `/api/certs` publishes no `severity` — an expiry is a date and a revision count is not a + graded number — so the page has no `severity` token at all, the one chip class map is + `status` word → `ok|fail` with `unissued` mapping to *nothing* (a certificate that does not exist + never borrows green), and a behind bundle is a neutral `stale` chip. (2) **No control the API + would refuse**: a row with `configured: false` prints why (`not in HOST_TARGETS`) instead of + three buttons that all answer `404`, a never-issued host has no **Revoke** and no **bundle** link + (that route answers `issued: false`), and the download is a plain `GET` link because a session + cookie is all a signed `GET` needs. Writes are `confirm()`d, not typed-`DANGER`d — that gate is + for cutting power and this page cannot — and the outcome line prints the route's own `hint`, so + `restart_required`'s restart sentence lives in `api.py` and nowhere else. Its `COLUMNS` table + names every payload key it indexes, and `tests/test_api.py` resolves that table against a live + `GET /api/certs` row in both directions (the `SLOT_FIELDS` device), so a rename on either side + fails a test instead of printing `n/a` beside a healthy fleet. The two CNs are markup, and pinned + to `certs.ROOT_CN`/`certs.CLIENT_CN` the way `/settings` pins its password floor. + `app.js`'s masthead status dots are one per dashboard job — including `disks`, because the same agent can answer the 10-second utilisation poll and have a wedged SMART pass, and one dot for both would be right about neither. `app.js` owns the colour vocabulary: `Dash.severityFrom(cellOrSeverity)` turns a payload verdict into an @@ -634,6 +792,11 @@ That scheme is the history API's key — treat it as a public interface. `/api/health` today reads **all nine sources `ok`**: `host:driveripper`, `host:deskwork`, `host:gateway`, `disks:driveripper`, `disks:deskwork`, `disks:gateway`, `nut`, `llm:gufo`, `llm:vllm`. + **Since phase 15 that bearer-token transport is the compatibility path, not the designed one:** + the agent can serve `https://` from an installed bundle and the dashboard dials it with its own + client certificate, but no bundle is installed on any host here, so `HOST_TARGETS` stays + `http://` + userinfo until somebody runs the rollout in README's *Protecting an agent, host by + host* (gap 11 says what that needs and why flipping the URL first turns a host red). Phase 10's proof that the old drop was not this repo's firewall stands as history in `data/logs/NOTES-verify.md` — do not re-add a tunnel on the strength of it, and re-run the probe rather than trusting either note. @@ -784,9 +947,9 @@ set -a; . ./.env.dev; set +a # dev dials the fleet the prod way, no ./scripts/dev-server.sh status # per-source ok/DOWN — unauthenticated: /api/health ./scripts/dev-server.sh logs -uv run pytest # hermetic: 1267 passed + 15 skipped (11 live, +uv run pytest # hermetic: 1427 passed + 15 skipped (11 live, # 4 store_perf), no network -uv run pytest --cov # coverage gate (98.75%; floor 94%) +uv run pytest --cov # coverage gate (98.94%; floor 94%) uv run pytest -m live --live # opt-in: real fleet / engines / upsd uv run pytest -m store_perf --store-perf # opt-in: the store hot-path timings over the seeded # store, priced against `store_perf.BUDGETS` (the cached @@ -804,6 +967,10 @@ uv run python scripts/store_perf_probe.py # ... as a report (data/logs/store-per # store read-only, DASH_PERF_STORE_DIR moves the cache, and # every pass that *writes* runs on a scratch copy so the # seeded baseline never moves under the next measurement +# the bundle for one host: the stored bytes (that host's private key included) and +# `install-agent.sh --cert-bundle`'s only input — signed-in GET, see README's rollout +# curl -s -H "Authorization: Bearer $SESSION" -o /tmp/agent-.bundle \ +# http://127.0.0.1:8090/api/certs//bundle uv run ruff check src tests agent && uv run pyright ./scripts/capture-fixtures.sh # re-record ground truth (agents + UPS + gufo /slots) SMART_HOSTS="deskwork-ops driveripper-ops" ./scripts/capture-fixtures.sh @@ -820,11 +987,23 @@ bash .agents/validate.sh # the phase gate: the four checks abov Deployment: `Containerfile` (two-stage, `uv sync --frozen`, runs as uid 10001 on `0.0.0.0:8090`, SQLite under `/var/lib/dashboards` — put a volume there) and `deploy/install-agent.sh` per monitored host (systemd unit, own user, `--bind`/`--port`/`--token`/`--rotate-token`/ -`--status`/`--uninstall`; it stores the token `0640 root:dashboards-agent` — the unit hands the -agent the *path* and the agent opens it as its own user, so that user must be able to read it and -no other account may — never puts a token on argv, and runs under `ProtectSystem=strict` with +`--cert-bundle`/`--status`/`--uninstall`; it stores the token `0640 root:dashboards-agent` — the +unit hands the agent the *path* and the agent opens it as its own user, so that user must be able +to read it and no other account may, and a `--cert-bundle` file is installed to +`/etc/dashboards-agent.certs` at that same mode after a shape check that runs before anything is +touched and never dials the dashboard (the agent re-reads the bundle live for the denylist, so +0600 would fail the unit exactly like it fails the token file; the task file's `0600` is the +intent — *root and the agent's own user only* — and 0640 is the mode that achieves it) — never +puts a token on argv, and runs under `ProtectSystem=strict` with `DevicePolicy=strict` reopening only the disk nodes plus the `/dev/null` family, because strict -without `/dev/null` kills the unit at spawn (`status=208/STDIN`). No root shell? The one sudoers +without `/dev/null` kills the unit at spawn (`status=208/STDIN`); a bundle-installed unit also +gets `--cert-bundle` plus `RuntimeDirectory=dashboards-agent` (mode 0700 — the bundle unpacks its +PEM blocks there and the agent deletes the key once the SSL context holds it), keeps the +`--token-file` line for a host-by-host rollout, and `--status` then probes `https` on the +bundle's CA with **no** client certificate via a `python3` `ssl` one-liner — task 02's +`CERT_OPTIONAL` + `/health` exemption is what makes that probeable, and there is no external TLS +binary. A re-run without the flag leaves an installed bundle in place, and `--uninstall` removes +it with the token. No root shell? The one sudoers line that lets `ops` run the installer is `deploy/agent-sudoers`; run the script without root and it prints that line with its own resolved path.) The quadlet unit + env template + nginx/SNI live in the deployment repo, like every other @@ -927,6 +1106,21 @@ branch `main`); there is no CI on it, so `bash .agents/validate.sh` is the only verbatim, the four config rows, the `DASH_COOKIE_SECURE=1` deployment note, the three auth gaps). No server starts and no request is made: a doc that stops naming the reset command is a locked-out admin, and that is worth a test even though it asserts nothing about Python. +- **A module that holds a private key is pinned structurally, not only behaviourally** + (phase 15). `tests/test_certs.py` reads `src/dashboards/certs.py` as text and fails if it grows a + `logging` import, a raise carrying an interpolated value, or an import from the rest of the + package — the same three pins `tests/test_auth.py` holds over `auth.py` — because this app turns + log lines and error strings into `event` rows and `/api/health` fields that a browser reads, and a + stored PEM is a key. Its behavioural half runs the whole lifecycle over a dict `MetaStore` (five + rows on first boot, no byte changed on the second, rotate/revoke/root-rotate and their denylists, + a bundle that carries the agent key and never the root key), and one test completes a real + client-certificate handshake through the material `write_material` lays down. The other two halves + run real code rather than a model: `tests/test_agent.py` drives the real `host_agent` handler over + a TLS-wrapped loopback port (the handshake matrix, the live denylist re-read, token mode unchanged) + and `tests/test_deploy_scripts.py` runs `install-agent.sh` in a sandbox whose `install`/`chown` are + faked, which is how a `0640` mode, a byte-identical bundle and the https `--status` probe are proven + without a root shell or a host. **A `0600` in a task file is not a fact about this repo**: the agent + opens its own bundle and re-reads it for the denylist, so the group bit is load-bearing. - **A cost claim is pinned as a shape and a control, never as a wall clock** (phase 14). The default run keeps the seeded-store tests hermetic and asserts **shape and determinism** — counts, retention windows, the statements a plan is taken from, idempotence — while the timings live behind @@ -1545,6 +1739,70 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl `/proc/devices` lists NVMe as `blkext`) — the template change is its own phase. `tests/fixtures/smartctl_driveripper_*.txt` (captured via `sudo`) remain the ground truth for the drives' shape. +11. **The CA is durable, the collector dials with it, and no bundle is installed with one** + (phase 15 tasks 01-05, landed 2026-10-20). `certs.py` mints the CA and the dashboard's client + certificate into `meta` on the first boot of a database — verified against the running dev + store: the five `certs.*` rows exist + and did not change across two restarts — signs per-host server certificates into + `certs.agents`, and does reissue, revocation, and root rotation. Task 02 gave + `agent/host_agent.py` its TLS mode (see the agent bullet above: one installed bundle, the + per-path client-cert check, the live `# REVOKED` denylist, `bundle_rev` in the snapshot) and + task 03 gave the collector the matching half: an `https://` `HOST_TARGETS` entry is fetched + with this app's client certificate and verified against its own root, and a host marked + `revoked` fails its `host:` and `disks:` jobs with `agent certificate revoked` **before any + socket opens** — spy-pinned in `tests/test_collectors.py` and again through the real + `build_hub` wiring in `tests/test_api.py`, where the revocation is written into the same `meta` + table the running app reads. So both halves are proven against each other with no network and + no hardware. What is still missing is an **installed** bundle: `HOST_TARGETS` is `http://` for + every host here and no agent runs `--cert-bundle`, so deleting `certs.*` costs nothing today + and everything once bundles are placed (which is what the reset message warns about). Proving + the dial also found one defect in task 01's output: the leaves carried no + `AuthorityKeyIdentifier`, and OpenSSL's strict mode — which is what Python's + `ssl.create_default_context()` turns on, and that is the context this dial builds — refuses + such a chain with `CERTIFICATE_VERIFY_FAILED: Missing Authority Key Identifier`. Fixed in + `certs.py` (SKI on the root and every leaf, AKI on every leaf, non-critical), pinned in + `tests/test_certs.py`, and the reason a bundle recorded before it must not be installed. Task + 04 put the same lifecycle behind the gate: `/api/certs` lists the **stored** authority (through + `certs.read_state`, so a page refresh cannot mint anything) with one row per configured host and + per existing entry, `GET /api/certs//bundle` serves the stored bytes as an `attachment`, + and the three writes rotate, revoke, and rotate the root with `restart_required: true` said out + loud — each leaving one fixed-text `certs`-source event and no PEM outside `meta` (pinned by a + grep of the stored rows and of every payload a browser reads). Task 05 put the page on top of it: + `/certs` (`⚷ Certs` in every masthead, including `/settings`) renders that document — the root + CN/serial/expiry, the display-only client line, one row per host with its status chip and + issued-vs-deployed bundle rev and its **Rotate** / **Revoke** / **bundle** controls — and keeps + two rules the API already keeps: no colour the payload did not send (no `severity` token on the + page at all; `unissued` is a grey chip, a behind bundle a neutral `stale` one) and no control + the API would refuse (no buttons on a `configured: false` row, no revoke or download on a + never-issued host, the download a plain session `GET`). Writes are `confirm()`d rather than + typed-`DANGER`d, and the restart sentence is the route's own `hint`. Task 06 has since landed + the installer half: `install-agent.sh --cert-bundle ` validates the bundle by the + agent's own text shape *before* `need_root` (a bad file says "wrong file", not "add a sudoers + line"; exit 2), installs it `0640 root:dashboards-agent` byte-identically — the task file's + literal `0600` cannot work, because the agent opens the bundle as itself and re-reads the + denylist live (the exact incident the token file's comment records), and the task's own + intent is "only root and the agent's own service user may read it", which is what 0640 + achieves — re-runs without the flag leave the installed bundle in place, the unit gains + `--cert-bundle` + `RuntimeDirectory=dashboards-agent` (`0700`) from one conditional branch + while keeping the token line, `--status` probes `https` with the bundle's CA and no client + certificate (`python3` `ssl`, never an external TLS binary; an unreadable bundle says *run + --status as root* instead of pretending), `--uninstall` sweeps the bundle with the token, and + `deploy/agent-sudoers` gained no line (the bundle travels through the same root invocation). + `tests/test_deploy_scripts.py` pins all of it, including the probe run end-to-end against a + real in-process TLS agent: CA-only reads `/health`, a foreign root is refused, token mode + keeps the plain probe. The final pass proved the composition rather than the halves: a real + `host_agent.py --cert-bundle` on loopback and a real dashboard process over a scratch database + (`HOST_TARGETS=loop=https://127.0.0.1:`) answer 27 checks — `host:loop`/`disks:loop` `ok` + through the certificate dial, `agent.tls`/`agent.bundle_rev` in both snapshots, the bundle + download byte-identical to the `meta` row, an unsigned `/certs` a `303` with `?next=`, a revoke + followed by `agent certificate revoked` on both jobs with no dial, the dashboard's own client + identity untouched by a *host* revoke (the list is serials, not hosts), and a revoked **client** + serial refused 403 by the running agent on `/health` too the moment the re-sealed bundle is in + place (`data/logs/verify-15-tls-e2e.py`, transcript `verify-15-tls-e2e.out`). What is left of + the phase is the live rollout (07): the fleet's agents still run token mode, so `HOST_TARGETS` + stays `http://` here until somebody installs a bundle on a host — which needs root on that host + (`ops` has no installer grant on `deskwork`, and `driveripper`'s unit must not be rewritten, gap + 10), and flipping the URL before the bundle is installed turns that host red by construction. ## Locked decisions (and why) @@ -1560,6 +1818,10 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl explicit gating. - **In-house Prometheus parser.** Only two upstreams use the format; keeps `prometheus_client` out of the dependency set. +- **`cryptography` for X.509, dashboard-side only** (phase 15). Minting a CA and signing leaves is + not something to hand-roll, and there is exactly one place in this app that does it + (`certs.py`). The agent gets no new dependency: it loads the PEMs the dashboard signed with stdlib + `ssl`, which is why the bundle is plain text and why the agent still runs on a host's Python 3.10. - **Port 8090**, inherited from the `llm_monitor` prototype this started from: 8000 is vLLM and 8731 is gufo in this fleet. - **`.env` is a seed, never an override** — container/systemd env must win. diff --git a/README.md b/README.md index adcd95a..dd50cf7 100644 --- a/README.md +++ b/README.md @@ -13,9 +13,11 @@ Four dashboards, one domain, one process, one SQLite file: No Prometheus, no Grafana, no CDN, no build step. Nothing is fetched from the internet at runtime; every asset is served by this app. -Two more pages are not dashboards: `/login` (the one thing an anonymous visitor sees) and +Three more pages are not dashboards: `/login` (the one thing an anonymous visitor sees), `/settings` — `⚙ Account` in the masthead of every dashboard — which shows this browser's -session, signs it out, and changes the admin password. +session, signs it out, and changes the admin password, and `/certs` — `⚷ Certs`, beside it — +which is the certificate authority the agents are dialled with: the root CA, this dashboard's +own client certificate, and one row per host with its rotate, revoke and bundle download. Working on this repo? **[`AGENTS.md`](AGENTS.md)** holds the invariants (read-only by construction, never invent a number, unconfigured ≠ broken), the locked decisions, the real @@ -70,14 +72,16 @@ and running the installer without root prints that exact line. upsd is dialled a `/sys/block`, `/sys/class/drm` and `statvfs` and answers `GET /snapshot` (everything, one JSON document; `?top=N` trims the process lists), `GET /info` (slow-changing identity: OS, kernel, CPU model, GPU list), `GET /schema` (the schema version plus this document) and -`GET /health` (the one route that never needs the token) as JSON, with a plain HTML status +`GET /health` (the one route that needs no credential) as JSON, with a plain HTML status page at `GET /` so a browser pointed at an agent learns what it is looking at. ```bash python3 host_agent.py --once # print one snapshot, exit python3 host_agent.py --host 0.0.0.0 --port 9120 --token-file /etc/dashboards-agent.token +python3 host_agent.py --host 0.0.0.0 --port 9120 --cert-bundle /etc/dashboards-agent.certs sudo deploy/install-agent.sh # systemd unit + user + token -deploy/install-agent.sh --status # unit state and /health +sudo deploy/install-agent.sh --cert-bundle agent.bundle # ...or the TLS install (bundle from /certs) +deploy/install-agent.sh --status # unit state and /health (https when TLS) sudo deploy/install-agent.sh --uninstall ``` @@ -106,8 +110,41 @@ one installed before phase 09 has no SMART pass to run. Opt out silently with `- The drive truth this is all developed against lives in `tests/fixtures/smartctl_driveripper_*.txt`: 16 real drives, recorded with serials scrubbed, each one parsed by a test. -Binding off-loopback without a token is refused (exit 2) rather than warned about. Prefer -`--token-file` over `--token`: argv is world-readable through `ps`, and this token guards +**TLS mode is `--cert-bundle` (env `AGENT_CERT_BUNDLE`)**: one installed PEM the dashboard +signed — this host's server certificate and key, the dashboard's root CA, a `# BUNDLE-REV ` +line and the `# REVOKED ` denylist. The agent then serves `https://` and answers every +path but `/health` only to a **client certificate signed by that root and not on the list**: 403 +`client certificate required` for a peer that proved nothing, 403 `client certificate revoked` +for one the operator withdrew (a revoked identity is refused on `/health` too — that path is open +to the installer's probe, which holds the CA and no certificate, not to a certificate that was +taken away). The denylist is re-read when the bundle file's stamp changes, so a re-deployed +bundle takes effect on the next request rather than the next restart; the certificate itself needs +a restart, because `ssl` cannot swap a chain under a live socket. The unpacked PEMs exist only +long enough to load into the context and are then deleted, and the private key is written `0600` +in the unit's `RuntimeDirectory` when it has one. In this mode `/snapshot` gains two additive +fields — `tls: true` and the `bundle_rev` the bundle carried, which is how the dashboard tells a +stale install from a current one (they reach `/api/system/snapshot` and `/api/disks/snapshot` as +`agent.tls` / `agent.bundle_rev`) — and **the bearer token is not read**: the handshake is the +auth, and a token that kept working after a revocation would defeat the list. The dashboard's half +is `collectors.AgentTls`: point that host's `HOST_TARGETS` entry at `https://` and the fetch +presents the dashboard's client certificate and verifies the agent against the same root, and a +host revoked in `certs.agents` is refused before a socket is opened. A bundle that will +not load is exit 2 naming the file; there is no "fall back to plain HTTP" path. + +`deploy/install-agent.sh --cert-bundle ` is how that file gets there. It validates the +bundle first by the same text shape (two certificates and exactly one key, plus the +`# BUNDLE-REV` line) — before touching anything, before the root check, and without ever +dialling the dashboard — then installs it to `/etc/dashboards-agent.certs` as +`0640 root:dashboards-agent`, the token-file idiom: the agent opens the bundle *itself* at +startup and re-reads it when its stamp changes, so the service user must be able to read it +and no other account may. The unit gains `--cert-bundle` and a private `RuntimeDirectory` for +the PEM blocks, `--status` probes `https://…/health` with the bundle's CA and no client +certificate (python3 `ssl`, never an external TLS binary), `--uninstall` removes the bundle +with the token, and a re-run without the flag leaves an installed bundle in place — the +token-reuse idiom. + +Binding off-loopback without a token *or* a bundle is refused (exit 2) rather than warned about. +Prefer `--token-file` over `--token`: argv is world-readable through `ps`, and this token guards hostname, mounts, disk and GPU models, and process names. `deploy/install-agent.sh` creates a system user, generates a token once (re-runs reuse it; `--rotate-token` for a new one), stores it `0640 root:dashboards-agent` — the unit hands the agent the *path*, and the agent @@ -116,13 +153,77 @@ and runs the agent under `ProtectSystem=strict`, `DevicePolicy=strict` with only nodes and the `/dev/null` family reopened (SMART needs the former; without `/dev/null` in the allow list systemd cannot start the service at all), with no capabilities. +### Managing the certificates + +The dashboard is the authority, and `/certs` (`⚷ Certs` in the masthead) is its page: the **Root +CA** block (CN, serial, expiry as a local timestamp *and* as time left, and **Rotate root CA**), +the **Dashboard client** line (the one certificate every agent verifies the dial against — +display-only, because it is minted with the root and has no route of its own), and one row per +host with its status chip, serial, expiry, issued-vs-deployed bundle revision, the addresses its +certificate is valid for, and **Rotate** / **Revoke** / **bundle**. + +Nothing there is coloured: `severity.py` rates no certificate, so an expiry is a date and a +bundle revision is a count. The one verdict on the page is the API's `status` word (`active` green, +`revoked` red, `unissued` grey — a certificate that does not exist never borrows a green pass). +Every write asks a `confirm()` first and prints the server's own sentence afterwards; a root +rotation answers `restart_required`, because the CA and the client pair are startup material — the +table is new at once, the running process keeps dialling with what it loaded, and every host then +needs its new bundle installed. A row whose host has left `HOST_TARGETS` is listed with no buttons +at all, because every per-host route refuses it. + +Under the page is the same API a script would call. Every route below is signed-in (and every write +is CSRF-gated like any other), and **none of them dials a host** — the effect of a revocation is +that the *next poll* refuses to dial (see `collectors.AgentTls` above): + +| Route | What it does | +| ----- | ------------ | +| `GET /api/certs` | `root` and `client` with serial and expiry, plus one row per host the operator could act on: every `HOST_TARGETS` name and every certificate still in the table. A configured host nobody has installed yet reads `status: "unissued"` with `null` serial rather than going missing; `bundle_rev` is what this dashboard issued and `deployed_bundle_rev` is what that host's agent reported back (so "the box is one bundle behind" is visible, `n/a` when nothing answered). Reads are pure — no page refresh ever mints a certificate | +| `GET /api/certs//bundle` | the exact bytes of the stored bundle as `application/x-pem-file` with `Content-Disposition: attachment; filename="agent-.bundle"`. That byte-equality is the point: `install-agent.sh` is handed the file and compares nothing else, so a route that re-signed on each read would make "is this the bundle I installed?" unanswerable. It carries that host's **private key**, which is why it is a download behind the session and `no-store` | +| `POST /api/certs//rotate` | fresh key, fresh serial, `bundle_rev` +1, `status: active` — and it is also how a host gets its **first** bundle (`bundle_rev` 1). The SAN comes from the current `HOST_TARGETS` URLs plus the name, not from the old certificate, because the address a host is dialled by is a deployment fact | +| `POST /api/certs//revoke` | marks the identity revoked, and **re-seals every other bundle** because the `# REVOKED` denylist lives inside each of them — so the fleet has new bundles to collect, while their `bundle_rev` does not move (their identity did not change). There is no unrevoke: the fix is a rotate | +| `POST /api/certs/root/rotate` | a new root, a new client certificate, every agent re-issued, and `{"ok": true, "restart_required": true}` — the CA and the client pair are startup material, so the table is new at once while the running process keeps dialling with what it loaded. Saying so beats pretending a hot-swap that does not exist | + +### Protecting an agent, host by host + +Three steps, and they are order-dependent because the dashboard picks its dial from the URL: + +```bash +# 1. one bundle per host, from the dashboard that owns the authority (a signed-in GET; it is +# byte-identical to the stored row and it carries that host's private key, so `curl -o` it +# somewhere private rather than `cat`ting it into a shell history) +curl -s -H "Authorization: Bearer $SESSION" -o agent-deskwork.bundle \ + http://127.0.0.1:8090/api/certs/deskwork/bundle + +# 2. install it beside the token — as root on the host, 0640 root:dashboards-agent. The unit +# gains `--cert-bundle` and its RuntimeDirectory, and restarts into TLS mode. +scp agent-deskwork.bundle -ops:/tmp/ +ssh -ops sudo /tmp/install-agent.sh --cert-bundle /tmp/agent-deskwork.bundle +ssh -ops sudo /tmp/install-agent.sh --status # https, with the CA and no client cert + +# 3. point that host's HOST_TARGETS entry at https with NO userinfo token — the certificate is +# the auth now — and restart the dashboard. `/certs` should then read issued == deployed rev. +export HOST_TARGETS="deskwork=https://:9120 …" +``` + +`install-agent.sh --bind … --port 9120` prints the `https://:9120` line for a bundle install, +so step 3 is a copy rather than a guess. A host left on `http://` keeps working over its bearer +token — the rollout is per host, and one revoked or expired certificate never blinks the others. + +A name that is not in `HOST_TARGETS` answers `404 {"configured": false}` **before** any signing, any +dial, and any `meta` write: a typo that minted a key would leave a certificate nobody can install +in the authority forever. Each accepted write leaves one fixed-text event — +`cert-rotated`, `cert-revoked`, `cert-root-rotated` under `source: "certs"`, readable at +`/api/events?source=certs` — and a certificate serial never leaves a `meta` row. Serials are served +as decimal **strings** for that reason: a 64-bit serial through a JS number is a serial the page +cannot match against a denylist. + ## Signing in One admin, one password, and the server **refuses to start** when it has neither a stored verifier nor `DASH_ADMIN_PASSWORD` — the domain is public and the fleet's hostnames and UPS controls sit behind this login, so there is no way to run it open by accident. There is no second account and no per-user view: the five pages (`/`, `/system`, `/disks`, `/llm`, `/ups`), -the `/settings` page and every data route belong to that one operator. +the two admin pages (`/settings`, `/certs`) and every data route belong to that one operator. A session is an **HMAC-signed cookie** (`dash_session`), not a row in a table: it carries the signed-in name and its own expiry, is valid for `DASH_SESSION_HOURS` (default `720` = 30 days), @@ -301,10 +402,10 @@ curl -s -H "Authorization: Bearer $SESSION" -X POST -H 'content-type: applicatio -d '{"ups":"apc","command":"test.panel.start"}' http://127.0.0.1:8090/api/ups/command ``` -That is the app's one write endpoint **that dials anything** (the `/api/auth/*` writes change only -this process and its SQLite `meta` table — invariant 1 is about the fleet), and the bearer header -changes nothing about its gates: a command that can cut output still needs `"confirm":"DANGER"`, -and is still refused while the UPS is on battery. +That is the app's one write endpoint **that dials anything** (the `/api/auth/*` and `/api/certs/*` +writes change only this process and its SQLite `meta` table — invariant 1 is about the fleet), and +the bearer header changes nothing about its gates: a command that can cut output still needs +`"confirm":"DANGER"`, and is still refused while the UPS is on battery. ### Lockout @@ -328,7 +429,7 @@ documentation and is never loaded, and `scripts/dev-server.sh` exports `.env.dev | Variable | Meaning | | ------------------------------------- | -------------------------------------------------------------- | -| `HOST_TARGETS` | `name=url` per host running the agent; a bearer token may ride in the URL userinfo (`http://TOKEN@host:9120`) | +| `HOST_TARGETS` | `name=url` per host running the agent; a bearer token may ride in the URL userinfo (`http://TOKEN@host:9120`). An `https://` URL is dialled as a *protected* agent instead: verified against the dashboard's own root CA and presented with its client certificate, with a revoked host refused before the dial (`certs.py`, `collectors.AgentTls`) | | `GUFO_BASE_URL` / `VLLM_BASE_URL` | LLM engines; unset means *not deployed here*, and its half of the page disappears | | `RATE_WINDOW_SECONDS` | the Δ window behind every measured rate — tokens/sec, bytes/sec, jiffies→CPU % (default 10); each value carries the window it actually got, and `/system`'s footnote prints that, not this default. A window the poll loop cannot honour — the ~0.2 s catch-up after a fetch slower than the interval, or a base older than 2× the window — answers one `n/a` instead of a spike, and a rate 8× above the last one is held until a window measured from a **different** sample confirms it (phase 13) | | `DASH_HOST` / `DASH_PORT` | bind address for the dashboard itself (default `127.0.0.1:8090`; the container sets `0.0.0.0`) | @@ -339,7 +440,7 @@ documentation and is never loaded, and `scripts/dev-server.sh` exports `.env.dev | `GUFO_API_KEY` / `VLLM_API_KEY` | only if an engine is behind a bearer token; they never leave the server | | `NUT_HOST` / `NUT_PORT` | upsd address — **whatever port it really listens on**, not always 3493. Reads are unauthenticated in stock NUT | | `NUT_USER` / `NUT_PASSWORD` | only needed for instant commands (battery tests) | -| `NUT_ALLOW_ACTIONS` | `1` enables the one write endpoint in the app | +| `NUT_ALLOW_ACTIONS` | `1` enables the one write endpoint that dials **anything** — the `/api/auth/*` and `/api/certs/*` writes touch only this process and its `meta` table | | `NUT_SIMULATE` | `1` replaces upsd with a **labelled** simulator (development only) | | `DASH_READ_ONLY` | `1` disables every mutating endpoint regardless of other flags | | `DASH_DB` | SQLite path (default `data/dashboards.sqlite3`) | @@ -372,6 +473,7 @@ sources/host.py · disks.py · llm.py · nut.py poller.py (one tas store.py ── SQLite (WAL): sample(source,host,metric,res,ts) + event + meta ── api.py → web/ ▲ auth.py ── verifier + session key, loaded once at startup (no row and no seed → no server) +certs.py ─ root CA + client cert, minted once at startup (unreadable row → no server) ``` * `poller.py` — every source gets its own asyncio task, its own health record, and its own @@ -404,13 +506,40 @@ auth.py ── verifier + session key, loaded once at startup (no row and no see once-a-minute write transaction from 5.4 s to a few milliseconds, which is what stopped a poller's insert losing its 5 s `busy_timeout` (known gap 4). Its `meta` table is the one place state that must outlive a restart goes (`get_meta`/ - `set_meta`/`delete_meta`), and its first user is the admin credential below. + `set_meta`/`delete_meta`). Its users are the admin credential below and, since phase 15, the + certificate authority in `certs.py` — five `certs.*` rows and nothing else. * `auth.py` — the admin login's decisions, in stdlib and without HTTP: a self-describing PBKDF2-HMAC-SHA256 verifier, an HMAC-signed stateless session cookie (no session table, and rotating the stored key is what evicts every other browser), a session-bound CSRF token, and an in-process login backoff. `load_auth_state()` is the startup half: the environment seeds a password **once**, the stored verifier wins from then on, and a server with neither refuses to start (see *Signing in*). +* `certs.py` — the fleet's certificate authority, in the same shape as `auth.py`: a pure lifecycle + over the `meta` table, no clock it cannot be handed, no logging, and one fixed-text raise. The + first boot of a database mints a **root CA** (`certs.root.key`/`.cert`, self-signed + `CN=dashboards-root`, 10 years) and the dashboard's own **client** certificate + (`certs.client.key`/`.cert`, `CN=dashboards`, 1 year); every later boot reads those rows and + regenerates nothing, because a root that changes on restart untrusts every certificate already + installed on a host. `issue_agent_cert`/`reissue_agent_cert` sign one server certificate per + `HOST_TARGETS` name (`CN` = the host, SAN = the addresses it is dialled by) into + `certs.agents`, and each entry's **bundle** — the agent's cert, the agent's key, the root *cert* + (never the root key), a `# BUNDLE-REV ` line and the current `# REVOKED ` lines — is + stored verbatim so the bytes an installer places are the bytes the row holds. `revoke` marks the + host and re-seals every bundle, because the denylist lives inside each of them; `rotate_root` + replaces the root, the client, and every agent at once. A stored PEM that will not parse (or a + client leaf the stored root did not sign) is a **fail-closed startup**, the admin-password rule: + the message is `sqlite3 "" "DELETE FROM meta WHERE key LIKE 'certs.%'"`, after which every + agent needs a fresh bundle. Every certificate carries a `SubjectKeyIdentifier` and every leaf an + `AuthorityKeyIdentifier` naming that root, because OpenSSL's strict mode — the one Python's + `ssl.create_default_context()` turns on, which is what the dial below builds — refuses a chain + without them. The **dial** is `collectors.AgentTls` (task 03): with an `https://` `HOST_TARGETS` + entry, the fetch verifies the agent against this root and presents this dashboard's own client + certificate, so only this dashboard can read the host; a host marked `revoked` fails its `host:` + and `disks:` jobs with `agent certificate revoked` *before any socket opens*, and because the + status is a fresh `certs.agents` read on every poll, revoking takes effect on the next poll with + no restart. An `http://` entry is untouched — same shared client, same bearer token — so the + rollout is per host. The private keys are credentials like any password: never served, never + logged, and the root key never leaves the row that holds it. * `severity.py` — the single green/amber/red table (`THRESHOLDS`, plus `INVERTED` for the metrics where lower is worse). Rated values in the API payloads travel with a sibling `severity` (`cpu.severity.usage_pct`, `memory.severity`, `disks.severity`, a UPS card's @@ -436,9 +565,16 @@ auth.py ── verifier + session key, loaded once at startup (no row and no see one per stale metric; the list dedupes per (source, metric) and is capped at 12 with a marker that says how many it hid. * `api.py` — the JSON every page reads: `/api/config`, `/api/health`, `/api/events`, - `/api/history`, `/api/metrics`, `/api/summary`, and one `snapshot` + `history` pair per - dashboard (`/api/system/*`, `/api/disks/*`, `/api/llm/*`, `/api/ups/*`). GETs everywhere - except the single `POST /api/ups/command`. Two rules make the pages legible: a source with + `/api/history`, `/api/metrics`, `/api/summary`, one `snapshot` + `history` pair per + dashboard (`/api/system/*`, `/api/disks/*`, `/api/llm/*`, `/api/ups/*`), and the certificate + authority's own surface (`/api/certs`, `/api/certs//bundle`, `POST …/rotate`, `POST + …/revoke`, `POST /api/certs/root/rotate`). GETs everywhere except `POST /api/ups/command`, + which stays the **only** route that dials anything: the `/api/certs/*` writes are `meta` row + writes through `certs.py`, the same class of write `POST /api/auth/password` is, and a + revocation's whole effect is that the next poll refuses to dial (see `certs.py` below). + Reads there are pure — `certs.read_state` is the request-time read and `certs.ensure_certs` the + startup bootstrap — so refreshing the list cannot mint a certificate, and it reports the + **stored** authority even after a root rotation the running process has not picked up yet. Two rules make the pages legible: a source with no env config answers `404 {"configured": false}` **without dialling** (so a page says "not configured" instead of going red), and a history query for a series that does not exist answers an empty series instead of an error (so a card asks for every plot it could @@ -458,7 +594,8 @@ auth.py ── verifier + session key, loaded once at startup (no row and no see per host — counts, `smartctl_available`, and `failed_devices`, so a notifier learns *which* drive the hardware called `FAILED` without a second call); it is additive, exactly as `alerts` was, and the key paths of both are pinned with and without targets configured. -* `web/` — five static pages (`/`, `/system`, `/disks`, `/llm`, `/ups`), one stylesheet, hand-rolled +* `web/` — five static dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`) plus the three + admin pages (`/login`, `/settings`, `/certs`), one stylesheet, hand-rolled SVG charts (`charts.js`), and a masthead status dot per dashboard job (`disks` has its own: the same agent can answer the 10-second utilisation poll and have a wedged SMART pass). No framework, no CDN, no bundler, and no narration: a page never explains its own pipeline, so @@ -640,6 +777,18 @@ local-testing word stays legal in prose, which is the only place it can be usefu the four public paths, the reset command, the four variables and the `DASH_COOKIE_SECURE=1` deployment note. Nothing starts a server there; a sentence that quietly disappears is the bug. +`tests/test_certs.py` is the same kind of file for the certificate authority, and two of its tests +are structural rather than behavioural on purpose: they read `certs.py` itself and fail if it gains +a `logging` import or a raise that interpolates anything, because a stored PEM is a private key and +this app turns log lines and error strings into rows a browser reads. The rest runs the whole +lifecycle over a dict store — first boot persists exactly five rows, a second boot changes no byte, +rotate/revoke/root-rotate and their denylists, the bundle never carrying the root key — and one test +completes a real client-certificate handshake with the material a startup would write. +`tests/test_agent.py` runs the matching TLS matrix against the real agent handler, and +`tests/test_deploy_scripts.py` runs the installer in a sandbox where `install`/`chown` are faked, so +the `0640` mode, the byte-identical bundle and the `--status` probe (against a live TLS agent) are +proven without a root shell. + `tests/store_perf.py` is the second opt-in marker's machinery (`live` is the other). It seeds a **production-shaped SQLite file** — `SampleStore`'s own retention (`raw 48 h / 1m 30 d / 1h 400 d`), the fleet's real @@ -699,8 +848,9 @@ the page never reads it. ## Battery-test buttons -Instant commands are the only mutating endpoint in the application (`POST /api/ups/command`) -and they are gated four times: `NUT_ALLOW_ACTIONS=1`, not `DASH_READ_ONLY`, an `upsd.users` +Instant commands are the only endpoint in the application that changes anything **outside this +process** (`POST /api/ups/command`; the session and certificate-authority writes change only this +app's own rows), and they are gated four times: `NUT_ALLOW_ACTIONS=1`, not `DASH_READ_ONLY`, an `upsd.users` account that actually has the command, and a confirmation in the UI — destructive ones (deep test, calibration, `load.off`) require typing `DANGER`. The server re-checks every gate; hiding a button is not a safety control. Commands that would cut power are refused diff --git a/TODO.md b/TODO.md index 4640904..71bf7ad 100644 --- a/TODO.md +++ b/TODO.md @@ -1 +1,4 @@ # TODO + +- [ ] The nav bar needs to collapse into a hamburger like brain of reese does +- [ ] The UPS tab should have an expandable, clear list of last incidents with dates/times \ No newline at end of file diff --git a/agent/host_agent.py b/agent/host_agent.py index 0cfa35f..62f0559 100644 --- a/agent/host_agent.py +++ b/agent/host_agent.py @@ -42,6 +42,8 @@ SCHEMA (GET /snapshot) ts float unix seconds (agent clock) duration_ms float collection cost agent_version str + tls true only when serving with --cert-bundle + bundle_rev int the `# BUNDLE-REV` of the bundle it loaded host {hostname, os_pretty, kernel, arch, python, boot_ts, uptime_seconds, cpu_model, cpu_cores_logical, cpu_cores_physical, container} cpu {modes: {user,nice,system,idle,iowait,irq,softirq,steal,guest, @@ -81,17 +83,26 @@ SCHEMA (GET /snapshot) errors [str] reads that failed; non-empty means the snapshot is partial Deployment: see deploy/install-agent.sh (systemd unit, binds 127.0.0.1 unless you pass ---host, requires a bearer token via --token or AGENT_TOKEN when bound off-loopback). +--host, requires a bearer token via --token or AGENT_TOKEN when bound off-loopback). The +other protection is `--cert-bundle`/AGENT_CERT_BUNDLE: one installed PEM (server cert, server +key, the dashboard's root CA, `# BUNDLE-REV ` and `# REVOKED ` lines) turns the +server into HTTPS that answers every path but `/health` only to a client certificate signed +by that root, and refuses one on the denylist on every path — the certificate is the auth, so +no token is read on that path. """ from __future__ import annotations import argparse +import contextlib import json import os import re +import shutil import socket +import ssl import subprocess +import tempfile import threading import time from collections.abc import Callable @@ -99,7 +110,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path from typing import Any -AGENT_VERSION = "1.0.0" +AGENT_VERSION = "1.1.0" SCHEMA_VERSION = 1 PAGE_SIZE = 0 # resolved from sysconf at runtime @@ -1136,6 +1147,267 @@ def _split_stat(stat: str) -> list[str] | None: return [match.group("comm"), *match.group("rest").split()] +# --------------------------------------------------------------------------- # +# TLS bundle: what the dashboard signs, what the installer places +# --------------------------------------------------------------------------- # + +#: The two metadata lines inside a bundle, spelled exactly as `src/dashboards/certs.py` writes +#: them. The agent stays stdlib (no `cryptography` on a monitored host), so the format is agreed +#: here by text rather than imported, and `tests/test_agent.py` parses a bundle that `certs.py` +#: actually produced — the only way the two halves can stay honest. +BUNDLE_REV_LINE = "# BUNDLE-REV " +REVOKED_LINE = "# REVOKED " + +#: Where the bundle is unpacked when systemd gave the unit a `RuntimeDirectory` (task 06): tmpfs, +#: private to the service user, gone when the unit stops. Otherwise a `mkdtemp` directory that is +#: removed as soon as the SSL context holds the material. +BUNDLE_RUNTIME_DIR = Path("/run/dashboards-agent") + +_PEM_BLOCK_RE = re.compile(r"-----BEGIN [A-Z0-9 ]+-----.*?-----END [A-Z0-9 ]+-----\n", re.DOTALL) +_CERT_HEADER = "-----BEGIN CERTIFICATE-----" +#: The unpacked file names, in the order `_bundle_blocks` hands the blocks back. +_BUNDLE_FILES = ("agent-cert.pem", "agent-key.pem", "root-ca.pem") + + +def _line_value(line: str, prefix: str) -> str: + """The first token after a bundle line's prefix, or "" when the line carries nothing.""" + rest = line[len(prefix):].strip() + fields = rest.split() + return fields[0] if fields else "" + + +def bundle_revision(text: str) -> int | None: + """The `# BUNDLE-REV ` number, or None when the line is absent or is not a number. + + Absent is an error rather than a default of 0: the revision is how the dashboard tells a stale + bundle from the one it just issued, so an agent serving "rev 0" would be reporting a number it + never had. + """ + for line in text.splitlines(): + stripped = line.strip() + if not stripped.startswith(BUNDLE_REV_LINE): + continue + token = _line_value(stripped, BUNDLE_REV_LINE) + try: + return int(token) + except ValueError: + return None + return None + + +def bundle_revoked(text: str) -> frozenset[str]: + """Every `# REVOKED ` value in a bundle, kept exactly as written.""" + serials: set[str] = set() + for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith(REVOKED_LINE): + serial = _line_value(stripped, REVOKED_LINE) + if serial: + serials.add(serial) + return frozenset(serials) + + +def _bundle_blocks(text: str) -> tuple[str, str, str]: + """(agent certificate, agent key, root certificate) — positionally, because that is the format. + + Two certificates and exactly one private key is the shape of a bundle. A bundle with the root + key in it is the failure this catches: an exporter needs to *verify* its clients, which is a + public-key operation, so a stolen agent install must never be a stolen CA. + """ + blocks = _PEM_BLOCK_RE.findall(text) + certs = [block for block in blocks if block.startswith(_CERT_HEADER)] + keys = [block for block in blocks if not block.startswith(_CERT_HEADER)] + if len(certs) != 2 or len(keys) != 1: + raise ValueError("expected the agent certificate, the agent key and the root certificate") + return certs[0], keys[0], certs[1] + + +def _file_stamp(path: str) -> tuple[int, int] | None: + """(mtime_ns, size) for the denylist re-read, or None when the file cannot be stated. + + Nanoseconds rather than the float `st_mtime`: two writes in the same millisecond are a real + redeploy, and a change this agent misses is a revoked certificate that keeps working. + """ + try: + info = os.stat(path) + except OSError: + return None + return (info.st_mtime_ns, info.st_size) + + +def _material_dir() -> Path: + """The private directory to unpack into: the unit's RuntimeDirectory when usable, else tmp.""" + if BUNDLE_RUNTIME_DIR.is_dir() and os.access(BUNDLE_RUNTIME_DIR, os.W_OK | os.X_OK): + return BUNDLE_RUNTIME_DIR + directory = Path(tempfile.mkdtemp(prefix="dashboards-agent-")) + os.chmod(directory, 0o700) # mkdtemp is 0700 already; the claim is this agent's, not the umask's + return directory + + +def _unlink(path: Path) -> None: + with contextlib.suppress(FileNotFoundError): + path.unlink() + + +def _write_private(path: Path, text: str) -> None: + """Create `path` at 0600 in one `open()`, so a private key never exists at the umask default.""" + _unlink(path) # O_EXCL would refuse a leftover from a killed run, and a symlink would be worse + handle = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + with os.fdopen(handle, "w", encoding="ascii") as stream: + stream.write(text) + + +def resolve_bundle(path: str) -> tuple[ssl.SSLContext | None, int, frozenset[str], str | None]: + """(context, bundle_rev, revoked serials, error) for one installed bundle. + + The `resolve_token` idiom: no globals, no server state, every failure an error string the + caller prints and exits on. A bundle this agent cannot load must stop the startup rather than + answer plain HTTP behind a token nobody configured — "degraded to no TLS" is the one answer + that is worse than a crash. + + The three PEM blocks are grepped out by text and written into a private directory, because + `ssl` loads a chain from paths; they are unlinked as soon as the context has them, so the key + then lives in this process's memory and nowhere else (and there is no shutdown path that could + forget to remove it). The root CA goes in as the *verify* list, which is the whole of what an + exporter needs: it authenticates its clients, it is never authenticated by them. + """ + try: + text = Path(path).read_text(encoding="utf-8", errors="replace") + except OSError as exc: + return None, 0, frozenset(), f"cannot read --cert-bundle {path}: {exc}" + if not text.strip(): + return None, 0, frozenset(), f"--cert-bundle {path} is empty" + revision = bundle_revision(text) + if revision is None: + return None, 0, frozenset(), ( + f"--cert-bundle {path} has no readable `# BUNDLE-REV ` line — the revision is how " + "the dashboard tells this bundle from a stale one, so a bundle without it is not one " + "this app signed") + try: + server_cert, server_key, root_cert = _bundle_blocks(text) + except ValueError as exc: + return None, 0, frozenset(), f"--cert-bundle {path} is not a bundle: {exc}" + directory: Path | None = None + ephemeral = False + try: + directory = _material_dir() + ephemeral = directory != BUNDLE_RUNTIME_DIR + written = [directory / name for name in _BUNDLE_FILES] + for target, body in zip(written, (server_cert, server_key, root_cert), strict=True): + _write_private(target, body) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.minimum_version = ssl.TLSVersion.TLSv1_2 + context.load_cert_chain(str(written[0]), str(written[1])) + context.load_verify_locations(str(written[2])) + # Optional at the handshake, enforced by the handler: `CERT_REQUIRED` would make the + # installer's `--status` and systemd's probe fail, because neither holds a client + # certificate the installer was never given, and a service that cannot health-check is a + # dead install. See `Handler._certificate_allows` for who may read what. + context.verify_mode = ssl.CERT_OPTIONAL + except (OSError, ValueError, ssl.SSLError) as exc: + return None, 0, frozenset(), f"cannot load --cert-bundle {path}: {exc}" + finally: + if directory is not None: + for name in _BUNDLE_FILES: + _unlink(directory / name) + if ephemeral: + shutil.rmtree(directory, ignore_errors=True) + return context, revision, bundle_revoked(text), None + + +class TlsBundle: + """One loaded bundle: the SSL context, the revision it carries, and the live denylist. + + The context and the revision are fixed for the process's life — `ssl` cannot swap a chain + under a live socket, so a new certificate needs a restart (and says so on `/certs`). The + `# REVOKED` list is not fixed: re-reading one small text file when its stamp changes costs + nothing per poll and is the difference between "revoked" and "revoked on the next deploy", + which is the only reason the lines live in the bundle instead of in the dashboard alone. + """ + + def __init__(self, path: str, context: ssl.SSLContext, bundle_rev: int, + revoked: frozenset[str]) -> None: + self.path = path + self.context = context + self.bundle_rev = bundle_rev + self._lock = threading.Lock() # ThreadingHTTPServer may re-read on several requests + self._revoked = revoked + self._stamp = _file_stamp(path) + + @classmethod + def load(cls, path: str) -> tuple[TlsBundle | None, str | None]: + """`resolve_bundle` plus the file stamp the denylist re-read compares against.""" + context, revision, revoked, error = resolve_bundle(path) + if error is not None or context is None: + return None, error + return cls(path, context, revision, revoked), None + + def revoked_serials(self) -> frozenset[str]: + """The denylist as the file holds it right now, re-read only when the file changed.""" + stamp = _file_stamp(self.path) + with self._lock: + if stamp is None or stamp == self._stamp: + return self._revoked + try: + text = Path(self.path).read_text(encoding="utf-8", errors="replace") + except OSError: + # Unreadable mid-redeploy: keep the list this agent already had. Forgetting a + # revocation because a file vanished for one second is the wrong direction. + return self._revoked + self._revoked = bundle_revoked(text) + self._stamp = stamp + return self._revoked + + +def _peer_certificate(connection: Any) -> dict[str, Any]: + """The verified client certificate of this connection, or {} when there is none. + + `getpeercert()` is only there on a TLS socket and only answers after a handshake, so a + plain-HTTP connection to a bundle-configured agent (a scanner, a port probe) reads as "no + certificate" and gets the same 403 as a certless TLS client rather than a traceback. + """ + getter = getattr(connection, "getpeercert", None) + if getter is None: + return {} + try: + peer = getter() + except (OSError, ValueError): # ValueError is `ssl` for "the handshake has not run" + return {} + return peer or {} + + +def _serial_forms(raw: str) -> frozenset[str]: + """Every spelling one certificate serial can be written in. + + `getpeercert()` renders a serial as **hexadecimal** text while `certs.py` writes the **decimal** + number into the bundle's `# REVOKED` line, and an agent that compared one spelling to the other + would accept a revoked client forever — a revocation that silently does nothing is worse than + no revocation list, because the operator believes it happened. Accepting both spellings on both + sides can only ever deny a certificate that is not revoked (two different serials whose hex and + decimal renderings coincide), never allow one that is. + """ + token = (raw or "").strip() + if not token: + return frozenset() + forms = {token} + for base in (10, 16): + try: + value = int(token, base) + except ValueError: + continue + forms.add(str(value)) + forms.add(format(value, "X")) + return frozenset(forms) + + +def serial_is_revoked(cert: dict[str, Any], revoked: frozenset[str]) -> bool: + """Is this peer certificate on the bundle's denylist?""" + peer = _serial_forms(str(cert.get("serialNumber") or "")) + if not peer: + return False + return any(_serial_forms(serial) & peer for serial in revoked) + + # --------------------------------------------------------------------------- # # HTTP server # --------------------------------------------------------------------------- # @@ -1165,10 +1437,18 @@ class Handler(BaseHTTPRequestHandler): server_version = f"host_agent/{AGENT_VERSION}" collector: Collector token: str | None = None + bundle: TlsBundle | None = None def do_GET(self) -> None: # the BaseHTTPRequestHandler API name — keep it path = self.path.split("?", 1)[0].rstrip("/") or "/" - if self.token and path != "/health": + bundle = self.bundle + if bundle is not None: + # The handshake is the auth on this path, so the bearer check below never runs: a + # bundle *and* a token would let a stolen token read a host whose certificate the + # operator just revoked, which is the one thing the denylist exists to prevent. + if not self._certificate_allows(bundle, path): + return + elif self.token and path != "/health": given = self.headers.get("Authorization", "") if given != f"Bearer {self.token}": self._send(401, {"ok": False, "detail": "missing or bad bearer token"}) @@ -1192,10 +1472,42 @@ class Handler(BaseHTTPRequestHandler): top = max(0, int(part[4:])) except ValueError: top = None - self._send(200, {"ok": True, **self.collector.snapshot(top=top)}) + body: dict[str, Any] = {"ok": True, **self.collector.snapshot(top=top)} + if bundle is not None: + # Additive, and only in TLS mode: the dashboard (task 03) reads the revision to + # tell a stale installed bundle from the one `/certs` just issued, and an absent + # field means "this agent predates certificates" rather than "revision 0". + body["tls"] = True + body["bundle_rev"] = bundle.bundle_rev + self._send(200, body) else: self._send(404, {"ok": False, "detail": f"no such path: {path}"}) + def _certificate_allows(self, bundle: TlsBundle, path: str) -> bool: + """Does this connection's client certificate authorise `path`? Sends the refusal if not. + + Two answers, because they are two different operator actions: 403 "required" is a client + that never proved who it is (nothing installed, or the wrong CA), 403 "revoked" is a client + that proved it and was withdrawn. + + The order is the design. A revoked identity is refused on **every** path, `/health` first: + the exemption is for the probe that holds the root CA and no client certificate at all, not + for a certificate the operator has just taken away. Then `/health` passes a certless probe, + because `install-agent.sh --status` and systemd's probe were never handed a client + certificate and a service that cannot health-check is a dead install. + """ + peer = _peer_certificate(self.connection) + present = bool(peer and peer.get("subject")) + if present and serial_is_revoked(peer, bundle.revoked_serials()): + self._send(403, {"ok": False, "detail": "client certificate revoked"}) + return False + if path == "/health": + return True + if not present: + self._send(403, {"ok": False, "detail": "client certificate required"}) + return False + return True + def _status_page(self) -> None: info = self.collector.host_info() uptime = info.get("uptime_seconds") @@ -1232,12 +1544,26 @@ class Handler(BaseHTTPRequestHandler): print(f"[agent] {self.address_string()} {fmt}", flush=True) -def serve(host: str, port: int, collector: Collector, token: str | None) -> None: +def serve(host: str, port: int, collector: Collector, token: str | None, + bundle: TlsBundle | None = None) -> None: + """Serve forever on host:port — plain HTTP with an optional token, or HTTPS with a bundle. + + All three class attributes are assigned on every call: `Handler` is process-wide state, so a + server started without a bundle must not inherit one from a server that had it (that would + answer a plain-HTTP socket with 403 on every path, or worse, accept a client certificate the + operator revoked). + """ Handler.collector = collector Handler.token = token + Handler.bundle = bundle httpd = ThreadingHTTPServer((host, port), Handler) - print(f"[agent] listening on http://{host}:{port} " - f"({'token required' if token else 'no token'}); pid {os.getpid()}", flush=True) + if bundle is not None: + httpd.socket = bundle.context.wrap_socket(httpd.socket, server_side=True) + print(f"[agent] listening on https://{host}:{port} with tls " + f"(client cert required, rev {bundle.bundle_rev}); pid {os.getpid()}", flush=True) + else: + print(f"[agent] listening on http://{host}:{port} " + f"({'token required' if token else 'no token'}); pid {os.getpid()}", flush=True) try: httpd.serve_forever() except KeyboardInterrupt: @@ -1271,6 +1597,12 @@ def main(argv: list[str] | None = None) -> int: parser.add_argument("--token-file", default=os.environ.get("AGENT_TOKEN_FILE", ""), help="read the bearer token from a file (keeps it out of ps); wins " "over --token") + parser.add_argument("--cert-bundle", default=os.environ.get("AGENT_CERT_BUNDLE", ""), + help="one installed PEM from the dashboard: this agent's server " + "certificate and key, the dashboard's root CA, a `# BUNDLE-REV` " + "line and the `# REVOKED` denylist. Serves https and answers " + "every path but /health only to a client certificate signed by " + "that root and not on the list; on that path the token is not read") parser.add_argument("--root", default="/", help="filesystem root to read (for tests)") parser.add_argument("--top", type=int, default=5, help="processes per top-N list") parser.add_argument("--net-all", action="store_true", help="include veth/bridge interfaces") @@ -1292,11 +1624,18 @@ def main(argv: list[str] | None = None) -> int: if error: print(f"[agent] {error}", flush=True) return 2 - if args.host not in ("127.0.0.1", "localhost", "::1") and not token: - print("[agent] refusing to bind off-loopback without --token (host metrics are " - "sensitive); pass --token, --token-file or set AGENT_TOKEN", flush=True) + bundle: TlsBundle | None = None + if args.cert_bundle: + bundle, bundle_error = TlsBundle.load(args.cert_bundle) + if bundle_error is not None: + print(f"[agent] {bundle_error}", flush=True) + return 2 + if args.host not in ("127.0.0.1", "localhost", "::1") and not token and bundle is None: + print("[agent] refusing to bind off-loopback without --token or --cert-bundle (host " + "metrics are sensitive); pass --token, --token-file, --cert-bundle or set " + "AGENT_TOKEN / AGENT_CERT_BUNDLE", flush=True) return 2 - serve(args.host, args.port, collector, token) + serve(args.host, args.port, collector, token, bundle) return 0 diff --git a/deploy/install-agent.sh b/deploy/install-agent.sh index 4fa143f..f592484 100755 --- a/deploy/install-agent.sh +++ b/deploy/install-agent.sh @@ -3,6 +3,7 @@ # # sudo ./deploy/install-agent.sh # loopback only (read it locally) # sudo ./deploy/install-agent.sh --bind 10.0.0.5 # serve the LAN, token required +# sudo ./deploy/install-agent.sh --cert-bundle agent.bundle # serve TLS (phase 15) # ./deploy/install-agent.sh --status # unit state + /health, no root # sudo ./deploy/install-agent.sh --uninstall # @@ -16,6 +17,21 @@ # names. Re-running the installer reuses the existing token — use --rotate-token when the old # one leaked. # +# The certificate bundle (--cert-bundle , fetched by the operator from the dashboard's +# GET /api/certs//bundle or the /certs page) is the TLS install: one PEM holding this +# host's server certificate and key, the dashboard's root CA, a `# BUNDLE-REV ` line and +# the revocation denylist. This script never dials the dashboard for it. The bundle is +# validated before anything is touched — a bundle the agent could not load must stop the +# install with its reason, not leave a unit that exits at startup — and then installed to +# /etc/dashboards-agent.certs, the token-file idiom: the agent opens the bundle *itself* at +# startup and re-reads it when its stamp changes (that is how a re-deployed revocation list +# lands without a restart), so the mode must let the agent's own service user read it and no +# other account. The unit then runs https with --cert-bundle and a RuntimeDirectory for the +# PEM blocks the agent unpacks (it deletes the key once the SSL context holds it), and +# --status probes /health over TLS with the bundle's CA and no client certificate — the path +# the agent exempts for probes. Re-running without the flag leaves an installed bundle in +# place, the token-reuse idiom. +# # The agent is read-only by construction (it answers GET and writes nothing), runs as its # own system user, and gets the usual systemd lockdown. It still needs to read /proc, /sys # and statvfs(), which the hardening below leaves alone. @@ -34,6 +50,7 @@ AGENT_USER="dashboards-agent" AGENT_DIR="/usr/local/lib/dashboards-agent" AGENT_TARGET="$AGENT_DIR/host_agent.py" TOKEN_FILE="/etc/dashboards-agent.token" +CERT_FILE="/etc/dashboards-agent.certs" UNIT="/etc/systemd/system/$SERVICE.service" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" AGENT_SOURCE="$SCRIPT_DIR/../agent/host_agent.py" @@ -51,6 +68,7 @@ BIND="127.0.0.1" PORT="9120" TOKEN="${AGENT_TOKEN:-}" ROTATE=0 +BUNDLE_SRC="" MODE="install" usage() { @@ -65,6 +83,7 @@ while [ $# -gt 0 ]; do --port) PORT="$2"; shift 2 ;; --token) TOKEN="$2"; shift 2 ;; --rotate-token) ROTATE=1; shift ;; + --cert-bundle) BUNDLE_SRC="$2"; shift 2 ;; --status) MODE="status"; shift ;; --uninstall) MODE="uninstall"; shift ;; -h|--help) usage 0 ;; @@ -81,7 +100,7 @@ need_root() { cat >&2 <&1 || true echo - if command -v curl >/dev/null 2>&1; then + if [ -r "$UNIT" ] && grep -q -- '--cert-bundle' "$UNIT"; then + # TLS install (phase 15): probe over https with the bundle's root CA and *no* client + # certificate. The agent's CERT_OPTIONAL handshake plus the /health exemption is what + # makes this possible — the installer holds no client identity, and a service that + # cannot health-check is a dead install. The CA is the bundle's last certificate block + # (the format is positional: agent cert, agent key, root CA). Chain-to-our-root is the + # claim; the leaf names the dial addresses rather than loopback, so the name check is + # off — forging the chain would still need the root key. python3 is the unit's own + # interpreter: there is no external TLS binary to install on a monitored host. + printf 'health: ' + python3 - "$BIND" "$PORT" "$CERT_FILE" <<'PY' || echo "(no answer)" +import re +import ssl +import sys +import urllib.request + +host, port, bundle = sys.argv[1], sys.argv[2], sys.argv[3] +try: + with open(bundle, encoding="utf-8", errors="replace") as handle: + text = handle.read() +except OSError as exc: + print(f"(cannot read {bundle}: {exc.strerror or exc} — run --status as root)") + raise SystemExit(0) +blocks = re.findall(r"-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----", text, re.DOTALL) +if len(blocks) < 2: + print(f"(no root CA block in {bundle})") + raise SystemExit(0) +context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) +context.check_hostname = False +context.verify_mode = ssl.CERT_REQUIRED +context.load_verify_locations(cadata=blocks[-1]) +try: + with urllib.request.urlopen( + f"https://{host}:{port}/health", timeout=4, context=context + ) as response: + print(response.read().decode("utf-8", "replace").strip()) +except Exception as exc: # the probe reports, it never propagates + print(f"(no answer: {exc})") +PY + elif command -v curl >/dev/null 2>&1; then + # token mode: the plain probe, unchanged printf 'health: ' curl -s --max-time 4 "http://$BIND:$PORT/health" || echo "(no answer)" echo @@ -115,7 +174,7 @@ status() { uninstall() { need_root --uninstall systemctl disable --now "$SERVICE" 2>/dev/null || true - rm -f "$UNIT" "$TOKEN_FILE" "$AGENT_TARGET" + rm -f "$UNIT" "$TOKEN_FILE" "$CERT_FILE" "$AGENT_TARGET" rmdir "$AGENT_DIR" 2>/dev/null || true systemctl daemon-reload echo "removed $SERVICE (the $AGENT_USER user was left in place; remove it with userdel)" @@ -127,11 +186,72 @@ new_token() { python3 -c 'import secrets; print(secrets.token_urlsafe(32).rstrip("="))' } +validate_bundle() { + # The `resolve_bundle` idiom from agent/host_agent.py, run before anything is touched: + # a bundle the agent could not load must stop the install with its reason, not leave a + # unit that exits at startup. Shape only, by text — the agent parses the same way (it is + # stdlib; a monitored host has no cryptography) and the dashboard is the only signer, so + # a second copy of this shape check would only ever disagree about nothing. The three PEM + # blocks are positional: agent certificate, agent key, root CA certificate. + if ! command -v python3 >/dev/null 2>&1; then + echo "--cert-bundle needs python3 (3.10 or newer) to be validated" >&2 + exit 2 + fi + if python3 - "$1" <<'PY' 1>&2 +import re +import sys + +path = sys.argv[1] +try: + with open(path, encoding="utf-8", errors="replace") as handle: + text = handle.read() +except OSError as exc: + print(f"--cert-bundle {path} cannot be read: {exc.strerror or exc}") + raise SystemExit(1) +if not text.strip(): + print(f"--cert-bundle {path} is empty") + raise SystemExit(1) +revision = None +for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith("# BUNDLE-REV "): + fields = stripped[len("# BUNDLE-REV "):].split() + if fields: + try: + revision = int(fields[0]) + except ValueError: + revision = None + break +if revision is None: + print(f"--cert-bundle {path} has no `# BUNDLE-REV ` line — that line is how the" + " dashboard tells the bundle it issued from a stale one, so a bundle without it" + " is not one this app signed") + raise SystemExit(1) +blocks = re.findall(r"-----BEGIN [A-Z0-9 ]+-----.*?-----END [A-Z0-9 ]+-----", text, re.DOTALL) +certificates = [b for b in blocks if b.startswith("-----BEGIN CERTIFICATE-----")] +keys = [b for b in blocks if not b.startswith("-----BEGIN CERTIFICATE-----")] +if len(certificates) != 2 or len(keys) != 1: + print(f"--cert-bundle {path} must hold exactly the agent certificate, the agent key and" + f" the root CA certificate — found {len(certificates)} certificates and" + f" {len(keys)} keys. Re-download it from GET /api/certs//bundle") + raise SystemExit(1) +PY + then + return 0 + fi + exit 2 +} + # NOT named `install`: bash resolves a function before an external command of the same name, # so an `install()` here turns the `install -d` / `install -m` calls below into infinite # recursion instead of creating $AGENT_DIR. Phase 10 task 02 found exactly that on a live # host: the run looped on the SMART block forever and installed nothing. do_install() { + # Validate before need_root: a bad bundle is refused with its own reason whether or not + # this shell is root, so a wrong file says "wrong file" and not "add a sudoers line". + if [ -n "$BUNDLE_SRC" ]; then + validate_bundle "$BUNDLE_SRC" + fi need_root --bind "$BIND" --port "$PORT" [ -f "$AGENT_SOURCE" ] || { echo "no agent next to this script: $AGENT_SOURCE" >&2; exit 1; } if ! command -v python3 >/dev/null 2>&1; then @@ -177,6 +297,36 @@ do_install() { chmod 0640 "$TOKEN_FILE" chown root:"$AGENT_USER" "$TOKEN_FILE" + # The certificate bundle is the token's twin and the mode is the whole design. It holds + # the agent's *private key*, so every other account — ops included — must not read it; + # but the agent process (running as $AGENT_USER) opens the bundle itself at startup and + # re-reads it when its stamp changes (that live re-read is how a re-deployed denylist + # lands without a restart), so a 0600 root-owned file would fail the unit exactly the way + # it would fail the token file. 0640 root:$AGENT_USER is the smallest grant that works. + # An existing bundle survives a re-run that was not given the flag — the token-reuse + # idiom — and this script never dials the dashboard to fetch one. + if [ -n "$BUNDLE_SRC" ]; then + install -m 0640 "$BUNDLE_SRC" "$CERT_FILE" + chown root:"$AGENT_USER" "$CERT_FILE" + echo " installed the certificate bundle in $CERT_FILE" + elif [ -f "$CERT_FILE" ]; then + echo "reusing the installed certificate bundle in $CERT_FILE (--cert-bundle for a new one)" + fi + + # TLS mode: the unit runs https with the installed bundle. The --token-file line stays — + # the agent skips the token check on the TLS path — so a rollout happens host by host + # instead of on a flag day. + TLS_EXEC_ARGS="" + TLS_UNIT_LINES="" + if [ -f "$CERT_FILE" ]; then + TLS_EXEC_ARGS=" --cert-bundle $CERT_FILE" + TLS_UNIT_LINES="# The TLS bundle unpacks its PEM blocks here: tmpfs, private to the unit, +# gone when it stops. ProtectSystem=strict leaves the agent nothing else writable, and the +# agent deletes the key the moment the SSL context holds it. +RuntimeDirectory=dashboards-agent +RuntimeDirectoryMode=0700" + fi + cat > "$UNIT" <&2 exit 1 fi - cat <=0.54.0", "pydantic>=2.14.1", "python-dotenv>=1.2.4", + # Phase 15: the dashboard mints its own root CA and signs the agent certs (`certs.py`). + # Dashboard-side ONLY — `agent/host_agent.py` stays stdlib and loads the PEMs with `ssl`. + # Wheels exist for every platform we build on, and it is the only sane way to issue X.509. + "cryptography>=50.0.2", ] [dependency-groups] diff --git a/src/dashboards/api.py b/src/dashboards/api.py index 3378c71..264195a 100644 --- a/src/dashboards/api.py +++ b/src/dashboards/api.py @@ -15,17 +15,20 @@ from __future__ import annotations import logging import math +import re import time from collections.abc import Sequence from dataclasses import dataclass, field, replace from typing import Any +from urllib.parse import urlsplit -from fastapi import APIRouter, Body, Depends, HTTPException, Query, Request +from fastapi import APIRouter, Body, Depends, HTTPException, Query, Request, Response from fastapi.responses import JSONResponse from . import alerts as alert_rules -from . import auth +from . import auth, certs from .auth import ADMIN_USER, LOGIN_PATH, AuthState, LoginThrottle, Session +from .certs import CertMaterial from .collectors import NutMonitor from .config import Settings, strip_userinfo from .poller import Hub, SourceState @@ -75,6 +78,14 @@ class AppContext: the session gate (task 03) may treat it as always present and still has to answer a rejection rather than a 500 if it somehow is not. + `certs` is the same story one level down (phase 15): the dashboard's own CA and client + certificate are loaded once at startup and written to a private temp directory, because + `httpx` takes paths and not PEM text. The `host:`/`disks:` jobs dial an `https://` + `HOST_TARGETS` entry with those paths (`collectors.AgentTls`, task 03), and an `http://` + entry still goes out on the shared client with its bearer token. It is `None` only on a + context assembled outside the lifespan, which is what makes `agent_tls_map` answer an empty + map instead of pointing every TLS host at files that were never written. + `login_throttle` is a field default rather than a lifespan assignment for one reason: the context is constructed exactly once per app in both assembly branches, so the default *is* "one throttle per process" — and `None` would be a live branch in the login route @@ -88,6 +99,7 @@ class AppContext: nut_monitor: NutMonitor | None = None started_at: float = 0.0 auth: AuthState | None = None + certs: CertMaterial | None = None login_throttle: LoginThrottle = field(default_factory=LoginThrottle) @@ -1001,3 +1013,288 @@ def _action_event(ups_name: str, command: str, advice: dict[str, str], detail: s severity = "warn" if advice["risk"] != nut.RISK_SAFE else "info" return Event(time.time(), "ups", ups_name, "action", severity, f"{advice['label']} sent to {ups_name}: {detail}", {"command": command}) + + +# --------------------------------------------------------------------------- # +# Certificate authority (phase 15) +# --------------------------------------------------------------------------- # + +#: The event source for every certificate action, the `auth` precedent: `/api/events?source=certs` +#: is the audit trail, and `alerts.py` promotes only `kind: "unreachable"`, so a rotation can never +#: blink the overview. No dashboard feed reads this source: `/certs` prints the outcome of the +#: write the operator just made (the route's own `hint`), and the audit trail of every past one is +#: this source at `/api/events?source=certs`. +CERT_SOURCE = "certs" + +#: The row status of a host in `HOST_TARGETS` the CA has never signed. Not a `certs.py` status +#: (nothing is stored for such a host): it is the API saying "issue it" in the same field the +#: issued rows use, instead of dropping the row and letting the page infer a machine from a gap +#: (invariant 3 — a configured thing that has not happened is not the same as an absent thing). +UNISSUED = "unissued" + +#: A bundle is a private key, so it is a file and never a rendered page. `application/x-pem-file` +#: plus `attachment` is what makes the browser save it instead of opening a key in a tab. +PEM_MEDIA_TYPE = "application/x-pem-file" + + +def _cert_host_names(ctx: AppContext) -> list[str]: + """Every `HOST_TARGETS` name, de-duplicated, in config order.""" + return list(dict.fromkeys(target.name for target in ctx.settings.host_targets)) + + +def _require_cert_host(ctx: AppContext, name: str) -> None: + """A name this dashboard does not dial is not a certificate it manages (invariant 3). + + The **first** line of every `//…` route, before a store read and before any signing: a + typo that minted a key would put a certificate nobody can ever install into the CA's table and + bump nothing, and a typo on the revoke half would mark revoked a host the dashboard never + dials. The answer is `404 {"configured": false}` with no side effect at all, which is the + `/api/disks` gate's shape and is spy-pinned from both directions in `tests/test_api.py`. + """ + if name not in _cert_host_names(ctx): + raise HTTPException(404, detail={"configured": False, + "detail": f"no host {name!r} in HOST_TARGETS"}) + + +def _cert_san(ctx: AppContext, name: str) -> list[str]: + """The addresses this dashboard dials for one host — the SAN its certificate must carry. + + A TLS peer is verified by *name*, so the SAN has to be whatever goes on the wire: the URL host + of every `HOST_TARGETS` entry naming this host (an IP is entered as an IP address, which is + `certs.py`'s decision, not this one's). `display_url` rather than `url` keeps userinfo out of + the name list even for a hand-built `Target`, and the host's own name is added by the signing + call — this function returns what to add, not the whole list. + """ + hosts: list[str] = [] + for target in ctx.settings.host_targets: + if target.name != name: + continue + host = urlsplit(target.display_url).hostname + if host and host not in hosts: + hosts.append(host) + return hosts + + +def _deployed_bundle_rev(ctx: AppContext, name: str) -> int | None: + """The `bundle_rev` this host's agent reported about itself, or `None`. + + Read out of the same in-memory `system` payload every panel reads (task 03 carries the agent's + own `tls`/`bundle_rev` into its `agent` block), so "is this box running the bundle I just + issued?" is answered with what the host said rather than what the CA hopes. `None` for a host + that has never answered and for a token-mode agent, which publishes nothing — the page prints + `n/a`, and `n/a` and "version 1" are different sentences (rule 2). + """ + for job in ctx.hub.jobs.values(): + if job.dashboard != "system" or job.label != name: + continue + agent = (job.state.payload or {}).get("agent") or {} + revision = agent.get("bundle_rev") + if isinstance(revision, int) and not isinstance(revision, bool): + return revision + return None + return None + + +def _cert_rows(ctx: AppContext, stored: Sequence[certs.AgentCert]) -> list[dict[str, Any]]: + """One row per host the operator could act on: every configured host, plus every entry. + + The union rather than the stored table alone, because each side has a fact the other cannot + show: a configured host with no certificate is the *issue it* case, and an entry whose host has + left `HOST_TARGETS` is a private key still sitting in the table. `configured` names which half + of the union a row came from, and the per-host routes keep the configured gate either way — a + machine this dashboard no longer dials is not one to install a bundle on, so its row is listed + and not writable (`tests/test_api.py` pins that asymmetry). Nulls mean "no certificate on + record", never `0`, and a row with `status: unissued` carries `san: []` because there are no + names on record rather than a certificate with no names in it (rule 2). + """ + entries = {entry.name: entry for entry in stored} + names = sorted(set(entries) | set(_cert_host_names(ctx))) + live = {name: _deployed_bundle_rev(ctx, name) for name in names} + rows: list[dict[str, Any]] = [] + for name in names: + entry = entries.get(name) + rows.append({ + "name": name, + "configured": name in set(_cert_host_names(ctx)), + # A certificate serial is up to 64 bits and JS numbers are 53, so it travels as the + # decimal text the bundle's `# REVOKED` lines use: a serial the page rounds is a serial + # the page cannot match against a denylist. + "serial": str(entry.serial) if entry else None, + "not_after": entry.not_after if entry else None, + "status": entry.status if entry else UNISSUED, + "bundle_rev": entry.bundle_rev if entry else None, + "deployed_bundle_rev": live[name], + "san": list(entry.san) if entry else [], + }) + return rows + + +def _cert_event(kind: str, host: str, severity: str, message: str, + data: dict[str, Any] | None = None) -> Event: + """One certificate audit line. + + `message` is fixed text naming the host and the action and nothing else: no PEM, no serial of + ours, no reason. An `event` row is served by `/api/events` to a browser and kept forever, so + the only facts in it are the ones an operator can act on (AGENTS.md invariant 6). The `data` + payload carries the bundle revision — a counter, not a credential — because the follow-up question + after "who rotated this?" is "which bundle do I have to install now?". + """ + return Event(time.time(), CERT_SOURCE, host, kind, severity, message, data) + + +def _bundle_filename(name: str) -> str: + """The `Content-Disposition` name for one host's bundle. + + The name is already constrained — `_require_cert_host` only lets through a `HOST_TARGETS` name, + and `parse_targets` validates those against `^[a-z0-9][a-z0-9_.-]*$` — so there is no quote or + newline to escape here. The filter is still applied because this string goes into a **header** + and the guarantee comes from a config value: a header built out of configuration is a header + injection waiting for somebody to loosen the config parser. + """ + safe = re.sub(r"[^A-Za-z0-9_.-]", "_", name) or "agent" + return f'attachment; filename="agent-{safe}.bundle"' + + +def certs_router() -> APIRouter: + """`/api/certs` — the CA's own surface: list it, download a bundle, rotate, revoke. + + **Nothing here dials an upstream** (invariant 1). The writes are `meta` row writes through + `certs.py` in this process, which is the same class of write `POST /api/auth/password` is: the + fleet is not touched, and the enforcement of a revocation happens later, in + `collectors.fetch_snapshot`, which refuses the host before it opens a socket (task 03). The + event row this route writes is therefore the audit line and not the control — worth saying + because it is the opposite of `/api/ups/command`, where the write *is* the effect. + + All four writes are gated by the session middleware like every other non-public route + (invariant 12): a session, a same-origin `Origin`/`Sec-Fetch-Site`, and — on the cookie — this + session's `X-CSRF-Token`. A revoked-token write changes no row, which is what + `tests/test_auth_api.py` measures. + """ + router = APIRouter(prefix="/api/certs", tags=["certs"]) + + @router.get("") + def listing(ctx: AppContext = Depends(get_context)) -> dict[str, Any]: + """Every certificate this CA holds, with its expiry and status — a read, never an issue. + + `root` and `client` are read from the **stored** rows (`certs.read_state`), not from the + material this process loaded at startup: after `POST /api/certs/root/rotate` the table holds + the new CA while the socket still uses the old one, and the page has to be able to show what + the authority actually contains. Reading the state is also what keeps a page refresh free of + side effects — a GET that minted a certificate would make "did anything change?" unanswerable. + + No `severity` anywhere: an expiry is a date the page prints, and `severity.py` rates no + certificate. A cert about to expire is worth reading, not colouring by a threshold nobody + measured — and the renewal is one button press away on the page this feeds. + """ + state = certs.read_state(ctx.store) + return { + "root": {"serial": str(state.root_serial), "not_after": state.root_not_after}, + "client": {"serial": str(state.client_serial), "not_after": state.client_not_after}, + "agents": _cert_rows(ctx, certs.read_agents(ctx.store)), + } + + # Declared before `/{name}/rotate` on purpose: Starlette matches in registration order, so the + # CA-wide route wins for the literal name `root`. A host called `root` is therefore refused + # below rather than silently rotated past — the wrong-direction failure there is a fleet-wide CA + # rotation nobody meant to do, not a missing button. + @router.post("/root/rotate") + def rotate_root(ctx: AppContext = Depends(get_context)) -> dict[str, Any]: + """A new root CA, a new client certificate, every agent re-issued under it. + + `restart_required: true` is the honest constraint rather than a hedge: the CA and the + client pair are startup material (`app.build_certs` writes them once, and `ssl` cannot swap + a chain under a live socket), so the stored authority is new immediately while this process + keeps dialling with the material it loaded. Saying so is the only way the operator learns to + restart before wondering why the fleet still chains to the old root — and every host needs + its new bundle installed, which is why the answer counts them. + """ + if "root" in _cert_host_names(ctx): + raise HTTPException(409, "a host in HOST_TARGETS is named `root`, which this route " + "would shadow — rename that host before rotating the CA") + state = certs.rotate_root(ctx.store, certs.ensure_certs(ctx.store)) + ctx.store.log_event(_cert_event( + "cert-root-rotated", "root", "warn", + "root certificate rotated — every agent certificate reissued, restart required", + {"agents": len(state.agents)})) + return {"ok": True, "restart_required": True, + "root": {"serial": str(state.root_serial), "not_after": state.root_not_after}, + "agents": len(state.agents), + "hint": "restart the dashboard, then install every host's new bundle"} + + @router.get("/{name}/bundle") + def bundle(name: str, ctx: AppContext = Depends(get_context)) -> Response: + """The exact bytes the agent should hold, as a file. + + Served from the stored `certs.agents` row verbatim, which is the whole point of storing the + bundle instead of re-signing it on demand: "is the box running the bundle I downloaded?" + has to be answerable with a checksum, and a route that re-signed would make it answerable + only by luck. The bundle contains that host's **private key** by design — it is the install + artifact — which is why the route is behind the session, `no-store`, and an `attachment`. + """ + _require_cert_host(ctx, name) + entry = next((item for item in certs.read_agents(ctx.store) if item.name == name), None) + if entry is None: + # A configured host with nothing issued is a different fact from an unconfigured name, + # and the body says which: `configured` stays true so no caller reads this as "this + # machine is not deployed" (invariant 3). + raise HTTPException(404, detail={ + "configured": True, "issued": False, + "detail": f"no certificate issued for {name} yet — " + f"POST /api/certs/{name}/rotate mints one"}) + return Response(content=entry.bundle, media_type=PEM_MEDIA_TYPE, + headers={"content-disposition": _bundle_filename(name), **NO_STORE}) + + @router.post("/{name}/rotate") + def rotate_agent(name: str, ctx: AppContext = Depends(get_context)) -> dict[str, Any]: + """Re-issue one host's identity: fresh key, fresh serial, `bundle_rev` +1. + + The SANs come from the config rather than from the previous certificate, because the address + a host is dialled by is a deployment fact that may well have changed — and a rotated + certificate that no longer matches the URL is a red source with a handshake error instead of + an answer. This is also how a host gets its **first** bundle (`bundle_rev` 1), which is why + there is no separate "issue" route. + """ + _require_cert_host(ctx, name) + state = certs.reissue_agent_cert(ctx.store, certs.ensure_certs(ctx.store), name, + _cert_san(ctx, name)) + entry = state.agent(name) + if entry is None: # pragma: no cover - reissue always returns the entry it just wrote + raise HTTPException(500, detail={"ok": False, "detail": "reissue wrote nothing"}) + ctx.store.log_event(_cert_event("cert-rotated", name, "warn", + f"agent certificate rotated for {name}", + {"bundle_rev": entry.bundle_rev})) + return {"ok": True, "name": name, "serial": str(entry.serial), + "not_after": entry.not_after, "status": entry.status, + "bundle_rev": entry.bundle_rev, "san": list(entry.san), + "deployed_bundle_rev": _deployed_bundle_rev(ctx, name), + "hint": f"install the new bundle on {name} — /api/certs/{name}/bundle"} + + @router.post("/{name}/revoke") + def revoke_agent(name: str, ctx: AppContext = Depends(get_context)) -> dict[str, Any]: + """Strike one host's identity out — and re-seal every other bundle with the new denylist. + + The effect is enforced on the next poll, by this dashboard refusing to dial the host at all + (task 03), and by any agent holding this serial on its `# REVOKED` list refusing that + client. Every other host's bundle is rewritten here because the denylist lives *inside* the + bundle: an agent that cannot see the serial keeps accepting the certificate it was given, + which is why the hint says the fleet has new bundles to collect and why their + `bundle_rev` did **not** move (their identity did not change). + + There is no unrevoke. The fix for a struck-out identity is a new one, which is `rotate`. + """ + _require_cert_host(ctx, name) + state = certs.revoke(ctx.store, certs.ensure_certs(ctx.store), name) + entry = state.agent(name) + if entry is None: # pragma: no cover - revoke always returns the entry it just wrote + raise HTTPException(500, detail={"ok": False, "detail": "revoke wrote nothing"}) + ctx.store.log_event(_cert_event("cert-revoked", name, "warn", + f"agent certificate revoked for {name}", + {"bundle_rev": entry.bundle_rev})) + return {"ok": True, "name": name, "serial": str(entry.serial), + "not_after": entry.not_after, "status": entry.status, + "bundle_rev": entry.bundle_rev, + "deployed_bundle_rev": _deployed_bundle_rev(ctx, name), + "hint": f"{name} is refused on the next poll without dialling; every other " + f"bundle now lists this serial and needs re-installing"} + + return router diff --git a/src/dashboards/app.py b/src/dashboards/app.py index 70b30bc..54022fc 100644 --- a/src/dashboards/app.py +++ b/src/dashboards/app.py @@ -3,18 +3,22 @@ `create_app()` takes injectable settings and store so tests can build an app with an in-memory DB and no targets — the same code path production uses, without the network. -Startup is fail-closed: the lifespan builds the admin :class:`~dashboards.auth.AuthState` -before the first poll, and a server with neither a stored verifier nor `DASH_ADMIN_PASSWORD` -raises out of the lifespan and exits non-zero instead of serving the fleet to whoever finds the -domain. The `RuntimeError` is deliberately not caught here — uvicorn prints it and leaves, and -the message is the operator's fix. +Startup is fail-closed twice over. The lifespan builds the admin +:class:`~dashboards.auth.AuthState` before the first poll, and a server with neither a stored +verifier nor `DASH_ADMIN_PASSWORD` raises out of the lifespan and exits non-zero instead of +serving the fleet to whoever finds the domain. It also builds the +:class:`~dashboards.certs.CertMaterial` before the poll loops start (phase 15): a stored root CA +or client certificate that will not parse raises the same way, because a dashboard that cannot +present its own client certificate would spend the whole fleet's TLS history reporting an upstream +problem it caused itself. Neither `RuntimeError` is caught here — uvicorn prints it and leaves, +and the message is the operator's fix. """ from __future__ import annotations import os import time -from collections.abc import AsyncIterator, Awaitable, Callable +from collections.abc import AsyncIterator, Awaitable, Callable, Mapping from contextlib import asynccontextmanager from pathlib import Path from typing import Any @@ -27,11 +31,12 @@ from fastapi.staticfiles import StaticFiles from starlette.exceptions import HTTPException as StarletteHTTPException from starlette.routing import Match -from . import auth +from . import auth, certs from .api import ( NO_STORE, AppContext, auth_router, + certs_router, common_router, disks_router, llm_router, @@ -40,9 +45,12 @@ from .api import ( ups_router, ) from .auth import load_auth_state +from .certs import CertMaterial from .collectors import ( + AgentTls, NutMonitor, UpstreamError, + agent_tls_map, disks_jobs, host_jobs, llm_jobs, @@ -62,9 +70,11 @@ WEB = Path(__file__).resolve().parent / "web" # `PUBLIC_PATHS` (so the gate guards it with no extra code — the route-enumeration test in # `tests/test_auth_api.py` proves that, and `tests/test_api.py` proves the anonymous refusal for # this page specifically). It carries no `aria-current` tab because it is not one of the four. +# `/certs` (phase 15 task 05) is the second of those admin pages, for the same reason: the +# certificate authority is not a fifth dashboard, it is the thing you administer past them. PAGES = {"/": "index.html", "/system": "system.html", "/disks": "disks.html", "/llm": "llm.html", "/ups": "ups.html", "/login": "login.html", - "/settings": "settings.html"} + "/settings": "settings.html", "/certs": "certs.html"} VERSION = "0.1.0" # Everything else requires a signed-in admin. This list is the whole attack surface, so @@ -89,8 +99,8 @@ def is_public(path: str) -> bool: return any(rooted.startswith(prefix) for prefix in PUBLIC_PREFIXES) -def build_hub(settings: Settings, store: SampleStore, - client: httpx.AsyncClient) -> tuple[Hub, NutMonitor | None]: +def build_hub(settings: Settings, store: SampleStore, client: httpx.AsyncClient, + tls: Mapping[str, AgentTls] | None = None) -> tuple[Hub, NutMonitor | None]: """Every poll loop the settings describe, on one hub. One function because the assembly is wired twice: `build_context()` for production, and @@ -98,6 +108,12 @@ def build_hub(settings: Settings, store: SampleStore, one that drifts is the one production does not run — which is how a source can end up polling in tests and not in the container. + `tls` is the per-host certificate material (`agent_tls_map`, built from `build_certs`), and + it is a parameter rather than something this function derives because the two `host:`/`disks:` + jobs for one host must hold **the same** entry: the same identity on the wire and the same + revocation answer, so a revoked agent fails both of its sources with one sentence and neither + of them opens a socket. + Order is host → disks → llm → ups so far only for readability: two jobs carry the same host label (`host:gw`, `disks:gw`), so anything that looks a job up by label alone gets the system record — `alerts.py` filters by dashboard instead and does not depend on it. @@ -105,10 +121,10 @@ def build_hub(settings: Settings, store: SampleStore, rates = RateEstimator(span=settings.rate_window) hub = Hub(store) for job in host_jobs(list(settings.host_targets), client, HostCollector(rates), - settings.host_interval): + settings.host_interval, tls=tls): hub.add(job) for job in disks_jobs(list(settings.host_targets), client, DisksCollector(), - settings.disks_interval, timeout=settings.disks_timeout): + settings.disks_interval, timeout=settings.disks_timeout, tls=tls): hub.add(job) for job in llm_jobs(settings, client, rates): hub.add(job) @@ -120,6 +136,21 @@ def build_hub(settings: Settings, store: SampleStore, return hub, nut_monitor +def build_certs(store: SampleStore) -> CertMaterial: + """This process's certificate material: read (or mint, once) the CA, then write it out. + + One helper because the wiring exists twice — here for production, and in the lifespan's + injected-store branch for tests — and two copies of a startup sequence drift into the state + where one branch has certificates and the other does not (the `build_hub` rule in AGENTS.md). + + It runs **before** `build_hub`, and a failure is fatal on purpose: `certs.ensure_certs` raises + when a stored PEM will not parse, and a dashboard that cannot present its own client + certificate must not start the loops and report every TLS host as an upstream problem (the + `ctx.auth` rule, AGENTS.md invariant 11). + """ + return certs.write_material(certs.ensure_certs(store)) + + def build_context(settings: Settings, client: httpx.AsyncClient) -> tuple[ AppContext, SampleStore, Callable[[], None]]: """Create the store, the collectors, and the hub's jobs. Returns (ctx, store, start).""" @@ -129,9 +160,11 @@ def build_context(settings: Settings, client: httpx.AsyncClient) -> tuple[ minute_keep_days=settings.minute_keep_days, hour_keep_days=settings.hour_keep_days, ) - hub, nut_monitor = build_hub(settings, store, client) + material = build_certs(store) + hub, nut_monitor = build_hub(settings, store, client, + tls=agent_tls_map(material, store, settings.host_targets)) ctx = AppContext(settings=settings, store=store, hub=hub, nut_monitor=nut_monitor, - started_at=time.time()) + started_at=time.time(), certs=material) return ctx, store, hub.start @@ -268,9 +301,14 @@ def create_app(settings: Settings | None = None, store: SampleStore | None = Non try: if store is not None: # injected: rebuild everything except the DB, with the production wiring - hub, nut_monitor = build_hub(settings, store, client) + # (certificates first, then the per-host TLS material the agent jobs dial with) + material = build_certs(store) + hub, nut_monitor = build_hub(settings, store, client, + tls=agent_tls_map(material, store, + settings.host_targets)) ctx = AppContext(settings=settings, store=store, hub=hub, - nut_monitor=nut_monitor, started_at=time.time()) + nut_monitor=nut_monitor, started_at=time.time(), + certs=material) start = hub.start else: ctx, owned_store, start = build_context(settings, client) @@ -290,6 +328,8 @@ def create_app(settings: Settings | None = None, store: SampleStore | None = Non if ctx is not None: await ctx.hub.stop() await client.aclose() + # The private directory holding the client key goes with the process that wrote it. + certs.remove_material(ctx.certs if ctx is not None else None) if owned_store is not None and store is None: owned_store.close() @@ -301,6 +341,7 @@ def create_app(settings: Settings | None = None, store: SampleStore | None = Non app.add_api_route(path, _page_handler(filename), methods=["GET"], name=f"page:{path}") app.include_router(auth_router()) + app.include_router(certs_router()) app.include_router(common_router()) app.include_router(system_router()) app.include_router(disks_router()) diff --git a/src/dashboards/auth.py b/src/dashboards/auth.py index 2d00790..4a8b623 100644 --- a/src/dashboards/auth.py +++ b/src/dashboards/auth.py @@ -15,7 +15,8 @@ the host agent: the alternative was `itsdangerous` + Starlette `SessionMiddlewar `bcrypt`, which buys a session store to operate and a native wheel to build in the Containerfile, and costs the one property that matters here — every rule readable in one file with no magic inside the cookie. `hmac`/`hashlib`/`secrets` are the primitives those -libraries wrap. No dependency was added to write this and none is coming. +libraries wrap. No dependency was added to write this and none is coming **to this file** — the +app's one later addition, `cryptography`, belongs to `certs.py` (phase 15) and is not imported here. **Credentials never leave the process.** A password, a verifier, a salt, or a session token must never appear in a log line, an exception message, or a value a caller could render. The diff --git a/src/dashboards/certs.py b/src/dashboards/certs.py new file mode 100644 index 0000000..65f8494 --- /dev/null +++ b/src/dashboards/certs.py @@ -0,0 +1,955 @@ +"""The dashboard's own certificate authority: the root, its client cert, and one per agent. + +Phase 15 makes every exporter this repo installs a *protected web server* whose protection is a +certificate instead of a bearer token: the dashboard holds a root CA, presents its own client +certificate when it dials an agent, and signs each agent's server certificate. "Protected" here +means *only this dashboard can read an agent* — the agent requires a client cert chained to this +root on every path but `/health` — so the root key and the client key are as sensitive as the +admin password and in one way more so: whoever holds them *is* the dashboard as far as the fleet +is concerned. + +Everything lives in the `meta` table, exactly how `auth.py` keeps the verifier and the session +signing key, for the same three reasons: a restart must not rotate it (a new root CA would +invalidate every bundle the agents hold, and a fleet that has to be re-installed on every deploy +is a fleet that will be run without TLS), there must be no second place durable state can hide in +(no config file, no keyring, no directory outside the DB), and the operator's reset path is one +`DELETE FROM meta` they can type. + +**The shapes.** The root CA is self-signed (`CN=dashboards-root`, 10 years) and can only sign. The +dashboard's client cert (`CN=dashboards`, 1 year) is the identity every agent checks, so there is +exactly one per dashboard — an agent trusts "signed by our root", not a per-process name. Each +agent gets a server cert (`CN=`, 1 year) whose SAN is the address(es) the dashboard +dials for it, because a TLS peer is verified by *name* and a certificate without the name in its +SAN is a handshake failure, not a warning. + +**The bundle** is the install artifact — the TODO's "root certificate installed with each +exporter", plus what that exporter needs to serve: the agent's server cert, the agent's key, the +root CA *cert* (never the root key), a `# BUNDLE-REV ` line, and the current `# REVOKED +` lines. It is stored in `meta` as text rather than re-derived on demand so the bytes an +operator downloads from `/api/certs//bundle` are byte-identical to the bytes the agent holds +— a page that re-signed on every read would make "is this host running the bundle I just issued?" +unanswerable. + +**Nothing here reads a clock** (`now` is an argument, the `auth.py` rule), nothing logs, and the +only raise is the fixed-text one for a `meta` row that will not parse: a dashboard whose own +credential is corrupt must stop rather than serve a fleet it cannot authenticate to, and the +message may not interpolate a value because a traceback goes to a log and, through the lifespan, +to whatever prints the startup failure (`AGENTS.md` invariant 6). + +This module is the *lifecycle* only. The agent's TLS mode, the dial with a client certificate, the +`/api/certs` surface and the `/certs` page are tasks 02-05. +""" + +from __future__ import annotations + +import ipaddress +import json +import os +import re +import shutil +import tempfile +import time +from collections.abc import Iterable, Sequence +from dataclasses import dataclass, replace +from datetime import UTC, datetime, timedelta +from pathlib import Path +from typing import Protocol + +from cryptography import x509 +from cryptography.exceptions import InvalidSignature, UnsupportedAlgorithm +from cryptography.hazmat.primitives.asymmetric import ec +from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey +from cryptography.hazmat.primitives.hashes import SHA256 +from cryptography.hazmat.primitives.serialization import ( + Encoding, + NoEncryption, + PrivateFormat, + PublicFormat, + load_pem_private_key, +) +from cryptography.x509.oid import ExtendedKeyUsageOID, NameOID + +# --- the vocabulary ---------------------------------------------------------------------- # + +#: Meta keys this module owns, named here because the operator has to be able to type them (they +#: are the reset command) and because task 04's API and `tests/test_certs.py` all read the same +#: five rows. One concern per key, the `auth.*` precedent — except the agents, which share the +#: single `certs.agents` row because a revocation changes the denylist *every* bundle carries: +#: updating one host's row and not the others would leave most of the fleet trusting a +#: certificate the operator just revoked. +META_ROOT_KEY = "certs.root.key" +META_ROOT_CERT = "certs.root.cert" +META_CLIENT_KEY = "certs.client.key" +META_CLIENT_CERT = "certs.client.cert" +META_AGENTS = "certs.agents" + +#: Every key this module writes, in the order the reset message names them. +META_KEYS = (META_ROOT_KEY, META_ROOT_CERT, META_CLIENT_KEY, META_CLIENT_CERT, META_AGENTS) + +#: The root CA's own CN and the client cert's. Nothing in this app compares them to anything; +#: they are what `openssl x509 -noout -subject` answers on a host, which is why they are words an +#: operator would want to read there. +ROOT_CN = "dashboards-root" +CLIENT_CN = "dashboards" + +#: Ten years for the root, one for everything it signs. The root's lifetime is the *install +#: lifetime of the fleet*: a bundle installed once has to still chain to the root years later, and +#: re-trusting a new root on N hosts is the expensive operation this number buys time against. A +#: leaf's lifetime is the opposite: a quietly expired agent certificate is a red source with a +#: confusing error, so it is short enough that rotation is routine and long enough that it is +#: never urgent. Neither is a security claim — the private keys never leave the `meta` table or an +#: agent's `0600` file. +ROOT_LIFETIME = timedelta(days=3650) +LEAF_LIFETIME = timedelta(days=365) + +#: The two values of an agent entry's `status`. `revoked` is enforced by the *dashboard* before it +#: dials (task 03) and by any agent holding that serial on its denylist. There is no "unrevoke": +#: the fix for a compromised identity is a new one (`reissue_agent_cert`). +ACTIVE = "active" +REVOKED = "revoked" + +#: The two metadata lines inside a bundle. Text lines rather than JSON because the bundle is a PEM +#: file an operator will `cat` and an agent parses by hand with stdlib only (task 02), and +#: mandatory because a bundle with no revision is not one this app signed — `# BUNDLE-REV 0` is +#: fine, its absence is not. +BUNDLE_REV_LINE = "# BUNDLE-REV " +REVOKED_LINE = "# REVOKED " + +#: The one message this module raises, and the only text it raises with. Fixed on purpose +#: (invariant 6): the row that failed to parse may be a private key, and an exception message goes +#: to a log and out to whoever prints the startup failure. It names the keys and the command +#: rather than the reason, because "which row?" is answerable by the operator in one `sqlite3` +#: call and the answer is always the same. +CORRUPT_STORE_MESSAGE = ( + "the stored certificates in this database cannot be read — refusing to start a dashboard\n" + " that cannot present its own credential.\n" + "\n" + " Delete this module's rows and start again; a fresh root CA and client certificate are\n" + " generated on the first boot of an empty table:\n" + "\n" + " sqlite3 \"\" \"DELETE FROM meta WHERE key LIKE 'certs.%'\"\n" + "\n" + " (certs.root.key, certs.root.cert, certs.client.key, certs.client.cert, certs.agents —\n" + " nothing outside this module's own rows.)\n" + "\n" + " Every installed agent bundle stops chaining to the new root, so each agent needs a fresh\n" + " bundle from /certs after the restart." +) + +#: What "I cannot read this PEM" looks like. `InvalidSignature` belongs here because a stored +#: client certificate the stored root did not sign is the same corruption by another route (a +#: root rotation that died halfway), and `UnsupportedAlgorithm` because somebody swapped in a key +#: type this build cannot use — also a row to delete, not a stack to debug. +_PEM_ERRORS = (ValueError, TypeError, OSError, InvalidSignature, UnsupportedAlgorithm) + +#: One PEM block, grepped out of a bundle by text. The agent does the same thing with stdlib +#: (task 02), so this module and the agent agree on the format by construction rather than by a +#: shared import they cannot have. +_BLOCK_RE = re.compile(r"-----BEGIN [A-Z0-9 ]+-----.*?-----END [A-Z0-9 ]+-----\n", re.DOTALL) +_CERT_HEADER = "-----BEGIN CERTIFICATE-----" + + +class MetaStore(Protocol): + """The slice of :class:`dashboards.store.SampleStore` the cert lifecycle needs. + + Declared rather than imported, exactly as `auth.MetaStore` is: this module then depends on + nothing else in the package, cannot reach a database connection from a unit test, and is + exercised with a dict. `delete_meta` is part of the shape because the reset path has to be + provable — "did the row you named actually go away" is a question with a rowcount answer. + """ + + def get_meta(self, key: str) -> str | None: ... + + def set_meta(self, key: str, value: str) -> None: ... + + def delete_meta(self, key: str) -> int: ... + + +# --- the state ------------------------------------------------------------------------- # + + +@dataclass(frozen=True) +class AgentCert: + """One host's issued identity, as stored in `certs.agents`. + + `bundle` is the whole install artifact (server cert + server key + root cert + the metadata + lines), so "what does this host hold?" is one string, and `bundle_rev` is the number that + question is actually asked with: task 05's page compares the issued rev against the rev the + agent reported in its snapshot and says *stale* without colouring it. + + `serial` is an int and the denylist lines in a bundle are decimal text — the same number + `getpeercert()` reports, so an agent's check is an integer membership test rather than a + comparison of formatted strings. + """ + + name: str + serial: int + not_after: float + status: str + bundle_rev: int + bundle: str + san: tuple[str, ...] + + def to_json(self) -> dict[str, object]: + """The stored form. `san` becomes a list because JSON has no tuple and this row is the record.""" + return { + "serial": self.serial, + "not_after": self.not_after, + "status": self.status, + "bundle_rev": self.bundle_rev, + "bundle": self.bundle, + "san": list(self.san), + } + + +@dataclass(frozen=True) +class CertsState: + """Everything one process knows about its own certificates, read once at startup. + + This is a bundle of secrets in the same sense `auth.AuthState` is: `root_key` and `client_key` + are unencrypted PEM text, so it is never rendered, never serialised into a response, and never + logged. Task 04's routes serve serials and expiries derived from it, never a field of it, and + the on-disk material (:func:`write_material`) deliberately leaves the root key out — signing + happens in this process and nowhere else. + + `agents` is the state of the fleet's identities **as of the read that produced this object**. + The write functions below return a new state instead of mutating one, and the store stays the + truth: :func:`read_agents` is how a caller that did not just write gets a current answer. + """ + + root_key: str + root_cert: str + root_serial: int + root_not_after: float + client_key: str + client_cert: str + client_serial: int + client_not_after: float + agents: tuple[AgentCert, ...] = () + + def agent(self, name: str) -> AgentCert | None: + """The entry for one host, or `None` when this state has never issued one for it.""" + for entry in self.agents: + if entry.name == name: + return entry + return None + + def with_agents(self, entries: Iterable[AgentCert]) -> CertsState: + """This state with a new agent list — sorted by name, so the stored row has one order.""" + return replace(self, agents=tuple(sorted(entries, key=lambda item: item.name))) + + +# --- PEM helpers ------------------------------------------------------------------------- # + + +def _moment(base: float) -> datetime: + """The injected epoch seconds as the timezone-aware UTC datetime X.509 wants.""" + return datetime.fromtimestamp(base, UTC) + + +def _epoch(moment: datetime) -> float: + """The inverse, for the numbers the API and the tests compare against.""" + return moment.timestamp() + + +def _text(store: MetaStore, key: str) -> str: + """A stored row exactly as it was written. + + Not stripped: a PEM document's trailing newline is part of the document, and a state whose + `root_cert` differs by one byte from the row it was read from would make every bundle built + from it differ from the bundle already installed on an agent. "Is this row present?" is + therefore a separate question — :func:`_present` — because a row of whitespace is not a + certificate, and that is the only thing stripping buys. + """ + return store.get_meta(key) or "" + + +def _present(text: str) -> bool: + """Is there anything in this row at all? A blank row is an absent row.""" + return bool(text.strip()) + + +def _new_key() -> EllipticCurvePrivateKey: + """A fresh P-256 key. + + EC rather than RSA for two reasons that both matter: it is generated in microseconds instead of + a half-second of prime hunting (this runs on every test app's startup, and a root rotation + mints a dozen keys at once), and every `ssl` build in this fleet does ECDSA in TLS 1.3, so + there is no old peer this excludes. + """ + return ec.generate_private_key(ec.SECP256R1()) + + +def _pem_key(key: EllipticCurvePrivateKey) -> str: + """PKCS#8 PEM, unencrypted. + + Unencrypted because nobody could type the passphrase: the dashboard signs unattended at + startup and the agent loads its key from a `0600` file in a systemd unit with no terminal. The + protection is the file mode and the `meta` table's permissions — the same bargain as the agent + bearer token this replaces, and the reason the mode is task 06's job rather than a comment. + """ + return key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption()).decode("ascii") + + +def _pem_cert(cert: x509.Certificate) -> str: + return cert.public_bytes(Encoding.PEM).decode("ascii") + + +def _load_key(text: str) -> EllipticCurvePrivateKey: + try: + key = load_pem_private_key(text.encode("utf-8", "surrogatepass"), password=None) + except _PEM_ERRORS as exc: + raise RuntimeError(CORRUPT_STORE_MESSAGE) from exc + if not isinstance(key, EllipticCurvePrivateKey): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + return key + + +def _load_cert(text: str) -> x509.Certificate: + try: + return x509.load_pem_x509_certificate(text.encode("utf-8", "surrogatepass")) + except _PEM_ERRORS as exc: + raise RuntimeError(CORRUPT_STORE_MESSAGE) from exc + + +def _same_public_key(key: EllipticCurvePrivateKey, cert: x509.Certificate) -> bool: + """Does this certificate belong to that key? A pair that disagrees is a corrupt row. + + The comparison is on the SPKI encoding — the only cheap identity a public key has — and it + returns a bool rather than asserting so the caller's answer stays the one fixed-text + `RuntimeError` whatever the mismatch turns out to be. + """ + left = key.public_key().public_bytes(Encoding.PEM, PublicFormat.SubjectPublicKeyInfo) + right = cert.public_key().public_bytes(Encoding.PEM, PublicFormat.SubjectPublicKeyInfo) + return left == right + + +def _chain_ok(cert: x509.Certificate, root: x509.Certificate) -> bool: + """Was `cert` signed by `root`? `False` rather than raise, so the caller picks the words.""" + try: + cert.verify_directly_issued_by(root) + except _PEM_ERRORS: + return False + return True + + +def _san_names(name: str, san: Sequence[str] | None) -> tuple[str, ...]: + """The SAN entries for a host: the addresses given, then the host name, deduplicated. + + The name is always in the list, so the result is never empty. That is not cosmetic: OpenSSL + refuses to build a certificate without a SAN, and a certificate whose SAN omits the address it + is dialled by fails hostname verification — the `n/a` of TLS, with an error message about the + handshake rather than about the name. + """ + values: list[str] = [] + for raw in [*(san or ()), name]: + value = (raw or "").strip() + if value and value not in values: + values.append(value) + return tuple(values) + + +def _san_extension(names: Sequence[str]) -> x509.SubjectAlternativeName: + """DNS name or IP address, decided by the stdlib parser rather than a hunch about the string.""" + items: list[x509.GeneralName] = [] + for value in names: + try: + items.append(x509.IPAddress(ipaddress.ip_address(value))) + except ValueError: + items.append(x509.DNSName(value)) + return x509.SubjectAlternativeName(items) + + +def _self_signed_root(key: EllipticCurvePrivateKey, moment: datetime) -> x509.Certificate: + subject = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, ROOT_CN)]) + return ( + x509.CertificateBuilder() + .subject_name(subject) + .issuer_name(subject) + .public_key(key.public_key()) + .serial_number(x509.random_serial_number()) + .not_valid_before(moment) + .not_valid_after(moment + ROOT_LIFETIME) + # The SKI is what a verifier matches an issued certificate's AuthorityKeyIdentifier + # against (RFC 5280 §4.2.1.2), and OpenSSL refuses a chain built from a CA that has none + # when it verifies in strict mode — which is what Python's `create_default_context` turns + # on. A CA whose leaves are rejected by a default client is not a CA this fleet can use, + # so the identifier is part of the shape rather than an optimisation. + .add_extension(x509.SubjectKeyIdentifier.from_public_key(key.public_key()), + critical=False) + .add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True) + .add_extension( + x509.KeyUsage( + digital_signature=False, + content_commitment=False, + key_encipherment=False, + data_encipherment=False, + key_agreement=False, + key_cert_sign=True, + crl_sign=True, + encipher_only=False, + decipher_only=False, + ), + critical=True, + ) + .sign(key, SHA256()) + ) + + +def _sign_leaf(*, key: EllipticCurvePrivateKey, cn: str, san: Sequence[str], + root_cert: x509.Certificate, root_key: EllipticCurvePrivateKey, + moment: datetime, client_auth: bool) -> x509.Certificate: + """Sign `key` into a leaf: named `cn`, valid for `san`, client or server purpose. + + The EKU matters more than it looks. An agent verifying a *client* certificate checks the + `clientAuth` purpose and the dashboard verifying a *server* checks `serverAuth`; a leaf with + the wrong one is a handshake failure no operator maps back to this function. Every leaf is + `CA=FALSE`, so nothing an agent is tricked into holding can sign a further certificate. + """ + builder = ( + x509.CertificateBuilder() + .subject_name(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, cn)])) + .issuer_name(root_cert.subject) + .public_key(key.public_key()) + .serial_number(x509.random_serial_number()) + .not_valid_before(moment) + .not_valid_after(moment + LEAF_LIFETIME) + .add_extension(x509.BasicConstraints(ca=False, path_length=None), critical=True) + # Both identifiers, on every leaf: AKI names the CA that signed this one (OpenSSL's + # strict mode rejects an issued certificate that omits it, and a Python client defaults + # to strict), SKI identifies this key for whatever chains to it. Non-critical both times, + # as RFC 5280 asks, so a verifier that ignores them still validates the signature. + .add_extension( + x509.AuthorityKeyIdentifier.from_issuer_public_key(root_key.public_key()), + critical=False, + ) + .add_extension(x509.SubjectKeyIdentifier.from_public_key(key.public_key()), critical=False) + .add_extension( + x509.ExtendedKeyUsage( + [ + ExtendedKeyUsageOID.CLIENT_AUTH + if client_auth + else ExtendedKeyUsageOID.SERVER_AUTH + ] + ), + critical=False, + ) + ) + if san: + builder = builder.add_extension(_san_extension(san), critical=False) + return builder.sign(root_key, SHA256()) + + +# --- the bundle -------------------------------------------------------------------------- # + + +def _bundle_text(*, cert_pem: str, key_pem: str, root_pem: str, bundle_rev: int, + revoked: Sequence[int]) -> str: + """The install artifact, as text. + + Order: the metadata lines, the agent's own certificate, the agent's key, then the root CA + cert. That order *is* the format — :func:`_bundle_parts` reads it back positionally and task + 02's agent parses the same way with stdlib only. The root **key** is never in a bundle: an + exporter needs to verify its clients, which is a public-key operation, so a stolen agent + install would otherwise be a stolen CA. + """ + lines = [f"{BUNDLE_REV_LINE}{bundle_rev}"] + lines.extend(f"{REVOKED_LINE}{serial}" for serial in sorted(set(revoked))) + return "\n".join(lines) + "\n" + cert_pem + key_pem + root_pem + + +def _bundle_parts(bundle: str) -> tuple[str, str, str]: + """`(agent cert, agent key, root cert)` out of a stored bundle, or the fixed-text raise. + + Two certificate blocks and exactly one key block is the shape; anything else means the row is + not a bundle this module wrote. Parsing by text instead of re-serialising is what lets a + revoked bundle be *re-sealed* (new denylist, same cert and key) without adding a seventh and + eighth meta row for material that is already in there. + """ + blocks = _BLOCK_RE.findall(bundle or "") + certs = [block for block in blocks if block.startswith(_CERT_HEADER)] + keys = [block for block in blocks if not block.startswith(_CERT_HEADER)] + if len(certs) != 2 or len(keys) != 1: + raise RuntimeError(CORRUPT_STORE_MESSAGE) + return certs[0], keys[0], certs[1] + + +def _revoked_serials(entries: Iterable[AgentCert]) -> tuple[int, ...]: + """Every serial currently on the denylist — the serial of each entry marked revoked.""" + return tuple(sorted({entry.serial for entry in entries if entry.status == REVOKED})) + + +def _seal(entries: Sequence[AgentCert], *, root_pem: str) -> tuple[AgentCert, ...]: + """Give every entry the bundle it should hold right now, for this root and this denylist. + + The revocation list is **fleet-wide**, which is why sealing every entry is part of revoking + one: the list lives inside the bundle, so an agent whose bundle omits a serial keeps accepting + that client certificate until somebody re-installs it. `bundle_rev` does not move when a + bundle is only re-sealed — the revision means "this host has a new identity", and a peer being + denied is not a new identity for anybody. + """ + revoked = _revoked_serials(entries) + sealed: list[AgentCert] = [] + for entry in entries: + cert_pem, key_pem, _root = _bundle_parts(entry.bundle) + sealed.append( + replace( + entry, + bundle=_bundle_text( + cert_pem=cert_pem, + key_pem=key_pem, + root_pem=root_pem, + bundle_rev=entry.bundle_rev, + revoked=revoked, + ), + ) + ) + return tuple(sealed) + + +# --- the stored agent table --------------------------------------------------------------- # + + +def _agent_from_json(name: str, raw: object) -> AgentCert: + """One entry out of the JSON row. Anything that is not the shape is the fixed-text raise.""" + if not isinstance(raw, dict): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + raw_san = raw.get("san") + if not isinstance(raw_san, list): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + try: + entry = AgentCert( + name=name, + serial=int(raw["serial"]), + not_after=float(raw["not_after"]), + status=str(raw["status"]), + bundle_rev=int(raw["bundle_rev"]), + bundle=str(raw["bundle"]), + san=tuple(str(item) for item in raw_san), + ) + except (KeyError, TypeError, ValueError) as exc: + raise RuntimeError(CORRUPT_STORE_MESSAGE) from exc + if not entry.bundle: + # An entry with no bundle is a host nobody can install: the identity exists but the + # artifact does not, which is exactly as broken as a missing key and gets the same answer. + raise RuntimeError(CORRUPT_STORE_MESSAGE) + return entry + + +def _agents_from_text(raw: str) -> tuple[AgentCert, ...]: + """The `certs.agents` row as entries, sorted by name (one order in memory and on disk).""" + if not raw.strip(): + return () + try: + parsed = json.loads(raw) + except json.JSONDecodeError as exc: + raise RuntimeError(CORRUPT_STORE_MESSAGE) from exc + if not isinstance(parsed, dict): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + return tuple(_agent_from_json(name, value) for name, value in sorted(parsed.items())) + + +def _store_agents(store: MetaStore, state: CertsState) -> CertsState: + """Persist the agent table and hand the state back — the only writer of `certs.agents`.""" + payload = {entry.name: entry.to_json() for entry in sorted(state.agents, key=lambda i: i.name)} + store.set_meta(META_AGENTS, json.dumps(payload)) + return state + + +# --- public lifecycle ---------------------------------------------------------------------- # + + +def ensure_certs(store: MetaStore, *, now: float | None = None) -> CertsState: + """Read the certificate state, generating and persisting whatever is missing. + + The rule that makes this safe to call on every start: **a stored row always wins.** A root CA + that changed between restarts would untrust every installed agent bundle and turn a redeploy + into a fleet-wide outage with no operator action available, so this function generates only + what is absent and never overwrites a value it read. + + Half-populated rows generate the missing half, which is what a crash between two `set_meta` + calls leaves behind — and a *newly generated* root takes its client certificate with it, because + a leaf signed by the previous root is a credential no agent in the fleet can verify. An empty + agent table is still written as `{}`, so the first boot leaves exactly the five rows this + module owns. + + What is *not* forgiven is a row that will not parse, a key that does not match its + certificate, or a client certificate the stored root did not sign: those raise + :class:`RuntimeError` with :data:`CORRUPT_STORE_MESSAGE` and nothing interpolated. A dashboard + that cannot present its own credential must not start the poll loops and imply it is watching + the fleet (the `ctx.auth` rule, `AGENTS.md` invariant 11); the fix is the `DELETE FROM meta` + the message names, after which every agent needs a new bundle. + + `now` is injected for the same reason `auth.mint_session` takes it: the expiries in the state + become a number a test asserts on instead of a sleep. + """ + moment = _moment(time.time() if now is None else now) + + root_key_text = _text(store, META_ROOT_KEY) + root_cert_text = _text(store, META_ROOT_CERT) + root_is_new = not (_present(root_key_text) and _present(root_cert_text)) + if not root_is_new: + root_key = _load_key(root_key_text) + root_cert = _load_cert(root_cert_text) + if not _same_public_key(root_key, root_cert): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + else: + root_key = _new_key() + root_cert = _self_signed_root(root_key, moment) + root_key_text, root_cert_text = _pem_key(root_key), _pem_cert(root_cert) + store.set_meta(META_ROOT_KEY, root_key_text) + store.set_meta(META_ROOT_CERT, root_cert_text) + + client_key_text = _text(store, META_CLIENT_KEY) + client_cert_text = _text(store, META_CLIENT_CERT) + # A new root invalidates the client certificate that was signed under the old one, so the + # "stored wins" rule does not extend across a generation: keeping that leaf would hand the + # process a credential nothing in the fleet can verify. + if not root_is_new and _present(client_key_text) and _present(client_cert_text): + client_key = _load_key(client_key_text) + client_cert = _load_cert(client_cert_text) + if not _same_public_key(client_key, client_cert) or not _chain_ok(client_cert, root_cert): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + else: + client_key = _new_key() + client_cert = _sign_leaf( + key=client_key, + cn=CLIENT_CN, + san=_san_names(CLIENT_CN, ()), + root_cert=root_cert, + root_key=root_key, + moment=moment, + client_auth=True, + ) + client_key_text, client_cert_text = _pem_key(client_key), _pem_cert(client_cert) + store.set_meta(META_CLIENT_KEY, client_key_text) + store.set_meta(META_CLIENT_CERT, client_cert_text) + + agents_text = _text(store, META_AGENTS) + if not _present(agents_text): + # The empty fleet is a row too. Writing it here means the first boot leaves exactly the + # five keys this module owns — an operator reading the table can tell "no agents issued + # yet" from "this module has never run", and task 04's API has a row to list. + store.set_meta(META_AGENTS, "{}") + agents_text = "{}" + + return CertsState( + root_key=root_key_text, + root_cert=root_cert_text, + root_serial=root_cert.serial_number, + root_not_after=_epoch(root_cert.not_valid_after_utc), + client_key=client_key_text, + client_cert=client_cert_text, + client_serial=client_cert.serial_number, + client_not_after=_epoch(client_cert.not_valid_after_utc), + agents=_agents_from_text(agents_text), + ) + + +def issue_agent_cert(store: MetaStore, state: CertsState, name: str, + san: Sequence[str] | None = (), *, now: float | None = None) -> CertsState: + """Mint (or re-mint) one host's server certificate and persist its bundle. + + A re-issue on a name that already has an entry bumps `bundle_rev` and replaces the serial: that + *is* rotation, and it is why there is no "unrevoke" — a revoked identity is replaced, never + forgiven. Nothing here issues on a read path: `GET /api/certs` lists what is stored, so + refreshing a page never mints a certificate nobody asked for. + + Returns the new state; `state.agent(name)` on the return value is the entry just written. + """ + return _issue(store, state, name, san=san, status=ACTIVE, now=now) + + +def reissue_agent_cert(store: MetaStore, state: CertsState, name: str, + san: Sequence[str] | None = (), *, now: float | None = None) -> CertsState: + """`issue_agent_cert`, named for `POST /api/certs//rotate`. + + One implementation on purpose: rotating a certificate and issuing one for the first time are + the same operation with the same result (fresh key, fresh serial, `bundle_rev` +1, active), and + two code paths would eventually disagree about one of those. Rotating a host with no + certificate yet is legal and yields `bundle_rev` 1 — the operator asked for a working cert. + """ + return _issue(store, state, name, san=san, status=ACTIVE, now=now) + + +def revoke(store: MetaStore, state: CertsState, name: str, *, + now: float | None = None) -> CertsState: + """Mark one host's identity revoked and re-seal every bundle so the fleet knows. + + The serial stays listed until a reissue replaces it. Revoking a name with no entry yet mints + one and marks it revoked immediately: what a revocation *does* is make the dashboard refuse to + dial that host (task 03), so "never issued" must not be a state in which the operator's click + silently does nothing. + + Every other bundle is rewritten because the denylist lives inside each of them — see + :func:`_seal`. No other host's `bundle_rev` moves: its identity did not change, only the list + of peers it must refuse. + """ + if state.agent(name) is None: + return _issue(store, state, name, san=None, status=REVOKED, now=now) + entries = [ + replace(entry, status=REVOKED) if entry.name == name else entry + for entry in state.agents + ] + return _store_agents(store, state.with_agents(_seal(entries, root_pem=state.root_cert))) + + +def rotate_root(store: MetaStore, state: CertsState, *, now: float | None = None) -> CertsState: + """A new root CA, a new client certificate, and every agent re-issued under it. + + One action re-arms the whole fleet, which is the point: a root whose key might have been + copied is worthless if replacing it means hand-signing N certificates somewhere else. Every + host's `bundle_rev` bumps and each operator installs the new bundle from `/certs`. The denylist + resets with the generation — every old serial is being replaced anyway, and a list of + certificates that no longer exist would only confuse the agent reading it. + + The write order is root key, root cert, client key, client cert, then the agent table. It + cannot be atomic (the `MetaStore` protocol has one row per call, and inventing a transaction + here would put a SQLite detail into a module tested with a dict), so a crash halfway leaves a + new root beside an old client — which the next :func:`ensure_certs` *detects* through the chain + check and answers with the fixed-text refusal rather than a dashboard dialing every agent with + a certificate nothing trusts. That is the intended failure: stop, print the `DELETE FROM meta` + command, start again. + + The running process keeps using the material it loaded at startup (its client certificate and + CA are already on disk from :func:`write_material`), which is why task 04's response answers + `restart_required: true` instead of pretending a hot-swap exists. + """ + moment = _moment(time.time() if now is None else now) + root_key = _new_key() + root_cert = _self_signed_root(root_key, moment) + root_key_text, root_cert_text = _pem_key(root_key), _pem_cert(root_cert) + store.set_meta(META_ROOT_KEY, root_key_text) + store.set_meta(META_ROOT_CERT, root_cert_text) + + client_key = _new_key() + client_cert = _sign_leaf( + key=client_key, + cn=CLIENT_CN, + san=_san_names(CLIENT_CN, ()), + root_cert=root_cert, + root_key=root_key, + moment=moment, + client_auth=True, + ) + client_key_text, client_cert_text = _pem_key(client_key), _pem_cert(client_cert) + store.set_meta(META_CLIENT_KEY, client_key_text) + store.set_meta(META_CLIENT_CERT, client_cert_text) + + issued: list[AgentCert] = [] + for entry in state.agents: + key = _new_key() + cert = _sign_leaf( + key=key, + cn=entry.name, + san=_san_names(entry.name, entry.san), + root_cert=root_cert, + root_key=root_key, + moment=moment, + client_auth=False, + ) + issued.append( + replace( + entry, + serial=cert.serial_number, + not_after=_epoch(cert.not_valid_after_utc), + status=ACTIVE, + bundle_rev=entry.bundle_rev + 1, + bundle=_bundle_text( + cert_pem=_pem_cert(cert), + key_pem=_pem_key(key), + root_pem=root_cert_text, + bundle_rev=entry.bundle_rev + 1, + revoked=(), + ), + ) + ) + fresh = CertsState( + root_key=root_key_text, + root_cert=root_cert_text, + root_serial=root_cert.serial_number, + root_not_after=_epoch(root_cert.not_valid_after_utc), + client_key=client_key_text, + client_cert=client_cert_text, + client_serial=client_cert.serial_number, + client_not_after=_epoch(client_cert.not_valid_after_utc), + ) + return _store_agents(store, fresh.with_agents(_seal(issued, root_pem=root_cert_text))) + + +def read_agents(store: MetaStore) -> tuple[AgentCert, ...]: + """The agent table exactly as stored — no generation, no writes, no PEM of ours. + + Task 04's `GET /api/certs` and task 03's per-poll revocation check both go through here, which + is what makes a revocation clicked in the browser take effect on the next poll without a + restart. + """ + return _agents_from_text(_text(store, META_AGENTS)) + + +def read_state(store: MetaStore) -> CertsState: + """The stored certificate state with **nothing generated and nothing written**. + + :func:`ensure_certs` is the bootstrapping read: it mints whatever a table is missing, which is + exactly right for a process start and wrong for `GET /api/certs` (task 04). A page refresh that + minted a root CA would be a side effect nobody asked for, and a root that appears the moment a + row is deleted is a root nobody can explain to the operator reading the page. + + So this is the same read without the generation: the rows are the truth, they are validated the + same way (a PEM that will not parse, a key that does not match its certificate, and a client + leaf the stored root did not sign are all the one fixed-text refusal), and a missing row is + that refusal too rather than an invitation to create one. The rows cannot legitimately be + missing while a request is being served — :func:`app.build_certs` runs :func:`ensure_certs` + before the app can answer anything — so "the table has no CA in it" really is the corrupt state + the message describes. + + Task 04's reads use this; its writes use :func:`ensure_certs`, because a write is allowed to + bootstrap the state it is about to change. + """ + root_key_text = _text(store, META_ROOT_KEY) + root_cert_text = _text(store, META_ROOT_CERT) + client_key_text = _text(store, META_CLIENT_KEY) + client_cert_text = _text(store, META_CLIENT_CERT) + if not all(_present(row) for row in + (root_key_text, root_cert_text, client_key_text, client_cert_text)): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + + root_key = _load_key(root_key_text) + root_cert = _load_cert(root_cert_text) + client_key = _load_key(client_key_text) + client_cert = _load_cert(client_cert_text) + if not _same_public_key(root_key, root_cert) or not _same_public_key(client_key, client_cert): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + if not _chain_ok(client_cert, root_cert): + raise RuntimeError(CORRUPT_STORE_MESSAGE) + + return CertsState( + root_key=root_key_text, + root_cert=root_cert_text, + root_serial=root_cert.serial_number, + root_not_after=_epoch(root_cert.not_valid_after_utc), + client_key=client_key_text, + client_cert=client_cert_text, + client_serial=client_cert.serial_number, + client_not_after=_epoch(client_cert.not_valid_after_utc), + agents=read_agents(store), + ) + + +def agent_status(store: MetaStore, name: str) -> str: + """`REVOKED` for a host the operator revoked, `ACTIVE` for every other host. + + Absent means active rather than unknown: a host that was never issued has no certificate to + revoke, and answering `REVOKED` because a row is missing from a half-written table would take + a working host's source red with no operator action behind it (invariant 3's shape). The + caller that cares about issued-versus-not — the page — reads the entry itself. + """ + for entry in read_agents(store): + if entry.name == name: + return REVOKED if entry.status == REVOKED else ACTIVE + return ACTIVE + + +def _issue(store: MetaStore, state: CertsState, name: str, *, san: Sequence[str] | None, + status: str, now: float | None) -> CertsState: + """The one implementation behind issue / reissue / revoke-of-an-unknown-host.""" + moment = _moment(time.time() if now is None else now) + root_key = _load_key(state.root_key) + root_cert = _load_cert(state.root_cert) + key = _new_key() + names = _san_names(name, san) + cert = _sign_leaf( + key=key, + cn=name, + san=names, + root_cert=root_cert, + root_key=root_key, + moment=moment, + client_auth=False, + ) + previous = state.agent(name) + # 1 on a first issue, +1 on every rotation: this counter is the only answer to "is the host + # running the bundle I just issued?", so it moves exactly when the identity does. + bundle_rev = 1 if previous is None else previous.bundle_rev + 1 + entry = AgentCert( + name=name, + serial=cert.serial_number, + not_after=_epoch(cert.not_valid_after_utc), + status=status, + bundle_rev=bundle_rev, + bundle=_bundle_text( + cert_pem=_pem_cert(cert), + key_pem=_pem_key(key), + root_pem=state.root_cert, + bundle_rev=bundle_rev, + revoked=(), + ), + san=names, + ) + entries = [entry if item.name == name else item for item in state.agents] + if previous is None: + entries.append(entry) + return _store_agents(store, state.with_agents(_seal(entries, root_pem=state.root_cert))) + + +# --- on-disk material ---------------------------------------------------------------------- # + + +@dataclass(frozen=True) +class CertMaterial: + """Where this process's own credential lives on disk, so `httpx` can be pointed at it. + + `httpx` (and `ssl` generally) wants *paths*, not PEM text, so the state has to be written + somewhere to be usable. That somewhere is a private temp directory from `mkdtemp` (0700 by + construction, and chmodded again so the claim is this module's and not inherited from the + platform) with every file `0600`, removed by :func:`remove_material` on shutdown. **The root + key is never written at all**: it is used to sign inside this process and nowhere else, so the + one copy of it stays the `meta` row. + """ + + dir: Path + ca: Path + client_cert: Path + client_key: Path + state: CertsState + + +def _write_private(path: Path, text: str) -> Path: + """Create `path` with mode 0600 in one `open()`, so there is no window at the umask default.""" + handle = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + with os.fdopen(handle, "w", encoding="ascii") as stream: + stream.write(text) + return path + + +def write_material(state: CertsState, + tmp_base: str | os.PathLike[str] | None = None) -> CertMaterial: + """Write the CA and the client pair into a fresh private directory. + + Called once per process (`app.build_certs`) and never per request: those paths go into every + dial, and a directory changing under a live connection pool is a poll that fails for a reason + no upstream error can explain. `tmp_base` exists for tests and for a deployment that wants the + material somewhere specific; it defaults to the environment's `TMPDIR`. + """ + directory = Path(tempfile.mkdtemp(prefix="dashboards-certs-", dir=tmp_base)) + os.chmod(directory, 0o700) + return CertMaterial( + dir=directory, + ca=_write_private(directory / "root-ca.pem", state.root_cert), + client_cert=_write_private(directory / "client-cert.pem", state.client_cert), + client_key=_write_private(directory / "client-key.pem", state.client_key), + state=state, + ) + + +def remove_material(material: CertMaterial | None) -> None: + """Best-effort removal of a process's material directory. + + Best effort because failing here must not turn a clean shutdown into a crash: the files are + `0600` inside a `0700` directory owned by this user, so a leftover is no exposure beyond what + the running process already had, and there is nothing an operator could act on from a + shutdown path. + """ + if material is None: + return + shutil.rmtree(material.dir, ignore_errors=True) diff --git a/src/dashboards/collectors.py b/src/dashboards/collectors.py index 329e414..355b4ff 100644 --- a/src/dashboards/collectors.py +++ b/src/dashboards/collectors.py @@ -7,12 +7,16 @@ red dot plus an event. Nothing here writes to any upstream — with one delibera from __future__ import annotations +import ssl import time +from collections.abc import Callable, Iterable, Mapping from dataclasses import dataclass, field +from functools import partial from typing import Any import httpx +from . import certs from .config import Target, strip_userinfo from .poller import Job, Result from .rates import RateEstimator @@ -35,8 +39,117 @@ class UpstreamError(Exception): # --------------------------------------------------------------------------- # # System hosts # --------------------------------------------------------------------------- # + +#: The scheme a protected exporter is dialled by. An `http://` target keeps the bearer-token +#: path byte-for-byte as it was before phase 15, so one deployment can hold token-mode and +#: certificate-mode agents at the same time — which is what a host-by-host rollout needs. +HTTPS_SCHEME = "https://" + + +@dataclass(frozen=True) +class AgentTls: + """How to dial one agent as a *protected web server* — and whether it may be dialled. + + `verify` and `cert` are **paths**, not PEM text, because that is what `ssl` consumes: they + are the files :func:`dashboards.certs.write_material` wrote once at startup, which is also + the only place this process's client private key exists outside the `meta` row. The root + **key** is not in the picture at all — signing happens in :mod:`dashboards.certs`, and a + dial needs only the CA to verify the agent and the leaf to identify itself. + + `status` is a callable rather than a string, and the reason is revocation. The operator + clicks revoke on `/certs`; a poll that had cached `"active"` at startup would go on dialling + a host the operator just struck out. Reading `certs.agents` fresh (one small `meta` row, at + a 10-second cadence) is what makes the next poll the one that notices — no restart, no + second process talking to the first. + + `transport` exists for the tests: handing a `httpx.MockTransport` here scripts an agent's + 403 the same way the plain-host tests hand one to the shared client. It is `None` in every + dial this app makes for real. + """ + + verify: str + cert: tuple[str, str] + status: Callable[[], str] + transport: httpx.AsyncBaseTransport | None = None + + @property + def revoked(self) -> bool: + """Has the operator struck this host's identity out? + + Only the one word refuses. A `status` callable that raises (a store that will not read) + propagates to the job and is reported as that store failure — answering `False` here + would turn a broken read into "not revoked, carry on dialling", which is the wrong way + for a security check to fail. + """ + return self.status() == certs.REVOKED + + def ssl_context(self) -> ssl.SSLContext: + """A client context that trusts **our** root and presents **our** leaf. + + Built here rather than passed to `httpx` as `verify=`/`cert=` because 0.28 + deprecates both of those spellings and the supported form is a context the caller made — + which is also the honest shape for this app: the trust decision is ours, made against one + named CA file, not a library default that would also accept whatever the host trusts. + `create_default_context` leaves `check_hostname` on and `verify_mode` at `CERT_REQUIRED`, + so an agent whose certificate does not chain to this root *or* is not valid for the name + it is dialled by fails the handshake instead of degrading to a warning. + + Per dial, so the client key lives in a context that goes away with the poll rather than + in a connection pool that outlives it. A missing or unreadable PEM raises `OSError`, + which is what the caller's failure branch turns into a red source. + """ + context = ssl.create_default_context(cafile=self.verify) + context.load_cert_chain(self.cert[0], self.cert[1]) + return context + + def client(self, *, shared: httpx.AsyncClient) -> httpx.AsyncClient: + """A client for **one** https dial, carrying this dashboard's own credential. + + Per dial rather than per process, because `httpx` resolves the TLS material when a client + is *constructed* — there is no per-request override to hand `get()` — and the app reads + plain agents, gufo and vLLM through one shared client whose own defaults must stay + untouched: setting `verify` there would make every engine read depend on a CA nobody + issued their certificate with. + + The cost is one TLS handshake per poll, which is what the plain path already pays: the + shared pool expires an idle keepalive connection after 5 s and an agent is polled every + 10 s. The gain is that no live client key sits in an open connection pool between polls. + """ + verify: ssl.SSLContext | bool = True + if self.transport is None: + # Only the real transport needs the material: a scripted one opens no socket, and a + # test of a refusal sentence has no reason to hold a key pair. That is the one thing + # the seam changes, and why the seam is a transport rather than a patched `get`. + verify = self.ssl_context() + return httpx.AsyncClient(verify=verify, timeout=shared.timeout, + follow_redirects=shared.follow_redirects, + transport=self.transport) + + +def refusal_detail(response: httpx.Response) -> str: + """Why an agent refused us, in its own words — the JSON `detail` the handler sends. + + The two sentences a TLS agent writes (`client certificate required`, `client certificate + revoked`) are exactly what the operator needs in a health record, and paraphrasing them here + would be the dashboard claiming to know more than the peer said. A body that is not the + agent's JSON — a proxy's HTML refusal, an empty reply — gives a bounded slice of it instead, + the way the non-JSON failure above quotes the body it could not parse. + """ + try: + body = response.json() + except ValueError: + body = None + if isinstance(body, dict): + detail = body.get("detail") + if isinstance(detail, str) and detail.strip(): + return detail.strip() + quoted = " ".join(response.text.split())[:80] + return quoted or "no reason given" + + async def fetch_snapshot(client: httpx.AsyncClient, target: Target, - *, timeout: float | None = None) -> dict[str, Any]: + *, timeout: float | None = None, + tls: AgentTls | None = None) -> dict[str, Any]: """One agent `/snapshot`, fetched and checked. Shared by every dashboard that reads one. `system` and `disks` dial the same URL on different cadences, so the failure sentences @@ -45,24 +158,49 @@ async def fetch_snapshot(client: httpx.AsyncClient, target: Target, health record and written to the event table, both of which the browser reads, and `target.url` may carry the agent's token as userinfo (invariant 6). + With `tls` given, an `https://` target is dialled as a protected web server: the agent is + verified against this dashboard's own root CA and this dashboard presents its own client + certificate, so "who may read this host" is decided by the certificate rather than by a + token in a URL. A host the operator revoked is refused **before any socket opens** — + invariant 3's shape, a source the app knows it must not talk to says so without a network + round trip — and the poller's last-good/stale semantics handle the rest (invariant 4). + `timeout` overrides the client default for this one request (`None` = keep the client's). The SMART pass lives inside the agent's snapshot handler, so the slow `disks` poll needs a budget a 4-second system poll cannot afford — and must not inherit one. `None` is translated to httpx's own sentinel because passing a literal `None` to `get()` means *no timeout at all*, which would hang the loop forever. """ + if tls is not None and tls.revoked: + raise UpstreamError(f"{target.name}: agent certificate revoked") request_timeout: Any = httpx.USE_CLIENT_DEFAULT if timeout is None else timeout url = f"{target.url}/snapshot" shown = f"{target.display_url}/snapshot" + # `None` for an `http://` target even when material exists: a token-mode host is dialled by + # the shared client exactly as it was before phase 15, and a 403 from it is not about a + # certificate this app never presented. + secure = tls if (tls is not None and target.url.startswith(HTTPS_SCHEME)) else None try: - response = await client.get(url, headers=target.headers, timeout=request_timeout) + if secure is not None: + async with secure.client(shared=client) as scoped: + response = await scoped.get(url, headers=target.headers, + timeout=request_timeout) + else: + response = await client.get(url, headers=target.headers, timeout=request_timeout) except httpx.TimeoutException as exc: raise UpstreamError(f"{target.name}: {shown} timed out", status=504) from exc - except httpx.HTTPError as exc: + except (httpx.HTTPError, OSError) as exc: + # `OSError` is not belt-and-braces: `httpx` loads the CA and the client chain when the + # client is built, so a material file that has gone missing surfaces here as a + # `FileNotFoundError`/`ssl.SSLError`, and a poll loop must report that as an upstream + # failure rather than lose the task to it (invariant 4). raise UpstreamError(f"{target.name}: {shown} unreachable ({type(exc).__name__})") from exc if response.status_code == 401: raise UpstreamError(f"{target.name}: agent rejected the token (401) — check the " f"userinfo token in HOST_TARGETS") + if secure is not None and response.status_code == 403: + raise UpstreamError(f"{target.name}: agent refused the client certificate " + f"({refusal_detail(response)}) at {shown}") if response.status_code >= 400: raise UpstreamError(f"{target.name}: {shown} -> HTTP {response.status_code}") try: @@ -76,8 +214,9 @@ async def fetch_snapshot(client: httpx.AsyncClient, target: Target, async def fetch_host_snapshot(client: httpx.AsyncClient, target: Target, - host_collector: HostCollector) -> Result: - body = await fetch_snapshot(client, target) + host_collector: HostCollector, + *, tls: AgentTls | None = None) -> Result: + body = await fetch_snapshot(client, target, tls=tls) result = parse_host_snapshot(target.name, body, host_collector) if body.get("errors"): # The agent degraded a section (a /proc read failed) but still answered. Say so @@ -99,15 +238,19 @@ async def fetch_host_snapshot(client: httpx.AsyncClient, target: Target, # --------------------------------------------------------------------------- # async def fetch_disk_snapshot(client: httpx.AsyncClient, target: Target, disks_collector: disks_source.DisksCollector, - *, timeout: float | None = None) -> Result: + *, timeout: float | None = None, + tls: AgentTls | None = None) -> Result: """The same `/snapshot`, projected onto physical drives and their SMART health. Deliberately a second read of an endpoint `system` already polls: the two dashboards want different cadences (SMART is a 5-minute signal, utilisation is a 10-second one), and sharing one fetch would either chart stale SMART or run `smartctl` every 10 seconds. Separate jobs also means a dead disk poll ages `/disks` alone (invariant 4). + + `tls` is the same per-host material `system` dials with: one revoked host fails both of + its jobs with the same sentence, and neither of them opens a socket. """ - body = await fetch_snapshot(client, target, timeout=timeout) + body = await fetch_snapshot(client, target, timeout=timeout, tls=tls) return disks_source.parse_disks_snapshot(target.name, body, disks_collector) @@ -272,27 +415,61 @@ class NutMonitor: # --------------------------------------------------------------------------- # # Job factories (what app.py wires into the Hub) # --------------------------------------------------------------------------- # +def agent_tls_map(material: certs.CertMaterial | None, store: certs.MetaStore, + targets: Iterable[Target]) -> dict[str, AgentTls]: + """One :class:`AgentTls` per configured host, from this process's own certificate material. + + The CA and the client certificate are the same for every host — the dashboard has **one** + identity, which is what an agent verifies — so only the revocation check is per name, and it + is a live read of the store the jobs already write to (`certs.agent_status`, one small + `meta` row). An absent `certs.agents` entry counts as active: a host that has never been + issued has no certificate to revoke, and refusing it because a row is missing would take a + working source red with no operator action behind it. + + No material means an empty map, which is what a context assembled without certificates (or + a test hub) gets — every host is then dialled on the pre-phase-15 path. + """ + if material is None: + return {} + verify = str(material.ca) + cert = (str(material.client_cert), str(material.client_key)) + return { + target.name: AgentTls(verify=verify, cert=cert, + status=partial(certs.agent_status, store, target.name)) + for target in targets + } + + def host_jobs(targets: list[Target], client: httpx.AsyncClient, host_collector: HostCollector, - interval: float) -> list[Job]: + interval: float, *, + tls: Mapping[str, AgentTls] | None = None) -> list[Job]: + """One `system` job per agent. `tls` is keyed by host name; a host missing from it is + dialled with no client certificate and no revocation check, exactly as before phase 15.""" + tls_map = tls or {} return [ Job(f"host:{target.name}", "system", target.name, interval, - lambda t=target: fetch_host_snapshot(client, t, host_collector)) + lambda t=target: fetch_host_snapshot(client, t, host_collector, + tls=tls_map.get(t.name))) for target in targets ] def disks_jobs(targets: list[Target], client: httpx.AsyncClient, disks_collector: disks_source.DisksCollector, interval: float, - *, timeout: float | None = None) -> list[Job]: + *, timeout: float | None = None, + tls: Mapping[str, AgentTls] | None = None) -> list[Job]: """One job per host agent — the same target list `system` polls, at its own cadence. No targets, no jobs: `disks` has no upstream of its own, so "unconfigured" means the same thing it means for `system` (invariant 3), and `/api/disks/*` 404s without ever - dialling (task 03). + dialling (task 03). `tls` is the same per-host map `system` gets — the two jobs dial one + agent, so a revoked host is refused by both or by neither. """ + tls_map = tls or {} return [ Job(f"disks:{target.name}", disks_source.DASHBOARD, target.name, interval, - lambda t=target: fetch_disk_snapshot(client, t, disks_collector, timeout=timeout)) + lambda t=target: fetch_disk_snapshot(client, t, disks_collector, timeout=timeout, + tls=tls_map.get(t.name))) for target in targets ] diff --git a/src/dashboards/sources/disks.py b/src/dashboards/sources/disks.py index 4a99ac8..43f9790 100644 --- a/src/dashboards/sources/disks.py +++ b/src/dashboards/sources/disks.py @@ -48,6 +48,7 @@ from typing import Any from ..poller import Result from ..severity import classify from ..store import Event, Sample +from .host import agent_tls_fields #: The sample family and the dashboard name. They are the same word here on purpose: #: `/api/events?source=disks` and `hub.states("disks")` must agree, which is the phase-03 @@ -333,5 +334,16 @@ def parse_disks_snapshot(name: str, snapshot: Mapping[str, Any], collector: Disk "disks": rows, "summary": disk_summary(rows), } + # The same `agent` block `/api/system/snapshot` carries, from the same snapshot, so the page + # that compares an issued bundle revision with the one this agent is running can read either + # dashboard. Added only when the agent published one — a token-mode host keeps the payload it + # had before phase 15 rather than gaining an empty block or an invented `false`. + tls_report = agent_tls_fields(snapshot) + if tls_report: + # `agent` because these are the agent's own words about the server it is running, which is + # the same key path `/api/system/snapshot` uses for them (there the block also carries + # `version` and `duration_ms`; this payload never did, so it grows only the two fields + # phase 15 added rather than a half-populated copy of the other page's block). + payload["agent"] = tls_report return Result(payload=payload, samples=samples, events=health_events(collector, name, rows, now)) diff --git a/src/dashboards/sources/host.py b/src/dashboards/sources/host.py index 365c319..317fca8 100644 --- a/src/dashboards/sources/host.py +++ b/src/dashboards/sources/host.py @@ -16,6 +16,7 @@ Metric names in the history DB use `@stream` for per-device series: from __future__ import annotations import time +from collections.abc import Mapping from dataclasses import dataclass, field from typing import Any @@ -47,6 +48,31 @@ class HostCollector: return self.targets.setdefault(name, HostTargetState()) +def agent_tls_fields(snapshot: Mapping[str, Any]) -> dict[str, Any]: + """The agent's own TLS self-report, or `{}` when it published none. + + `host_agent.py` adds `tls: true` and `bundle_rev` **only** in TLS mode, so a token-mode (or + pre-1.1.0) agent keeps the payload byte-identical to before phase 15 — the `degraded` idiom, + a key that appears when the fact exists rather than a `false` the agent never asserted. + + This is not a verdict and carries no `severity`: `bundle_rev` is an identity counter, and the + question task 05 asks with it ("is the installed bundle the one `/certs` just issued?") is + answered beside the number, not by colouring it — the fleet-header rule that a count is not a + graded number. + + Shared with `sources/disks.py` because both dashboards read the same snapshot and task 05 has + to be able to read the deployed revision out of either payload. + """ + fields: dict[str, Any] = {} + tls = snapshot.get("tls") + if isinstance(tls, bool): + fields["tls"] = tls + revision = snapshot.get("bundle_rev") + if isinstance(revision, int) and not isinstance(revision, bool): + fields["bundle_rev"] = revision + return fields + + def _rate(collector: HostCollector, host: str, key: str, value: float | None, now: float) -> tuple[float | None, float | None]: return collector.rates.rate(f"{host}:{key}", value, now) @@ -382,7 +408,7 @@ def parse_host_snapshot(name: str, snapshot: dict[str, Any], collector: HostColl "top_rss": processes.get("top_rss") or [], }, "agent": {"duration_ms": snapshot.get("duration_ms"), "errors": snapshot.get("errors") or [], - "version": snapshot.get("agent_version")}, + "version": snapshot.get("agent_version"), **agent_tls_fields(snapshot)}, } return Result(payload=payload, samples=samples, events=events) diff --git a/src/dashboards/web/pages/certs.html b/src/dashboards/web/pages/certs.html new file mode 100644 index 0000000..f58594c --- /dev/null +++ b/src/dashboards/web/pages/certs.html @@ -0,0 +1,101 @@ + + + + + +Homelab · certificates + + + + +
+ ◆ Homelab + + +
+ --:--:-- + + +
+
+ +
+
+
+

Certificates

+
the authority this dashboard signs its agents with
+
+
+ +
+ +
+
+

Root CA

+
+
+ +
Common name
dashboards-root
+
Serial
…
+
Expires
…
+
Time left
…
+
+
+ +
+

+
+
+ +
+ +

Dashboard client

+
+
+
Common name
dashboards
+
Serial
…
+
Expires
…
+
Time left
…
+
+
+
+
+ +
+

Agents

+
+ + + + + + + + +
HostStatusSerialExpiresBundleDialled addressesActions
+ +

+
+
+
+ + + + + diff --git a/src/dashboards/web/pages/disks.html b/src/dashboards/web/pages/disks.html index 5eaaa81..fe4e875 100644 --- a/src/dashboards/web/pages/disks.html +++ b/src/dashboards/web/pages/disks.html @@ -21,6 +21,7 @@ --:--:-- + diff --git a/src/dashboards/web/pages/index.html b/src/dashboards/web/pages/index.html index 708bbe5..fa85cb8 100644 --- a/src/dashboards/web/pages/index.html +++ b/src/dashboards/web/pages/index.html @@ -20,6 +20,7 @@
--:--:-- +
diff --git a/src/dashboards/web/pages/llm.html b/src/dashboards/web/pages/llm.html index 1827363..19553af 100644 --- a/src/dashboards/web/pages/llm.html +++ b/src/dashboards/web/pages/llm.html @@ -21,6 +21,7 @@ --:--:-- + diff --git a/src/dashboards/web/pages/settings.html b/src/dashboards/web/pages/settings.html index a0be56c..9896b69 100644 --- a/src/dashboards/web/pages/settings.html +++ b/src/dashboards/web/pages/settings.html @@ -24,6 +24,7 @@ --:--:-- + diff --git a/src/dashboards/web/pages/system.html b/src/dashboards/web/pages/system.html index 338d0ff..b72b946 100644 --- a/src/dashboards/web/pages/system.html +++ b/src/dashboards/web/pages/system.html @@ -21,6 +21,7 @@ --:--:-- + diff --git a/src/dashboards/web/pages/ups.html b/src/dashboards/web/pages/ups.html index b07a8af..b50af27 100644 --- a/src/dashboards/web/pages/ups.html +++ b/src/dashboards/web/pages/ups.html @@ -21,6 +21,7 @@ --:--:-- + diff --git a/src/dashboards/web/static/dash.css b/src/dashboards/web/static/dash.css index 19858b4..273d064 100644 --- a/src/dashboards/web/static/dash.css +++ b/src/dashboards/web/static/dash.css @@ -871,12 +871,22 @@ input[type="text"].confirm { .settings-form button { justify-self: start; margin-top: .35rem; } /* The outcome line: no space until it has something to say, and its colour is the shared word * for what happened (`ok` landed, `warn` was refused before the wire, `fail` was refused by the - * credential), not a third palette. */ + * credential), not a third palette. `/certs` borrows the rule for the outcome of a rotate or a + * revoke — same sentence shape, and a second near-copy is how one of them stops matching the + * theme. */ .settings-result { margin: .8rem 0 0; font-size: 12.5px; } .settings-result.ok { color: var(--ok); } .settings-result.warn { color: var(--warn); } .settings-result.fail { color: var(--fail); } +/* ---- certificates ------------------------------------------------------- + * `/certs` (phase 15 task 05) is built out of parts the sheet already owns — `.card` for the + * panels, `.kv` for the two authority rows, `table.data` for the agents, `.chip` for a status + * word, `button.ghost` / `button.action.caution` for the writes, `.stale` for a bundle that is + * behind, and the account page's `.settings-result` for the outcome line. One shape is missing, + * and it is the one cell a table cannot nowrap: an address list. */ +table.data td.san { white-space: normal; } + /* ---- dialog ------------------------------------------------------------- */ dialog { border: 1px solid var(--border-strong); border-radius: var(--radius); background: var(--bg-raised); diff --git a/src/dashboards/web/static/page-certs.js b/src/dashboards/web/static/page-certs.js new file mode 100644 index 0000000..29c64f0 --- /dev/null +++ b/src/dashboards/web/static/page-certs.js @@ -0,0 +1,288 @@ +/* page-certs.js — the certificate authority's own page: read the table, rotate, revoke, + * download a bundle. + * + * Every value on it comes from `GET /api/certs` and every action is one of the routes task 04 + * serves, so the page holds no opinion about certificates of its own — it has buttons. There is + * no colour on it that the API did not send, either: the only verdict here is the payload's own + * `status` word put through a string-to-class map, and an expiry is a date the page prints + * (`severity.py` rates no certificate), so no row can be coloured green or red by a threshold + * nobody measured. + */ +'use strict'; + +(function () { + /** + * The payload's `status` word to the chip's class. + * + * `unissued` maps to no class at all, which is the neutral chip: a certificate that does not + * exist must not borrow the `ok` green (`Dash.severityFrom`'s rule, and the reason this is a + * word map rather than a comparison). + */ + const STATUS_CHIP = { active: 'ok', revoked: 'fail', unissued: '' }; + + /** + * One entry per column of `#agents-table`, in the order `` prints them, naming every + * `agents[]` key it reads. + * + * This table is the page's whole idea of a row, which is what makes it worth extracting: + * `tests/test_api.py` reads the served `fields` lists and resolves them against a live + * `GET /api/certs` row in both directions, so a rename on either side fails a test instead of + * printing a dash beside a real certificate (the `SLOT_FIELDS` device on `/llm`). + */ + const COLUMNS = [ + { fields: ['name'], build: (entry) => textCell('name', entry.name) }, + { fields: ['status'], build: statusCell }, + { fields: ['serial'], build: (entry) => textCell('num', decimal(entry.serial)) }, + { fields: ['not_after'], build: expiryCell }, + { fields: ['bundle_rev', 'deployed_bundle_rev'], build: bundleCell }, + { fields: ['san'], build: (entry) => textCell('num san', (entry.san || []).join(' ') || null) }, + // The actions column is the only one that reads more than its own fact: `configured` is the + // API's gate on all three controls, and there is nothing to download or revoke without a + // serial. + { fields: ['configured', 'status', 'serial'], build: actionCell }, + ]; + + const els = { + rootSerial: document.getElementById('root-serial'), + rootExpires: document.getElementById('root-expires'), + rootLeft: document.getElementById('root-left'), + rootResult: document.getElementById('root-result'), + rotateRoot: document.getElementById('rotate-root'), + clientSerial: document.getElementById('client-serial'), + clientExpires: document.getElementById('client-expires'), + clientLeft: document.getElementById('client-left'), + reload: document.getElementById('reload'), + table: document.getElementById('agents-table'), + body: document.querySelector('#agents-table tbody'), + empty: document.getElementById('agents-empty'), + result: document.getElementById('agents-result'), + }; + + /** + * The question asked before each write: what the press does to the fleet, in one sentence. + * + * `confirm()` and not the typed `DANGER` word: that gate is for commands that cut power, and + * the worst thing a certificate action does is stop one host being dialled until its new + * bundle is installed. Fixed sentences from the page — the API's answer arrives afterwards. + */ + const CONFIRM = { + rotate: (name) => 'Issue a new certificate for ' + name + '? The dashboard dials with it ' + + 'from the next poll, so that host needs its new bundle installed before it answers ' + + 'again.', + revoke: (name) => 'Revoke ' + name + '? It stops the dashboard trusting that host until a ' + + 'reissue and a re-delivered bundle, and every other bundle is re-sealed to list this ' + + 'one.', + root: 'Rotate the root CA? Every agent certificate and this dashboard\'s own client ' + + 'certificate are replaced. The dashboard has to restart, and every host needs its ' + + 'bundle re-installed.', + }; + + function missing(value) { + return value === null || value === undefined || value === ''; + } + + function localTime(epoch) { + if (typeof epoch !== 'number' || !isFinite(epoch)) return null; + return new Date(epoch * 1000).toLocaleString(undefined, + { dateStyle: 'medium', timeStyle: 'short' }); + } + + /** `in 364d`, or the truth about a certificate whose clock has already run out. */ + function timeLeft(epoch) { + if (typeof epoch !== 'number' || !isFinite(epoch)) return null; + const seconds = epoch - Date.now() / 1000; + return seconds <= 0 ? 'expired' : 'in ' + Dash.duration(seconds); + } + + /** + * A serial as text. A certificate serial is up to 64 bits and JS numbers are 53, so the API + * sends the decimal string and the page never lets it near an arithmetic operator. + */ + function decimal(serial) { + return missing(serial) ? null : String(serial); + } + + function textCell(className, value) { + const td = document.createElement('td'); + if (className) td.className = className; + Dash.setTxt(td, value); + return td; + } + + function statusCell(entry) { + const td = document.createElement('td'); + const word = missing(entry.status) ? 'unknown' : String(entry.status); + td.appendChild(Dash.chip(word, STATUS_CHIP[word] || '')); + return td; + } + + /** The expiry twice: the moment it happens, and how long is left from now. */ + function expiryCell(entry) { + const td = document.createElement('td'); + td.textContent = localTime(entry.not_after) || Dash.naText; + const left = timeLeft(entry.not_after); + if (left) { + const faint = document.createElement('span'); + faint.className = 'faint'; + faint.textContent = ' · ' + left; + td.appendChild(faint); + } + return td; + } + + /** + * The bundle pair: what the CA issued, and what that agent says it is running. + * + * A `null` `deployed_bundle_rev` means the agent has never published one — a token-mode host + * sends no `agent.tls` block at all — which is `n/a` and never revision 0. A pair that differs + * gets a neutral chip with no verdict class: a revision count is not a graded number, exactly + * the `/llm` fleet-header rule, so "install the new bundle" is a fact in grey and not an alarm. + */ + function bundleCell(entry) { + const issued = entry.bundle_rev; + const deployed = entry.deployed_bundle_rev; + const td = document.createElement('td'); + td.className = 'num'; + if (missing(issued)) { + Dash.setTxt(td, null); + return td; + } + td.textContent = '#' + issued + ' → ' + (missing(deployed) ? Dash.naText : '#' + deployed); + if (!missing(deployed) && deployed < issued) { + td.appendChild(document.createTextNode(' ')); + td.appendChild(Dash.chip('stale', '')); + } + return td; + } + + /** + * The three controls, minus any the API would refuse. + * + * A row whose host has left `HOST_TARGETS` is listed and not writable — every per-host route + * answers `404 {"configured": false}` before it signs or serves anything — so it gets the + * reason instead of three buttons whose only outcome is an error. A never-issued host has no + * bundle to download and nothing to strike out, so it gets **Rotate**, which is the action that + * mints revision 1. + */ + function actionCell(entry) { + const td = document.createElement('td'); + if (!entry.configured) { + const note = document.createElement('span'); + note.className = 'faint'; + note.textContent = 'not in HOST_TARGETS'; + td.appendChild(note); + return td; + } + const controls = document.createElement('div'); + controls.className = 'gap'; + td.appendChild(controls); + + controls.appendChild(button('Rotate', () => write( + '/api/certs/' + encodeURIComponent(entry.name) + '/rotate', + CONFIRM.rotate(entry.name), entry.name))); + + if (entry.status !== 'unissued') { + controls.appendChild(button('Revoke', () => write( + '/api/certs/' + encodeURIComponent(entry.name) + '/revoke', + CONFIRM.revoke(entry.name), entry.name))); + } + + if (!missing(entry.serial)) { + const bundle = document.createElement('a'); + bundle.className = 'ghost'; + bundle.href = '/api/certs/' + encodeURIComponent(entry.name) + '/bundle'; + bundle.textContent = 'bundle'; + bundle.title = 'the stored bytes this agent must hold, private key included'; + controls.appendChild(bundle); + } + return td; + } + + function button(label, onClick) { + const el = document.createElement('button'); + el.className = 'ghost'; + el.type = 'button'; + el.textContent = label; + el.addEventListener('click', onClick); + return el; + } + + function row(entry) { + const tr = document.createElement('tr'); + for (const column of COLUMNS) tr.appendChild(column.build(entry)); + return tr; + } + + /** Outcome of a write: the server's own `hint`, then a re-read so the table cannot lie. + * + * Both write routes answer with the sentence about what happens next (`install the new bundle + * on deskwork — /api/certs/deskwork/bundle`), which is the only reason the line exists: a + * generic "done" would be the page narrating its own buttons. Every word goes out through + * `textContent`. + */ + async function write(url, question, subject) { + if (!window.confirm(question)) return; + say(els.result, subject + '…', 'warn'); + try { + const body = await Dash.fetchJSON(url, { method: 'POST' }); + say(els.result, body.hint || subject + ': ok', 'ok'); + await read(); + } catch (err) { + say(els.result, err.message || 'the write was refused', 'fail'); + } + } + + async function rotateRoot() { + if (!window.confirm(CONFIRM.root)) return; + say(els.rootResult, 'rotating the root CA…', 'warn'); + try { + const body = await Dash.fetchJSON('/api/certs/root/rotate', { method: 'POST' }); + // `restart_required` is why the answer is amber and not green: the table holds the new CA + // while this process goes on dialling with the material it loaded at startup. The sentence + // printed is the route's own `hint`, so the page cannot drift from it. + say(els.rootResult, body.hint || 'root certificate rotated', + body.restart_required ? 'warn' : 'ok'); + await read(); + } catch (err) { + say(els.rootResult, err.message || 'the write was refused', 'fail'); + } + } + + function say(el, text, kind) { + if (!el) return; + el.textContent = text || ''; + el.classList.remove('ok', 'warn', 'fail'); + if (kind) el.classList.add(kind); + } + + /** The two certificates that are not per-host, printed from the stored rows. */ + function paintAuthority(block, serialEl, expiresEl, leftEl) { + Dash.setTxt(serialEl, decimal(block.serial)); + Dash.setTxt(expiresEl, localTime(block.not_after)); + Dash.setTxt(leftEl, timeLeft(block.not_after)); + } + + async function read() { + let body; + try { + body = await Dash.fetchJSON('/api/certs'); + } catch (err) { + if (err.status === 401) return; // `fetchJSON` has already navigated + paintAuthority({}, els.rootSerial, els.rootExpires, els.rootLeft); + paintAuthority({}, els.clientSerial, els.clientExpires, els.clientLeft); + say(els.result, err.message || 'the authority could not be read', 'fail'); + return; + } + paintAuthority(body.root, els.rootSerial, els.rootExpires, els.rootLeft); + paintAuthority(body.client, els.clientSerial, els.clientExpires, els.clientLeft); + + const agents = Array.isArray(body.agents) ? body.agents : []; + els.body.replaceChildren(...agents.map(row)); + els.table.hidden = agents.length === 0; + els.empty.hidden = agents.length !== 0; + } + + if (els.reload) els.reload.addEventListener('click', read); + if (els.rotateRoot) els.rotateRoot.addEventListener('click', rotateRoot); + read(); +}()); diff --git a/tests/conftest.py b/tests/conftest.py index e56fb1e..338fda1 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -156,6 +156,18 @@ SMART_MISSING_BINARY = ("smart: /usr/sbin/smartctl not found — install smartmo "give this user device read access (disk group) for SMART data") +@pytest.fixture(scope="session") +def host_agent_module() -> Any: + """The stdlib agent as a module, for the tests that run it outside `tests/test_agent.py`. + + Loaded by path because it is one file that ships to every host and must not become a package + (see `_load_host_agent`). The phase-15 collector tests need one module object for both the + bundle they load and the handler they serve, so the TLS dial runs the artifact the fleet + really runs rather than a look-alike. + """ + return _load_host_agent() + + @pytest.fixture def smart_shape() -> Any: """`smart_shape('nvme')` / `smart_shape('ata_hdd')` -> the capture, parsed by the agent. diff --git a/tests/test_agent.py b/tests/test_agent.py index ee558d5..8b691d7 100644 --- a/tests/test_agent.py +++ b/tests/test_agent.py @@ -1,26 +1,49 @@ """The host agent: stdlib-only, read-only by construction, and wrong on no box. -Three things are asserted here: +Four things are asserted here: * parsing of /proc and /sys files exactly as Linux writes them (fake roots, so the weird cases are testable on any machine); * the collector produces a complete, non-crashing snapshot against the *real* /proc of whatever runs the tests, because that is the code that ships; - * the HTTP surface answers GETs, needs a token when it is exposed, and has no write path. + * the HTTP surface answers GETs, needs a token when it is exposed, and has no write path; + * the TLS mode (phase 15) answers a real handshake only to a client certificate the bundle's + root signed and the bundle's denylist does not name, with certificates minted by the + dashboard's own `certs.py` — there is no second generator to disagree with. """ from __future__ import annotations +import http.client import importlib.util import json import re +import socket +import ssl +import subprocess import sys import threading import time import urllib.error import urllib.request +from datetime import UTC, datetime, timedelta from http.server import ThreadingHTTPServer from pathlib import Path +from types import SimpleNamespace +from typing import Any, cast import pytest +from cryptography import x509 +from cryptography.hazmat.primitives.asymmetric import ec +from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey +from cryptography.hazmat.primitives.hashes import SHA256 +from cryptography.hazmat.primitives.serialization import ( + Encoding, + NoEncryption, + PrivateFormat, + load_pem_private_key, +) +from cryptography.x509.oid import NameOID + +from dashboards import certs ROOT = Path(__file__).resolve().parents[1] AGENT_PATH = ROOT / "agent" / "host_agent.py" @@ -414,6 +437,141 @@ def test_the_pass_budget_skips_the_rest_with_one_error(tmp_path, monkeypatch) -> assert any("budget" in e and "3 disk(s) skipped" in e for e in snap["errors"]) +class _TimeProxy: + """The agent's `time` module with one instrumented function: `monotonic`. + + Everything else (`time()`, `sleep()`) stays the stdlib one, so patching the clock cannot + leak into the rest of the snapshot or into pytest. + """ + + def __init__(self, clock: FakeSmartPassClock) -> None: + self._clock = clock + + def monotonic(self) -> float: + return self._clock.now + + def __getattr__(self, name: str) -> Any: + """Everything the agent's `time` module offers besides `monotonic`, unchanged.""" + return getattr(time, name) + + +class FakeSmartPassClock: + """A clock plus a `smartctl` spawn the test drives, so a **deadline** is assertable. + + The budget arithmetic in `_smart_pass` is invisible to a fake binary: `subprocess.run`'s + `timeout` never reaches the child, and burning 80 real seconds to watch a 16-disk pass + truncate is not a measurement anybody keeps. So this drives the two things the pass + actually reads — `time.monotonic()` and `subprocess.run()` — and records the timeout each + device was given, which is `min(deadline - now, 5 s)`: the pass's own view of how much + budget was left when it asked. Both patches go on `host_agent`'s own module globals + through proxies, never onto the stdlib modules themselves. + """ + + def __init__(self, *, seconds_per_device: float, start: float = 10_000.0) -> None: + self.seconds_per_device = seconds_per_device + self.now = start + self.calls: list[tuple[str, float]] = [] + self.output = read_fixture(NVME_CAPTURE) + + def run(self, argv: list[str], **kwargs: Any) -> subprocess.CompletedProcess[str]: + device = str(argv[-1]).rsplit("/", 1)[-1] + timeout = float(kwargs["timeout"]) + self.calls.append((device, timeout)) + self.now += self.seconds_per_device + return subprocess.CompletedProcess(argv, 0, self.output, "") + + def install(self, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(host_agent, "time", _TimeProxy(self)) + monkeypatch.setattr(host_agent, "subprocess", SimpleNamespace( + run=self.run, + TimeoutExpired=subprocess.TimeoutExpired, + CompletedProcess=subprocess.CompletedProcess)) + + +# driveripper's real inventory: 8 NVMe + 8 spinning, which is the shape the budget scales for +SHELF_DISKS = [f"nvme{i}n1" for i in range(8)] + [f"sd{c}" for c in "abcdefgh"] + + +def write_disk_shelf(root: Path, names: list[str]) -> None: + for name in names: + write_fake_disk(root, name, model=f"SYSFS {name}") + + +def test_the_pass_budget_scales_with_the_disk_count(tmp_path, monkeypatch) -> None: + """A 16-disk pass gets 5 s x 16, not the 20 s floor — because a truncated pass nulls the + tail drives on every single poll (the bug that put this on the operator's screen on + 2026-10-19). The clock here burns 1.5 s per device, so the pass costs 24 s of budget: + over the floor, far under the scale. With the flat budget this test truncates. + """ + root = tmp_path / "root" + write_disk_shelf(root, SHELF_DISKS) + clock = FakeSmartPassClock(seconds_per_device=1.5) + clock.install(monkeypatch) + collector = host_agent.Collector(str(root)) + snap = collector.snapshot() + + asked = [name for name, _ in clock.calls] + assert asked == snap["disk_io"]["device_names"], "every disk got its smartctl run" + assert len(asked) == 16 + assert clock.seconds_per_device * len(asked) > host_agent.SMARTCTL_BUDGET_SECONDS, ( + "this pass costs more than the 20 s floor — that is what makes it a scaling test") + assert [e for e in snap["errors"] if "budget" in e] == [], snap["errors"] + assert all(device["smart"] is not None for device in snap["disk_io"]["devices"]) + # the *late* devices still hold the whole per-device bound, which is the point of scaling + assert clock.calls[-1][1] == host_agent.SMARTCTL_TIMEOUT_SECONDS + assert clock.calls[0][1] == host_agent.SMARTCTL_TIMEOUT_SECONDS + + +def test_the_pass_budget_floor_is_twenty_seconds(tmp_path, monkeypatch) -> None: + """Scaling is `max(20 s, 5 s x disks)`, so one or two disks still get the 20 s floor and + not a 5 s or 10 s window. The second device's timeout is the visible half: with a 20 s + floor it is `min(20 - 18, 5)` = 2 s, where a budget of 5 s x 2 would have skipped it and a + 30 s floor would still have handed out the full 5 s. + """ + root = tmp_path / "root" + write_disk_shelf(root, ["nvme0n1", "sda"]) + clock = FakeSmartPassClock(seconds_per_device=18.0) + clock.install(monkeypatch) + collector = host_agent.Collector(str(root)) + snap = collector.snapshot() + + assert [name for name, _ in clock.calls] == ["nvme0n1", "sda"] + assert clock.calls[0][1] == host_agent.SMARTCTL_TIMEOUT_SECONDS + assert clock.calls[1][1] == pytest.approx( + host_agent.SMARTCTL_BUDGET_SECONDS - clock.seconds_per_device) + assert [e for e in snap["errors"] if "budget" in e] == [] + assert all(device["smart"] is not None for device in snap["disk_io"]["devices"]) + + +def test_the_pass_budget_error_line_names_the_scaled_figure(tmp_path, monkeypatch) -> None: + """When a scaled pass does run out, the operator's one line says *which* budget ran out: + 80 s on a 16-disk shelf, never the 20 s floor. A test that asserted only "a budget error + appears" would pass on the pre-scaling code; this one cannot. + """ + root = tmp_path / "root" + write_disk_shelf(root, SHELF_DISKS) + clock = FakeSmartPassClock(seconds_per_device=6.0) + clock.install(monkeypatch) + collector = host_agent.Collector(str(root)) + snap = collector.snapshot() + + scaled = max(host_agent.SMARTCTL_BUDGET_SECONDS, + host_agent.SMARTCTL_TIMEOUT_SECONDS * len(SHELF_DISKS)) + assert scaled == 80.0 + # 6 s per device: devices 1-14 fit inside 80 s (78 s spent), the 15th finds the deadline + # gone, and its sibling is skipped with it. + assert len(clock.calls) == 14, [name for name, _ in clock.calls] + assert clock.calls[-1][1] == pytest.approx(scaled - 6.0 * 13) + budget_lines = [e for e in snap["errors"] if "budget" in e] + assert len(budget_lines) == 1, snap["errors"] + assert budget_lines[0] == (f"smart: pass budget of {scaled:.0f}s exhausted, " + f"2 disk(s) skipped") + assert f"{host_agent.SMARTCTL_BUDGET_SECONDS:.0f}s" not in budget_lines[0] + by_name = {device["name"]: device for device in snap["disk_io"]["devices"]} + assert [by_name[name]["smart"] is None for name in SHELF_DISKS[:14]] == [False] * 14 + assert [by_name[name]["smart"] is None for name in SHELF_DISKS[14:]] == [True, True] + + def test_unexecutable_smartctl_degrades_like_a_missing_one(tmp_path, monkeypatch) -> None: root = tmp_path / "root" write_fake_disk(root, "nvme0n1") @@ -606,6 +764,9 @@ def agent_server(): def start(token: str | None = None) -> tuple[str, int]: host_agent.Handler.collector = host_agent.Collector("/") host_agent.Handler.token = token + # Cleared on purpose: `Handler` is process-wide state, so a token-mode server started + # after a TLS test in this file must not inherit that test's bundle. + host_agent.Handler.bundle = None httpd = ThreadingHTTPServer(("127.0.0.1", 0), host_agent.Handler) servers.append(httpd) threading.Thread(target=httpd.serve_forever, daemon=True).start() @@ -687,8 +848,10 @@ def test_agent_has_no_write_path(agent_server) -> None: assert excinfo.value.code in (405, 501) -def test_refuses_to_bind_off_loopback_without_a_token() -> None: +def test_refuses_to_bind_off_loopback_without_a_token(monkeypatch) -> None: """0.0.0.0 with no token would hand host inventory to the whole LAN.""" + monkeypatch.delenv("AGENT_TOKEN", raising=False) + monkeypatch.delenv("AGENT_CERT_BUNDLE", raising=False) assert host_agent.main(["--host", "0.0.0.0", "--port", "0"]) == 2 @@ -712,9 +875,10 @@ def test_agent_is_dependency_free() -> None: top = {name.split(".")[0] for name in imports} third_party = top - set(sys.stdlib_module_names) - {"host_agent"} assert not third_party, f"agent grew a non-stdlib import: {third_party}" - assert top <= {"__future__", "argparse", "collections", "dataclasses", "http", "json", - "os", "pathlib", "re", "socket", "subprocess", "threading", "time", - "typing"}, f"agent import footprint grew: {sorted(top)}" + assert top <= {"__future__", "argparse", "collections", "contextlib", "dataclasses", "http", + "json", "os", "pathlib", "re", "shutil", "socket", "ssl", "subprocess", + "tempfile", "threading", "time", "typing"}, ( + f"agent import footprint grew: {sorted(top)}") # ---- how the token is supplied ------------------------------------------- # @@ -746,6 +910,477 @@ def test_no_token_at_all_is_allowed_on_loopback(tmp_path) -> None: assert (token, error) == (None, None) -def test_main_refuses_to_bind_off_loopback_without_a_token(tmp_path) -> None: +def test_main_refuses_to_bind_off_loopback_without_a_token(tmp_path, monkeypatch) -> None: + monkeypatch.delenv("AGENT_TOKEN", raising=False) + monkeypatch.delenv("AGENT_CERT_BUNDLE", raising=False) code = host_agent.main(["--host", "0.0.0.0", "--token-file", str(tmp_path / "missing")]) assert code == 2, "the agent must not start exposed and unauthenticated" + + +# ---- TLS mode: one installed bundle, and its client certificate is the auth --- # +# +# Everything below runs a real `ssl` handshake against the real handler, because the claims +# here are handshake claims: "optional at the handshake, enforced by the handler" is exactly +# the kind of thing that reads fine and then makes `/health` unprobeable. The certificates come +# from `dashboards.certs` over a dict store, so the agent is tested against the artifact the +# dashboard actually signs rather than against a second generator that could drift from it. + + +#: The agent module is loaded by `importlib` (it is one file with no importable package), so +#: pyright sees `host_agent` as an untyped module and rejects `host_agent.TlsBundle` in a type +#: position. This alias is the honest spelling: the object is that class, and the tests below +#: assert `isinstance` where the claim actually matters. +Bundle = Any + + +class DictMeta: + """The three-method `certs.MetaStore` slice, backed by a dict (the `test_certs.py` idiom).""" + + def __init__(self) -> None: + self.rows: dict[str, str] = {} + + def get_meta(self, key: str) -> str | None: + return self.rows.get(key) + + def set_meta(self, key: str, value: str) -> None: + self.rows[key] = value + + def delete_meta(self, key: str) -> int: + return 1 if self.rows.pop(key, None) is not None else 0 + + +def sign_client_leaf(state: certs.CertsState, cn: str, where: Path) -> SimpleNamespace: + """A second client identity under the dashboard's own root, so it verifies and can be revoked. + + `certs.py` mints exactly one client certificate on purpose (the dashboard has one identity), + so the revoked-peer and wrong-authority cases need a leaf signed by hand. The serial comes + back because the denylist names a serial, not a name. + """ + root_key = cast(EllipticCurvePrivateKey, + load_pem_private_key(state.root_key.encode("utf-8"), password=None)) + root_cert = x509.load_pem_x509_certificate(state.root_cert.encode("utf-8")) + key = ec.generate_private_key(ec.SECP256R1()) + moment = datetime.now(UTC) + cert = ( + x509.CertificateBuilder() + .subject_name(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, cn)])) + .issuer_name(root_cert.subject) + .public_key(key.public_key()) + .serial_number(x509.random_serial_number()) + .not_valid_before(moment - timedelta(days=1)) + .not_valid_after(moment + timedelta(days=365)) + .add_extension(x509.BasicConstraints(ca=False, path_length=None), critical=True) + .sign(root_key, SHA256()) + ) + cert_path = where / f"{cn}-cert.pem" + key_path = where / f"{cn}-key.pem" + cert_path.write_bytes(cert.public_bytes(Encoding.PEM)) + key_path.write_bytes(key.private_bytes(Encoding.PEM, PrivateFormat.PKCS8, NoEncryption())) + return SimpleNamespace(cert=cert_path, key=key_path, serial=cert.serial_number) + + +@pytest.fixture(scope="module") +def tls_fleet(tmp_path_factory: pytest.TempPathFactory) -> SimpleNamespace: + """A dashboard CA, its client pair, one issued agent bundle, and one foreign authority.""" + base = tmp_path_factory.mktemp("agent-tls") + store = DictMeta() + state = certs.ensure_certs(store) + state = certs.issue_agent_cert(store, state, "localhost", ["127.0.0.1", "localhost"]) + entry = state.agent("localhost") + assert entry is not None + for name in ("dashboard", "foreign"): + (base / name).mkdir() + dashboard = certs.write_material(state, base / "dashboard") + foreign = certs.write_material(certs.ensure_certs(DictMeta()), base / "foreign") + return SimpleNamespace( + root_cert=state.root_cert, + bundle=entry.bundle, + bundle_rev=entry.bundle_rev, + client_cert=str(dashboard.client_cert), + client_key=str(dashboard.client_key), + foreign_cert=str(foreign.client_cert), + foreign_key=str(foreign.client_key), + stolen=sign_client_leaf(state, "stolen", base), + ) + + +def write_bundle(tmp_path: Path, text: str) -> Path: + """An installed bundle: one PEM file, exactly as `install-agent.sh` will place it.""" + path = tmp_path / "dashboards-agent.certs" + path.write_text(text, encoding="ascii") + return path + + +def load_bundle(tmp_path: Path, text: str) -> Bundle: + path = write_bundle(tmp_path, text) + bundle, error = host_agent.TlsBundle.load(str(path)) + assert error is None and bundle is not None + return bundle + + +def client_context(ca: str, cert: str | None = None, key: str | None = None) -> ssl.SSLContext: + """A client that trusts `ca`, and presents a client certificate only when given one.""" + context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + context.load_verify_locations(cadata=ca) + if cert and key: + context.load_cert_chain(cert, key) + return context + + +@pytest.fixture +def tls_agent_server(): + """The real handler on a TLS-wrapped loopback port; yields the base URL.""" + servers: list[ThreadingHTTPServer] = [] + + def start(bundle: Bundle, token: str | None = None) -> str: + host_agent.Handler.collector = host_agent.Collector("/", smart=False) + host_agent.Handler.token = token + host_agent.Handler.bundle = bundle + httpd = ThreadingHTTPServer(("127.0.0.1", 0), host_agent.Handler) + httpd.socket = bundle.context.wrap_socket(httpd.socket, server_side=True) + servers.append(httpd) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + return f"https://127.0.0.1:{httpd.server_address[1]}" + + yield start + for httpd in servers: + httpd.shutdown() + httpd.server_close() + host_agent.Handler.bundle = None + + +def _json_or_raw(body: bytes) -> dict: + """JSON when the handler answered with JSON; the raw text for the status page.""" + try: + return json.loads(body) + except json.JSONDecodeError: + return {"raw": body.decode(errors="replace")} + + +def _https_get(url: str, context: ssl.SSLContext) -> tuple[int, dict]: + request = urllib.request.Request(url) + try: + with urllib.request.urlopen(request, timeout=15, context=context) as response: + return response.status, _json_or_raw(response.read()) + except urllib.error.HTTPError as error: + return error.code, _json_or_raw(error.read()) + + +def test_the_agent_and_the_dashboard_agree_on_the_bundle_lines() -> None: + """The agent parses a PEM by text with stdlib only, so the two literals are the contract.""" + assert host_agent.BUNDLE_REV_LINE == certs.BUNDLE_REV_LINE + assert host_agent.REVOKED_LINE == certs.REVOKED_LINE + + +def test_the_bundle_lines_are_read_as_written() -> None: + text = "# BUNDLE-REV 12\n# REVOKED 5\n# REVOKED 77\n" + assert host_agent.bundle_revision(text) == 12 + assert host_agent.bundle_revoked(text) == frozenset({"5", "77"}) + assert host_agent.bundle_revision("nothing at all\n") is None + assert host_agent.bundle_revision("# BUNDLE-REV not-a-number\n") is None + assert host_agent.bundle_revoked("# REVOKED\n") == frozenset() + + +@pytest.mark.parametrize("revoked, peer_hex, expected", [ + # the shape a real `x509.random_serial_number()` serial arrives in, both spellings + ("57989923439447977599989997426575336710275559875", + "A285BAC84EB9C5FC282EF7CDB8D962F38BF71C3", True), + ("12648430", "C0FFEE", True), + ("14", "0E", True), + ("0E", "0E", True), + ("15", "0E", False), + ("0", "0E", False), +]) +def test_the_denylist_matches_a_serial_whichever_way_it_is_written(revoked: str, + peer_hex: str, + expected: bool) -> None: + """`getpeercert()` says hex, `certs.py` writes decimal, and the agent must meet in the middle. + + An agent comparing one spelling to the other accepts a revoked client forever: the operator + clicked revoke, `/certs` says revoked, and the fleet keeps trusting the stolen identity. + """ + cert = {"serialNumber": peer_hex} if peer_hex else {} + assert host_agent.serial_is_revoked(cert, frozenset({revoked})) is expected + + +def test_a_certificate_with_no_serial_is_never_treated_as_revoked() -> None: + assert host_agent.serial_is_revoked({}, frozenset({"1", "0"})) is False + + +def test_resolve_bundle_reports_the_revision_and_the_denylist_of_a_real_bundle( + tls_fleet: SimpleNamespace, tmp_path: Path) -> None: + text = tls_fleet.bundle + f"{host_agent.REVOKED_LINE}{tls_fleet.stolen.serial}\n" + context, revision, revoked, error = host_agent.resolve_bundle(str(write_bundle(tmp_path, text))) + assert error is None + assert context is not None and isinstance(context, ssl.SSLContext) + assert revision == tls_fleet.bundle_rev + assert revoked == frozenset({str(tls_fleet.stolen.serial)}) + + +@pytest.mark.parametrize("shape", ["missing", "empty", "no-key", "no-revision"]) +def test_a_bundle_that_cannot_be_loaded_is_an_error_naming_the_file( + tls_fleet: SimpleNamespace, tmp_path: Path, shape: str) -> None: + """Every failure is a message the operator can act on, and none of them is "no TLS today".""" + if shape == "missing": + path = tmp_path / "absent.certs" + elif shape == "empty": + path = write_bundle(tmp_path, " \n") + elif shape == "no-key": + body = [block for block in re.findall( + r"-----BEGIN [A-Z0-9 ]+-----.*?-----END [A-Z0-9 ]+-----\n", tls_fleet.bundle, re.S) + if block.startswith("-----BEGIN CERTIFICATE")] + path = write_bundle(tmp_path, "# BUNDLE-REV 1\n" + "".join(body)) + else: + path = write_bundle(tmp_path, tls_fleet.bundle.replace("# BUNDLE-REV 1\n", "")) + context, revision, revoked, error = host_agent.resolve_bundle(str(path)) + assert context is None and revision == 0 and revoked == frozenset() + assert error is not None and ("absent" in error if shape == "missing" else str(path) in error) + if shape == "no-revision": + assert "BUNDLE-REV" in error + + +def test_the_unpacked_pems_are_gone_once_the_context_has_them( + tls_fleet: SimpleNamespace, tmp_path: Path, monkeypatch) -> None: + """The unit's RuntimeDirectory is used when it exists, and holds nothing afterwards. + + `ssl` loads a chain from paths, so the key is written; leaving it there means the private key + outlives the process on a disk an operator did not choose, and `kill -9` has no cleanup path. + """ + runtime = tmp_path / "run" + runtime.mkdir() + monkeypatch.setattr(host_agent, "BUNDLE_RUNTIME_DIR", runtime) + context, _revision, _revoked, error = host_agent.resolve_bundle(str(write_bundle(tmp_path, + tls_fleet.bundle))) + assert error is None and context is not None + assert list(runtime.iterdir()) == [], "the private key must not stay on disk once loaded" + + +def test_a_missing_runtime_directory_falls_back_to_a_temporary_one( + tls_fleet: SimpleNamespace, tmp_path: Path, monkeypatch) -> None: + monkeypatch.setattr(host_agent, "BUNDLE_RUNTIME_DIR", tmp_path / "not-mounted") + context, revision, _revoked, error = host_agent.resolve_bundle(str(write_bundle(tmp_path, + tls_fleet.bundle))) + assert error is None and context is not None + assert revision == tls_fleet.bundle_rev + + +def test_a_client_certificate_signed_by_the_root_reads_the_snapshot( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """The whole point of the phase, plus the two fields task 03 reads. + + `bundle_rev` is how `/certs` tells "this host runs an old bundle" from "this host is current", + so it is the number the bundle carried, not a value the agent made up. + """ + base = tls_agent_server(load_bundle(tmp_path, tls_fleet.bundle)) + status, body = _https_get(base + "/snapshot", + client_context(tls_fleet.root_cert, tls_fleet.client_cert, + tls_fleet.client_key)) + assert status == 200 + assert body["ok"] is True and body["host"]["hostname"] + assert body["tls"] is True + assert body["bundle_rev"] == tls_fleet.bundle_rev + for path in ("/info", "/schema", "/"): + status, _body = _https_get(base + path, client_context(tls_fleet.root_cert, + tls_fleet.client_cert, + tls_fleet.client_key)) + assert status == 200, path + + +def test_health_is_probeable_with_the_root_ca_alone_while_every_other_path_is_not( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """The `CERT_OPTIONAL` assumption: a probe with no client certificate must still work. + + `install-agent.sh --status` and the systemd probe hold the CA and were never handed a client + certificate; `CERT_REQUIRED` would make every TLS install look dead. Everything that carries + metrics is 403 — including the status page, which repeats hostname, kernel and CPU model. + """ + base = tls_agent_server(load_bundle(tmp_path, tls_fleet.bundle)) + probe = client_context(tls_fleet.root_cert) + status, body = _https_get(base + "/health", probe) + assert status == 200 and body["ok"] is True + for path in ("/snapshot", "/info", "/schema", "/", "/nope"): + refused, detail = _https_get(base + path, probe) + assert refused == 403, path + assert detail["detail"] == "client certificate required", path + + +def test_a_client_signed_by_another_authority_is_refused( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """A certificate that does not chain to the bundle's root never reaches a handler. + + The server verifies at the handshake, so the answer is an aborted connection rather than an + HTTP code — the assertion is only ever `not 200` because which alert arrives is the peer + OpenSSL's choice, not ours. + """ + base = tls_agent_server(load_bundle(tmp_path, tls_fleet.bundle)) + stranger = client_context(tls_fleet.root_cert, tls_fleet.foreign_cert, tls_fleet.foreign_key) + with pytest.raises((OSError, http.client.HTTPException)) as caught: + _https_get(base + "/snapshot", stranger) + assert not isinstance(caught.value, urllib.error.HTTPError) + + +def test_a_plaintext_request_to_a_tls_agent_gets_nothing( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """A port probe cannot read the answer, and does not get a helpful error either.""" + base = tls_agent_server(load_bundle(tmp_path, tls_fleet.bundle)) + port = int(base.rsplit(":", 1)[-1]) + with socket.create_connection(("127.0.0.1", port), timeout=15) as raw: + raw.sendall(b"GET /snapshot HTTP/1.1\r\nHost: x\r\n\r\n") + raw.settimeout(5) + try: + payload = raw.recv(200) + except OSError: + payload = b"" + assert b"hostname" not in payload + + +def test_a_revoked_client_certificate_is_refused_and_the_bundle_is_re_read( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """The two halves: the denylist bites, and re-deploying the bundle takes effect no later + than the next request — no restart, which is the only reason the lines live in the file.""" + path = write_bundle(tmp_path, tls_fleet.bundle + + f"{host_agent.REVOKED_LINE}{tls_fleet.stolen.serial}\n") + bundle, error = host_agent.TlsBundle.load(str(path)) + assert error is None and bundle is not None + base = tls_agent_server(bundle) + stolen = client_context(tls_fleet.root_cert, str(tls_fleet.stolen.cert), str(tls_fleet.stolen.key)) + + status, body = _https_get(base + "/snapshot", stolen) + assert status == 403 + assert body["detail"] == "client certificate revoked" + # A withdrawn identity is refused even on the open path: the exemption is for a probe that + # holds the CA and no certificate, not for a certificate the operator has just taken away. + assert _https_get(base + "/health", stolen)[0] == 403 + # The dashboard's own identity is untouched — the list names a serial, not a host. + assert _https_get(base + "/snapshot", client_context(tls_fleet.root_cert, tls_fleet.client_cert, + tls_fleet.client_key))[0] == 200 + + path.write_text(tls_fleet.bundle, encoding="ascii") # a redeploy: the line is gone + status, body = _https_get(base + "/snapshot", stolen) + assert status == 200, "the denylist is re-read when the file changes, without a restart" + + +def test_a_renamed_denylist_entry_is_re_read_on_the_next_request( + tls_fleet: SimpleNamespace, tmp_path: Path) -> None: + """An unreadable bundle keeps the list this agent already had rather than forget it.""" + path = write_bundle(tmp_path, tls_fleet.bundle + + f"{host_agent.REVOKED_LINE}{tls_fleet.stolen.serial}\n") + bundle, error = host_agent.TlsBundle.load(str(path)) + assert error is None and bundle is not None + assert bundle.revoked_serials() == frozenset({str(tls_fleet.stolen.serial)}) + path.unlink() + assert bundle.revoked_serials() == frozenset({str(tls_fleet.stolen.serial)}) + + +def test_the_token_is_not_read_on_the_tls_path( + tls_fleet: SimpleNamespace, tls_agent_server, tmp_path: Path) -> None: + """Bundle plus token: the handshake is the auth, so a stolen token cannot outlive a revoke. + + Checking the bearer on the TLS path too would not be defence in depth — it would be a second + credential that keeps working after the operator revoked the certificate, which is the one + thing the denylist exists to prevent. + """ + base = tls_agent_server(load_bundle(tmp_path, tls_fleet.bundle), token="s3cret") + status, _body = _https_get(base + "/snapshot", + client_context(tls_fleet.root_cert, tls_fleet.client_cert, + tls_fleet.client_key)) + assert status == 200 + + +def test_token_mode_reports_no_tls_fields(agent_server) -> None: + """Token mode is byte-identical: no `tls`, no `bundle_rev`, no new field that an agent + installed before phase 15 does not have.""" + assert host_agent.Handler.bundle is None, "the token-mode fixture must not inherit a bundle" + base, _port = agent_server(token="s3cret") + _status, body = _get(base + "/snapshot", {"Authorization": "Bearer s3cret"}) + assert body["ok"] is True + assert "tls" not in body and "bundle_rev" not in body + + +def test_serve_wraps_the_socket_and_says_tls_and_the_revision( + tls_fleet: SimpleNamespace, tmp_path: Path, capsys, monkeypatch) -> None: + """`serve()` is the function the unit runs, so its start line is what an operator reads. + + The server is recorded rather than guessed at a port: the test needs the ephemeral port the + OS chose *and* a way to stop `serve_forever`, which is the same object. + """ + created: list[ThreadingHTTPServer] = [] + + class Recording(ThreadingHTTPServer): + def __init__(self, *args: object, **kwargs: object) -> None: + super().__init__(*args, **kwargs) # type: ignore[arg-type] + created.append(self) + + monkeypatch.setattr(host_agent, "ThreadingHTTPServer", Recording) + bundle = load_bundle(tmp_path, tls_fleet.bundle) + thread = threading.Thread(target=host_agent.serve, + args=("127.0.0.1", 0, host_agent.Collector("/", smart=False), + None, bundle), + daemon=True) + thread.start() + deadline = time.monotonic() + 15 + while not created and time.monotonic() < deadline: + time.sleep(0.01) + assert created, "serve() never built its server" + httpd = created[0] + port = httpd.server_address[1] + status, _body = _https_get(f"https://127.0.0.1:{port}/health", + client_context(tls_fleet.root_cert)) + assert status == 200 + httpd.shutdown() + thread.join(timeout=15) + printed = capsys.readouterr().out + assert "tls (client cert required, rev " in printed + # the scheme is the whole claim here (the port in the line is the requested one, which is 0 + # when the OS picks, exactly as in token mode) + assert "[agent] listening on https://127.0.0.1" in printed + assert "token" not in printed, "a TLS agent does not read a token" + + +def test_a_bundle_allows_the_off_loopback_bind_without_a_token( + tls_fleet: SimpleNamespace, tmp_path: Path, monkeypatch) -> None: + """The client certificate *is* the protection, so the exit-2 refusal does not apply.""" + served: list[tuple] = [] + monkeypatch.setattr(host_agent, "serve", + lambda *args: served.append(args)) + path = write_bundle(tmp_path, tls_fleet.bundle) + assert host_agent.main(["--host", "0.0.0.0", "--port", "0", "--cert-bundle", str(path)]) == 0 + assert served, "a bundle-configured agent must start" + host_arg, _port, _collector, token_arg, bundle_arg = served[0] + assert (host_arg, token_arg) == ("0.0.0.0", None) + assert isinstance(bundle_arg, host_agent.TlsBundle) + + +@pytest.mark.parametrize("extra", [[], ["--cert-bundle", ""]]) +def test_off_loopback_without_a_token_or_a_bundle_is_refused_and_names_both( + tmp_path: Path, capsys, monkeypatch, extra: list[str]) -> None: + monkeypatch.delenv("AGENT_TOKEN", raising=False) + monkeypatch.delenv("AGENT_CERT_BUNDLE", raising=False) + args = ["--host", "0.0.0.0", "--port", "0", *extra] + assert host_agent.main(args) == 2 + printed = capsys.readouterr().out + assert "--token" in printed and "--cert-bundle" in printed + + +def test_an_unreadable_bundle_is_fatal_at_startup(tmp_path: Path, capsys) -> None: + """Exit 2 naming the file: a degraded-to-plain-HTTP agent is the answer that is worse.""" + missing = tmp_path / "absent.certs" + assert host_agent.main(["--host", "127.0.0.1", "--port", "0", + "--cert-bundle", str(missing)]) == 2 + assert str(missing) in capsys.readouterr().out + + +def schema_documents(field: str) -> bool: + """Is this field in the SCHEMA block? `GET /schema` answers with that text, self-documenting + an install for whoever has to debug it, so a field the payload carries belongs there.""" + doc = host_agent.__doc__ or "" + schema = doc.split("SCHEMA (GET /snapshot)", 1)[-1].split("Deployment:", 1)[0] + return re.search(rf"^\s+{field}\s", schema, re.M) is not None + + +def test_the_agent_version_and_schema_document_the_new_fields() -> None: + """1.1.0 because the payload grew, and the docstring is what `GET /schema` answers.""" + assert host_agent.AGENT_VERSION == "1.1.0" + assert schema_documents("tls") and schema_documents("bundle_rev") + assert "--cert-bundle" in (host_agent.__doc__ or "") + diff --git a/tests/test_api.py b/tests/test_api.py index 6275e3f..8cb2f81 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -7,6 +7,8 @@ asset the network can fail to deliver. """ from __future__ import annotations +import ipaddress +import json import re import time from collections.abc import Iterator @@ -16,13 +18,15 @@ from typing import Any import httpx import pytest +from cryptography import x509 +from cryptography.x509.oid import NameOID from fastapi import HTTPException from fastapi.testclient import TestClient import store_perf from auth_client import sign_in from auth_contract import ADMIN_PASSWORD, FAST_PBKDF2 -from dashboards import alerts, auth +from dashboards import alerts, auth, certs from dashboards.api import MIN_PASSWORD_LENGTH, RANGE_SECONDS, parse_range from dashboards.app import create_app from dashboards.collectors import UpstreamError @@ -61,7 +65,11 @@ PAGES = ("/", "/system", "/disks", "/llm", "/ups") #: tab, a fixed tab order — which this page deliberately does not have. `MARKUP_PAGES` is the #: list for the checks that are about the bytes rather than the nav. SETTINGS_PAGE = "/settings" -MARKUP_PAGES = (*PAGES, SETTINGS_PAGE) +#: `/certs` is the second admin page (phase 15 task 05), kept out of `PAGES` for exactly the same +#: reason: it is not a tab. It is in `MARKUP_PAGES` and `PAGE_SCRIPTS`, so the CDN, `.sev-`, +#: flavour-text and orphan-`id` audits cover it like the five dashboards. +CERTS_PAGE = "/certs" +MARKUP_PAGES = (*PAGES, SETTINGS_PAGE, CERTS_PAGE) @contextmanager @@ -105,6 +113,25 @@ def ups_client(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Iterator[Test yield c +def wait_for_source_error(client: TestClient, key: str, fragment: str, + *, timeout: float = 15.0) -> str: + """Wait for one poll loop to record an error saying `fragment`, and hand back the sentence. + + The poll loops run in the app's event loop, so an assertion about what a source said has to + wait for a poll rather than read the state at import time. The failure names the last error + seen, because "it never said that" and "it said something else entirely" are different bugs. + """ + deadline = time.time() + timeout + error = "" + while time.time() < deadline: + rows = {row["key"]: row for row in client.get("/api/health").json()["sources"]} + error = str((rows.get(key) or {}).get("error") or "") + if fragment in error: + return error + time.sleep(0.05) + pytest.fail(f"{key} never reported {fragment!r} (last error: {error!r})") + + def wait_for_payload(client: TestClient, path: str, *, timeout: float = 5.0) -> dict[str, Any]: deadline = time.time() + timeout while time.time() < deadline: @@ -721,6 +748,49 @@ def test_a_dead_host_never_writes_its_token_into_the_feed(tmp_path: Path) -> Non store.close() +def test_a_revoked_agent_fails_both_of_its_sources_without_a_dial( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The chain end to end, wiring included: revoke it, and the next poll stops talking to it. + + `collectors.agent_tls_map` is what `build_hub` hands the jobs, so this is the proof that a + revocation reaches a *running* app through its own `meta` table — no restart, and no second + process telling the first. The spy replaces `httpx.AsyncClient.get` on the class, so it + catches the shared client and the per-dial TLS client alike: what must not happen is any + request at all, because a dial that then gets refused still put a socket on a host the + operator struck out. + """ + db = str(tmp_path / "revoked-agent.sqlite3") + store = SampleStore(db) + settings = Settings(db_path=db, admin_password=ADMIN_PASSWORD, + pbkdf2_iterations=FAST_PBKDF2, + host_interval=0.05, disks_interval=0.05, + host_targets=(Target(name="deskwork", url="https://127.0.0.1:1"),)) + with TestClient(create_app(settings, store)) as client: + sign_in(client) + # Before the revoke the same dead host fails as an unreachable upstream. That sentence is + # the control: it is what a dial produces, so the one after it has to mean something else. + wait_for_source_error(client, "host:deskwork", "unreachable") + state = certs.issue_agent_cert(store, certs.ensure_certs(store), "deskwork", + ["127.0.0.1"]) + certs.revoke(store, state, "deskwork") + + calls: list[str] = [] + + async def spy_get(self: Any, url: Any, **kwargs: Any) -> Any: + calls.append(str(url)) + raise AssertionError(f"a revoked agent was dialled at {url}") + + monkeypatch.setattr(httpx.AsyncClient, "get", spy_get) + for key in ("host:deskwork", "disks:deskwork"): + error = wait_for_source_error(client, key, "agent certificate revoked") + assert "unreachable" not in error, "the refusal replaced the dial's sentence" + assert calls == [], "the gate is in front of the network, not behind it" + # and the overview says the operator revoked it rather than that the box went down + messages = [row["message"] for row in client.get("/api/summary").json()["alerts"]] + assert any("certificate revoked" in text for text in messages), messages + store.close() + + def test_engine_urls_are_redacted_the_same_way_hosts_are(tmp_path: Path) -> None: """`GUFO_BASE_URL`/`VLLM_BASE_URL` go to the browser verbatim in `/api/config`, and unlike `HOST_TARGETS` nothing parses a userinfo token out of them into a header — so a URL that @@ -926,25 +996,176 @@ def test_the_settings_page_refuses_a_caller_with_no_session(tmp_path: Path) -> N store.close() -def test_every_dashboard_page_links_to_settings(client) -> None: - """The `⚙ Account` link is the only way to the session page, so it is on every dashboard. +def test_every_page_links_to_both_admin_pages(client) -> None: + """`⚙ Account` and `⚷ Certs` are the only way to the two admin pages, so they are everywhere. - It sits in `.masthead-right` before the theme toggle and **outside** `.tabs`, because the tab + They sit in `.masthead-right` before the theme toggle and **outside** `.tabs`, because the tab strip's structure — five dashboards, one of them current — is pinned by `test_the_disks_tab_is_in_the_masthead_of_every_page` and a sixth entry with no `aria-current` would break what that marker means. + + Neither page links to itself, which is the `/settings` precedent from phase 11 and the reason + the two loops below are over different tuples: an `⚷ Certs` link on `/certs` is a no-op the + operator learns to ignore, and ignored chrome is worse than none. """ - link = ']*aria-current="page"', html) + assert tabs == [], f"/certs marked a tab current: {tabs}" + assert 'id="nav-status"' not in html, "/certs grew fleet dots it has nothing to fill" + for path in PAGES: + assert f'href="{path}"' in html, f"/certs cannot navigate to {path}" + assert '/static/dash.css' in html + + +def test_the_certs_page_carries_the_authority_and_the_agents_table(client) -> None: + """The three blocks the page is: the root, the client line, and one row per host. + + Pinned on the served bytes rather than on a browser: the root and client panels and the table + are markup, and the two rows of the table body are built from `GET /api/certs` by + `page-certs.js` — so the per-row action labels are asserted on the served **script**, the way + `/ups` pins its command buttons. `tests/test_api.py` has no JS runner (a locked anchor), and a + label that shipped in neither file would be an invisible button either way. + """ + html = client.get(CERTS_PAGE).text + js = client.get("/static/page-certs.js").text + + assert "Root CA" in html and 'id="root-serial"' in html + assert 'id="root-expires"' in html and 'id="root-left"' in html + assert "Rotate root CA" in html, "the CA-wide action disappeared" + assert "Dashboard client" in html and 'id="client-serial"' in html + assert 'id="client-expires"' in html, "/certs lost the client expiry" + assert 'id="agents-table"' in html and "" in html + assert 'id="agents-empty"' in html and "hidden" in html + + for label in ("Rotate", "Revoke", "bundle"): + assert f"'{label}'" in js, f"the {label} action lost its label" + for route in ("/rotate", "/revoke", "/bundle"): + assert route in js, f"the page no longer reaches {route}" + assert "/api/certs/root/rotate" in js + + +def test_the_certs_page_prints_the_names_the_authority_mints(client) -> None: + """The CN of each certificate is markup, so the literal is pinned to `certs.py`. + + `MIN_PASSWORD_LENGTH`'s idiom: there is no build step to share a Python constant with a page, + so a page that restates one says it in markup and a test compares the two. Printing a CN the + module did not sign with would be the page inventing a fact about a stored certificate. + """ + html = client.get(CERTS_PAGE).text + assert f">{certs.ROOT_CN}<" in html, "the root panel's CN is not what certs.py signs" + assert f">{certs.CLIENT_CN}<" in html, "the client line's CN is not what certs.py signs" + + +def test_the_certs_page_reads_only_fields_the_cert_api_publishes( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """End to end, in both directions: the page's column table and `GET /api/certs` are one shape. + + The served script's `COLUMNS` entries name every payload key the page indexes, so this is the + `SLOT_FIELDS` device on the surface where a rename is worst: a `deployed_bundle_rev` that + quietly became `agent_bundle_rev` would print `n/a` beside a healthy fleet forever. + """ + script = "/static/page-certs.js" + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(3), + host_interval=3600.0) as client: + js = client.get(script).text + names: set[str] = set() + for listed in re.findall(r"fields: \[([^\]]*)\]", js): + names.update(re.findall(r"'([^']+)'", listed)) + assert names == CERT_AGENT_KEYS, sorted(names ^ CERT_AGENT_KEYS) + + client.post("/api/certs/deskwork/rotate") + for row in cert_agent_rows(client): + assert len(row) == 8 + for name in names: + assert name in row, f"the page indexes `{name}` and the API does not publish it" + + # …and the two authority blocks: the page prints a serial and an expiry for each, which is + # exactly the key set the API answers there. + body = cert_document(client) + for block in ("root", "client"): + assert set(body[block]) == CERT_TIMESERIAL_KEYS + for name in CERT_TIMESERIAL_KEYS: + assert f".{name}" in js, f"the page stopped reading `{name}`" + + +def test_the_certs_page_never_colours_a_certificate(client) -> None: + """No severity vocabulary reaches this page at all — an expiry is a date, not a verdict. + + `/api/certs` sends no `severity` anywhere (task 04, and `severity.py` rates no certificate), + so a page that coloured an expiring cert would be inventing a threshold in JavaScript. The one + colour on the page is the payload's `status` word through a word→class map, and `unissued` + maps to no class at all — a certificate that does not exist must not borrow green. + """ + js = _code(client.get("/static/page-certs.js").text) + assert "severity" not in js, "/certs grew a verdict the API does not send" + assert "severityFrom" not in js and "severityFor" not in js + table = js[js.index("const STATUS_CHIP = {"):js.index("};")] + assert "active: 'ok'" in table and "revoked: 'fail'" in table + assert "unissued: ''" in table, "an unissued host would be coloured" + assert "not_after" not in table and "serial" not in table, "an expiry became a verdict" + + +def test_the_certs_page_asks_before_it_writes_and_offers_nothing_the_api_refuses( + client) -> None: + """Every write is confirmed, and no row offers a control its own payload rules out. + + `confirm()` and not the typed `DANGER` word: that gate belongs to commands that cut power, + and this page has none — the worst outcome here is a host that is not dialled until somebody + installs a bundle. The guards are the API's, restated: a row outside `HOST_TARGETS` gets no + buttons at all (`404 {"configured": false}` is all three routes' first line), a never-issued + host has nothing to revoke and no bundle to download, and the bundle download is a plain + `GET` link because a session cookie is all a signed `GET` needs. + """ + js = _code(client.get("/static/page-certs.js").text) + assert "window.confirm(" in js + assert "DANGER" not in js, "the power-cut word migrated to a page with no power cut" + assert "entry.configured" in js and "not in HOST_TARGETS" in js + assert "entry.status !== 'unissued'" in js, "a row could revoke nothing" + assert "entry.serial" in js + assert "bundle.href = '/api/certs/'" in js + assert "method: 'POST'" in js + # …and the refusal is text, never markup: these are server sentences + assert "innerHTML" not in js + + +def test_the_certs_page_prints_the_restart_sentence_the_api_spoke(client) -> None: + """`restart_required` prints the route's `hint`, not a sentence the page wrote for itself. + + The restart fact lives in `api.py` (the CA and the client pair are startup material, so the + table is new before the socket is). A page copy of it is the version that quietly stops + saying "and every bundle needs re-installing". + """ + js = client.get("/static/page-certs.js").text + assert "restart_required" in js + assert "body.hint" in js + assert "restart the dashboard" not in js.lower(), "the page invented the restart sentence" def test_the_password_floor_the_form_checks_is_the_one_the_api_enforces(client) -> None: @@ -1000,7 +1221,7 @@ def test_a_rotated_secret_clears_the_cached_session_every_page_uses(client) -> N ASSETS = ("/static/dash.css", "/static/app.js", "/static/charts.js", "/static/page-index.js", "/static/page-disks.js", "/static/page-system.js", "/static/page-llm.js", "/static/page-ups.js", "/static/page-login.js", - "/static/page-settings.js") + "/static/page-settings.js", "/static/page-certs.js") def _css(client: TestClient) -> str: @@ -2894,6 +3115,9 @@ PAGE_SCRIPTS = { # `/settings` is audited like a dashboard even though it is not one: an id nothing reads is # dead markup whatever page it sits on, and this page has five new ones. SETTINGS_PAGE: "page-settings.js", + # `/certs` too (phase 15 task 05): an authority panel and an agents table are markup, and + # every id in them is written by this page's own script. + CERTS_PAGE: "page-certs.js", } # `/login` is deliberately **not** in this table even though it is audited (phase 11): a # signed-in client asking for it gets a 303 to the dashboard, so this helper's own @@ -3459,14 +3683,19 @@ def test_the_disks_surface_is_gets_only_and_the_write_stays_the_ups_one( assert response.status_code == 405, f"{verb} /api/disks/snapshot was accepted" writes = sorted(str(route.path) for route in routes if "POST" in (getattr(route, "methods", None) or set())) - # Invariant 1 is about the fleet, and phase 11 did not touch the fleet: the three `/auth/` - # writes create, re-credential, and clear a session in this process and dial nothing. They - # are named here instead of filtered out of sight, because the failure this assertion catches - # is a POST arriving quietly — a list the reader recognises is worth more than a short one. - upstream = [path for path in writes if not path.startswith("/api/auth/")] + # Invariant 1 is about the fleet. The `/auth/` writes create, re-credential, and clear a + # session in this process; the `/certs/` writes (phase 15) mint, reissue, and revoke a + # certificate in this process's `meta` table. Neither dials anything — and a revocation's whole + # effect is that the *next poll* refuses to dial — so they are named here rather than filtered + # out of sight, because the failure this assertion catches is a POST arriving quietly. + authority = [path for path in writes if path.startswith("/api/certs/")] session = [path for path in writes if path.startswith("/api/auth/")] + upstream = [path for path in writes + if not path.startswith(("/api/auth/", "/api/certs/"))] assert upstream == ["/api/ups/command"], "one write endpoint, four gates, unchanged" assert session == ["/api/auth/login", "/api/auth/logout", "/api/auth/password"] + assert authority == ["/api/certs/root/rotate", "/api/certs/{name}/revoke", + "/api/certs/{name}/rotate"], "the CA surface grew a write nobody named" # …and nothing anywhere grew the verbs this app has no use for, session included. assert not sorted(path for route in routes for path in [str(route.path)] @@ -3478,7 +3707,8 @@ def test_the_disks_surface_is_gets_only_and_the_write_stays_the_ups_one( and not str(route.path).startswith("/api/")) # `/settings` (phase 11) is a fifth dashboard-shaped **page** and it grew no verb: a page # route is a GET, so the write surface this test guards stays the one UPS command. - assert pages == ["/", "/disks", "/llm", "/login", "/settings", "/system", "/ups"] + assert pages == ["/", "/certs", "/disks", "/llm", "/login", "/settings", "/system", + "/ups"] def test_config_prints_the_disks_cadence_the_page_explains_itself_with( @@ -4160,6 +4390,487 @@ def test_samples_written_by_the_poller_are_queryable(tmp_path: Path, store.close() +# --------------------------------------------------------------------------- # +# The certificate authority surface (phase 15 task 04) +# +# These routes mint, re-sign, and revoke the identities every agent in the fleet is dialled with, +# so the assertions are about **stored rows and wire effects** rather than about JSON being shaped +# nicely: a read that issued a certificate, a typo that wrote a row, a revocation that did not +# reach the collector, and a key that left `meta` are the four ways this surface can be wrong while +# every status code still reads 200. +# +# The fleet here is loopback ports nobody listens on. That is deliberate: nothing on this surface +# is allowed to need a live agent, so a refused connection is both the whole environment and the +# control that says "a dial was attempted, and it was not this". +# --------------------------------------------------------------------------- # + +#: Three configured hosts, none of them reachable, on the scheme a protected exporter is dialled +#: by. The URL host is also what a rotated certificate's SAN must carry, so these are the SANs the +#: assertions below read. +DEAD_TARGETS = (Target(name="deskwork", url="https://127.0.0.1:1"), + Target(name="driveripper", url="https://127.0.0.1:2"), + Target(name="gateway", url="https://127.0.0.1:3")) + +#: The whole `GET /api/certs` document and one agent row, pinned as a table so a panel that +#: retires can never shrink the payload (`/api/disks`' idiom) and task 05's page has a contract. +CERT_DOCUMENT_KEYS = {"root", "client", "agents"} +CERT_TIMESERIAL_KEYS = {"serial", "not_after"} +CERT_AGENT_KEYS = {"name", "configured", "serial", "not_after", "status", "bundle_rev", + "deployed_bundle_rev", "san"} + + +def cert_document(client: TestClient) -> dict[str, Any]: + """`GET /api/certs`, with the status asserted: a refusal here would pass a `.get()` chain.""" + response = client.get("/api/certs") + assert response.status_code == 200, response.text + body = response.json() + assert set(body) == CERT_DOCUMENT_KEYS, "the CA document grew or lost a top-level key" + for block in ("root", "client"): + assert set(body[block]) == CERT_TIMESERIAL_KEYS, block + assert int(body[block]["serial"]) > 0, "a serial is the decimal text of a positive int" + assert body[block]["not_after"] > time.time(), "a stored CA may not already be expired" + return body + + +def cert_agent_rows(client: TestClient, name: str | None = None) -> list[dict[str, Any]]: + """The agent rows, every one of them shaped, optionally just one host's.""" + rows = cert_document(client)["agents"] + for row in rows: + assert set(row) == CERT_AGENT_KEYS, sorted(row) + if name is None: + return rows + return [row for row in rows if row["name"] == name] + + +def cert_context(client: TestClient) -> Any: + """The app's `AppContext`. + + `TestClient.app` is typed as the ASGI callable it wraps, so the one cast lives here instead of + one `# type: ignore` at every assertion that needs the store or a job. + """ + return client.app.state.ctx # type: ignore[attr-defined] + + +def cert_meta(store: SampleStore) -> dict[str, str]: + """Every `certs.*` row, for the before/after comparisons that prove a route wrote nothing. + + Coalesced to `""` because the comparison is "did any byte of these five rows move", and an + absent row is the empty answer to that question rather than a second kind of nothing. + """ + return {key: store.get_meta(key) or "" for key in certs.META_KEYS} + + +def revoked_serials(bundle: str) -> list[str]: + """The denylist of a bundle, as the agent reads it: the `# REVOKED ` lines.""" + return [line.removeprefix(certs.REVOKED_LINE).strip() + for line in bundle.splitlines() if line.startswith(certs.REVOKED_LINE)] + + +#: One PEM block out of a bundle, by text — the same read task 02's agent does with stdlib. +_PEM_BLOCK = re.compile(r"-----BEGIN [A-Z0-9 ]+-----.*?-----END [A-Z0-9 ]+-----\n", re.DOTALL) + + +def bundle_certs(bundle: str) -> list[x509.Certificate]: + """The two certificates of a bundle, in the order the format writes them. + + `[0]` is the agent's own server certificate and `[1]` the root it chains to (`certs.py`'s + `_bundle_text` order: metadata lines, agent cert, agent key, root cert). Reading them out of the + served bytes rather than trusting the response's echo is what makes "did the rotation reach the + artifact?" a checked claim. + """ + blocks = [block for block in _PEM_BLOCK.findall(bundle) if "CERTIFICATE" in block] + assert len(blocks) == 2, f"a bundle is two certificates and a key, got {len(blocks)}" + return [x509.load_pem_x509_certificate(block.encode()) for block in blocks] + + +def bundle_of(client: TestClient, name: str) -> str: + """The served bundle, asserted to still be a bundle (2 certs, 1 key) before it is compared.""" + response = client.get(f"/api/certs/{name}/bundle") + assert response.status_code == 200, response.text + assert response.headers["content-type"].startswith("application/x-pem-file") + assert "no-store" in response.headers["cache-control"], "a private key may not be cached" + blocks = re.findall(r"-----BEGIN [A-Z0-9 ]+-----", response.text) + assert blocks.count("-----BEGIN CERTIFICATE-----") == 2, blocks + assert blocks.count("-----BEGIN PRIVATE KEY-----") == 1, blocks + return response.text + + +def stored_bundle(store: SampleStore, name: str) -> str: + """The bundle exactly as the `certs.agents` row holds it.""" + row: dict[str, Any] = json.loads(store.get_meta(certs.META_AGENTS) or "{}") + return str(row[name]["bundle"]) + + +def cert_events(client: TestClient) -> list[dict[str, Any]]: + return client.get("/api/events", params={"source": "certs"}).json()["events"] + + +def cert_hosts(count: int) -> tuple[Target, ...]: + return DEAD_TARGETS[:count] + + +@pytest.mark.parametrize("hosts", [0, 1, 3], ids=["none", "one", "three"]) +def test_the_cert_list_is_the_stored_table_for_zero_one_and_three_hosts( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, hosts: int) -> None: + """The list works with no fleet at all, and says *unissued* rather than omitting a host. + + The CA exists whether or not `HOST_TARGETS` does — a fresh database boots with a root and a + client certificate — so this is not the `/api/disks` "not configured" surface and does not 404. + A configured host with nothing issued still gets a row (invariant 3: a machine that is deployed + and not yet installed is not an absent machine), with `None` and not `0` where there is no + certificate (rule 2). + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(hosts), + host_interval=3600.0) as client: + rows = cert_agent_rows(client) + assert len(rows) == hosts, "one row per configured host, and no more" + assert [row["name"] for row in rows] == sorted(row["name"] for row in rows), ( + "one order in the payload and one in the table") + for row in rows: + assert row["configured"] is True + assert row["status"] == "unissued", "nothing has been signed for it yet" + assert row["serial"] is None and row["bundle_rev"] is None, "no invented zeros" + assert row["deployed_bundle_rev"] is None, "nobody has answered, so nobody reported" + assert row["san"] == [], "no names on record, not a certificate with no names" + + +def test_the_cert_list_carries_the_rev_the_agent_reported( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """`deployed_bundle_rev` is what the host said about itself, not what the CA hopes. + + Task 05's "stale bundle" sentence is a comparison of two numbers that come from two different + places: the row this dashboard issued and the `agent` block of the last `system` payload. Both + halves are pinned here, including the direction that matters — issued 2 against deployed 1 is + the operator's "install the new bundle" case, and it has to be visible rather than smoothed + over. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + assert client.post("/api/certs/deskwork/rotate").status_code == 200 + job = cert_context(client).hub.jobs["host:deskwork"] + assert cert_agent_rows(client, "deskwork")[0]["deployed_bundle_rev"] is None, ( + "a host that has never answered reports nothing") + job.state.payload = {"name": "deskwork", "agent": {"tls": True, "bundle_rev": 1}} + assert cert_agent_rows(client, "deskwork")[0]["deployed_bundle_rev"] == 1 + assert client.post("/api/certs/deskwork/rotate").json()["bundle_rev"] == 2 + row = cert_agent_rows(client, "deskwork")[0] + assert (row["bundle_rev"], row["deployed_bundle_rev"]) == (2, 1), "the stale pair, both" + # A token-mode agent publishes no `agent.tls` block at all, and that is `n/a`, not rev 0. + job.state.payload = {"name": "deskwork", "agent": {"version": "1.0.0"}} + assert cert_agent_rows(client, "deskwork")[0]["deployed_bundle_rev"] is None + + +def test_an_entry_for_a_host_that_left_the_config_is_listed_and_not_writable( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """A row issued for a machine that has gone is a key still sitting in the table. + + Listing it is the honest answer (`/certs` is where you learn you still hold one); refusing to + rotate or revoke it is the same invariant from the other side — `HOST_TARGETS` is what this + dashboard dials, and a name it no longer dials is not a name to install a bundle on. The row is + written straight to the store because that is the only way this state arises: a host removed + from the config, never a route that created it. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + certs.issue_agent_cert(store, certs.read_state(store), "old-host", ["10.9.8.7"]) + + rows = cert_agent_rows(client) + assert [row["name"] for row in rows] == ["deskwork", "old-host"] + ghost = rows[1] + assert ghost["configured"] is False, "listed, but this dashboard does not dial it" + assert ghost["status"] == certs.ACTIVE and ghost["bundle_rev"] == 1 + assert ghost["deployed_bundle_rev"] is None, "no job means no report, not rev 0" + assert client.post("/api/certs/old-host/rotate").status_code == 404 + assert client.post("/api/certs/old-host/revoke").status_code == 404 + assert cert_events(client) == [], "a refusal is not an audit line" + + +def test_a_cert_route_for_an_unknown_name_dials_nothing_and_writes_nothing( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The invariant-3 gate on the per-host routes, taken before anything can have a side effect. + + `not-installed` is refused in the first line of each route, so no client is built, no store row + is written, and no key is minted. All three are measured: the two network spies raise, `set_meta` + is watched for any `certs.*` write, and the five rows are compared byte for byte. A typo that + minted a certificate would be invisible on every status code and permanent in the table. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + calls: list[str] = [] + written: list[str] = [] + + async def spy_get(self: Any, url: Any, **kwargs: Any) -> Any: + calls.append(f"GET {url}") + raise AssertionError(f"an unconfigured cert name dialled {url}") + + def spy_fetch(*args: Any, **kwargs: Any) -> Any: + calls.append("fetch_snapshot") + raise AssertionError("an unconfigured cert name fetched an agent snapshot") + + real_set_meta = SampleStore.set_meta + + def spy_set_meta(self: SampleStore, key: str, value: str) -> None: + written.append(key) + real_set_meta(self, key, value) + + monkeypatch.setattr(httpx.AsyncClient, "get", spy_get) + monkeypatch.setattr("dashboards.collectors.fetch_snapshot", spy_fetch) + monkeypatch.setattr(SampleStore, "set_meta", spy_set_meta) + before = cert_meta(store) + + for response in (client.get("/api/certs/not-configured/bundle"), + client.post("/api/certs/not-configured/rotate"), + client.post("/api/certs/not-configured/revoke")): + assert response.status_code == 404, response.text + body = response.json() + assert body["configured"] is False + assert "HOST_TARGETS" in body["detail"], "the sentence names the fix" + + assert calls == [], "the gate is in front of the network, not behind it" + assert [key for key in written if key.startswith("certs.")] == [], "a write happened" + assert cert_meta(store) == before, "the stored CA and agent table did not move" + assert cert_events(client) == [], "and no audit line was written for a name that is not here" + # …and the same route still works for the host that *is* configured, so the refusal above + # is the gate and not a broken router. + assert client.post("/api/certs/deskwork/rotate").status_code == 200 + + +def test_rotating_a_host_mints_the_bundle_it_asks_for( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Rotate is also *issue*: first call is `bundle_rev` 1, and the SAN is the config's address. + + The SAN assertion is the one a rollout fails on silently — a certificate that does not name the + address the dashboard dials is a handshake error and a red source, not a warning — so it is read + out of the served bundle's own certificate rather than out of the response's echo. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + body = client.post("/api/certs/deskwork/rotate").json() + assert body["ok"] is True and body["status"] == certs.ACTIVE + assert body["bundle_rev"] == 1 and body["serial"] and body["not_after"] > time.time() + assert body["san"] == ["127.0.0.1", "deskwork"], "the dialled address and the name" + + cert = bundle_certs(bundle_of(client, "deskwork"))[0] + names = cert.extensions.get_extension_for_class( + x509.SubjectAlternativeName).value.get_values_for_type(x509.IPAddress) + assert names == [ipaddress.ip_address("127.0.0.1")], "an IP SAN, or an IP dial fails" + assert cert.subject == x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "deskwork")]) + + events = cert_events(client) + assert len(events) == 1 and events[0]["kind"] == "cert-rotated" + assert events[0]["host"] == "deskwork" and events[0]["message"] == ( + "agent certificate rotated for deskwork") + assert events[0]["data"] == {"bundle_rev": 1} + + second = client.post("/api/certs/deskwork/rotate").json() + assert second["bundle_rev"] == 2 and second["serial"] != body["serial"] + # The new identity is not on the denylist: rotation replaces an identity, it does not strike + # one out, and a bundle that listed its own serial would refuse the client it ships. + bundle = bundle_of(client, "deskwork") + assert revoked_serials(bundle) == [], revoked_serials(bundle) + assert second["serial"] not in bundle.splitlines(), "no denylist line for the live serial" + assert stored_bundle(store, "deskwork") == bundle, "served bytes are the stored bytes" + + +def test_revoking_a_host_marks_the_table_and_the_next_poll_refuses_without_a_dial( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """The API half of the revocation chain: this route is what task 03's refusal reads. + + The control is the sentence from *before* the click (`unreachable`, which is what a dial to a + dead port says), and after it the poll has to say `agent certificate revoked` with a spy on + `httpx.AsyncClient.get` proving no socket was opened. The stored denylist is the other half: an + agent holding a bundle that omits the serial keeps accepting the client it was given. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=0.05, disks_interval=0.05) as client: + store: SampleStore = cert_context(client).store + wait_for_source_error(client, "host:deskwork", "unreachable") + serial = client.post("/api/certs/deskwork/rotate").json()["serial"] + + body = client.post("/api/certs/deskwork/revoke").json() + assert body["ok"] is True and body["status"] == certs.REVOKED + assert body["serial"] == serial, "revocation marks the identity it has, it does not replace" + assert body["hint"] and "next poll" in body["hint"] + + calls: list[str] = [] + + async def spy_get(self: Any, url: Any, **kwargs: Any) -> Any: + calls.append(str(url)) + raise AssertionError(f"a revoked agent was dialled at {url}") + + monkeypatch.setattr(httpx.AsyncClient, "get", spy_get) + for key in ("host:deskwork", "disks:deskwork"): + error = wait_for_source_error(client, key, "agent certificate revoked") + assert "unreachable" not in error + assert calls == [] + + assert revoked_serials(bundle_of(client, "deskwork")) == [serial] + events = cert_events(client) + assert [row["kind"] for row in events] == ["cert-revoked", "cert-rotated"] + assert events[0]["message"] == "agent certificate revoked for deskwork" + assert certs.agent_status(store, "deskwork") == certs.REVOKED, ( + "the collector's own read agrees with the row the route wrote") + + +def test_root_rotation_reissues_every_host_and_answers_restart_required( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """One action re-arms the fleet — and says the part it cannot do from inside the process. + + `restart_required` is not decoration: the CA and the client pair are startup material, so the + table is new immediately while the sockets keep the old identity. The page (task 05) prints + that sentence because the API answered it, and the alternative is an operator who rotated a + root and watches a fleet that still chains to the previous one. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(3), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + for target in DEAD_TARGETS: + assert client.post(f"/api/certs/{target.name}/rotate").status_code == 200 + before = cert_document(client) + rows_before = {row["name"]: row for row in before["agents"]} + root_pem_before = store.get_meta(certs.META_ROOT_CERT) or "" + key_before = store.get_meta(certs.META_ROOT_KEY) or "" + assert root_pem_before and key_before, "both halves of the CA exist before the rotation" + + body = client.post("/api/certs/root/rotate").json() + assert body["ok"] is True and body["restart_required"] is True + assert body["agents"] == 3 + assert body["root"]["serial"] != before["root"]["serial"], "a fresh root" + + after = cert_document(client) + assert after["root"] == body["root"], "the list reports what the rotation wrote" + assert after["client"]["serial"] != before["client"]["serial"], "and a fresh client leaf" + for row in after["agents"]: + was = rows_before[row["name"]] + assert row["bundle_rev"] == was["bundle_rev"] + 1, "every host reissued, rev bumped" + assert row["serial"] != was["serial"] and row["status"] == certs.ACTIVE + bundle = bundle_of(client, row["name"]) + leaf, root = bundle_certs(bundle) + assert leaf.serial_number == int(row["serial"]), "the served bundle is the listed one" + assert root.serial_number == int(after["root"]["serial"]), ( + "and it chains to the authority the list now reports") + assert revoked_serials(bundle) == [], "the denylist resets with the generation" + assert store.get_meta(certs.META_ROOT_KEY) != key_before + assert store.get_meta(certs.META_ROOT_CERT) != root_pem_before + assert root_pem_before not in "".join(cert_meta(store).values()), ( + "the old root is gone") + events = cert_events(client) + assert events[0]["kind"] == "cert-root-rotated" + assert events[0]["message"] == ( + "root certificate rotated — every agent certificate reissued, restart required") + assert events[0]["data"] == {"agents": 3} + + +def test_the_bundle_route_names_a_file_and_changes_when_the_identity_does( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Byte-identical to the stored row, an `attachment`, and different bytes after a reissue. + + This is the installer's input (task 06), so the claim is about bytes and not about a bundle + that merely re-verifies: `install-agent.sh` is handed the file and compares nothing else. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + missing = client.get("/api/certs/deskwork/bundle") + assert missing.status_code == 404, "nothing issued yet, so nothing to download" + body = missing.json() + assert body["configured"] is True and body["issued"] is False, "not the same 404" + assert "rotate" in body["detail"], "and the answer names the way to get one" + + first = client.post("/api/certs/deskwork/rotate").json()["serial"] + response = client.get("/api/certs/deskwork/bundle") + assert response.headers["content-disposition"] == ( + 'attachment; filename="agent-deskwork.bundle"') + assert response.text == stored_bundle(store, "deskwork") + + client.post("/api/certs/deskwork/rotate") + second = client.post("/api/certs/deskwork/rotate").json()["serial"] + assert second != first + updated = client.get("/api/certs/deskwork/bundle") + assert updated.text != response.text, "a reissue changes the served bytes" + assert updated.text == stored_bundle(store, "deskwork") + + +def test_no_certificate_material_leaves_meta_through_the_cert_surface( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """invariant 6 on the API that owns the keys: serials and dates go out, PEMs do not. + + Every write here is exercised first, so the grep is over a table that has been rotated and + revoked rather than a pristine one, and it covers both the payloads a browser reads and the + stored event rows those payloads are built from — an event row is kept forever, which is what + makes a PEM in a message permanent rather than transient. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(2), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + client.post("/api/certs/deskwork/rotate") + client.post("/api/certs/driveripper/rotate") + client.post("/api/certs/driveripper/revoke") + client.post("/api/certs/root/rotate") + secrets = [value for value in cert_meta(store).values() if value] + assert any("PRIVATE KEY" in value for value in secrets), "the table really does hold keys" + + for path in ("/api/certs", "/api/health", "/api/events", "/api/summary", "/api/config"): + text = client.get(path).text + for marker in ("BEGIN CERTIFICATE", "PRIVATE KEY", "BUNDLE-REV"): + assert marker not in text, f"{path} served {marker}" + assert client.get("/api/health").json()["db"], "health still answers, keys or no keys" + for event in store.recent_events(50, source="certs"): + assert "BEGIN" not in event.message and "PRIVATE KEY" not in event.message + assert not any(secret in json.dumps(event.as_json()) for secret in secrets) + + +def test_a_host_named_root_cannot_be_rotated_past_the_certificate_authority( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """`/api/certs/root/rotate` and `/api/certs/{name}/rotate` are one URL for a host called root. + + The CA route is registered first, so it wins — which means the wrong button would rotate the + whole authority instead of one host's leaf. Refusing with the reason is the only honest answer: + the alternative is a fleet-wide outage behind a click that looked like routine maintenance. + """ + with build_client(tmp_path, monkeypatch, + host_targets=(Target(name="root", url="https://127.0.0.1:1"),), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + before = cert_document(client)["root"]["serial"] + refused = client.post("/api/certs/root/rotate") + assert refused.status_code == 409, refused.text + assert "rename" in refused.json()["detail"], "the sentence says what to do" + assert cert_document(client)["root"]["serial"] == before, "and the CA did not move" + assert cert_events(client) == [] + assert cert_meta(store)[certs.META_AGENTS] == "{}", "nothing was issued for the name either" + + +def test_a_read_of_the_cert_surface_issues_nothing_and_writes_nothing( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """A GET that mints a certificate is the surprise this route must never be. + + `certs.ensure_certs` is the bootstrapping read and `certs.read_state` is the request-time one; + making `ensure_certs` explode is what proves the difference rather than asserting it in prose, + and the meta rows are compared byte for byte across three reads. + """ + with build_client(tmp_path, monkeypatch, host_targets=cert_hosts(1), + host_interval=3600.0) as client: + store: SampleStore = cert_context(client).store + before = cert_meta(store) + + def explode(*args: Any, **kwargs: Any) -> Any: + raise AssertionError("a GET tried to bootstrap the certificate authority") + + monkeypatch.setattr(certs, "ensure_certs", explode) + assert cert_agent_rows(client) + assert cert_document(client)["root"]["serial"] == str( + x509.load_pem_x509_certificate( + str(before[certs.META_ROOT_CERT]).encode()).serial_number) + # A bundle that has not been issued is still a 404 rather than a mint, on a read too. + assert client.get("/api/certs/deskwork/bundle").status_code == 404 + assert cert_meta(store) == before, "three reads, no write" + assert cert_events(client) == [] + + # --------------------------------------------------------------------------- # # phase 14 task 04: the endpoint budgets # diff --git a/tests/test_auth_api.py b/tests/test_auth_api.py index 8ea7d89..4af037e 100644 --- a/tests/test_auth_api.py +++ b/tests/test_auth_api.py @@ -42,7 +42,7 @@ from auth_contract import FAST_PBKDF2 from dashboards import auth from dashboards.api import MIN_PASSWORD_LENGTH, AppContext from dashboards.app import PUBLIC_PATHS, PUBLIC_PREFIXES, create_app, is_public -from dashboards.config import Settings +from dashboards.config import Settings, Target from dashboards.poller import Hub from dashboards.sources import nut from dashboards.store import SampleStore @@ -58,11 +58,17 @@ PROTECTED_API = ( "/api/system/snapshot", "/api/system/history", "/api/disks/snapshot", "/api/disks/history", "/api/llm/snapshot", "/api/llm/history", "/api/ups/snapshot", "/api/ups/history", "/api/ups/commands", "/api/auth/status", + # Phase 15: the certificate authority's own surface. `/api/certs//bundle` is every + # agent's private key, so "it needs a session" is the whole point of listing it here. + "/api/certs", "/api/certs/{name}/bundle", "/api/certs/{name}/rotate", + "/api/certs/{name}/revoke", "/api/certs/root/rotate", ) -#: Every HTML page behind the gate. `/settings` is a **page** and not one of the four -#: dashboards (it carries no `aria-current` tab), which is exactly why it is in this list: the -#: one thing that must never regress is a new route answering anonymously. -PROTECTED_PAGES = ("/", "/system", "/disks", "/llm", "/ups", "/settings") +#: Every HTML page behind the gate. `/settings` and `/certs` are **pages** and not dashboards +#: (neither carries an `aria-current` tab), which is exactly why they are in this list: the one +#: thing that must never regress is a new route answering anonymously. `/certs` is the sharper +#: case of the two — an anonymous caller reaching it would see the CA's serials and every host +#: the dashboard dials. +PROTECTED_PAGES = ("/", "/system", "/disks", "/llm", "/ups", "/settings", "/certs") #: The one write this app has, with the fields the route needs (the UPS is unconfigured, so a @@ -1256,6 +1262,105 @@ def test_the_danger_word_is_still_what_cuts_power_after_the_gate(nut_app: TestCl assert WireRecorder.sent == [("apc", "driver.killpower")] +# --------------------------------------------------------------------------- # +# The certificate authority's writes (phase 15 task 04) +# --------------------------------------------------------------------------- # + + +@pytest.fixture +def cert_app(tmp_path: Path) -> Iterator[TestClient]: + """A signed-in client on an app with **one configured agent** and no live host. + + What these routes *do* is `tests/test_api.py`'s business; this file asks only whether the gate + stops a write before it reaches `certs.py`. The target is a dead loopback port because a + certificate route is never allowed to dial anything (invariant 1) — the stored agent table is + the wire here, which is what the assertions below read. + """ + db = tmp_path / "certs-gate.sqlite3" + store = SampleStore(str(db)) + try: + app = create_app(Settings(db_path=str(db), admin_password=ADMIN_PASSWORD, + pbkdf2_iterations=FAST_PBKDF2, + host_targets=(Target(name="deskwork", + url="https://127.0.0.1:1"),), + host_interval=3600.0, disks_interval=3600.0), store) + with TestClient(app, follow_redirects=False) as client: + sign_in(client) + yield client + finally: + store.close() + + +def test_anonymous_calls_see_no_certificate_list_and_no_bundle(anon: TestClient) -> None: + """The named, explicit version of the route-walk proof. + + `test_every_route_that_is_not_public_requires_a_session` walks the served table and cannot miss + a route; this test is here so the *bundle* route — every agent's private key, behind a login — + has one assertion that says what it is about, and so a redirect that would leak the existence of + a host name into a `Location` header is caught as deliberately as the 401 is. + """ + listing = anon.get("/api/certs", headers=JSON_ACCEPT) + assert listing.status_code == 401 + assert listing.json() == {"ok": False, "detail": "sign in required", "login": "/login"} + assert "no-store" in listing.headers["cache-control"] + + bundle = anon.get("/api/certs/deskwork/bundle", headers=JSON_ACCEPT) + assert bundle.status_code == 401, "a bundle is a private key with a filename on it" + assert "location" not in bundle.headers + + for write in (anon.post("/api/certs/deskwork/rotate"), + anon.post("/api/certs/deskwork/revoke"), + anon.post("/api/certs/root/rotate")): + assert write.status_code == 401, write.url + assert write.json() == {"ok": False, "detail": "sign in required", "login": "/login"} + assert "no-store" in write.headers["cache-control"] + + +def test_a_cert_write_without_the_csrf_token_changes_no_stored_row( + cert_app: TestClient) -> None: + """The UPS wire test's shape on the CA's own writes, with the row as the wire. + + Proven in both directions on purpose: a gate that refused every write would pass a test that + only asserted the refusal, and a route that wrote before checking would pass one that only + asserted the success. The stored agent table is what a rotation changes, so it is what is read. + """ + def issued() -> dict[str, Any]: + rows: list[dict[str, Any]] = cert_app.get("/api/certs").json()["agents"] + return rows[0] + + assert issued()["status"] == "unissued" and issued()["bundle_rev"] is None + + csrf = str(cert_app.headers.get(auth.CSRF_HEADER)) + del cert_app.headers[auth.CSRF_HEADER] + refused = cert_app.post("/api/certs/deskwork/rotate") + assert refused.status_code == 403 + assert "csrf" in refused.json()["detail"].lower() + assert issued()["status"] == "unissued", "a key was minted by a write the gate refused" + + cert_app.headers[auth.CSRF_HEADER] = csrf + accepted = cert_app.post("/api/certs/deskwork/rotate") + assert accepted.status_code == 200, accepted.text + assert accepted.json()["bundle_rev"] == 1 + assert issued()["bundle_rev"] == 1 + + +def test_a_cross_origin_cert_write_is_refused_before_the_authority_is_touched( + cert_app: TestClient) -> None: + """The origin half of the gate on a route whose effect is durable and fleet-wide.""" + cross_site = cert_app.post("/api/certs/deskwork/revoke", + headers={"origin": "https://elsewhere.example"}) + assert cross_site.status_code == 403 + assert "cross-origin" in cross_site.json()["detail"] + + stalled = cert_app.post("/api/certs/root/rotate", headers={"sec-fetch-site": "cross-site"}) + assert stalled.status_code == 403 + assert cert_app.get("/api/certs").json()["agents"][0]["status"] == "unissued", ( + "nothing moved: not the host's identity and not the root") + + # …and the same request with nothing forged is an ordinary gated write. + assert cert_app.post("/api/certs/deskwork/revoke").status_code == 200 + + def test_a_cross_origin_write_is_refused_even_with_a_session_and_a_csrf_token( authed: TestClient) -> None: response = authed.post("/api/ups/command", json=WRITE, @@ -1493,7 +1598,8 @@ def test_the_served_route_table_is_the_surface_the_gate_describes(anon: TestClie pages = sorted(str(route.path) for route in served_routes(anon) if isinstance(getattr(route, "path", None), str) and not str(route.path).startswith(("/api", "/static"))) - assert pages == ["/", "/disks", "/llm", "/login", "/settings", "/system", "/ups"] + assert pages == ["/", "/certs", "/disks", "/llm", "/login", "/settings", "/system", + "/ups"] for path in pages: response = anon.get(path, headers=BROWSER_HEADERS) if path == "/login": diff --git a/tests/test_certs.py b/tests/test_certs.py new file mode 100644 index 0000000..eb55c36 --- /dev/null +++ b/tests/test_certs.py @@ -0,0 +1,1094 @@ +"""`src/dashboards/certs.py` — the dashboard's CA, its client cert, and every agent's, tested. + +The lifecycle here is the part of phase 15 that cannot be checked in a browser: a root CA that +regenerates on restart silently untrusts every installed agent bundle; a bundle carrying the root +*key* turns one stolen agent install into a forged fleet; a revocation that lands on one host's +bundle but not the others is a revocation that did not happen. So this file is mostly claims about +**stored bytes**, and the store underneath them is a dict. + +That is the `tests/test_auth_bootstrap.py` idiom, and it is why `certs.py` talks to a three-method +`MetaStore` protocol: the security-relevant branch has no SQLite in it, so the decisions are +asserted directly. Three tests at the end put something real underneath instead — the real +`SampleStore` for durability, a real `ssl` handshake for usability, and the real `create_app()` +lifespan for the fail-closed startup — because "it parses" is not the same claim as "an agent +accepts it". + +Nothing sleeps and nothing reads the wall clock: `now` is an argument everywhere, so a 10-year root +and a 1-year leaf are asserted as numbers. +""" + +from __future__ import annotations + +import ast +import json +import socket +import ssl +import tempfile +import threading +from pathlib import Path +from typing import Any + +import pytest +from cryptography import x509 +from cryptography.exceptions import InvalidSignature +from cryptography.hazmat.primitives.asymmetric import rsa +from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey +from cryptography.hazmat.primitives.asymmetric.types import PublicKeyTypes +from cryptography.hazmat.primitives.serialization import ( + Encoding, + NoEncryption, + PrivateFormat, + PublicFormat, + load_pem_private_key, +) +from cryptography.x509.oid import NameOID +from fastapi.testclient import TestClient + +from auth_contract import ADMIN_PASSWORD, FAST_PBKDF2 +from dashboards import auth, certs +from dashboards.app import create_app +from dashboards.config import Settings +from dashboards.store import SampleStore + +CERTS_SOURCE = Path(__file__).resolve().parents[1] / "src" / "dashboards" / "certs.py" + +#: One fixed instant for every lifetime assertion. `certs.py` never reads a clock, so an expiry is +#: this number plus the documented lifetime — no sleeps, no flaky window. +NOW = 1_700_000_000.0 + +CLIENT_CERT_FILE = "client-cert.pem" +CLIENT_KEY_FILE = "client-key.pem" +CA_FILE = "root-ca.pem" + + +class FakeMeta: + """The three-method shape `certs.py` asks for, with no database in it. + + Exactly how the auth bootstrap is tested: the lifecycle is a function over a key/value store, + so the store is a dict and the assertions are about the rows. + """ + + def __init__(self, rows: dict[str, str] | None = None) -> None: + self.rows: dict[str, str] = dict(rows or {}) + + def get_meta(self, key: str) -> str | None: + return self.rows.get(key) + + def set_meta(self, key: str, value: str) -> None: + self.rows[key] = value + + def delete_meta(self, key: str) -> int: + return 1 if self.rows.pop(key, None) is not None else 0 + + +def boot(*, now: float = NOW, rows: dict[str, str] | None = None + ) -> tuple[FakeMeta, certs.CertsState]: + """A store and the state one process start produced against it.""" + store = FakeMeta(rows) + return store, certs.ensure_certs(store, now=now) + + +def issue(store: FakeMeta, state: certs.CertsState, name: str = "deskwork", + san: list[str] | None = None) -> certs.CertsState: + return certs.issue_agent_cert(store, state, name, san or [], now=NOW) + + +def rotate(store: FakeMeta, state: certs.CertsState, name: str = "deskwork", + san: list[str] | None = None) -> certs.CertsState: + return certs.reissue_agent_cert(store, state, name, san or [], now=NOW) + + +def revoke(store: FakeMeta, state: certs.CertsState, name: str) -> certs.CertsState: + return certs.revoke(store, state, name, now=NOW) + + +def root_of(state: certs.CertsState) -> x509.Certificate: + return cert_of(state.root_cert) + + +# --- reading the artifacts back, independently of the module -------------------------------- # + + +def blocks(text: str) -> list[str]: + """Every PEM block in `text`, in order. + + Reimplemented with a split rather than by calling the module's parser: a test that reads a + bundle with the code that wrote it cannot notice the format changing, and task 02's agent has + to parse it with its own stdlib code anyway. + """ + out: list[str] = [] + for part in text.split("-----BEGIN ")[1:]: + label, _, rest = part.partition("-----\n") + body, _, _tail = rest.partition("-----END ") + out.append(f"-----BEGIN {label}-----\n{body}-----END {label}-----\n") + return out + + +def cert_of(text: str) -> x509.Certificate: + return x509.load_pem_x509_certificate(text.encode("ascii")) + + +def key_of(text: str) -> EllipticCurvePrivateKey: + key = load_pem_private_key(text.encode("ascii"), password=None) + assert isinstance(key, EllipticCurvePrivateKey), "this module issues P-256 keys" + return key + + +def public_id(public_key: PublicKeyTypes) -> bytes: + """A public key's identity as its SPKI encoding. + + Compared instead of `public_numbers()` because that attribute exists per algorithm and + `certificate.public_key()` is typed as the union of all of them; the encoding is the same + claim ("same key") and needs no cast. + """ + return public_key.public_bytes(Encoding.PEM, PublicFormat.SubjectPublicKeyInfo) + + +def entry_of(state: certs.CertsState, name: str) -> certs.AgentCert: + """The entry for one host, asserted present — a missing row is the failure, not a `None`.""" + entry = state.agent(name) + assert entry is not None, f"no entry stored for {name}" + return entry + + +def bundle_pems(bundle: str) -> dict[str, str]: + """A bundle as `{cert, key, root}` — the agent's view of the same three blocks.""" + found = blocks(bundle) + certs_found = [b for b in found if b.startswith("-----BEGIN CERTIFICATE")] + keys = [b for b in found if not b.startswith("-----BEGIN CERTIFICATE")] + assert len(certs_found) == 2 and len(keys) == 1, f"unexpected bundle shape: {found}" + return {"cert": certs_found[0], "key": keys[0], "root": certs_found[1]} + + +def revoked_lines(bundle: str) -> list[int]: + """The `# REVOKED` denylist, as serials, in the order the bundle carries them.""" + return [int(line.removeprefix(certs.REVOKED_LINE)) + for line in bundle.splitlines() if line.startswith(certs.REVOKED_LINE)] + + +def rev_of(bundle: str) -> int: + lines = [line for line in bundle.splitlines() if line.startswith(certs.BUNDLE_REV_LINE)] + assert len(lines) == 1, "exactly one revision line" + return int(lines[0].removeprefix(certs.BUNDLE_REV_LINE)) + + +def san_names(cert: x509.Certificate) -> list[str]: + return [str(item.value) + for item in cert.extensions.get_extension_for_class(x509.SubjectAlternativeName).value] + + +def eku_of(cert: x509.Certificate) -> list[str]: + extension = cert.extensions.get_extension_for_class(x509.ExtendedKeyUsage).value + return [str(item.dotted_string) for item in extension] + + +def cn(cert: x509.Certificate) -> str: + return str(cert.subject.get_attributes_for_oid(NameOID.COMMON_NAME)[0].value) + + +# --- the module's own hygiene ---------------------------------------------------------------- # + + +def test_the_module_owns_five_meta_keys_and_names_them_all_in_the_reset() -> None: + """The rows an operator has to be able to type, and the message that types them for them.""" + assert certs.META_KEYS == ("certs.root.key", "certs.root.cert", "certs.client.key", + "certs.client.cert", "certs.agents") + for key in certs.META_KEYS: + assert key in certs.CORRUPT_STORE_MESSAGE + assert "DELETE FROM meta" in certs.CORRUPT_STORE_MESSAGE + assert "sqlite3" in certs.CORRUPT_STORE_MESSAGE + + +def _tree() -> ast.Module: + return ast.parse(CERTS_SOURCE.read_text()) + + +def test_the_module_imports_nothing_from_the_package_and_nothing_logs() -> None: + """The `auth.py` rule: standalone, testable with a dict, with no path from a key to a log line. + + A private key is a credential the same way a password is, and this app turns log lines and + exception messages into things a browser can read (`AGENTS.md` invariant 6). Keeping `logging` + out of the file entirely is the version of that rule a test can actually hold. + """ + names: set[str] = set() + for node in ast.walk(_tree()): + if isinstance(node, ast.Import): + names.update(alias.name.split(".")[0] for alias in node.names) + elif isinstance(node, ast.ImportFrom): + assert node.level == 0, "no relative import: this module imports nothing in the package" + if node.module: + names.add(node.module.split(".")[0]) + assert "logging" not in names + assert "cryptography" in names, "the one third-party dependency this phase adds" + + +def test_the_only_raise_is_the_fixed_text_refusal() -> None: + """Every raise site is `RuntimeError(CORRUPT_STORE_MESSAGE)`: no f-string, no interpolated value. + + The row that failed to parse may be a private key, and a message quoting even a prefix of it + would go into the startup output and wherever the app prints failures. Enumerating the raise + sites is what makes the *next* one somebody adds a test failure instead of a leak. + """ + raised = [node for node in ast.walk(_tree()) if isinstance(node, ast.Raise)] + assert raised, "the fail-closed refusal exists" + for node in raised: + assert isinstance(node.exc, ast.Call) and isinstance(node.exc.func, ast.Name) + assert node.exc.func.id == "RuntimeError" + assert len(node.exc.args) == 1 + message = node.exc.args[0] + assert isinstance(message, ast.Name) and message.id == "CORRUPT_STORE_MESSAGE", ( + "the refusal is the module-level literal, never a built string") + assert node.cause is None or isinstance(node.cause, ast.Name), "no interpolated context" + assert not any(isinstance(child, ast.JoinedStr) for child in ast.walk(node)), ( + "a raise may not interpolate anything") + # ...and the literal behind that name is a literal, not something assembled at import time. + assigned = [stmt for stmt in _tree().body if isinstance(stmt, ast.Assign) + and any(isinstance(t, ast.Name) and t.id == "CORRUPT_STORE_MESSAGE" + for t in stmt.targets)] + assert len(assigned) == 1 and isinstance(assigned[0].value, ast.Constant) + assert isinstance(assigned[0].value.value, str) + + +# --- first boot: generate once, then never again ---------------------------------------------- # + + +def test_the_first_boot_persists_exactly_the_five_cert_rows() -> None: + store, state = boot() + + assert sorted(store.rows) == sorted(certs.META_KEYS) + assert state.root_cert == store.rows[certs.META_ROOT_CERT] + assert state.client_key == store.rows[certs.META_CLIENT_KEY] + assert state.agents == () + assert json.loads(store.rows[certs.META_AGENTS]) == {}, "the empty fleet is a row too" + # PEM text, not DER and not a path: `sqlite3 … SELECT value` shows a certificate. + assert store.rows[certs.META_ROOT_CERT].startswith("-----BEGIN CERTIFICATE-----") + assert store.rows[certs.META_CLIENT_KEY].startswith("-----BEGIN PRIVATE KEY-----") + + +def test_a_second_boot_changes_no_byte_of_any_stored_certificate() -> None: + """The completion criterion for the task, and the one that makes a redeploy survivable. + + A root CA regenerated per start would untrust every agent bundle in the fleet, so this is a + read that only fills gaps. The later `now` is part of the test: regenerating would also move + the expiry, and a state that merely *looks* current is worse than one that refuses. + """ + store, first = boot() + before = dict(store.rows) + + second = certs.ensure_certs(store, now=NOW + 3_600) + + assert store.rows == before, "no row moved, including the root key" + assert (second.root_cert, second.root_key) == (first.root_cert, first.root_key) + assert (second.client_cert, second.client_key) == (first.client_cert, first.client_key) + assert (second.root_serial, second.client_serial) == (first.root_serial, first.client_serial) + assert second.root_not_after == first.root_not_after + + +def test_a_missing_client_pair_is_generated_under_the_stored_root() -> None: + """A crash between two `set_meta` calls leaves half a credential; the fix is the other half.""" + store, first = boot() + del store.rows[certs.META_CLIENT_KEY] + del store.rows[certs.META_CLIENT_CERT] + + second = certs.ensure_certs(store, now=NOW) + + assert second.root_cert == first.root_cert, "the root is never regenerated to match a leaf" + assert second.client_cert != first.client_cert + cert_of(second.client_cert).verify_directly_issued_by(root_of(second)) + + +def test_a_new_root_takes_the_client_certificate_with_it() -> None: + """Keeping a leaf signed by a dead root would hand the process a credential nothing trusts.""" + store, first = boot() + del store.rows[certs.META_ROOT_KEY] + del store.rows[certs.META_ROOT_CERT] + + second = certs.ensure_certs(store, now=NOW) + + assert second.root_serial != first.root_serial + assert second.client_cert != first.client_cert + cert_of(second.client_cert).verify_directly_issued_by(root_of(second)) + with pytest.raises(InvalidSignature): + cert_of(first.client_cert).verify_directly_issued_by(root_of(second)) + + +def test_a_row_of_whitespace_is_an_absent_row() -> None: + store, state = boot(rows={certs.META_ROOT_CERT: " \n "}) + assert state.root_cert.startswith("-----BEGIN CERTIFICATE-----") + assert cert_of(state.root_cert).serial_number == state.root_serial + assert store.rows[certs.META_ROOT_CERT] == state.root_cert + + +# --- the shapes ------------------------------------------------------------------------------- # + + +def test_the_root_is_self_signed_and_can_only_sign() -> None: + _store, state = boot() + root = root_of(state) + + assert root.subject == root.issuer + assert cn(root) == certs.ROOT_CN + assert root.extensions.get_extension_for_class(x509.BasicConstraints).value.ca is True + usage = root.extensions.get_extension_for_class(x509.KeyUsage).value + assert usage.key_cert_sign is True and usage.crl_sign is True + assert usage.digital_signature is False, "a CA that signs data is a CA being used as a leaf" + assert root.not_valid_after_utc.timestamp() == NOW + certs.ROOT_LIFETIME.total_seconds() + assert certs.ROOT_LIFETIME.days == 3650 and certs.LEAF_LIFETIME.days == 365 + assert state.root_not_after == root.not_valid_after_utc.timestamp() + + +def test_every_certificate_carries_the_identifiers_a_strict_verifier_requires() -> None: + """AKI and SKI are not decoration: OpenSSL refuses a chain without them in strict mode. + + Python's `ssl.create_default_context()` (which is what the dashboard's own https dial builds, + and what `curl --cacert`-style clients reach for) turns `X509_STRICT` on, and a strict + verification of a leaf with no AuthorityKeyIdentifier fails with + `CERTIFICATE_VERIFY_FAILED: Missing Authority Key Identifier` — every TLS agent would read + red, with an error that names neither the extension nor the module that left it out. The + agent's own server-side check does not use strict mode, which is why the TLS tests pass + without this and why it is pinned here instead. + """ + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + entry = state.agent("deskwork") + assert entry is not None + root = root_of(state) + + root_ski = root.extensions.get_extension_for_class(x509.SubjectKeyIdentifier).value + assert root_ski.digest, "a CA whose leaves cannot name it cannot be chained to strictly" + + for label, cert in (("client", cert_of(state.client_cert)), + ("agent", cert_of(bundle_pems(entry.bundle)["cert"]))): + identifier = cert.extensions.get_extension_for_class(x509.SubjectKeyIdentifier) + authority = cert.extensions.get_extension_for_class(x509.AuthorityKeyIdentifier) + assert authority.value.key_identifier == root_ski.digest, f"{label}: AKI must name our root" + assert not identifier.critical and not authority.critical, ( + "RFC 5280: the identifiers are non-critical, so a verifier that ignores them " + "still validates the signature") + + # and a strict client context accepts what that produced, which is the claim the dial makes + context = ssl.create_default_context(cadata=state.root_cert) + assert context.verify_flags & ssl.VERIFY_X509_STRICT, "this is the mode being satisfied" + + +def test_the_client_certificate_chains_to_the_root_and_is_a_client_identity() -> None: + _store, state = boot() + client = cert_of(state.client_cert) + assert client.verify_directly_issued_by(root_of(state)) is None + + assert cn(client) == certs.CLIENT_CN + assert eku_of(client) == [x509.oid.ExtendedKeyUsageOID.CLIENT_AUTH.dotted_string], ( + "an agent verifies a client certificate against the clientAuth purpose") + assert client.extensions.get_extension_for_class(x509.BasicConstraints).value.ca is False + assert client.not_valid_after_utc.timestamp() == NOW + certs.LEAF_LIFETIME.total_seconds() + assert san_names(client) == [certs.CLIENT_CN] + assert state.client_serial == client.serial_number + assert state.client_not_after == client.not_valid_after_utc.timestamp() + + +def test_an_agent_certificate_chains_to_the_root_names_the_host_and_holds_the_san() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51", "deskwork.reeselink.com"]) + entry = state.agent("deskwork") + assert entry is not None + + agent = cert_of(bundle_pems(entry.bundle)["cert"]) + agent.verify_directly_issued_by(root_of(state)) + assert cn(agent) == "deskwork" + assert san_names(agent) == ["10.1.0.51", "deskwork.reeselink.com", "deskwork"] + assert eku_of(agent) == [x509.oid.ExtendedKeyUsageOID.SERVER_AUTH.dotted_string] + assert agent.serial_number == entry.serial + assert agent.not_valid_after_utc.timestamp() == entry.not_after == ( + NOW + certs.LEAF_LIFETIME.total_seconds()) + # The key in the bundle is the key the certificate was issued for. An agent that loads a + # mismatched pair fails the handshake with an error about nothing it can see. + assert public_id(key_of(bundle_pems(entry.bundle)["key"]).public_key()) == public_id(agent.public_key()) + + +def test_the_san_holds_ip_addresses_as_ips_and_names_as_dns() -> None: + """`10.1.0.126` has to be an iPAddress SAN or hostname verification of an IP dial fails.""" + store, state = boot() + state = issue(store, state, "gateway", ["10.1.1.27"]) + entry = state.agent("gateway") + assert entry is not None + general = cert_of(bundle_pems(entry.bundle)["cert"]).extensions.get_extension_for_class( + x509.SubjectAlternativeName).value + assert {type(item).__name__ for item in general} == {"IPAddress", "DNSName"} + + +def test_the_host_name_is_always_in_the_san_and_nothing_is_listed_twice() -> None: + """A caller that passes no address still gets a verifiable cert, and a duplicate is one name.""" + store, state = boot() + state = issue(store, state, "driveripper", ["driveripper", " driveripper ", ""]) + entry = state.agent("driveripper") + assert entry is not None + assert entry.san == ("driveripper",) + assert san_names(cert_of(bundle_pems(entry.bundle)["cert"])) == ["driveripper"] + + +# --- issue, rotate, revoke ---------------------------------------------------------------------- # + + +def test_a_first_issue_is_revision_one_active_and_one_row() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + entry = state.agent("deskwork") + assert entry is not None + + assert entry.status == certs.ACTIVE + assert entry.bundle_rev == 1 + assert entry.serial > 0 + assert entry.not_after == NOW + certs.LEAF_LIFETIME.total_seconds() + assert rev_of(entry.bundle) == 1 + + stored = json.loads(store.rows[certs.META_AGENTS]) + assert list(stored) == ["deskwork"] + assert sorted(stored["deskwork"]) == ["bundle", "bundle_rev", "not_after", "san", "serial", + "status"] + assert stored["deskwork"]["bundle"] == entry.bundle, "the agent holds exactly what the row holds" + + +def test_a_reissue_bumps_the_revision_and_replaces_the_serial_and_the_bundle() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + first = state.agent("deskwork") + assert first is not None + + state = rotate(store, state, "deskwork", ["10.1.0.51"]) + second = state.agent("deskwork") + assert second is not None + + assert second.bundle_rev == first.bundle_rev + 1 + assert second.serial != first.serial + assert second.bundle != first.bundle + assert rev_of(second.bundle) == second.bundle_rev + assert key_of(bundle_pems(second.bundle)["key"]) != key_of(bundle_pems(first.bundle)["key"]), ( + "rotation replaces the key too, or the stolen one still works") + assert len(state.agents) == 1, "an issue is not an append" + + +def test_issue_and_reissue_are_one_operation_under_two_names() -> None: + """Rotating a host with nothing issued yet is legal and yields revision 1 (the task's wording).""" + store, state = boot() + rotated = rotate(store, state, "deskwork", ["10.1.0.51"]) + issued = issue(store, state, "deskwork", ["10.1.0.51"]) + + assert entry_of(rotated, "deskwork").bundle_rev == 1 + assert entry_of(issued, "deskwork").bundle_rev == 1 + assert entry_of(rotated, "deskwork").status == certs.ACTIVE + assert entry_of(issued, "deskwork").status == certs.ACTIVE + + +def test_revoking_puts_the_serial_on_every_bundles_denylist() -> None: + """The list lives inside each bundle, so revoking one host rewrites them all — that is the point.""" + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + state = issue(store, state, "gateway", ["10.1.1.27"]) + gateway = state.agent("gateway") + deskwork = state.agent("deskwork") + assert gateway is not None and deskwork is not None + + state = revoke(store, state, "gateway") + + assert entry_of(state, "gateway").status == certs.REVOKED + assert entry_of(state, "gateway").serial == gateway.serial, "the serial stays the denylisted one" + assert revoked_lines(entry_of(state, "gateway").bundle) == [gateway.serial] + assert revoked_lines(entry_of(state, "deskwork").bundle) == [gateway.serial], ( + "an agent that cannot see the revoked serial keeps accepting that client certificate") + assert entry_of(state, "deskwork").bundle_rev == deskwork.bundle_rev, ( + "a peer being revoked is not a new identity for this host") + assert entry_of(state, "deskwork").status == certs.ACTIVE + assert [entry.name for entry in certs.read_agents(store)] == ["deskwork", "gateway"] + + +def test_revoking_a_host_with_no_certificate_still_records_the_revocation() -> None: + """What a revoke *does* is stop the dashboard dialing; never-issued must not be a no-op.""" + store, state = boot() + state = revoke(store, state, "deskwork") + entry = state.agent("deskwork") + + assert entry is not None + assert entry.status == certs.REVOKED + assert entry.bundle_rev == 1 + assert revoked_lines(entry.bundle) == [entry.serial] + assert certs.agent_status(store, "deskwork") == certs.REVOKED + + +def test_a_reissue_after_a_revoke_gives_a_serial_that_is_not_listed() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + state = issue(store, state, "gateway", ["10.1.1.27"]) + revoked_serial = entry_of(state, "gateway").serial + state = revoke(store, state, "gateway") + + state = rotate(store, state, "gateway", ["10.1.1.27"]) + + fresh = state.agent("gateway") + assert fresh is not None and fresh.status == certs.ACTIVE + assert fresh.serial != revoked_serial + assert revoked_lines(fresh.bundle) == [], "the fix for a revoked identity is a new one" + assert revoked_lines(entry_of(state, "deskwork").bundle) == [] + assert certs.agent_status(store, "gateway") == certs.ACTIVE + + +def test_two_revocations_are_both_listed_and_in_one_order() -> None: + store, state = boot() + for name in ("deskwork", "gateway", "driveripper"): + state = issue(store, state, name, [f"10.1.0.{len(name)}"]) + state = revoke(store, state, "deskwork") + state = revoke(store, state, "driveripper") + + expected = sorted([entry_of(state, "deskwork").serial, entry_of(state, "driveripper").serial]) + for name in ("deskwork", "gateway", "driveripper"): + assert revoked_lines(entry_of(state, name).bundle) == expected, "one order, in every bundle" + assert entry_of(state, "gateway").status == certs.ACTIVE + + +def test_agent_status_names_only_what_the_operator_revoked() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + state = revoke(store, state, "deskwork") + + assert certs.agent_status(store, "deskwork") == certs.REVOKED + assert certs.agent_status(store, "gateway") == certs.ACTIVE + assert certs.agent_status(FakeMeta(), "anything") == certs.ACTIVE, ( + "no row is not a revocation: the safe answer is to keep dialing") + + +def test_read_agents_is_a_fresh_view_of_the_table() -> None: + """Task 03 re-reads this each poll; a cached view would delay a revocation by a restart.""" + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + assert [entry.name for entry in certs.read_agents(store)] == ["deskwork"] + snapshot = state + + state = revoke(store, state, "deskwork") + + assert entry_of(snapshot, "deskwork").status == certs.ACTIVE, "a returned state is a snapshot" + assert certs.read_agents(store)[0].status == certs.REVOKED + + +def test_read_agents_sorts_by_name_so_one_order_reaches_the_page() -> None: + store, state = boot() + for name in ("gateway", "deskwork", "driveripper"): + state = issue(store, state, name) + assert [entry.name for entry in certs.read_agents(store)] == [ + "deskwork", "driveripper", "gateway"] + assert [entry.name for entry in state.agents] == ["deskwork", "driveripper", "gateway"] + + +def test_nothing_is_issued_on_a_read() -> None: + """A `GET` that minted a certificate would make "did anyone rotate this?" unanswerable.""" + store, state = boot() + before = dict(store.rows) + for _ in range(5): + certs.ensure_certs(store, now=NOW) + certs.read_state(store) + certs.read_agents(store) + certs.agent_status(store, "deskwork") + assert store.rows == before + assert state.agents == () + + +def test_read_state_is_the_stored_authority_rather_than_the_bootstrapped_one() -> None: + """`read_state` answers "what does the table hold now?", which is what `GET /api/certs` prints. + + After a root rotation the process still holds the state it started with, so a page reading that + object would show a CA the operator has just replaced. The read goes to the rows, which is the + same reason task 03 reads `agent_status` per poll instead of caching it at startup. + """ + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + stale_root = state.root_serial + + rotated = certs.rotate_root(store, state, now=NOW) + + fresh = certs.read_state(store) + assert (fresh.root_serial, fresh.client_serial) == (rotated.root_serial, + rotated.client_serial) + assert fresh.root_serial != stale_root, "the old authority is not what the read answers" + assert fresh.root_cert == store.rows[certs.META_ROOT_CERT], "byte for byte, as stored" + assert [entry.name for entry in fresh.agents] == [entry.name for entry in rotated.agents] + + +def test_read_state_refuses_a_missing_row_instead_of_minting_one() -> None: + """The one behaviour that separates this read from `ensure_certs`, pinned from both sides. + + A GET that quietly regenerated a deleted row would both write on a read and replace the CA with + one nothing in the fleet trusts — the failure `ensure_certs` exists to bootstrap at startup and + the failure a request must never cause. `ensure_certs` on the same table generates; `read_state` + refuses, and the message is the fixed reset text that names the rows. + """ + store, _state = boot() + store.rows.pop(certs.META_CLIENT_CERT) + + with pytest.raises(RuntimeError) as caught: + certs.read_state(store) + + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + assert store.rows.get(certs.META_CLIENT_CERT) is None, "the refusal wrote nothing back" + + regenerated = certs.ensure_certs(store, now=NOW) # the startup read does mint it + assert regenerated.client_cert + + +@pytest.mark.parametrize("key", [certs.META_ROOT_KEY, certs.META_ROOT_CERT, + certs.META_CLIENT_KEY, certs.META_CLIENT_CERT]) +def test_read_state_validates_the_rows_it_reports(key: str) -> None: + """A page listing a corrupt authority would be a calmer way to hide the outage.""" + store, _state = boot() + store.rows[key] = "not a PEM document" + + with pytest.raises(RuntimeError) as caught: + certs.read_state(store) + + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_read_state_refuses_a_key_that_is_not_the_one_its_certificate_certifies() -> None: + """A row copied over from another row is corruption with a plausible look to it. + + The two halves of a pair are separate `meta` rows, so this is exactly what a hand-edit produces, + and a process that started with a mismatched pair would sign leaves with a key nothing verifies. + """ + store, _state = boot() + store.rows[certs.META_CLIENT_KEY] = store.rows[certs.META_ROOT_KEY] + + with pytest.raises(RuntimeError) as caught: + certs.read_state(store) + + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_read_state_refuses_a_leaf_the_stored_root_did_not_sign() -> None: + """A crash between the root write and the client write is the corrupt state, not a half-state.""" + store, _state = boot() + other, _other_state = boot() + store.rows[certs.META_CLIENT_CERT] = other.rows[certs.META_CLIENT_CERT] + store.rows[certs.META_CLIENT_KEY] = other.rows[certs.META_CLIENT_KEY] + + with pytest.raises(RuntimeError) as caught: + certs.read_state(store) + + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +# --- root rotation ------------------------------------------------------------------------------ # + + +def test_rotating_the_root_reissues_every_agent_and_resets_the_denylist() -> None: + store, state = boot() + for name in ("deskwork", "gateway"): + state = issue(store, state, name, [f"10.1.0.{len(name)}"]) + state = revoke(store, state, "gateway") + old_root = root_of(state) + old_client_serial = state.client_serial + before = {entry.name: entry for entry in state.agents} + + state = certs.rotate_root(store, state, now=NOW) + + new_root = root_of(state) + assert new_root.serial_number != old_root.serial_number + assert state.client_serial != old_client_serial + cert_of(state.client_cert).verify_directly_issued_by(new_root) + + for name, previous in before.items(): + entry = state.agent(name) + assert entry is not None + assert entry.status == certs.ACTIVE, "every identity is replaced, so nothing stays revoked" + assert entry.bundle_rev == previous.bundle_rev + 1 + assert entry.serial != previous.serial + assert entry.san == previous.san, "rotation must not change the names the dashboard dials" + cert_of(bundle_pems(entry.bundle)["cert"]).verify_directly_issued_by(new_root) + assert bundle_pems(entry.bundle)["root"] == state.root_cert + assert revoked_lines(entry.bundle) == [], "the denylist belongs to one generation" + assert json.loads(store.rows[certs.META_AGENTS])["gateway"]["bundle_rev"] == ( + before["gateway"].bundle_rev + 1) + + +def test_no_stored_value_still_carries_the_old_generation() -> None: + """A rotated root has to be *gone*: a stale key or cert left in a row is the leak and the bug.""" + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + old_root_key, old_root_cert, old_client_key = state.root_key, state.root_cert, state.client_key + + state = certs.rotate_root(store, state, now=NOW) + + for key, value in store.rows.items(): + assert old_root_key not in value, key + assert old_root_cert not in value, key + assert old_client_key not in value, key + assert state.root_key != old_root_key + assert root_of(state).serial_number != cert_of(old_root_cert).serial_number + entry = state.agent("deskwork") + assert entry is not None and old_root_cert not in entry.bundle + + +def test_rotating_the_root_with_no_agents_yet_still_leaves_a_working_credential() -> None: + store, state = boot() + state = certs.rotate_root(store, state, now=NOW) + assert state.agents == () + cert_of(state.client_cert).verify_directly_issued_by(root_of(state)) + assert sorted(store.rows) == sorted(certs.META_KEYS) + + +# --- the bundle --------------------------------------------------------------------------------- # + + +def test_the_bundle_is_metadata_then_agent_cert_then_agent_key_then_root_cert() -> None: + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + entry = state.agent("deskwork") + assert entry is not None + bundle = entry.bundle + + assert bundle.startswith(f"{certs.BUNDLE_REV_LINE}1\n") + assert "# REVOKED" not in bundle, "nothing is revoked on a fresh root with nothing revoked" + assert [block.split("-----")[1] for block in blocks(bundle)] == [ + "BEGIN CERTIFICATE", "BEGIN PRIVATE KEY", "BEGIN CERTIFICATE"] + assert bundle_pems(bundle)["root"] == state.root_cert, "the trust anchor is the stored root" + assert bundle_pems(bundle)["cert"] == cert_of( + bundle_pems(bundle)["cert"]).public_bytes(Encoding.PEM).decode("ascii") + + +def test_the_bundle_carries_the_agent_key_and_never_the_root_key() -> None: + """The TODO's "root certificate installed with each exporter" is the root *cert*. + + Asserted twice on purpose: by header count (one key block, so a second private key is visible) + and by parsing that key against both keypairs, so a swap that kept the count honest still + fails here. + """ + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + entry = state.agent("deskwork") + assert entry is not None + bundle = entry.bundle + + assert bundle.count("-----BEGIN PRIVATE KEY-----") == 1 + assert state.root_key not in bundle + in_bundle = key_of(bundle_pems(bundle)["key"]) + agent_cert = cert_of(bundle_pems(bundle)["cert"]) + assert public_id(in_bundle.public_key()) == public_id(agent_cert.public_key()) + assert public_id(in_bundle.public_key()) != public_id(root_of(state).public_key()) + + +def test_a_bundle_that_is_not_the_stored_shape_is_the_fixed_text_refusal() -> None: + """A hand-edited or truncated row cannot be re-sealed — and re-sealing is what revoking does.""" + store, state = boot() + state = issue(store, state, "deskwork", ["10.1.0.51"]) + payload = json.loads(store.rows[certs.META_AGENTS]) + payload["deskwork"]["bundle"] = "-----BEGIN CERTIFICATE-----\nbroken\n" + store.set_meta(certs.META_AGENTS, json.dumps(payload)) + tampered = certs.ensure_certs(store, now=NOW) + + with pytest.raises(RuntimeError) as caught: + revoke(store, tampered, "deskwork") + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +# --- fail closed on a corrupt row ---------------------------------------------------------------- # + + +@pytest.mark.parametrize("key", [certs.META_ROOT_KEY, certs.META_ROOT_CERT, + certs.META_CLIENT_KEY, certs.META_CLIENT_CERT]) +def test_a_stored_pem_that_will_not_parse_refuses_to_start(key: str) -> None: + store, _state = boot() + store.rows[key] = "not a PEM document" + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + + message = str(caught.value) + assert message == certs.CORRUPT_STORE_MESSAGE + for name in certs.META_KEYS: + assert name in message, "the operator is told which rows to delete" + + +def test_a_key_type_this_build_cannot_use_refuses_to_start() -> None: + """Somebody hand-inserting an RSA key gets the reset message, not a stack trace at first dial.""" + store, _state = boot() + rsa_pem = rsa.generate_private_key(public_exponent=65537, key_size=2048).private_bytes( + Encoding.PEM, PrivateFormat.PKCS8, NoEncryption()).decode("ascii") + store.rows[certs.META_CLIENT_KEY] = rsa_pem + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_a_key_that_does_not_match_its_certificate_refuses_to_start() -> None: + store, _state = boot() + other = FakeMeta() + other_state = certs.ensure_certs(other, now=NOW) + store.rows[certs.META_CLIENT_KEY] = other_state.client_key + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_a_client_certificate_the_stored_root_did_not_sign_refuses_to_start() -> None: + """The half-finished root rotation: a new root beside an old leaf is a dashboard that reaches nothing.""" + store, _state = boot() + other = FakeMeta() + other_state = certs.ensure_certs(other, now=NOW) + store.rows[certs.META_CLIENT_CERT] = other_state.client_cert + store.rows[certs.META_CLIENT_KEY] = other_state.client_key + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_a_root_key_and_cert_from_two_generations_refuse_to_start() -> None: + store, _state = boot() + other = FakeMeta() + other_state = certs.ensure_certs(other, now=NOW) + store.rows[certs.META_ROOT_KEY] = other_state.root_key + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +@pytest.mark.parametrize("raw", [ + "not json at all", + "[]", + '{"deskwork": "nope"}', + '{"deskwork": {"serial": 1, "not_after": 2, "status": "active", "bundle_rev": 1, ' + '"bundle": "x"}}', + '{"deskwork": {"serial": 1, "not_after": 2, "status": "active", "bundle_rev": 1, ' + '"bundle": "x", "san": "deskwork"}}', + '{"deskwork": {"serial": "abc", "not_after": 2, "status": "active", "bundle_rev": 1, ' + '"bundle": "x", "san": []}}', + '{"deskwork": {"serial": 1, "not_after": 2, "status": "active", "bundle_rev": 1, ' + '"bundle": "", "san": []}}', +]) +def test_an_agent_table_that_is_not_the_stored_shape_refuses_to_start(raw: str) -> None: + """Every one of these is what a partial write or a hand edit could leave behind.""" + store, _state = boot() + store.rows[certs.META_AGENTS] = raw + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + + +def test_the_refusal_quotes_nothing_from_the_database() -> None: + """invariant 6, against the value that matters most in this module: the root key.""" + store, _state = boot() + secret = store.rows[certs.META_ROOT_KEY] + store.rows[certs.META_ROOT_CERT] = "truncated-----" + + with pytest.raises(RuntimeError) as caught: + certs.ensure_certs(store, now=NOW) + + message = str(caught.value) + body = "\n".join(secret.splitlines()[1:-1]) + assert body not in message + assert body[:32] not in message + assert store.rows[certs.META_ROOT_KEY] == secret, "refusing must not rewrite the rows" + + +# --- durability through the real store -------------------------------------------------------------- # + + +def test_the_certificates_survive_a_reopen_of_the_real_database(tmp_path: Path) -> None: + """The whole reason this lives in SQLite: a restart re-reads it and an agent install stays valid.""" + path = tmp_path / "dashboards.sqlite3" + first_store = SampleStore(path) + try: + first = certs.ensure_certs(first_store, now=NOW) + issued = certs.issue_agent_cert(first_store, first, "deskwork", ["10.1.0.51"], now=NOW) + finally: + first_store.close() + + reopened = SampleStore(path) + try: + second = certs.ensure_certs(reopened, now=NOW + 10) + finally: + reopened.close() + + assert second.root_cert == first.root_cert + assert second.root_key == first.root_key + assert second.client_cert == first.client_cert + entry = second.agent("deskwork") + assert entry is not None + assert entry.bundle == entry_of(issued, "deskwork").bundle + assert entry.bundle_rev == 1 + + +# --- on-disk material ------------------------------------------------------------------------------ # + + +def test_the_material_is_three_private_files_and_never_the_root_key(tmp_path: Path) -> None: + _store, state = boot() + material = certs.write_material(state, tmp_path) + try: + assert sorted(p.name for p in material.dir.iterdir()) == [ + CLIENT_CERT_FILE, CLIENT_KEY_FILE, CA_FILE] + assert material.ca.read_text() == state.root_cert + assert material.client_cert.read_text() == state.client_cert + assert material.client_key.read_text() == state.client_key + assert (material.dir.stat().st_mode & 0o777) == 0o700 + for path in (material.ca, material.client_cert, material.client_key): + assert (path.stat().st_mode & 0o777) == 0o600, str(path) + assert state.root_key not in "\n".join(p.read_text() for p in material.dir.iterdir()) + assert material.state is state + finally: + certs.remove_material(material) + + +def test_the_material_defaults_to_the_environment_tmp_dir() -> None: + _store, state = boot() + material = certs.write_material(state) + try: + assert material.dir.name.startswith("dashboards-certs-") + assert material.dir.parent.resolve() == Path(tempfile.gettempdir()).resolve() + assert material.ca.exists() + finally: + certs.remove_material(material) + + +def test_removing_the_material_takes_the_directory_and_tolerates_none(tmp_path: Path) -> None: + _store, state = boot() + material = certs.write_material(state, tmp_path) + certs.remove_material(material) + assert not material.dir.exists() + certs.remove_material(None) + + +# --- the material works in a real TLS handshake ----------------------------------------------------- # + + +def test_the_material_and_a_bundle_complete_a_real_client_certificate_handshake( + tmp_path: Path) -> None: + """The claim tasks 02 and 03 rest on, checked with `ssl` instead of with this module's parser. + + A certificate `cryptography` wrote that `ssl` will not load is the worst bug available here: + every other test in this file would stay green while no agent in the fleet could handshake. + The server side runs with `CERT_OPTIONAL`, which is task 02's assumption that `/health` stays + probeable without a client certificate, so both answers are asserted: a peer certificate for + the dashboard, and none for a bare probe. + """ + # Real clock for this one test: `ssl` checks the validity window itself, so the fixed `NOW` + # this file mints everything else at would be "certificate has expired" by 2027. + store = FakeMeta() + state = certs.ensure_certs(store) + state = certs.issue_agent_cert(store, state, "localhost", ["127.0.0.1", "localhost"]) + entry = state.agent("localhost") + assert entry is not None + pems = bundle_pems(entry.bundle) + (tmp_path / "agent-cert.pem").write_text(pems["cert"]) + (tmp_path / "agent-key.pem").write_text(pems["key"]) + (tmp_path / CA_FILE).write_text(state.root_cert) + base = tmp_path / "material" + base.mkdir() + material = certs.write_material(state, base) + try: + (tmp_path / CLIENT_CERT_FILE).write_text(material.client_cert.read_text()) + (tmp_path / CLIENT_KEY_FILE).write_text(material.client_key.read_text()) + + server = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + server.load_cert_chain(str(tmp_path / "agent-cert.pem"), str(tmp_path / "agent-key.pem")) + server.load_verify_locations(str(tmp_path / CA_FILE)) + server.verify_mode = ssl.CERT_OPTIONAL + + with_cert = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + with_cert.load_verify_locations(str(tmp_path / CA_FILE)) + with_cert.load_cert_chain(str(tmp_path / CLIENT_CERT_FILE), str(tmp_path / CLIENT_KEY_FILE)) + without_cert = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + without_cert.load_verify_locations(str(tmp_path / CA_FILE)) + + seen = _handshake(server, [with_cert, without_cert], tmp_path) + finally: + certs.remove_material(material) + + assert seen[0], "the dashboard's client certificate reached the agent" + assert any(name == "commonName" and value == certs.CLIENT_CN + for name, value in seen[0]["subject"][0]) + # `getpeercert()` says "no certificate" as `{}` on some builds and `None` on others; the + # handler's check (task 02) is `if not cert`, which is what this pins rather than the empty + # value's exact shape. + assert not seen[1], "a probe with no client certificate still handshakes: the /health path" + + +def _handshake(server: ssl.SSLContext, clients: list[ssl.SSLContext], + tmp_path: Path) -> list[Any]: + """Serve `server` on a loopback port and run one handshake per client context.""" + assert len(clients) == 2 + listener = socket.socket() + listener.bind(("127.0.0.1", 0)) + listener.listen(2) + listener.settimeout(15) + port = listener.getsockname()[1] + seen: list[Any] = [] + + def serve() -> None: + for _ in clients: + try: + raw, _peer = listener.accept() + except OSError: + return + try: + with server.wrap_socket(raw, server_side=True) as tls: + seen.append(tls.getpeercert()) + tls.sendall(b"ok") + except OSError as exc: # a refused handshake is itself an answer under test + seen.append(exc) + + thread = threading.Thread(target=serve, daemon=True) + thread.start() + try: + for context in clients: + with socket.create_connection(("127.0.0.1", port), timeout=15) as raw, \ + context.wrap_socket(raw, server_hostname="localhost") as tls: + assert tls.recv(2) == b"ok" + finally: + listener.close() + thread.join(timeout=15) + return seen + + +# --- the app wiring (work item 3) ---------------------------------------------------------------- # + + +def settings_for(db: str) -> Settings: + return Settings(db_path=db, admin_password=ADMIN_PASSWORD, pbkdf2_iterations=FAST_PBKDF2) + + +def test_the_lifespan_builds_the_material_before_the_loops_start(tmp_path: Path) -> None: + """`ctx.certs` is there for task 03 to dial with, in the injected-store branch too.""" + db = str(tmp_path / "app.sqlite3") + store = SampleStore(db) + app = create_app(settings_for(db), store) + try: + with TestClient(app): + context = app.state.ctx + assert context.certs is not None + assert context.certs.ca.exists() and context.certs.client_cert.exists() + assert context.certs.client_key.exists() + directory = context.certs.dir + for key in certs.META_KEYS: + assert store.get_meta(key), f"{key} is durable, not only in memory" + assert context.auth is not None + assert auth.verify_password(ADMIN_PASSWORD, context.auth.verifier) is True + finally: + store.close() + assert not directory.exists(), "the private directory goes with the process" + + +def test_a_corrupt_stored_certificate_makes_the_server_refuse_to_start(tmp_path: Path) -> None: + """Fail closed, before the loops: a dashboard with no credential cannot report anything truthfully.""" + db = str(tmp_path / "corrupt.sqlite3") + store = SampleStore(db) + certs.ensure_certs(store, now=NOW) + store.set_meta(certs.META_ROOT_CERT, "the certificate that is not one") + app = create_app(settings_for(db), store) + try: + with pytest.raises(RuntimeError) as caught, TestClient(app): + pytest.fail("the lifespan yielded with unreadable certificates") + assert str(caught.value) == certs.CORRUPT_STORE_MESSAGE + assert not hasattr(app.state, "ctx"), "nothing is published, so nothing can serve" + finally: + store.close() diff --git a/tests/test_collectors.py b/tests/test_collectors.py index c4bbd31..fe83d56 100644 --- a/tests/test_collectors.py +++ b/tests/test_collectors.py @@ -10,21 +10,30 @@ transitions); the socket protocol itself is in test_nut.py. from __future__ import annotations import json +import threading +from http.server import ThreadingHTTPServer from pathlib import Path +from types import SimpleNamespace from typing import Any, ClassVar import httpx import pytest +from dashboards import certs from dashboards.collectors import ( + AgentTls, NutMonitor, UpstreamError, + agent_tls_map, + disks_jobs, fetch_disk_snapshot, fetch_engine, fetch_host_snapshot, fetch_snapshot, + host_jobs, ) from dashboards.config import Target +from dashboards.poller import Result from dashboards.rates import RateEstimator from dashboards.sources import nut from dashboards.sources.disks import DisksCollector @@ -351,6 +360,403 @@ async def test_smart_addon_errors_reach_the_disk_payload_verbatim( assert "degraded" not in system.payload, "and /system stays unbothered by it" +# --------------------------------------------------------------------------- # +# TLS: the dashboard dials a protected exporter with its own client certificate +# --------------------------------------------------------------------------- # +# +# The certificates come from `dashboards.certs` over a dict store and the server is the real +# `agent/host_agent.py` in the TLS mode task 02 added, so this proves the dial itself: the +# client certificate is presented, the agent is verified against the root this dashboard minted, +# and a host the operator revoked is refused before a socket opens. `httpx.MockTransport` is +# kept for the sentences (a 403 body is the agent's words, not a handshake). + + +class DictMeta: + """The three-method `certs.MetaStore` slice, backed by a dict (the `test_certs.py` idiom).""" + + def __init__(self) -> None: + self.rows: dict[str, str] = {} + + def get_meta(self, key: str) -> str | None: + return self.rows.get(key) + + def set_meta(self, key: str, value: str) -> None: + self.rows[key] = value + + def delete_meta(self, key: str) -> int: + return 1 if self.rows.pop(key, None) is not None else 0 + + +@pytest.fixture +def tls_fleet(tmp_path: Path, host_agent_module: Any) -> Any: + """One dashboard CA + client pair, one issued agent bundle, and the agent serving it. + + Everything production has: the material written to a private directory (`httpx` takes + paths), the bundle installed where `install-agent.sh` will put it, and the `AgentTls` the + jobs get — built by `agent_tls_map`, the same factory `app.build_hub` is handed, so the + tests dial with the production wiring and not a hand-built guess of it. + """ + store = DictMeta() + state = certs.ensure_certs(store) + state = certs.issue_agent_cert(store, state, "localhost", ["127.0.0.1", "localhost"]) + entry = state.agent("localhost") + assert entry is not None + material = certs.write_material(state, tmp_path) + bundle_path = tmp_path / "dashboards-agent.certs" + bundle_path.write_text(entry.bundle, encoding="ascii") + bundle, error = host_agent_module.TlsBundle.load(str(bundle_path)) + assert error is None and bundle is not None + host_agent_module.Handler.collector = host_agent_module.Collector("/", smart=False) + host_agent_module.Handler.token = None + host_agent_module.Handler.bundle = bundle + httpd = ThreadingHTTPServer(("127.0.0.1", 0), host_agent_module.Handler) + httpd.socket = bundle.context.wrap_socket(httpd.socket, server_side=True) + thread = threading.Thread(target=httpd.serve_forever, daemon=True) + thread.start() + base = f"https://127.0.0.1:{httpd.server_address[1]}" + target = Target("localhost", base) + try: + yield SimpleNamespace( + store=store, + state=state, + material=material, + bundle=bundle, + bundle_rev=entry.bundle_rev, + base=base, + target=target, + tls=agent_tls_map(material, store, [target])["localhost"], + ) + finally: + httpd.shutdown() + httpd.server_close() + # `Handler` is process-wide state: a later test must not inherit this bundle. + host_agent_module.Handler.bundle = None + host_agent_module.Handler.token = None + certs.remove_material(material) + + +async def test_an_https_agent_answers_the_dashboard_that_presents_its_certificate( + tls_fleet: SimpleNamespace) -> None: + """The whole point of the phase, collector side: our cert opens the agent. + + The two fields the agent adds in TLS mode come back too, because task 05's page compares the + revision `/certs` issued with the revision the host is actually running. + """ + async with httpx.AsyncClient(timeout=15.0) as client: + body = await fetch_snapshot(client, tls_fleet.target, tls=tls_fleet.tls) + assert body["ok"] is True + assert body["tls"] is True + assert body["bundle_rev"] == tls_fleet.bundle_rev + result = await fetch_host_snapshot(client, tls_fleet.target, + HostCollector(RateEstimator()), tls=tls_fleet.tls) + assert result.payload["agent"]["tls"] is True + assert result.payload["agent"]["bundle_rev"] == tls_fleet.bundle_rev + assert result.payload["memory"]["total_bytes"] > 0, "the snapshot is a real one" + + +async def test_an_agent_signed_by_another_authority_is_not_trusted( + tls_fleet: SimpleNamespace, tmp_path: Path) -> None: + """A root that is not ours verifies nothing here, so the dial fails instead of accepting. + + Trusting any certificate would make the whole phase a no-op with extra steps, and the + failure has to reach the poller as an `UpstreamError` naming the host rather than as a + traceback out of `ssl`. + """ + stranger_base = tmp_path / "stranger" + stranger_base.mkdir() + stranger = certs.write_material(certs.ensure_certs(DictMeta()), stranger_base) + foreign = agent_tls_map(stranger, tls_fleet.store, [tls_fleet.target])["localhost"] + async with httpx.AsyncClient(timeout=15.0) as client: + with pytest.raises(UpstreamError) as caught: + await fetch_snapshot(client, tls_fleet.target, tls=foreign) + assert "unreachable" in caught.value.message + assert tls_fleet.base in caught.value.message + certs.remove_material(stranger) + + +async def test_dashboard_material_that_cannot_be_read_fails_the_poll( + tls_fleet: SimpleNamespace, tmp_path: Path) -> None: + """A missing client-cert file is a red source with a sentence, not a dead poll loop. + + `httpx` loads the CA and the client chain when the client is *built*, so this arrives as a + `FileNotFoundError` from inside the fetch. A TLS dial that can only fail with a non-HTTP + exception is the invariant-4 failure mode: the task would die and the source would look + healthy while its charts stopped. + """ + broken = AgentTls(verify=str(tls_fleet.material.ca), + cert=(str(tmp_path / "gone-cert.pem"), str(tmp_path / "gone-key.pem")), + status=lambda: certs.ACTIVE) + async with httpx.AsyncClient(timeout=15.0) as client: + with pytest.raises(UpstreamError, match="unreachable") as caught: + await fetch_snapshot(client, tls_fleet.target, tls=broken) + assert "FileNotFoundError" in caught.value.message + assert tls_fleet.base in caught.value.message, "the failure still names the host's URL" + + +async def test_an_https_agent_is_not_readable_without_the_material(tls_fleet: SimpleNamespace) -> None: + """A shared client with no client certificate gets nowhere — "only this dashboard can read + an agent" has to hold for the collector's own client too, not just for a stranger's. + """ + async with httpx.AsyncClient(timeout=15.0) as client: + with pytest.raises(UpstreamError) as caught: + await fetch_snapshot(client, tls_fleet.target) + assert "unreachable" in caught.value.message + + +async def test_a_revoked_agent_is_refused_before_anything_is_dialled( + tls_fleet: SimpleNamespace, monkeypatch: pytest.MonkeyPatch) -> None: + """The revocation gate is in front of the network, not behind it. + + The spy replaces `httpx.AsyncClient.get` on the class, so it catches the shared client and + the per-dial TLS client alike: what is being proved is that no request is built at all, not + that one particular client was unused. A dial-then-refuse would still put a socket on a host + the operator just struck out. + """ + calls: list[str] = [] + + async def spy_get(self: Any, url: Any, **kwargs: Any) -> Any: + calls.append(str(url)) + raise AssertionError(f"a revoked agent was dialled at {url}") + + monkeypatch.setattr(httpx.AsyncClient, "get", spy_get) + certs.revoke(tls_fleet.store, tls_fleet.state, "localhost") + async with httpx.AsyncClient(timeout=15.0) as client: + with pytest.raises(UpstreamError, match="agent certificate revoked") as caught: + await fetch_snapshot(client, tls_fleet.target, tls=tls_fleet.tls) + with pytest.raises(UpstreamError, match="agent certificate revoked"): + await fetch_disk_snapshot(client, tls_fleet.target, DisksCollector(), + tls=tls_fleet.tls) + assert calls == [], "a revoked host must not be dialled on either cadence" + assert "revoked" in caught.value.message + + +async def test_the_revocation_answer_is_a_live_read_of_the_store(tls_fleet: SimpleNamespace) -> None: + """`status` is a callable because the operator clicks revoke while this process is running. + + A status string read at startup would keep dialling a struck-out host until the next + restart, and a dashboard that needs a restart to enforce a security decision does not + enforce it. + """ + tls = tls_fleet.tls + assert tls.status() == certs.ACTIVE + assert tls.revoked is False + certs.revoke(tls_fleet.store, tls_fleet.state, "localhost") + assert tls.status() == certs.REVOKED + assert tls.revoked is True, "the same object, with no rebuild and no restart" + + +def test_the_revocation_is_per_host_and_the_material_is_shared(tls_fleet: SimpleNamespace) -> None: + """One identity for every dial, one revocation answer per name. + + `certs.py` gives the dashboard a single client certificate, so `verify`/`cert` are the same + paths for every host; what differs is whose identity the operator struck out. A name with no + entry is active rather than refused — never-issued is not the same fact as revoked, and + answering revoked for it would take a working host red with no operator action behind it + (invariant 3). + """ + targets = [tls_fleet.target, Target("deskwork", "https://10.1.0.51:9120")] + mapping = agent_tls_map(tls_fleet.material, tls_fleet.store, targets) + assert mapping["localhost"].verify == str(tls_fleet.material.ca) + assert mapping["localhost"].cert == mapping["deskwork"].cert == ( + str(tls_fleet.material.client_cert), str(tls_fleet.material.client_key)) + certs.revoke(tls_fleet.store, tls_fleet.state, "localhost") + assert mapping["localhost"].status() == certs.REVOKED + assert mapping["deskwork"].status() == certs.ACTIVE + assert agent_tls_map(None, tls_fleet.store, targets) == {}, ( + "no material means no TLS dial, exactly the pre-phase-15 path") + + +def scripted_refusal(body: Any, *, status: int = 403) -> AgentTls: + """An agent that answers every `/snapshot` with a refusal, in whatever shape it sent the body.""" + def handler(request: httpx.Request) -> httpx.Response: + if isinstance(body, dict): + return httpx.Response(status, json=body) + return httpx.Response(status, text=body) + + return AgentTls(verify="/unused/root.pem", cert=("/unused/cert.pem", "/unused/key.pem"), + status=lambda: certs.ACTIVE, transport=httpx.MockTransport(handler)) + + +@pytest.mark.parametrize("body,expected", [ + ({"ok": False, "detail": "client certificate required"}, "client certificate required"), + ({"ok": False, "detail": "client certificate revoked"}, "client certificate revoked"), + ("403 Forbidden", "403 Forbidden"), + ("", "no reason given"), +]) +async def test_a_refusal_from_a_protected_agent_quotes_its_own_words( + body: Any, expected: str) -> None: + """The 403 sentence names the refusal the *agent* made, redacted and never blank. + + An `UpstreamError` string goes into the health record and the event table, so it has to say + which of the agent's two refusals this was (a required certificate and a revoked one are two + different operator actions) while quoting nothing but the URL it was dialling + (AGENTS.md invariant 6). The `verify`/`cert` paths here do not exist: a scripted refusal + proves the wording, and the tests above prove the handshake. + """ + tls = scripted_refusal(body) + target = Target("gw", "https://SEKRIT-AGENT-TOKEN@agent.test") + async with httpx.AsyncClient(timeout=15.0) as client: + with pytest.raises(UpstreamError) as caught: + await fetch_snapshot(client, target, tls=tls) + message = caught.value.message + assert "agent refused the client certificate" in message + assert expected in message + assert "SEKRIT-AGENT-TOKEN" not in message, "the token leaked into a browser-readable string" + assert "https://agent.test/snapshot" in message, "and it still says where" + + +@pytest.mark.parametrize("status", [401, 403, 500]) +async def test_a_refusal_from_an_unprotected_agent_keeps_the_status_wording(status: int) -> None: + """The certificate sentence is only for a dial where a certificate was presented. + + A token-mode host answers 401 for a bad token and would answer 403 for something that has + nothing to do with certificates; calling that "refused the client certificate" would send + the operator to `/certs` for a token problem. + """ + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(status, text="nope") + + target = Target("gw", "http://agent.test") + async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as client: + with pytest.raises(UpstreamError) as caught: + await fetch_snapshot(client, target, + tls=AgentTls(verify="/unused/root.pem", + cert=("/unused/c.pem", "/unused/k.pem"), + status=lambda: certs.ACTIVE)) + if status == 401: + assert "token" in caught.value.message + else: + assert f"HTTP {status}" in caught.value.message + assert "client certificate" not in caught.value.message + + +async def test_a_token_mode_host_still_uses_the_shared_client(tls_fleet: SimpleNamespace) -> None: + """An `http://` entry is dialled exactly as it was before phase 15. + + The material exists in this deployment (one CA, one client pair, every host), so the scheme + is the only thing that may decide the path — a rollout is per host, and the plain hosts + cannot be allowed to pick up a client certificate they never asked for. + """ + seen: list[str] = [] + + def handler(request: httpx.Request) -> httpx.Response: + seen.append(str(request.url)) + return httpx.Response(200, json=agent_body()) + + plain = Target("localhost", "http://agent.test") + async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as client: + body = await fetch_snapshot(client, plain, tls=tls_fleet.tls) + assert body["ok"] is True + assert seen == ["http://agent.test/snapshot"], "the shared client, the shared transport" + + +async def test_a_revoked_agent_fails_both_jobs_with_one_sentence(store: Any, + tls_fleet: SimpleNamespace, + monkeypatch: pytest.MonkeyPatch) -> None: + """The phase's criterion, at the level the health record is written: `host:` and `disks:`. + + The two jobs dial one agent on two cadences, so a revocation has to fail both with the same + sentence (one dead source must not blink the other, and both must stop talking to it). + """ + calls: list[str] = [] + + async def spy_get(self: Any, url: Any, **kwargs: Any) -> Any: + calls.append(str(url)) + raise AssertionError(f"a revoked agent was dialled at {url}") + + monkeypatch.setattr(httpx.AsyncClient, "get", spy_get) + certs.revoke(tls_fleet.store, tls_fleet.state, "localhost") + async with httpx.AsyncClient(timeout=15.0) as client: + jobs = host_jobs([tls_fleet.target], client, HostCollector(RateEstimator()), 10.0, + tls={"localhost": tls_fleet.tls}) + jobs += disks_jobs([tls_fleet.target], client, DisksCollector(), 300.0, + tls={"localhost": tls_fleet.tls}) + assert [job.key for job in jobs] == ["host:localhost", "disks:localhost"] + for job in jobs: + assert await job.run_once(store) is False + assert [job.state.error for job in jobs] == [ + "UpstreamError: localhost: agent certificate revoked"] * 2 + assert calls == [] + + +async def test_host_and_disk_jobs_hand_each_host_its_own_material( + monkeypatch: pytest.MonkeyPatch) -> None: + """The plumbing, pinned: a host in the map gets its entry, a host outside it gets `None`. + + Two hosts and one wrong-key miss is the minimum that shows the lookup is by name rather than + "whatever was built first", which is how a revoked host would keep being dialled with + another host's revocation answer. + """ + seen: dict[str, Any] = {} + + def record(name: str) -> Any: + async def scrape(client: Any, target: Target, *_args: Any, **kwargs: Any) -> Result: + seen[f"{name}:{target.name}"] = kwargs.get("tls") + return Result(payload={}, samples=[], events=[]) + + return scrape + + fetch_host = record("system") + fetch_disk = record("disks") + monkeypatch.setattr("dashboards.collectors.fetch_host_snapshot", fetch_host) + monkeypatch.setattr("dashboards.collectors.fetch_disk_snapshot", fetch_disk) + entry = AgentTls(verify="/ca.pem", cert=("/c.pem", "/k.pem"), + status=lambda: certs.ACTIVE) + targets = [Target("gw", "https://gw.test"), Target("desk", "http://desk.test")] + async with httpx.AsyncClient() as client: + for job in (host_jobs(targets, client, HostCollector(RateEstimator()), 10.0, + tls={"gw": entry}) + + disks_jobs(targets, client, DisksCollector(), 300.0, tls={"gw": entry})): + await job.fn() + assert seen == {"system:gw": entry, "system:desk": None, + "disks:gw": entry, "disks:desk": None} + + +async def test_the_tls_self_report_reaches_both_payloads(disk_snapshot: Any) -> None: + """`tls` + `bundle_rev` travel into `/api/system/snapshot` and `/api/disks/snapshot`. + + Task 05's page has to answer "is the installed bundle the one I just issued?", and both + dashboards read the same snapshot, so both carry the deployed revision under the same + `agent` key. Nothing grades it: a revision is an identity counter, not a threshold. + """ + def stamped(snapshot: dict[str, Any]) -> dict[str, Any]: + """What a TLS-mode agent's `/snapshot` looks like on the wire.""" + snapshot["tls"] = True + snapshot["bundle_rev"] = 3 + return snapshot + + target = Target("driveripper", "http://agent.test") + async with client_for(lambda request: httpx.Response(200, + json=stamped(agent_body()))) as client: + system = (await fetch_host_snapshot(client, target, + HostCollector(RateEstimator()))).payload + assert system["agent"]["tls"] is True + assert system["agent"]["bundle_rev"] == 3 + + # the `/disks` payload is built from a capture that also carries SMART blocks, so it gets + # its own stamped snapshot rather than sharing the one above + with_smart = disk_snapshot(mutate=stamped) + async with client_for(lambda request: httpx.Response(200, json=with_smart)) as client: + disks = (await fetch_disk_snapshot(client, target, DisksCollector())).payload + assert disks["agent"] == {"tls": True, "bundle_rev": 3} + + +async def test_a_token_mode_snapshot_gains_no_tls_fields() -> None: + """The captures are token-mode agents, and their payloads must stay byte-identical. + + `false` and `0` would be numbers the agent never published (rule 2), and an empty block + would be a container that means nothing. + """ + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(200, json=agent_body()) + + target = Target("driveripper", "http://agent.test") + async with client_for(handler) as client: + system = (await fetch_host_snapshot(client, target, HostCollector(RateEstimator()))).payload + assert "tls" not in system["agent"] and "bundle_rev" not in system["agent"] + assert system["agent"]["version"], "the block is still the block it was" + + # --------------------------------------------------------------------------- # # LLM engines # --------------------------------------------------------------------------- # diff --git a/tests/test_deploy_scripts.py b/tests/test_deploy_scripts.py index 98ca3ce..a72bc01 100644 --- a/tests/test_deploy_scripts.py +++ b/tests/test_deploy_scripts.py @@ -12,15 +12,21 @@ apart from the path the script was invoked through. from __future__ import annotations +import json import os import re import shutil import stat import subprocess +import threading +from http.server import ThreadingHTTPServer from pathlib import Path +from typing import Any import pytest +from dashboards import certs + ROOT = Path(__file__).resolve().parent.parent DEPLOY = ROOT / "deploy" INSTALLER = DEPLOY / "install-agent.sh" @@ -291,6 +297,7 @@ ROOT_BOUNDARY_STUBS = ("id", "getent", "useradd", "usermod", "chown", "systemctl SANDBOX_REWRITES = { 'AGENT_DIR="/usr/local/lib/dashboards-agent"': 'AGENT_DIR="{root}/lib"', 'TOKEN_FILE="/etc/dashboards-agent.token"': 'TOKEN_FILE="{root}/token"', + 'CERT_FILE="/etc/dashboards-agent.certs"': 'CERT_FILE="{root}/certs"', 'UNIT="/etc/systemd/system/$SERVICE.service"': 'UNIT="{root}/dashboards-agent.service"', } @@ -361,6 +368,10 @@ def test_install_path_installs_into_its_own_paths_and_prints_the_token_once(tmp_ assert "User=dashboards-agent" in unit assert f"--token-file {token_path}" in unit, "the unit must use the file, never --token" assert "--host 10.0.0.5 --port 9120" in unit + # A token-only install runs no TLS, so it gets neither the bundle flag nor the directory + # the bundle unpacks into (the agent falls back to a private mkdtemp without one). + assert "--cert-bundle" not in unit, "no bundle installed means no TLS flag on a token install" + assert "RuntimeDirectory=dashboards-agent" not in unit # DevicePolicy=strict denies every node not listed, and systemd opens /dev/null for stdin # before it execs python: without this line the unit dies at spawn (status 208/STDIN). assert "DevicePolicy=strict" in unit @@ -396,3 +407,370 @@ def test_token_file_is_readable_by_the_service_user_and_nobody_else(tmp_path: Pa mode = stat.S_IMODE((tmp_path / "target" / "token").stat().st_mode) assert mode == 0o640, f"token file mode is {oct(mode)}, expected 0o640 (group = the agent user)" assert not mode & 0o007, "the token file must never be world-readable" + + +# --- the certificate bundle (phase 15 task 06) -------------------------------------------- + +#: Where the installed bundle lives — one constant in the script, pinned here so a rename +#: fails a test instead of stranding every already-written unit file. +CERT_FILE_PATH = "/etc/dashboards-agent.certs" + + +def _fake_bundle_text(revision: str = "3") -> str: + """A syntactically real bundle: agent cert, agent key, root CA, and the rev line. + + Shape only. The installer's validator greps PEM blocks by text exactly like + `resolve_bundle` does (a monitored host has no `cryptography`), so a well-formed *fake* + is all an unprivileged validation run needs; the functional probe tests below use a + real bundle minted by `dashboards.certs`. + """ + + def block(kind: str, body: str) -> str: + return f"-----BEGIN {kind}-----\n{body}\n-----END {kind}-----\n" + + return ( + f"# BUNDLE-REV {revision}\n" + + block("CERTIFICATE", "AGENTCERT") + + block("PRIVATE KEY", "AGENTKEY") + + block("CERTIFICATE", "ROOTCA") + ) + + +def _installer_status(stdout: str) -> str: + """The line `status()` prints as `health: …`, parsed as JSON when it is JSON.""" + for line in stdout.splitlines(): + if line.startswith("health: "): + body = line.removeprefix("health: ").strip() + try: + return str(json.loads(body)["ok"]) + except (json.JSONDecodeError, KeyError): + return body + pytest.fail(f"the probe printed no health line:\n{stdout}") + + +@pytest.mark.parametrize( + ("shape", "reason"), + [ + ("missing", "cannot be read"), + ("empty", "is empty"), + ("no-revision", "BUNDLE-REV"), + ("two-keys", "the agent certificate, the agent key and the root CA"), + ], +) +def test_a_bad_cert_bundle_is_refused_with_its_reason_before_anything_else( + tmp_path: Path, shape: str, reason: str +) -> None: + """Completion criterion: `--cert-bundle /nonexistent` refuses with the reason, touches nothing. + + The validation runs *before* `need_root`, so the failure names the file rather than + sending the operator to add a sudoers line for a typo in the flag's argument — and + nothing after the validation (user, files, unit) can have run. + """ + if shape == "missing": + path = tmp_path / "absent.bundle" + elif shape == "empty": + path = tmp_path / "empty.bundle" + path.write_text("\n", encoding="ascii") + elif shape == "no-revision": + path = tmp_path / "revless.bundle" + path.write_text(_fake_bundle_text().replace("# BUNDLE-REV 3\n", ""), encoding="ascii") + else: # a bundle with the root *key* in it is exactly what the shape check refuses + path = tmp_path / "leaky.bundle" + extra = "-----BEGIN PRIVATE KEY-----\nROOTKEY\n-----END PRIVATE KEY-----\n" + path.write_text(_fake_bundle_text() + extra, encoding="ascii") + result = _installer_without_root("--cert-bundle", str(path)) + assert result.returncode == 2, "the bundle refusal keeps its own exit code (2)" + assert result.stdout == "", "a refusal goes to stderr, not stdout" + assert reason in result.stderr + assert str(path) in result.stderr, "the message names the file the operator passed" + assert GRANT_PREFIX not in result.stderr, "a bad bundle is not a sudoers problem" + + +def test_a_well_formed_bundle_passes_validation_and_reaches_the_root_check( + tmp_path: Path, +) -> None: + """Validation is shape-only and never dials: the next refusal is the usual sudoers one.""" + path = tmp_path / "agent.bundle" + path.write_text(_fake_bundle_text(), encoding="ascii") + result = _installer_without_root("--bind", "10.0.0.5", "--cert-bundle", str(path)) + assert result.returncode == 1 + assert _grant_lines(result.stderr), "the bundle validated; what remains is the root boundary" + + +def test_installer_help_documents_the_cert_bundle_and_its_path() -> None: + result = _run([BASH, str(INSTALLER), "--help"]) + assert result.returncode == 0 + assert "--cert-bundle" in result.stdout + assert CERT_FILE_PATH in result.stdout + + +def test_the_bundle_lands_at_the_token_file_idiom_in_the_script() -> None: + """`0640 root:$AGENT_USER` + chown: the agent opens the bundle itself and re-reads it.""" + text = INSTALLER.read_text() + assert f'CERT_FILE="{CERT_FILE_PATH}"' in text + assert 'install -m 0640 "$BUNDLE_SRC" "$CERT_FILE"' in text + assert 'chown root:"$AGENT_USER" "$CERT_FILE"' in text + # 0600 would fail the unit the way it would fail the token file: the service user opens + # the bundle at startup and re-reads the denylist live. Pin the reasoning, not just the mode. + assert "0600 root-owned file would fail the unit" in text + assert "re-reads it when its stamp changes" in text + + +def test_the_tls_unit_lines_come_from_one_bundle_branch() -> None: + """Flag + RuntimeDirectory are emitted together, or the unit is a token-mode unit.""" + text = INSTALLER.read_text() + match = re.search(r'\n if \[ -f "\$CERT_FILE" \]; then(.*?)\n fi\n', text, re.DOTALL) + assert match, "the TLS unit lines must come from one `if [ -f \"$CERT_FILE\" ]` branch" + block = match.group(1) + assert 'TLS_EXEC_ARGS=" --cert-bundle $CERT_FILE"' in block + assert "RuntimeDirectory=dashboards-agent" in block + assert "${TLS_EXEC_ARGS}" in text, "ExecStart must carry the flag from that branch" + + +def test_status_probes_tls_with_the_bundle_ca_and_no_client_cert() -> None: + """The status branch: unit says --cert-bundle → python3 ssl probe, never openssl, no cert.""" + status_body = INSTALLER.read_text().split("status() {", 1)[1].split("\n}", 1)[0] + assert "grep -q -- '--cert-bundle'" in status_body + assert "PROTOCOL_TLS_CLIENT" in status_body + assert "load_verify_locations" in status_body + assert "https://" in status_body + assert "load_cert_chain" not in status_body, "the installer holds no client certificate" + assert 'http://$BIND:$PORT/health' in status_body, "token mode keeps the plain probe" + assert "openssl" not in INSTALLER.read_text(), "no new binary dependency on a monitored host" + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_a_bundle_install_lands_the_file_the_unit_flag_and_the_runtime_directory( + tmp_path: Path, +) -> None: + """The install body with a bundle: byte-identical file at 0640, TLS unit, https hint.""" + bundle = tmp_path / "agent.bundle" + bundle.write_text(_fake_bundle_text(), encoding="ascii") + script, call_log, env = _sandbox_installer(tmp_path) + result = subprocess.run( + [BASH, str(script), "--bind", "10.0.0.5", "--port", "9120", "--cert-bundle", str(bundle)], + capture_output=True, + text=True, + env=env, + timeout=60, + check=False, + ) + assert result.returncode == 0, f"install failed:\n{result.stdout}\n{result.stderr}" + + cert_path = tmp_path / "target" / "certs" + assert cert_path.is_file(), "install -m never landed the bundle" + assert cert_path.read_text(encoding="ascii") == _fake_bundle_text(), ( + "the installed bundle must be byte-identical to the downloaded one" + ) + mode = stat.S_IMODE(cert_path.stat().st_mode) + assert mode == 0o640, f"bundle mode is {oct(mode)}, expected 0o640 (group = the agent user)" + assert not mode & 0o007, "the bundle holds the agent's key: never world-readable" + + unit = (tmp_path / "target" / "dashboards-agent.service").read_text() + assert f"--cert-bundle {cert_path}" in unit + assert "RuntimeDirectory=dashboards-agent" in unit, ( + "ProtectSystem=strict leaves the agent no writable file but this, and the bundle " + "unpacks its PEM blocks there" + ) + assert "--token-file" in unit, "the dual-mode line stays — rollout is host by host" + assert 'HOST_TARGETS="hostname=https://10.0.0.5:9120"' in result.stdout, ( + "a TLS install points the dashboard at https, with no token in the URL" + ) + + calls = call_log.read_text() + assert calls.count("chown root:dashboards-agent") == 2, "token and bundle both chowned" + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_an_unrelated_rerun_reuses_the_installed_bundle(tmp_path: Path) -> None: + """The token-reuse idiom: a re-run without the flag neither rewrites nor drops the bundle.""" + script, _call_log, env = _sandbox_installer(tmp_path) + target = tmp_path / "target" + target.mkdir(parents=True) + cert_path = target / "certs" + cert_path.write_text(_fake_bundle_text(revision="7"), encoding="ascii") + result = subprocess.run( + [BASH, str(script), "--bind", "10.0.0.5"], + capture_output=True, + text=True, + env=env, + timeout=60, + check=False, + ) + assert result.returncode == 0, f"re-run failed:\n{result.stdout}\n{result.stderr}" + assert cert_path.read_text(encoding="ascii") == _fake_bundle_text(revision="7") + assert "reusing the installed certificate bundle" in result.stdout + unit = (target / "dashboards-agent.service").read_text() + assert f"--cert-bundle {cert_path}" in unit, "an installed bundle keeps the unit in TLS mode" + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_uninstall_removes_the_bundle_with_the_token(tmp_path: Path) -> None: + """An uninstalled agent leaves no private key behind.""" + script, call_log, env = _sandbox_installer(tmp_path) + target = tmp_path / "target" + (target / "lib").mkdir(parents=True) + leftovers = { + target / "token": "tok\n", + target / "certs": _fake_bundle_text(), + target / "lib" / "host_agent.py": "# stub\n", + target / "dashboards-agent.service": "[Unit]\n", + } + for path, body in leftovers.items(): + path.write_text(body, encoding="ascii") + result = subprocess.run( + [BASH, str(script), "--uninstall"], + capture_output=True, + text=True, + env=env, + timeout=60, + check=False, + ) + assert result.returncode == 0, f"uninstall failed:\n{result.stdout}\n{result.stderr}" + for path in leftovers: + assert not path.exists(), f"uninstall left {path} on disk" + assert "systemctl disable --now dashboards-agent" in call_log.read_text() + + +# --- the --status probe, end to end against the real TLS agent ----------------------------- + + +class DictMeta: + """The three-method `certs.MetaStore` slice, backed by a dict (the `test_auth.py` idiom).""" + + def __init__(self) -> None: + self.rows: dict[str, str] = {} + + def get_meta(self, key: str) -> str | None: + return self.rows.get(key) + + def set_meta(self, key: str, value: str) -> None: + self.rows[key] = value + + def delete_meta(self, key: str) -> int: + return 1 if self.rows.pop(key, None) is not None else 0 + + +def _mint_bundle() -> str: + """A real bundle from the dashboard's own CA — the artifact the operator downloads.""" + store = DictMeta() + state = certs.ensure_certs(store) + state = certs.issue_agent_cert(store, state, "localhost", ["127.0.0.1", "localhost"]) + entry = state.agent("localhost") + assert entry is not None + return entry.bundle + + +def _serve_tls_agent(host_agent: Any, bundle_text: str, where: Path) -> tuple[Any, int]: + """The real agent handler on a TLS-wrapped loopback port, exactly as the unit runs it.""" + path = where / "installed.certs" + path.write_text(bundle_text, encoding="ascii") + bundle, error = host_agent.TlsBundle.load(str(path)) + assert error is None and bundle is not None + host_agent.Handler.collector = host_agent.Collector("/", smart=False) + host_agent.Handler.token = None + host_agent.Handler.bundle = bundle + httpd = ThreadingHTTPServer(("127.0.0.1", 0), host_agent.Handler) + httpd.socket = bundle.context.wrap_socket(httpd.socket, server_side=True) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + return httpd, httpd.server_address[1] + + +def _run_status(script: Path, port: int, env: dict[str, str]) -> str: + result = subprocess.run( + [BASH, str(script), "--status", "--bind", "127.0.0.1", "--port", str(port)], + capture_output=True, + text=True, + env=env, + timeout=60, + check=False, + ) + assert result.returncode == 0, f"--status exited {result.returncode}:\n{result.stderr}" + return result.stdout + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_the_tls_status_probe_reads_health_with_the_ca_and_no_client_cert( + tmp_path: Path, host_agent_module: Any +) -> None: + """The task-02 ASSUMPTION, proven through the installer: CERT_OPTIONAL + the /health + exemption let a probe that holds only the bundle's CA read /health on a TLS agent.""" + bundle = _mint_bundle() + httpd, port = _serve_tls_agent(host_agent_module, bundle, tmp_path) + try: + script, _call_log, env = _sandbox_installer(tmp_path / "installer") + target = tmp_path / "installer" / "target" + target.mkdir(parents=True) + (target / "dashboards-agent.service").write_text( + f"ExecStart=python3 host_agent.py --cert-bundle {target / 'certs'}\n", encoding="ascii" + ) + (target / "certs").write_text(bundle, encoding="ascii") + assert _installer_status(_run_status(script, port, env)) == "True" + finally: + httpd.shutdown() + httpd.server_close() + host_agent_module.Handler.bundle = None + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_the_tls_status_probe_verifies_the_chain_and_says_no_to_another_authority( + tmp_path: Path, host_agent_module: Any +) -> None: + """The probe loads the CA for a reason: a server signed by a different root is not health.""" + httpd, port = _serve_tls_agent(host_agent_module, _mint_bundle(), tmp_path) + try: + script, _call_log, env = _sandbox_installer(tmp_path / "installer") + target = tmp_path / "installer" / "target" + target.mkdir(parents=True) + (target / "dashboards-agent.service").write_text( + f"ExecStart=python3 host_agent.py --cert-bundle {target / 'certs'}\n", encoding="ascii" + ) + (target / "certs").write_text(_mint_bundle(), encoding="ascii") # a *different* CA + stdout = _run_status(script, port, env) + finally: + httpd.shutdown() + httpd.server_close() + host_agent_module.Handler.bundle = None + line = _installer_status(stdout) + assert line != "True" and line.startswith("(no answer"), f"the probe must refuse: {line}" + + +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_the_tls_status_probe_names_the_bundle_when_it_cannot_read_it(tmp_path: Path) -> None: + """The bundle is 0640 root:agent — an unprivileged --status cannot read it, and says so.""" + script, _call_log, env = _sandbox_installer(tmp_path) + target = tmp_path / "target" + target.mkdir(parents=True) + (target / "dashboards-agent.service").write_text( + f"ExecStart=python3 host_agent.py --cert-bundle {target / 'certs'}\n", encoding="ascii" + ) + cert = target / "certs" + cert.write_text(_fake_bundle_text(), encoding="ascii") + cert.chmod(0o000) + line = _installer_status(_run_status(script, 1, env)) + assert "cannot read" in line and "run --status as root" in line, line + + +@pytest.mark.skipif(shutil.which("curl") is None, reason="curl is not installed on this box") +@pytest.mark.skipif(hasattr(os, "geteuid") and os.geteuid() == 0, reason="running as root") +def test_a_token_mode_status_probe_stays_plain_http(tmp_path: Path) -> None: + """The other branch: no bundle in the unit → the unchanged plain probe, not https.""" + script, _call_log, env = _sandbox_installer(tmp_path) + target = tmp_path / "target" + target.mkdir(parents=True) + (target / "dashboards-agent.service").write_text( + "ExecStart=python3 host_agent.py --token-file /etc/dashboards-agent.token\n", + encoding="ascii", + ) + result = subprocess.run( + [BASH, str(script), "--status", "--bind", "127.0.0.1", "--port", "1"], + capture_output=True, + text=True, + env=env, + timeout=60, + check=False, + ) + assert result.returncode == 0 + assert _installer_status(result.stdout).startswith("(no answer"), ( + "token mode must probe plain http — nothing listens here, and the refusal is the point" + ) diff --git a/uv.lock b/uv.lock index 102d9f8..1b21d04 100644 --- a/uv.lock +++ b/uv.lock @@ -35,11 +35,70 @@ wheels = [ [[package]] name = "certifi" -version = "2026.7.22" +version = "2026.10.19" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz", hash = "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55", size = 138112, upload-time = "2026-07-22T03:35:12.644Z" } +sdist = { url = "https://files.pythonhosted.org/packages/a6/bd/1436c50cc46699de0230fbaacc1d1de7bfec413176099949e9d5112e88f2/certifi-2026.10.19.tar.gz", hash = "sha256:035c1e85572b141bf1c3a3eff0e33a8eb74917522f2559075ff8d2fea780a1ae", size = 137598, upload-time = "2026-10-20T02:24:16.604Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" }, + { url = "https://files.pythonhosted.org/packages/c4/51/29046e48f693c54e5b7412216d447a1d8362c9653148d98c02cdf65fa733/certifi-2026.10.19-py3-none-any.whl", hash = "sha256:42d35340cc5b9ba78e818ddbff0c5b1c2b779de448a340fd1b989f5fa8670e1b", size = 136464, upload-time = "2026-10-20T02:24:15.072Z" }, +] + +[[package]] +name = "cffi" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pycparser", marker = "implementation_name != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d3/7b/d6bbf82b8b96e7391438898c42f5bd96dd02030fd5b64937d248220003e2/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c", size = 194064, upload-time = "2026-08-03T21:20:17.148Z" }, + { url = "https://files.pythonhosted.org/packages/94/e6/bcc91b283be94735e268487a054004f0aa19947b6348fa367db53230abc8/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb", size = 196720, upload-time = "2026-08-03T21:20:18.268Z" }, + { url = "https://files.pythonhosted.org/packages/d9/99/c4b0c17cacdc9c3b8f280026286a9826d6a208c0f047591a3c3ce99b91fd/cffi-2.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54", size = 184964, upload-time = "2026-08-03T21:20:19.708Z" }, + { url = "https://files.pythonhosted.org/packages/b3/a9/9db617d05d7367c1ad0ab00b3aa6e6f9281edd689b4ee9ea0e5a84e89c97/cffi-2.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72", size = 184962, upload-time = "2026-08-03T21:20:20.833Z" }, + { url = "https://files.pythonhosted.org/packages/67/b8/b42132ca113dc567d37684437b46ca1dafc885902b02a110a02d5b511857/cffi-2.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1", size = 222328, upload-time = "2026-08-03T21:20:22.118Z" }, + { url = "https://files.pythonhosted.org/packages/80/10/c5c0cbf0a657aecf59ef511409734230bf556f05a0d6c9eed7aa5c0a0166/cffi-2.1.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062", size = 209985, upload-time = "2026-08-03T21:20:23.401Z" }, + { url = "https://files.pythonhosted.org/packages/d5/6c/bfa0b87b03b9238148beca990292843c9396ba069b54496596594173de7b/cffi-2.1.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03", size = 208530, upload-time = "2026-08-03T21:20:24.628Z" }, + { url = "https://files.pythonhosted.org/packages/e9/02/4e7d553a7ac4b4238b38b3c1b80d486e9d4436f8d2acbf87a0997fe3f402/cffi-2.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96", size = 221525, upload-time = "2026-08-03T21:20:25.758Z" }, + { url = "https://files.pythonhosted.org/packages/82/1d/a4aaf9babd75acb4d5f223bff71533bee748dd770a382619a798960ee9ba/cffi-2.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527", size = 225053, upload-time = "2026-08-03T21:20:26.985Z" }, + { url = "https://files.pythonhosted.org/packages/81/10/5dc0e7bdd18e22107054288283380fc97a06ae3f1656a106908d666a3c88/cffi-2.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13", size = 223213, upload-time = "2026-08-03T21:20:28.277Z" }, + { url = "https://files.pythonhosted.org/packages/0b/e9/d0061c364cde06ee43168a0d076ac1da512cbc380d44767b844ba34fe2b6/cffi-2.1.1-cp314-cp314-win32.whl", hash = "sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c", size = 177682, upload-time = "2026-08-03T21:20:44.288Z" }, + { url = "https://files.pythonhosted.org/packages/a7/06/1c3e01e3ba14c39f6d10bfbac52753b7e22259e38088e5cfe1d704918690/cffi-2.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48", size = 187949, upload-time = "2026-08-03T21:20:45.623Z" }, + { url = "https://files.pythonhosted.org/packages/87/5b/da4e39efe18eeb89cf580ea9cfc66b6a7c3eadb808fc0cc1d3a295cb5a5d/cffi-2.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836", size = 182947, upload-time = "2026-08-03T21:20:46.955Z" }, + { url = "https://files.pythonhosted.org/packages/23/59/40338bf421c5accea1d45158170c87006ef1cd371b05c077e76476949728/cffi-2.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3", size = 188504, upload-time = "2026-08-03T21:20:29.495Z" }, + { url = "https://files.pythonhosted.org/packages/7d/47/5ecf1023850036e674c77ec4de86182d309ae344e39e7cba984b7df5d647/cffi-2.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2", size = 188259, upload-time = "2026-08-03T21:20:31.291Z" }, + { url = "https://files.pythonhosted.org/packages/2a/9c/92934c3bea9f785b23eba304538c0b4d37a2a96d2431eb3a1bc87a11aa19/cffi-2.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94", size = 223864, upload-time = "2026-08-03T21:20:32.571Z" }, + { url = "https://files.pythonhosted.org/packages/4d/45/ba4c93527bc38616a8bd36488acb69a2212d60486794f0c1f318949bbb76/cffi-2.1.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc", size = 211538, upload-time = "2026-08-03T21:20:33.808Z" }, + { url = "https://files.pythonhosted.org/packages/80/e9/b6ef565e452acb932fb0cb5443f44a78efbd1233e566f02b5a83855e9115/cffi-2.1.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29", size = 210688, upload-time = "2026-08-03T21:20:34.974Z" }, + { url = "https://files.pythonhosted.org/packages/9a/95/eff5f0cee78d2eabc7eebffec40d3fc1876b5f3c95582e018bb4b99601f2/cffi-2.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676", size = 223803, upload-time = "2026-08-03T21:20:36.564Z" }, + { url = "https://files.pythonhosted.org/packages/fa/01/579d39fb8bef00a335a23d83757b44feb24cd6345a2c451b64cb67b9c362/cffi-2.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e", size = 226763, upload-time = "2026-08-03T21:20:37.816Z" }, + { url = "https://files.pythonhosted.org/packages/8d/b0/0b44f47c60b01b57b6e2bbd92343f13a85a1d93bc46ccf6e47e244acd99c/cffi-2.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f", size = 225688, upload-time = "2026-08-03T21:20:38.959Z" }, + { url = "https://files.pythonhosted.org/packages/eb/d2/3b7176cb570a1d3e27faf67b72f591af508036e0d8b2be2ef9af9e8c84bb/cffi-2.1.1-cp314-cp314t-win32.whl", hash = "sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4", size = 182868, upload-time = "2026-08-03T21:20:40.388Z" }, + { url = "https://files.pythonhosted.org/packages/56/78/31f00c1bcd97c9bbf55f1bfdf5bc809a5de8887473e90bb9960dca825e80/cffi-2.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e", size = 194104, upload-time = "2026-08-03T21:20:41.725Z" }, + { url = "https://files.pythonhosted.org/packages/7b/1b/58496f2ed0a35de575250c02a43ab3cc2c04d494a88fed31c1cabc0fd176/cffi-2.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5", size = 186402, upload-time = "2026-08-03T21:20:43.042Z" }, + { url = "https://files.pythonhosted.org/packages/c1/8f/9ebe220eab48a093d1a5a5e339ab0dc7316eef3bb04d63c42f0251b61f50/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d", size = 194043, upload-time = "2026-08-03T21:20:48.179Z" }, + { url = "https://files.pythonhosted.org/packages/ff/69/844bad3ece306c4782c2ecb93597035b6690d48704b803914c199da1e8b3/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b", size = 196737, upload-time = "2026-08-03T21:20:49.457Z" }, + { url = "https://files.pythonhosted.org/packages/1b/8a/af668013284634733f02d683458a0728739c7d6ddb5e14cb0c20832266fe/cffi-2.1.1-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4", size = 184933, upload-time = "2026-08-03T21:20:50.639Z" }, + { url = "https://files.pythonhosted.org/packages/0c/75/2f5207ff6d1a613133b23a5203cc0c2a628313b5eb3974d7956ae3c57950/cffi-2.1.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8", size = 185002, upload-time = "2026-08-03T21:20:52.173Z" }, + { url = "https://files.pythonhosted.org/packages/e2/31/9e1313b0a6e30e91b3b3d3fff51ae99c857c07738e3afcce1f7334e1b7ab/cffi-2.1.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6", size = 222271, upload-time = "2026-08-03T21:20:53.462Z" }, + { url = "https://files.pythonhosted.org/packages/50/e3/f6234a833e6e08c7007003074723c406559eecf9b48dfc97471e5a8eb7a0/cffi-2.1.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80", size = 209919, upload-time = "2026-08-03T21:20:54.783Z" }, + { url = "https://files.pythonhosted.org/packages/0d/fc/5f74e293fced6edb51af3a46c4ccf6c23c9943774ecb375ddbd522c76add/cffi-2.1.1-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779", size = 208529, upload-time = "2026-08-03T21:20:56.066Z" }, + { url = "https://files.pythonhosted.org/packages/44/16/29e6d01b388bef055ecd6ca8244b3f4d336bd09e92d5d892187b9601084e/cffi-2.1.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399", size = 221630, upload-time = "2026-08-03T21:20:57.336Z" }, + { url = "https://files.pythonhosted.org/packages/a4/18/fa7f1f6857d5eb88a4ca99ffcbfb7c387a287ccc154c64a73e86314745d7/cffi-2.1.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688", size = 225134, upload-time = "2026-08-03T21:20:58.675Z" }, + { url = "https://files.pythonhosted.org/packages/e0/9f/e8e3dfa04a1b4c241f8c91faacad872b4d4efd051d49764ad4e2fd4b9fea/cffi-2.1.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7", size = 223197, upload-time = "2026-08-03T21:20:59.968Z" }, + { url = "https://files.pythonhosted.org/packages/f8/7e/8debeb04f1ab9fe2a6963964cd6f1aaf7192627b83926586a6a4e089c9fa/cffi-2.1.1-cp315-cp315-win32.whl", hash = "sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac", size = 177683, upload-time = "2026-08-03T21:21:14.901Z" }, + { url = "https://files.pythonhosted.org/packages/e0/31/5158704cc474ab65c1647932e88be78dc0873f47130e253be38bcaf13d01/cffi-2.1.1-cp315-cp315-win_amd64.whl", hash = "sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960", size = 187897, upload-time = "2026-08-03T21:21:16.108Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4b/b3a2da8570c704ffc0f9762cdc3ec0f02c8573798e0b5cf7f11c82bbb70f/cffi-2.1.1-cp315-cp315-win_arm64.whl", hash = "sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1", size = 182935, upload-time = "2026-08-03T21:21:17.271Z" }, + { url = "https://files.pythonhosted.org/packages/d0/ef/5443574510a1207e6f6bc38ba6e1f1de36cb48fef07b2728bb896a21f430/cffi-2.1.1-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc", size = 188464, upload-time = "2026-08-03T21:21:01.163Z" }, + { url = "https://files.pythonhosted.org/packages/7e/ae/a56fa8c4686ad50e148fcbc8d3ae0d03915ff5c30d795058988c24118cef/cffi-2.1.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab", size = 188262, upload-time = "2026-08-03T21:21:02.382Z" }, + { url = "https://files.pythonhosted.org/packages/53/b2/6187f46f2912276a3ae284076109cc5c8680482f11f766ccf26db4a86427/cffi-2.1.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e", size = 223779, upload-time = "2026-08-03T21:21:03.553Z" }, + { url = "https://files.pythonhosted.org/packages/8a/f6/c3ad28bd19f77047a03084424fbd4cbe997303267c14423737324be0385d/cffi-2.1.1-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358", size = 211520, upload-time = "2026-08-03T21:21:04.863Z" }, + { url = "https://files.pythonhosted.org/packages/a0/cd/ccac9013a5bd9fd764de118674ab9c805b5ca10c19270d90ee273f8b2240/cffi-2.1.1-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231", size = 210673, upload-time = "2026-08-03T21:21:06.223Z" }, + { url = "https://files.pythonhosted.org/packages/52/86/2976131c639aead931c5bee5aba67e4b09fbeb8018b6f282f70803f923a7/cffi-2.1.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6", size = 223835, upload-time = "2026-08-03T21:21:07.539Z" }, + { url = "https://files.pythonhosted.org/packages/ac/0c/33a7aeab2f9c76918c52e084beb39c570db3588133412929e8ec06fab90b/cffi-2.1.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94", size = 226705, upload-time = "2026-08-03T21:21:08.774Z" }, + { url = "https://files.pythonhosted.org/packages/e3/26/2cde30fdde421130bfc18f70395731a6e6b2053c6a1978a5258ff04e72fa/cffi-2.1.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5", size = 225539, upload-time = "2026-08-03T21:21:09.911Z" }, + { url = "https://files.pythonhosted.org/packages/6d/cd/a361394c94b2129d604bb846f624a8e88255a3ee33129c434a00d715e64f/cffi-2.1.1-cp315-cp315t-win32.whl", hash = "sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66", size = 182707, upload-time = "2026-08-03T21:21:11.226Z" }, + { url = "https://files.pythonhosted.org/packages/9b/b5/ba2b299993c26577d529b6ae29841f9e15b9fcf004d65f423f4fcf94ade9/cffi-2.1.1-cp315-cp315t-win_amd64.whl", hash = "sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3", size = 193772, upload-time = "2026-08-03T21:21:12.39Z" }, + { url = "https://files.pythonhosted.org/packages/aa/29/35e016098c814cd93de9cd320c66b5bfba14dc6ecedd3cb518fa7c408c69/cffi-2.1.1-cp315-cp315t-win_arm64.whl", hash = "sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692", size = 186360, upload-time = "2026-08-03T21:21:13.636Z" }, ] [[package]] @@ -129,11 +188,75 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/3f/0c/7a64e1ac90541a8edf50daef0914848011fb057a5bf55284a4811e21939a/coverage-7.16.2-py3-none-any.whl", hash = "sha256:11d28e9123a9156cb405d8d27b44256c9a58fb5decc2073a8f17862057e3aa0f", size = 215754, upload-time = "2026-09-27T12:28:59.075Z" }, ] +[[package]] +name = "cryptography" +version = "50.0.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "platform_python_implementation != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9d/af/182eb91b0df3fe75c4d9f26fe70684569566745f6ba7e5c9c73a862c5252/cryptography-50.0.2.tar.gz", hash = "sha256:7b46165bb56eb4704e2eaaf86f3c940d19154535d9b0ca7d6d590b04060e00d5", size = 880623, upload-time = "2026-09-30T15:30:04.884Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e5/56/d194340cc4a57535e82e1bee9e89667ac4b7c13b5d3f59686deae3094dd5/cryptography-50.0.2-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:fa8f5efb344d6908a1ce62f4a24e2e5780f825d6f53f5f50ec5ffacac72936cb", size = 3914904, upload-time = "2026-09-30T14:43:44.339Z" }, + { url = "https://files.pythonhosted.org/packages/d9/69/c9bd862c3bf43d6399c433caf002df16e2dffd4be49bdf515cda38038711/cryptography-50.0.2-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:79def8d059362e7831389ed3be0ecdf58a89386e1271e35dd9f5af84e81bffd0", size = 4731146, upload-time = "2026-09-30T14:43:47.113Z" }, + { url = "https://files.pythonhosted.org/packages/21/69/64cef1f702bf6657e0cc186ed1a2891d50d29fb41586b254e1c07adea261/cryptography-50.0.2-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:630ebfea3bf689d075f82316324ff7433dc447fe6bc1bfc76524b74b4a9567d2", size = 4719841, upload-time = "2026-09-30T14:43:49.01Z" }, + { url = "https://files.pythonhosted.org/packages/38/6b/61a3f8d8c5e1e49a6cddccafc4015cc1c0021360ab0acb4080e7a423644a/cryptography-50.0.2-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f9f6143a8c75945eb960d9eb98905a441394abfa24afaae239d514ffb2586480", size = 4738340, upload-time = "2026-09-30T14:43:50.932Z" }, + { url = "https://files.pythonhosted.org/packages/7b/2e/7212ca32fd43dc91f2f41db20160b268098874b4c9a0e7be94d6835f5b2e/cryptography-50.0.2-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:a582ab2ae1d34f67112cadc86702774c9ea4374df6bca6afe672817203c99134", size = 5367029, upload-time = "2026-09-30T14:43:52.911Z" }, + { url = "https://files.pythonhosted.org/packages/1a/f1/b474e930c4d910328780e3940da76f5aa5cbc48ce1fc14e44d239d9ea9db/cryptography-50.0.2-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:4061c0079120205fb760c58acab6443e217307dcf05e3702cf970e0689972856", size = 4753050, upload-time = "2026-09-30T14:43:55.272Z" }, + { url = "https://files.pythonhosted.org/packages/7c/52/9af10e80ac16b0fcc2123f9cbd5e7afbd0fd5075bb7a607c592258a39cda/cryptography-50.0.2-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:ac9ed99d81760c62fe89d5f0815cdfa1ba9a35141cf30f1c2d044f04b4803d2e", size = 4376724, upload-time = "2026-09-30T14:43:57.24Z" }, + { url = "https://files.pythonhosted.org/packages/71/37/6202e488cc1eb625ea110c292c6bda92823176e023f427d8d5660ce8d632/cryptography-50.0.2-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:87e9ce85beb6b328ba370cc6e6aea483c92617b4c95b1d33a49297eb662bfb04", size = 4737859, upload-time = "2026-09-30T14:43:59.541Z" }, + { url = "https://files.pythonhosted.org/packages/8f/30/e86d7d518489b0ae2497091a35287abcb1a2ce4037837a34afbe9b1d6964/cryptography-50.0.2-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:f265528741e048bce55c3463ed721fb0aa45a5888d8add8cfeccb3035451bbdc", size = 5324103, upload-time = "2026-09-30T14:44:01.901Z" }, + { url = "https://files.pythonhosted.org/packages/d3/69/2c833a049475e0a3444e94c7d0aca0aa51d166374a449b09e92ac98138de/cryptography-50.0.2-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:9dab55f57c74c3cad24c323bacbbd04be4705ba6eb0d92e920b1fc4837ed5079", size = 4752576, upload-time = "2026-09-30T14:44:04.545Z" }, + { url = "https://files.pythonhosted.org/packages/6c/5d/906970b83bbfc1f5bbfb677a143c181f2801f23b6a7204a3b47c42c97e65/cryptography-50.0.2-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:25784ce8b9621c90c643efb9e1e2162ab3b0224cae446ad5e70e7fcb1ce18b51", size = 4870819, upload-time = "2026-09-30T14:44:06.884Z" }, + { url = "https://files.pythonhosted.org/packages/68/e3/f2298d3bb55e0c4a91841ec4d01b3f020ba8c5fbf15ccdcc6dcf03f97025/cryptography-50.0.2-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:85d0d9a31b9098e98534226d5686b47264b95e62ce459dc2e62fdfc809f9fe93", size = 5030152, upload-time = "2026-09-30T14:44:09.443Z" }, + { url = "https://files.pythonhosted.org/packages/9a/4f/adfc442765721292fff86d314ce385d3249d22db42295c0dd057727b60f3/cryptography-50.0.2-cp311-abi3-win_amd64.whl", hash = "sha256:7afa5a6602a9f29af1f3a2965f831bae7c9d5d597b7cbb716d41ab3b7d89879c", size = 3824692, upload-time = "2026-09-30T14:44:11.671Z" }, + { url = "https://files.pythonhosted.org/packages/ce/cb/52eb3770c0d0be2702a98c6e96065ddc0a2877cf0845aa9c23397c142cd4/cryptography-50.0.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f785f6161f202ab04d8ca194158968798e480ca058943907972da5f12e2881e8", size = 3892731, upload-time = "2026-09-30T14:44:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/19/8e/aa1fc533d4546b127b45de8aa024eb5933d23eff9debfe25931e56861095/cryptography-50.0.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:0ecbc5652bdb6fc9eaf89a7d196e20941adfe812f43bc4ca05d9150496821047", size = 4710431, upload-time = "2026-09-30T14:44:15.427Z" }, + { url = "https://files.pythonhosted.org/packages/6a/64/72bc3f75176e7e406b748a3e3830432b8c51297b38368713df04dc04898a/cryptography-50.0.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ab50ee449bf968271e820086f10a33d101dd060370abc10bcd22279be2656539", size = 4694824, upload-time = "2026-09-30T14:44:17.69Z" }, + { url = "https://files.pythonhosted.org/packages/4e/c6/62c77550edfa5ca3f14bf44a1e6739b9fa09d6e998a11d97ed8213bccc98/cryptography-50.0.2-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:a9f7355e6fab51f6c369b86fb7571cffa05edee2c2121e0380a37fb9ac1cd5c1", size = 4716967, upload-time = "2026-09-30T14:44:19.661Z" }, + { url = "https://files.pythonhosted.org/packages/f4/37/cce70f150c432914460157a6ecc161752e053aa5ec0ef3b3f7dc6e31039a/cryptography-50.0.2-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:94e5e9f108ee10471288214d3d233fbfbb492840a8457eb85178d643ddeb32c7", size = 5328676, upload-time = "2026-09-30T14:44:21.744Z" }, + { url = "https://files.pythonhosted.org/packages/aa/9a/6f2f0304d634ceafdeaf23e84537336664ac419b5d07611675c2ad3f6b7a/cryptography-50.0.2-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:241449bf940a5d27309bd317e6f9a2af6932113818bb2b8f5c59ddc7ef16da18", size = 4727698, upload-time = "2026-09-30T14:44:24.178Z" }, + { url = "https://files.pythonhosted.org/packages/1d/de/66bcf9244d118663b2e1aaded8990f4640e3d7b7411870a5765f252074d2/cryptography-50.0.2-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:d8947001be83df1394050758ce0e745dd74fb134eef0a4b5124208dfc3a68c37", size = 4354821, upload-time = "2026-09-30T14:44:26.263Z" }, + { url = "https://files.pythonhosted.org/packages/bd/e6/db28a28c7b6c676addce89136de3d8db49ea825a8c863472e36e42ead4ad/cryptography-50.0.2-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:4a20ce1e5cb4284a86692fdcba7cb8754185c6b2e5c56fcef3751cf451d3cdc2", size = 4716748, upload-time = "2026-09-30T14:44:28.447Z" }, + { url = "https://files.pythonhosted.org/packages/30/96/01546c7f69ea0e2ab790a2e4f0934a4052fb9b388147fbf83c2fd72f1e57/cryptography-50.0.2-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:84f964e537f916e2cc85199e5a88742e964939b575ac8598b3f9d6cc416cdaf1", size = 5285085, upload-time = "2026-09-30T14:44:30.704Z" }, + { url = "https://files.pythonhosted.org/packages/6c/01/03263395f74d50b071e9e66daace3f8bef80493e5d410726f2ba8554736b/cryptography-50.0.2-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:828d49b0ff5a0e3975865571c5d91dbbdd0d38d8289b249a163e9425413a5e05", size = 4727268, upload-time = "2026-09-30T14:44:32.92Z" }, + { url = "https://files.pythonhosted.org/packages/eb/94/2bfe8f29ec0cc9c0d99359c4161adf32858e4934b72c6d100d2ac0bbe962/cryptography-50.0.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:deb9fde5c60e437ee4821bc9bc39ff31b42135c27e1dc61ef0a629389c1de62e", size = 4849503, upload-time = "2026-09-30T14:44:34.969Z" }, + { url = "https://files.pythonhosted.org/packages/54/44/e80651ecbf0e42b62e2bb5f5768916e07eea72e1297338956a61df361f88/cryptography-50.0.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:8c71ba2cd31fc93748c38e1b613200ff1c2665cbfd5341fe3a61cfde35a1430e", size = 5004057, upload-time = "2026-09-30T14:44:37.064Z" }, + { url = "https://files.pythonhosted.org/packages/f8/cc/1d33befb3cd7ea7e77d2d73f43f2066471da1b21f24a6156efcaabf6d2e8/cryptography-50.0.2-cp314-cp314t-win_amd64.whl", hash = "sha256:78198641e5be9521beea5aa782bb551a58068d10e6eb04c9c680c1b69f2e7d45", size = 3795868, upload-time = "2026-09-30T14:44:39.71Z" }, + { url = "https://files.pythonhosted.org/packages/2d/49/93f6a6e7a87c9aa68d44d3e1cdb5fe8f60c90d5d2f46acae9a56892816b8/cryptography-50.0.2-cp315-abi3.abi3t-macosx_11_0_arm64.whl", hash = "sha256:edc3342adf8f697fc5f59c887a304356f147b397809440ed64e2fa6af2f50f37", size = 4133708, upload-time = "2026-09-30T14:44:41.807Z" }, + { url = "https://files.pythonhosted.org/packages/8c/75/32ac2a56243d778805c16ca6a32b8f74fb757df7e28d7ecb560afafb59cf/cryptography-50.0.2-cp315-abi3.abi3t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d370b8d1dfcdf7130178137f6fbee6140774a1acc6cacefc4b42643ec11d0a3a", size = 4956267, upload-time = "2026-09-30T14:44:43.693Z" }, + { url = "https://files.pythonhosted.org/packages/aa/a4/2c8d734e43d97f0842ee9f1b7b4bfb3d0cf5e19edebf43c2afe6675c2320/cryptography-50.0.2-cp315-abi3.abi3t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f2f9bd7f90c64fe89253f0a2c05e3c4856072660429ce8831b4235bf29403a67", size = 4966465, upload-time = "2026-09-30T14:44:45.769Z" }, + { url = "https://files.pythonhosted.org/packages/c2/58/ee288c829a6f41f6235ae9dd33d82fd19b45442b65b4c8a3da36963d9f7a/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_aarch64.whl", hash = "sha256:e275096ea1e60cc595cda2836fd4a6c725d1125108b868be17f53684d164e2cc", size = 4959356, upload-time = "2026-09-30T14:44:48.211Z" }, + { url = "https://files.pythonhosted.org/packages/92/20/9ded6d51ddd9897f6b6e81fb9ebea7951d7cc5d6c890b0ed8abf77a51a80/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_ppc64le.whl", hash = "sha256:b13478603dcd0a2479ff8e87e2c19a7d525734686fe3c49542472293a204212d", size = 5548822, upload-time = "2026-09-30T14:44:50.86Z" }, + { url = "https://files.pythonhosted.org/packages/02/a8/8df951850d6b31d2a00218f19e2b3f999523437ed7a819df7fa427942fca/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_28_x86_64.whl", hash = "sha256:58a0c478eeca76fe5e07993c5a0703def34a6dc6a0cda4f5564639b33112ffe7", size = 5001199, upload-time = "2026-09-30T14:44:53.379Z" }, + { url = "https://files.pythonhosted.org/packages/8b/f9/36b3022218ce75b7cdf068fb95f809f9bd0d820e4955ef43b90c255cc7ac/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_31_armv7l.whl", hash = "sha256:d38cdff612d06fa6a32840d5e1b1f7a27cee4a349aa9085d94a67789d6bfd408", size = 4629333, upload-time = "2026-09-30T14:44:55.635Z" }, + { url = "https://files.pythonhosted.org/packages/8c/72/20f99a219f6af47cdd1cbd978c243b92d71496e168a746138af44ded4f29/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_aarch64.whl", hash = "sha256:fdd28f912fccfec1846a94e2e1e8f9b0012f557f0c46fe4f3eb0d7a87afcf90b", size = 4958822, upload-time = "2026-09-30T14:44:59.639Z" }, + { url = "https://files.pythonhosted.org/packages/f2/20/196f112617fb08eb4d608a2a6c422373d46f9cc2857f38fc0667033c0899/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_ppc64le.whl", hash = "sha256:cbc8738fd8526d80f35cb3a40d41f41a2e7030bb3b18b09a6778ef63d291c2fd", size = 5506351, upload-time = "2026-09-30T14:45:02.267Z" }, + { url = "https://files.pythonhosted.org/packages/24/95/83378121ef3eaaaf71d4b781577ff794acb39b9e1b87a3f156898c8497ed/cryptography-50.0.2-cp315-abi3.abi3t-manylinux_2_34_x86_64.whl", hash = "sha256:e105ab60406787da31fccc883fc0f733af1efd78f0136a4599692c4083a73d0c", size = 5000859, upload-time = "2026-09-30T14:45:05.009Z" }, + { url = "https://files.pythonhosted.org/packages/22/f7/70fd7ae4d1dbfa7ba29b02e1b9068771519a86027756510b700ce81086a8/cryptography-50.0.2-cp315-abi3.abi3t-musllinux_1_2_aarch64.whl", hash = "sha256:6f8700550aa1474a91e5dc07049c46f98b423b5b1ddd0483e0b51362eeeaf5be", size = 5092151, upload-time = "2026-09-30T15:29:15.932Z" }, + { url = "https://files.pythonhosted.org/packages/d4/be/688367b74de86984bd58d8efacfc7c9e68b89a6a22ced0fb4f38db50254a/cryptography-50.0.2-cp315-abi3.abi3t-musllinux_1_2_x86_64.whl", hash = "sha256:c71be1cbfa5cd9a41ee452acf1eccd82b2c05950358b106ec8ceb83411d1a020", size = 5286120, upload-time = "2026-09-30T15:29:18.309Z" }, + { url = "https://files.pythonhosted.org/packages/39/d1/55f8a3f2ef5d1529e16835ef10cf0fe3d559ce237b46dddc440c0bba3649/cryptography-50.0.2-cp315-abi3.abi3t-win_amd64.whl", hash = "sha256:c423ab384a46c4dff7217b2ea5ba2e11cffdeab6441acd04cf65a369caf0366c", size = 4111557, upload-time = "2026-09-30T15:29:20.155Z" }, + { url = "https://files.pythonhosted.org/packages/23/ad/ac987755d00e1e64273760228d2635ae38dae2be83e3c6e0d3289d91dec3/cryptography-50.0.2-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:0ec5f09541743261e66e291b4a0cbf0fb2997aeaab6d9e9c740b9dba1b58d1c2", size = 3943588, upload-time = "2026-09-30T15:29:22.265Z" }, + { url = "https://files.pythonhosted.org/packages/d5/8d/6d585339bedf85d45044c85d8412dac53f2bb6f918e8b7777efba1787844/cryptography-50.0.2-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c5e67125c7dca78d199ec4e116aa93dbb83494808ecbb8211a2cb09b1bf41dbd", size = 4756166, upload-time = "2026-09-30T15:29:24.58Z" }, + { url = "https://files.pythonhosted.org/packages/bf/f1/1c1f6874e8550cfddd4b688ceb38cefb6ed15ceed224d56f133f3d88c214/cryptography-50.0.2-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ee247f5c245c9a2fe7c8e2214e295918838e44e00a45a6718451e4004219e767", size = 4749145, upload-time = "2026-09-30T15:29:26.807Z" }, + { url = "https://files.pythonhosted.org/packages/c1/63/61b15dc1a8de03fe0adbe3fd7608b3ad5c73bf50993bbcb1faaa930afe33/cryptography-50.0.2-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:dfe9763530994147d9af1def057a5b9658b00e8f8fe8743d144d1e0911c2e454", size = 4763638, upload-time = "2026-09-30T15:29:28.588Z" }, + { url = "https://files.pythonhosted.org/packages/fc/35/b345bdfa40c9126df1a9d33236aa98418367931b8725f84fc3ae2b98dc59/cryptography-50.0.2-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:58ddb5a8e3179d12f19e4ea34d2d32e9d63a4baa142c875c1eb59f41b7243acd", size = 5382217, upload-time = "2026-09-30T15:29:30.589Z" }, + { url = "https://files.pythonhosted.org/packages/4f/87/ef344a9e616871f2519c22d6afcda79ddd5d35e9592d95eb6e677608d055/cryptography-50.0.2-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:f21e8a22c8605750c7af886bab299a363721264061b4ac0a30efb73cfd58efc5", size = 4781387, upload-time = "2026-09-30T15:29:32.605Z" }, + { url = "https://files.pythonhosted.org/packages/90/5b/f2fdb13cd0b96f6f932c8627bb292a45f11c64d21620a8e120aee9a3b848/cryptography-50.0.2-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:9c8402a82ea0dc4ceeab793db05f0fafa8ca139ca34fcde5df0f596103c74107", size = 4403790, upload-time = "2026-09-30T15:29:34.374Z" }, + { url = "https://files.pythonhosted.org/packages/bc/ce/7e4f662b1e3c393513569e402cfc85ac7da0bd3d5435e122a3140219eb2d/cryptography-50.0.2-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:0ddc924c04591c2811ca024d62ecad4f7f6f08af8939c211438f48a16bd23602", size = 4764319, upload-time = "2026-09-30T15:29:36.149Z" }, + { url = "https://files.pythonhosted.org/packages/3c/3f/86ff33ce34cc0de6847fb96e035a1a760d81652e38643f617c02ad32ef7a/cryptography-50.0.2-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:a6557e5f38e065ca9fbdaf7cfc7435ecb1d113aa81a022d1b51921ee7432e227", size = 5338560, upload-time = "2026-09-30T15:29:39.053Z" }, + { url = "https://files.pythonhosted.org/packages/40/cf/6b5c8e2fd9202d98988ab7cb5cc5c991704c4ad55f492ff408e4969f83f1/cryptography-50.0.2-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:1981f1db4630889b9ef7803fadef12b056f428cb6b85c27ba57b774793b6093c", size = 4780973, upload-time = "2026-09-30T15:29:41.251Z" }, + { url = "https://files.pythonhosted.org/packages/10/bf/8d6ebc7dded797bd0f0160d52188021211f011a2b164ef0ae1dac4587465/cryptography-50.0.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:7a8701d6b584d76e909e3d305b7d126b41439876a5aaf76cddc67fc230eafa2e", size = 4897738, upload-time = "2026-09-30T15:29:43.106Z" }, + { url = "https://files.pythonhosted.org/packages/d4/aa/f3f6e0de7e6253b8baa8b2d8fb9d50924fa75cee3d4624bd4bc1208ee923/cryptography-50.0.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:ce47f66801c20ec6c6632453bb5960fe38939e9306970b48b3a5a26de7745d94", size = 5058280, upload-time = "2026-09-30T15:29:44.827Z" }, + { url = "https://files.pythonhosted.org/packages/f6/b6/a1faf3a27ae9405fb34b1713cc73b2d8a26b04d5c561578fa2e6ef3e5bb9/cryptography-50.0.2-cp39-abi3-win_amd64.whl", hash = "sha256:4e81d95e5bafc2d6e34e4bed780e53e4d5b9a2f928573428aa4d35fbec1eb0de", size = 3854095, upload-time = "2026-09-30T15:29:46.782Z" }, +] + [[package]] name = "dashboards" version = "0.1.0" source = { editable = "." } dependencies = [ + { name = "cryptography" }, { name = "fastapi" }, { name = "httpx" }, { name = "pydantic" }, @@ -153,6 +276,7 @@ dev = [ [package.metadata] requires-dist = [ + { name = "cryptography", specifier = ">=50.0.2" }, { name = "fastapi", specifier = ">=0.143.3" }, { name = "httpx", specifier = ">=0.28.1" }, { name = "pydantic", specifier = ">=2.14.1" }, @@ -189,7 +313,7 @@ wheels = [ [[package]] name = "fastapi" -version = "0.143.5" +version = "0.153.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "annotated-doc" }, @@ -199,9 +323,9 @@ dependencies = [ { name = "typing-extensions" }, { name = "typing-inspection" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/8f/6a/ac6ec87026038fb7583018e0905f6c9d55bc8ee07d30d44c743f18b19435/fastapi-0.143.5.tar.gz", hash = "sha256:0b5c8e52d243cd2f015376b41e78db5bee585578782481abd145e1ab5a62a3ba", size = 470247, upload-time = "2026-10-18T09:12:55.556Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0a/69/8461d4c25490b5efe65b2178c857273785bd8b42a5555264b6b4f9bf2ed0/fastapi-0.153.0.tar.gz", hash = "sha256:5dbf5dcf6cf86388326c147566077c56a14c55b0f1bef370195da98e724a503f", size = 489663, upload-time = "2026-10-20T13:47:40.029Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/eb/4e/5f7d1268900eb14a6697a42d94c8f7e3b8450d27301761f231672f928419/fastapi-0.143.5-py3-none-any.whl", hash = "sha256:09cc38ed8c6a95f407783ef4d98ceeb350e192fe8fbb8ed711070bcd5208fdfa", size = 144894, upload-time = "2026-10-18T09:12:54.125Z" }, + { url = "https://files.pythonhosted.org/packages/93/41/c2c2c10be9068304a522dd64893916b5b21549af084ea942c26388aaa5b0/fastapi-0.153.0-py3-none-any.whl", hash = "sha256:77fbc9f41d2e5cd81f0c0a20482e627f8ebbb9b9005ac7586e415ee69dfea4ec", size = 146107, upload-time = "2026-10-20T13:47:38.295Z" }, ] [[package]] @@ -358,6 +482,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/40/9e/2b38731e0fc536806f16490e1a12d7f0dc2a1235aa8cc07bcc75416a7daa/pluggy-1.7.0-py3-none-any.whl", hash = "sha256:7dd7b0d8832ba3cb632c306926ded123429211b83641b35dc5c41ad2d34f9bec", size = 27082, upload-time = "2026-10-15T09:50:56.808Z" }, ] +[[package]] +name = "pycparser" +version = "3.11" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/da/a8/c5fdbeee588bb8ada9458774f43adf1bdd30bd59157055142183e769a024/pycparser-3.11.tar.gz", hash = "sha256:d875f09c3507d00e1aba0eecc6dcadc1352f30fff09dc6bff2f1c2935e97c2bc", size = 113796, upload-time = "2026-10-09T12:56:59.539Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/90/11/0e6f11117525ff0eec40ebac3d313376f102df93ca44ad9e893ee85e4f89/pycparser-3.11-py3-none-any.whl", hash = "sha256:51d5a8ba2be0bbe440b99d2112604c95bbbc3c2748a64260186c541e1729cd80", size = 51178, upload-time = "2026-10-09T12:56:58.131Z" }, +] + [[package]] name = "pydantic" version = "2.14.1" @@ -571,14 +704,14 @@ wheels = [ [[package]] name = "starlette" -version = "1.8.0" +version = "1.9.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anyio" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/e9/0c/6efb252d091ecccd7d62048ae11f0ea35cd75a4fbaeea5e30f9c3bf91d10/starlette-1.8.0.tar.gz", hash = "sha256:1565dc0b35d5737a271ed1e0e04e949f4e81198799f216d2667b0a0fb9cf9522", size = 2730457, upload-time = "2026-10-13T07:54:39.53Z" } +sdist = { url = "https://files.pythonhosted.org/packages/ad/eb/4e197bf26b023abafc5e9043ec8710d7de0fcb032d0969045236d72aa0b1/starlette-1.9.0.tar.gz", hash = "sha256:3d931de246ed870e35ce605974440816ef895e592f255d1396cfdf191d4c4318", size = 2732466, upload-time = "2026-10-18T15:50:33.189Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c1/b0/5742e4ac7af5eb58ec3470a537a49d7aa507e5539413e504b3a65ef50ba8/starlette-1.8.0-py3-none-any.whl", hash = "sha256:dfdd6b29c26483288088d990eee59631dedadd66ce20d203402a7ca8e3c4656f", size = 79612, upload-time = "2026-10-13T07:54:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/f3/93/181df62699d9b5aca1d87e5cfda131dac87e3cdaebf668662f7206ca3470/starlette-1.9.0-py3-none-any.whl", hash = "sha256:fd3f159936e882d06c1108f84688f7cda2cf85fc6a592a50edbd175e882c2ddc", size = 80335, upload-time = "2026-10-18T15:50:31.526Z" }, ] [[package]] @@ -698,89 +831,89 @@ wheels = [ [[package]] name = "websockets" -version = "17.2" +version = "17.3" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/01/89/3f825ab71c242fffb62ea8fe638741c290f62f8d7aadf8125ff897747af3/websockets-17.2.tar.gz", hash = "sha256:36c2fb94c990cc2545143b12690e2de6c16300f9dbe5b4f33fa300cf57dc8792", size = 188355, upload-time = "2026-10-03T14:56:53.5Z" } +sdist = { url = "https://files.pythonhosted.org/packages/30/4b/4692be6d5bf739ed049b3813901c42091f8c054df89d5c089343823dea0d/websockets-17.3.tar.gz", hash = "sha256:1c7dd93a1c9f89297471493674b1fc1d9467cafd45374f2172213046bdb3a329", size = 188878, upload-time = "2026-10-19T16:58:05.813Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8b/74/6bc991a28ac983600e65de408ebd1b1413d554ed0468ae5c831bc52dded6/websockets-17.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:ecb748910e9ba4624ebe2057791df51dcbffb48c37108ab94a3c593472023c9e", size = 217791, upload-time = "2026-10-03T14:54:26.381Z" }, - { url = "https://files.pythonhosted.org/packages/cb/2f/158e99426be6e71d09520bae53f29294fbb614b2fc5fbf8867b1d08395a7/websockets-17.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2ab9af5cb7265899e659f079eb71691375a1025b6d5fbd3caa495dd08f70833a", size = 215486, upload-time = "2026-10-03T14:54:27.962Z" }, - { url = "https://files.pythonhosted.org/packages/5c/09/1abf942723c0001d9c2fca1551907dade6304517b982b0bf10bba107fa81/websockets-17.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:06e46da092bca3a52e98f0458c66b247993ce501a07cd09c858be3296511ab7d", size = 215699, upload-time = "2026-10-03T14:54:29.523Z" }, - { url = "https://files.pythonhosted.org/packages/a7/1d/1ade03963ef497c47e6bad79e24370827b2fe6145fa8f58070ff2b7dcbac/websockets-17.2-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:fcce735ffd72ac4056db05325d9f0232382b74826f0196eb6a15ca903abdaa0f", size = 225081, upload-time = "2026-10-03T14:54:31.278Z" }, - { url = "https://files.pythonhosted.org/packages/9f/fd/47b8a0361c49da939b976a07b27a72a9f893d01dfcf4d2a28b53419ce1ef/websockets-17.2-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:42cbca10f82a8b2fb1536e8a0830ca6ceeb6bb3d8d64b766e0795369135654a8", size = 225430, upload-time = "2026-10-03T14:54:32.917Z" }, - { url = "https://files.pythonhosted.org/packages/f0/26/f4d4c76264ee037c5556ab5f50fcba302746dabf7528955534e4dda9965e/websockets-17.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c63ff5a21f26bd0e6a8464b53fadbe174825c8718ac14180df45665eaacdb6af", size = 226676, upload-time = "2026-10-03T14:54:34.833Z" }, - { url = "https://files.pythonhosted.org/packages/37/b3/c8b1c981322a050c4babfd327ffc9880f9c3834f5b15d2574e37eeb8768c/websockets-17.2-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:63f543463601c1558b755f8dd7618b6ec3dd0934dda051d3b7030d8c76e54de2", size = 228048, upload-time = "2026-10-03T14:54:36.424Z" }, - { url = "https://files.pythonhosted.org/packages/f0/5a/1cb29ddb23e6bc27ffd1c5316cd3616360d1ba0c3854eaa134ee3207bd28/websockets-17.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4c32eb565ad9ce8a6444248e5b7a19dbb86a81c811fe5fcc2fba7a735aed5163", size = 227281, upload-time = "2026-10-03T14:54:38.01Z" }, - { url = "https://files.pythonhosted.org/packages/ba/64/135274572dc0c845fc1111e2b932c807c395daac75d6eae6cfa148d8a208/websockets-17.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5d459bbb6c22f26dcebea56924a362aba50d453b9867912862c970434fcf0d94", size = 226025, upload-time = "2026-10-03T14:54:39.613Z" }, - { url = "https://files.pythonhosted.org/packages/58/75/f1e386aec3124489411caf5138cdd5a2bc43d3fd4a681c69adcf5f6272a5/websockets-17.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f19ca1a21871f024e38faf4107b433047df27558dff1b72a1dac31481e2c1fe5", size = 223277, upload-time = "2026-10-03T14:54:41.165Z" }, - { url = "https://files.pythonhosted.org/packages/60/eb/24733a0f568c2eb99e60f9faa620a98fb228c06a01e7e2f348b33290ed9c/websockets-17.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c76b4bcbf0f713194591673fc86a42820e14da6bbd1bb445d3d002cc4d1e4521", size = 226148, upload-time = "2026-10-03T14:54:42.779Z" }, - { url = "https://files.pythonhosted.org/packages/55/6d/ea66a30af74f5983cae31ebb9ef78b178b366a12856a414e1472225c4a34/websockets-17.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:30201a7f69833b015556c72feb69ea501b645986fd0b90dab13f589e995ff428", size = 224615, upload-time = "2026-10-03T14:54:44.41Z" }, - { url = "https://files.pythonhosted.org/packages/87/80/c6f2228ad89774429d270179375ebddb657119215f52d1df7c680d65cad7/websockets-17.2-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:0c8600aec354cc259f1691b0b42816f04a9886a953f82cb227246df76057f97a", size = 225398, upload-time = "2026-10-03T14:54:46.063Z" }, - { url = "https://files.pythonhosted.org/packages/f7/4a/3d8da19732ad468d4be7f1e3ac298078b60bdda55edde6589bef84a5eb7e/websockets-17.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:307fc22ea496be8542d67b82ae8c867a978dfd19ac35573d4f15943fd9277dfe", size = 226571, upload-time = "2026-10-03T14:54:47.672Z" }, - { url = "https://files.pythonhosted.org/packages/58/22/1231657122d9cc24791bb90af13cc2f4e84cf0d3a454cb37e3abfdcb2fd9/websockets-17.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:9c88697fa943bd4ef67cc919a17d81de6581846f52bfa8c6f64a916098986556", size = 224125, upload-time = "2026-10-03T14:54:49.537Z" }, - { url = "https://files.pythonhosted.org/packages/1a/04/350ca2445da758bc42cdb4218b44d4ce0d5a9c1d5e4cc4a58d64348ad9da/websockets-17.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:f7eac84d4969da82166d5e90d9c38d2f416fe24f9708a7013569b193745b9a31", size = 225081, upload-time = "2026-10-03T14:54:51.075Z" }, - { url = "https://files.pythonhosted.org/packages/da/c4/dec952b0df3a5d918ed2a545abb0c25ae519c3bc2d9aba3b7c46abae8f05/websockets-17.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:313f6703023d53baabab6d6c5c37cf637b2c4fee255acf2ed5e92ad69e28f1b7", size = 225376, upload-time = "2026-10-03T14:54:52.675Z" }, - { url = "https://files.pythonhosted.org/packages/f2/b4/198a260afbcc086ff4979774e51834ed7fb5b95f9ef305e0c4924630b857/websockets-17.2-cp314-cp314-win32.whl", hash = "sha256:08d90cf344bdb971ba3a826b78d4da9bfd56cc6a97a604d9b88cbd40bfa6c735", size = 217760, upload-time = "2026-10-03T14:54:54.247Z" }, - { url = "https://files.pythonhosted.org/packages/e5/9e/0523f8bc2f7aaddf39562d4fa01b4d38fa61b23d980917a16d2dd19c8dac/websockets-17.2-cp314-cp314-win_amd64.whl", hash = "sha256:dac93bf7a9beb215be3282b8441173cd50806c41c007b8be9bb24e03c60ad563", size = 218104, upload-time = "2026-10-03T14:54:55.845Z" }, - { url = "https://files.pythonhosted.org/packages/55/17/7b8bb4cb64a199e7082f1f9be784d657842fefc327ac777d6c1493504804/websockets-17.2-cp314-cp314-win_arm64.whl", hash = "sha256:2ab742249f953d148a9ba696c8b9944361e8cb92e8bc61ba2dd53a178403afd3", size = 217989, upload-time = "2026-10-03T14:54:57.376Z" }, - { url = "https://files.pythonhosted.org/packages/ee/76/f54ed054b6e860f1e0bbc7019542a048352d41231fdff6d904b379f881c7/websockets-17.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:a69ce25be5f1330ee1c74eb6fabbbceaa96b384beedd2627cecded7546490c40", size = 218125, upload-time = "2026-10-03T14:54:58.943Z" }, - { url = "https://files.pythonhosted.org/packages/e6/4c/0f3375cea66a125ae01d21fb9c537aae955ef499bfe7e2b2376a34362f2a/websockets-17.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:8e24b878cf54843a63985d90480f163ca7f692689fbcbe9cdbd8165521083a8b", size = 215658, upload-time = "2026-10-03T14:55:00.674Z" }, - { url = "https://files.pythonhosted.org/packages/0c/05/7c871a67bfb4b61adc1fe13583db97803f87dfeca644fe6ef51df7bb276d/websockets-17.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f33c7908a6885dcae9f462a4a8347b637053b4ff2b96beb4c23fba1cf7818e5f", size = 215858, upload-time = "2026-10-03T14:55:02.379Z" }, - { url = "https://files.pythonhosted.org/packages/41/8e/59df4d9cd357e902d1c74b13c3c0c3841c8df6e4b1b3d131bf26a23fdcb1/websockets-17.2-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:c796a1bb3e4015249639849f30e8e680df8a431b45d417ba8acf843d2451d95f", size = 225443, upload-time = "2026-10-03T14:55:03.966Z" }, - { url = "https://files.pythonhosted.org/packages/5c/64/5e486a3a44e041203c62eccf1fc89c7f8824e21104a7b82b182e5b21c228/websockets-17.2-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:983bcdc898662f6ba9d6a025c30d29946ff0986d9ad60d400af0da3671f7cbf3", size = 225726, upload-time = "2026-10-03T14:55:05.797Z" }, - { url = "https://files.pythonhosted.org/packages/f0/98/b6eb53121c91fbe8b6897aba06861ce60f9ab58faffc6bca5750cbc21681/websockets-17.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:35e0f088ddfd9d9bc5019e27ff3767411779e92b59db5bb1507f2731a5b61158", size = 226895, upload-time = "2026-10-03T14:55:07.626Z" }, - { url = "https://files.pythonhosted.org/packages/8a/18/8c091321b99c91eb3eaec9acbd940e69308b4e465b5605c430af0cf7d3a5/websockets-17.2-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:19e2511412ad3393191de652513bc7a0ca3c93af143b32d96d46e59fbbddf1d4", size = 229040, upload-time = "2026-10-03T14:55:09.321Z" }, - { url = "https://files.pythonhosted.org/packages/1a/96/3a92f944305b7de42fcb7530b9fa69607b4b4ce993c36a9f2330dbc318ba/websockets-17.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb5e2bf969ac99a6ae3c71208a5eb05cfde973192540ffa6e1068b57fb78c4f8", size = 227469, upload-time = "2026-10-03T14:55:10.935Z" }, - { url = "https://files.pythonhosted.org/packages/ea/a9/624f6d75ba326c22d03698b34c0ada984f1d76196322a62f6c22903b831d/websockets-17.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:691780fca2be3dec512cb603cb91060271968cb4af86b51d07c57445c5754a37", size = 226202, upload-time = "2026-10-03T14:55:12.536Z" }, - { url = "https://files.pythonhosted.org/packages/47/af/1e6e8c625aeb268830af2c4227fe05e8db59f4f4debe1dadfd0ada214895/websockets-17.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:2d39c19b1ba6a6791050383fd69efdd3b63533e2254693d0263879cd5f5921ba", size = 223743, upload-time = "2026-10-03T14:55:14.164Z" }, - { url = "https://files.pythonhosted.org/packages/dd/81/33c5280f4f6f81637c93ae065c6a594dfe35935622af135a5f7c3768bf22/websockets-17.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e48ac2b302986c6f55cf61e8e36b4dd97d0132c5078a713a697a940934ba422e", size = 226492, upload-time = "2026-10-03T14:55:15.796Z" }, - { url = "https://files.pythonhosted.org/packages/1d/f3/7aa9fc36e67caccbcfee2c48f4ada41e9da512d41523c024d039f0f22ba3/websockets-17.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:e136197f1262620ef2e507afc3ea759c1ae7d221886da20eec5f4c9f2618c2aa", size = 224940, upload-time = "2026-10-03T14:55:17.661Z" }, - { url = "https://files.pythonhosted.org/packages/3f/8c/457aff7081a63d1261608bb4d7b0b0f9dfe780697a2a334671745742850b/websockets-17.2-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:3eb44019a2b0b3b91bac95998f1e4e5589730421170e060fe654a2b7be727dc7", size = 225835, upload-time = "2026-10-03T14:55:19.607Z" }, - { url = "https://files.pythonhosted.org/packages/3e/c3/7a13a3b3050db2c36772ded49f8d48f99eb080948e9f6f762e7529925ab5/websockets-17.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:e5855e574804398859c5fbaf4fc7882b96278b7f6572a3d889627e6eb6cfca59", size = 226848, upload-time = "2026-10-03T14:55:21.274Z" }, - { url = "https://files.pythonhosted.org/packages/c4/3e/d5b2c1e473b1031a4a0ec0e10de69df5b981ab4a10aa482bb45c18dd43f5/websockets-17.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:5dc29815520c329f5662f6eb3ebadecf0d4f8c82dfa416d4d6efbf8f39245559", size = 224541, upload-time = "2026-10-03T14:55:22.874Z" }, - { url = "https://files.pythonhosted.org/packages/79/5d/bb81976cc1aa546afb51395ce42913521e9dea062bb34a61308cfff30726/websockets-17.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:d1a4f9462da6496b6cb79bbb09c60d17f7e63e8a1df136797b3afabec9560e4d", size = 225315, upload-time = "2026-10-03T14:55:24.443Z" }, - { url = "https://files.pythonhosted.org/packages/f4/6b/314962d5440c61b4c107914599c13ceeecc6bdb6e2e73a5f7e566a7d1f26/websockets-17.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9496bff5541086478264678bac73c0a75b2fde94fdf6568893bca1f7c6d50d18", size = 225747, upload-time = "2026-10-03T14:55:26.033Z" }, - { url = "https://files.pythonhosted.org/packages/98/fc/9eb64b34a3a4458eb08f3f24bde01508f72a00790330723c158ebb965048/websockets-17.2-cp314-cp314t-win32.whl", hash = "sha256:e1e3bc8090a7eae79fdf634b63bdbfa3c93999991023c37c6fd3b469fc8ff5dc", size = 217891, upload-time = "2026-10-03T14:55:27.681Z" }, - { url = "https://files.pythonhosted.org/packages/ba/ed/3a4e2a09b0822d6e525cbc6e44a4885669bad5b22ab9c64fa2444bc15325/websockets-17.2-cp314-cp314t-win_amd64.whl", hash = "sha256:65a89a5bde227bfe908016f35b5bd347970cd1e5b0360f389502eba1c7fde6e0", size = 218229, upload-time = "2026-10-03T14:55:29.314Z" }, - { url = "https://files.pythonhosted.org/packages/b5/66/cffb75ee746dd060984c3c3e2eac7f875a866225a30dfa53e2cd18232565/websockets-17.2-cp314-cp314t-win_arm64.whl", hash = "sha256:1c27339934109dfaca83f18ab2c23db06714e9d5deca2c8e37e8f492ab90d20b", size = 218146, upload-time = "2026-10-03T14:55:31.001Z" }, - { url = "https://files.pythonhosted.org/packages/12/e9/10a9b1633b63594054c87b97af048628cea2b21b5089a52a9fc1e0af60a3/websockets-17.2-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:a7c4bb26de6ef496d24822aee4f6a305d97cd33d21a2b85f290292d69ba1c25e", size = 217719, upload-time = "2026-10-03T14:55:32.674Z" }, - { url = "https://files.pythonhosted.org/packages/0c/00/ff4020fe0886dac7199a16ce2805c7afd7b981bd2e81d3fa18dff5d9863a/websockets-17.2-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:c08da1f15040bd1e1a6074bd4518a6ef20e67b1594ecfb0aa75e5b45f87e6d6d", size = 215448, upload-time = "2026-10-03T14:55:34.338Z" }, - { url = "https://files.pythonhosted.org/packages/66/06/bc7b944f81514378b2c2ab96c17df19e871cd33b9be0f1f6dfc975457e5e/websockets-17.2-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:3117abfd32b183bdb6194df9317766d32c6517f3d1c0aa8c62d5c6ccfda0b4a8", size = 215674, upload-time = "2026-10-03T14:55:35.918Z" }, - { url = "https://files.pythonhosted.org/packages/a8/da/2b2b76faa2f10c4813e3872c9577fd13a798f5918b1785b86ff7d635eb2a/websockets-17.2-cp315-cp315-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:a046227daa7f191e843d26b911c1146233e9a33d249e0c954dcb3ac7c398710e", size = 225119, upload-time = "2026-10-03T14:55:37.777Z" }, - { url = "https://files.pythonhosted.org/packages/ae/d4/22cbe288c0d5cef7620503be92c0098d82220353fc7e188034a19c517240/websockets-17.2-cp315-cp315-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:2901bdf24f20bc884124b3e88c61f7ece260c20c81e610f2196007395264a4aa", size = 225549, upload-time = "2026-10-03T14:55:39.364Z" }, - { url = "https://files.pythonhosted.org/packages/4c/0a/504b0d3063679f2c60430c3539482d42a4cb8bd1a76646baf742030a93cc/websockets-17.2-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f60e39adfecf998488166aca8ff24ab1ac406c9ecbecbcf9b3bcfc43cb1ec9a1", size = 226717, upload-time = "2026-10-03T14:55:40.942Z" }, - { url = "https://files.pythonhosted.org/packages/4e/ea/5da9309cc55c2665a6eebc22c369d9918c0d77258c61e92058e6b08d5ff1/websockets-17.2-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:d4df62fd8448a85c752bbea1803cb3a2785e6fc8352009ab64ad7447af079b3c", size = 228413, upload-time = "2026-10-03T14:55:42.54Z" }, - { url = "https://files.pythonhosted.org/packages/a6/74/5a24df72aa5500f311105687af864c27f1f9da910e968e97818c6149e6b0/websockets-17.2-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c8eea55fdfa9ba65c6981eea38bd20c800bce2f092a2803d82de764ecf0f071a", size = 227196, upload-time = "2026-10-03T14:55:44.251Z" }, - { url = "https://files.pythonhosted.org/packages/5e/ee/ca32cc1ed892dc4ac30a922e8f648048233fbdb8b0bce7048860ec4c60ec/websockets-17.2-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3f0def1279644acaa9bc861d4234af3f82ea9cee7e460dffac5cb63e691501e9", size = 226092, upload-time = "2026-10-03T14:55:45.842Z" }, - { url = "https://files.pythonhosted.org/packages/7d/0c/12d4a73324aa9798d5165d20c088f9dba66c75c871960e5d921ec66694e4/websockets-17.2-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fb78fb4158c12f77a934a003006784108a27a6553cfc0c6f10483c9c02e94f48", size = 223486, upload-time = "2026-10-03T14:55:47.45Z" }, - { url = "https://files.pythonhosted.org/packages/bc/a4/7fe15da5abb8f0f61e6a357593f7f2ed55724825b7db0ffe72b5c5fad68d/websockets-17.2-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:f8969ad228115ad8869b5fed801f899e52ab8ad376fdb165ba4760a277c8258a", size = 226200, upload-time = "2026-10-03T14:55:49.126Z" }, - { url = "https://files.pythonhosted.org/packages/08/b9/4cd3a311f96a2eea0ed458bc01fe2cce42f9cd50aa9e64315dfc855d63a9/websockets-17.2-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:4a49ca342efc0800e6ae94ed5c9cbdcb319308f75e73c21181e4c24d6710e8dd", size = 224862, upload-time = "2026-10-03T14:55:50.674Z" }, - { url = "https://files.pythonhosted.org/packages/41/b5/22caa3460f75e42bfcc74028870b556d22847ea9a9034aa03986f07f16a9/websockets-17.2-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:06fa3ce9c3154826c33d4395b225b2994aa64f1f3bcd8be8ed932019175d9268", size = 225391, upload-time = "2026-10-03T14:55:52.393Z" }, - { url = "https://files.pythonhosted.org/packages/95/be/8d28f92092076abf1ddfb3206b0ce956120a22e7c3105f6a3029d727deae/websockets-17.2-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:50644d8715be7e0ec0682f9d7744b63008e199c5e1618a48fa153756a332235f", size = 226545, upload-time = "2026-10-03T14:55:54.127Z" }, - { url = "https://files.pythonhosted.org/packages/cb/7b/ff943fa383e540fe17f066cc10a3eeedef26e50fd45aae2bdc6746d6f95a/websockets-17.2-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:60deca33e584c09e91f70f8b55a0b1de7d671d6a63f051d154920f48bed717c7", size = 224352, upload-time = "2026-10-03T14:55:55.856Z" }, - { url = "https://files.pythonhosted.org/packages/e9/df/1e6c3e06c473c9fd833a5c1620b15e2c3b37647b91b7d41871d20bc098de/websockets-17.2-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:b5f79366a8d8dbb981d53ba800bb54a95454595ab8a4548c2b95501b32a08326", size = 225255, upload-time = "2026-10-03T14:55:57.497Z" }, - { url = "https://files.pythonhosted.org/packages/db/f8/d8a4f988f7cbb568d8bd69da4632c5b6010aa9cd9366f285e23b73b678d9/websockets-17.2-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:f2bbf3f28d0b63157577c8b774b9136f076afa6797e1a52a2ecd477f23cad3a8", size = 225513, upload-time = "2026-10-03T14:55:59.338Z" }, - { url = "https://files.pythonhosted.org/packages/75/e0/920357165b2797a2530fc9e271d79a9b5fee2b750b154c990c740f767af3/websockets-17.2-cp315-cp315-win32.whl", hash = "sha256:74836317b7010b579522bb52426f1e225608b042c9e78cbe2493522bebb8a318", size = 217722, upload-time = "2026-10-03T14:56:01.307Z" }, - { url = "https://files.pythonhosted.org/packages/5f/eb/25bdca25bbc329ffb330ef33993397d6556a871e40a0d196e757699ea3f7/websockets-17.2-cp315-cp315-win_amd64.whl", hash = "sha256:aaead3d926e9ab4124ada727d20cd62d396649917822df4f771d1f07f1079b40", size = 218017, upload-time = "2026-10-03T14:56:02.914Z" }, - { url = "https://files.pythonhosted.org/packages/fa/cb/ea30a552bbcd1c75f0d14bfce6c884ee36187030b85b74a242aacc02406e/websockets-17.2-cp315-cp315-win_arm64.whl", hash = "sha256:40960554e60eb60c3eec4ff9e42a80f84f8cd3ca9bc80a5481a61f1e64d807c9", size = 217929, upload-time = "2026-10-03T14:56:04.604Z" }, - { url = "https://files.pythonhosted.org/packages/4a/01/477664c619af8aa3c908d482e2a95e13ceed9d78f21d15902013c3bc6c28/websockets-17.2-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:9a2a60a7f0ea5f239efb6391d2b28630a640d82dad63e3bee47cf2c623c4495d", size = 218029, upload-time = "2026-10-03T14:56:06.336Z" }, - { url = "https://files.pythonhosted.org/packages/2a/a9/b0be62ff1c0e2bc966da56b36d3d820c7e2ad3c0c4a4ac414fc7335b214f/websockets-17.2-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:cca2fcb72c007103740fa4fc3df19fdb1a318c641c69f3b0cc47ed63a889336e", size = 215607, upload-time = "2026-10-03T14:56:08.035Z" }, - { url = "https://files.pythonhosted.org/packages/fc/2b/a6738530de0437a31c1b168e4096ecf790aafaf561f33a009886c7d8042e/websockets-17.2-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:b789356bc4e2e6c20ba52817f92c3fed74e24657654237ecd536c54843b80c6c", size = 215817, upload-time = "2026-10-03T14:56:09.852Z" }, - { url = "https://files.pythonhosted.org/packages/c3/c2/2fc44ddc419cbb09ee1708af3e78d8a4b018db01fc7e4f91bd730e2f8d9e/websockets-17.2-cp315-cp315t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:222fb626fa15701a850eccc778be17312142b2f6a0e16aea80770b7459adb784", size = 225979, upload-time = "2026-10-03T14:56:11.85Z" }, - { url = "https://files.pythonhosted.org/packages/2e/91/a215b14caa7ea65bc36db81609108899c259503300d1560dae9c70a135e7/websockets-17.2-cp315-cp315t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:4497e87c34a2d21cbec1227858fec3af8e514dd70c47625557a122fcebc081dc", size = 226250, upload-time = "2026-10-03T14:56:13.548Z" }, - { url = "https://files.pythonhosted.org/packages/65/b9/9406a18e9edf558ed504d2a7679371d0f8107e4ef526c80b154ea4ec9752/websockets-17.2-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6281c171557ce0e408e19d9a223f22d915117ac38a5a7f32ed83809e7492316c", size = 227579, upload-time = "2026-10-03T14:56:15.143Z" }, - { url = "https://files.pythonhosted.org/packages/fe/45/a73af119244f46f5130005d7ab63f1c75890c890141a0ca2adc9d97d4671/websockets-17.2-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:08d97098644728bd1895caa7ecf3090b8e563d70809870d2adb33a107bd061d0", size = 229205, upload-time = "2026-10-03T14:56:17.086Z" }, - { url = "https://files.pythonhosted.org/packages/c1/92/ccd8e2e921d134a56f1ed4642d276500d9e33b3dc4d6deb63d614b3e53a6/websockets-17.2-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:1fdb8d5a1660307dc6d36d0b7fc725213cbd7f80800904dc4896aa3208b89121", size = 228011, upload-time = "2026-10-03T14:56:18.716Z" }, - { url = "https://files.pythonhosted.org/packages/e0/ef/7d71105d19a7aaab5ff87b9c712f6c1dda44e72ea56aa0e7b777f2fc274b/websockets-17.2-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:18b0a46e5e9b315e2b54ce8c3bafdeef0e1388ca363114fa868e6aab2dc58512", size = 226892, upload-time = "2026-10-03T14:56:20.412Z" }, - { url = "https://files.pythonhosted.org/packages/56/f7/87012d628b21e66e699440f39bfa7cc55fae7f52b2c532ab62184a589624/websockets-17.2-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:7f115d5d804a2163dd89245710049078b0e726a58c1f44a1f86c2c6e79055d76", size = 224241, upload-time = "2026-10-03T14:56:22.257Z" }, - { url = "https://files.pythonhosted.org/packages/55/f5/495371068b27ee5f7c435187f9dafd62402f195e2c76063bdd4653da1565/websockets-17.2-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:1d829946a2e7630f92f9d7b45b62f3abe9f393cc2dea6a35edb3988f865e75f2", size = 227076, upload-time = "2026-10-03T14:56:23.909Z" }, - { url = "https://files.pythonhosted.org/packages/18/18/3dce3cc6099be5e044e0fd5d0e0c9931c8e3387511cdec8014a345f619e5/websockets-17.2-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:6c274fc1572edf7c197094a0eb1887d45fdc95254bc80597dc7599550486c06a", size = 225727, upload-time = "2026-10-03T14:56:25.689Z" }, - { url = "https://files.pythonhosted.org/packages/47/30/57d0c7aaf8d4473926fa8829b8136483f561388d1e747ae71c9f2a83d5fd/websockets-17.2-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:4173a4b8a025ae44313d9d9b4ecf31e886c7b7faf45386d51a8ca4ff2dcf3f2a", size = 226225, upload-time = "2026-10-03T14:56:27.246Z" }, - { url = "https://files.pythonhosted.org/packages/0c/9f/9dce1203756756c00b407b9a6b13a7500fcd38f2634d4daa3f65575814ec/websockets-17.2-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:d8cfe9522ad69b6abb26b413ed1deca43cb915cefc588433d557cb3ae1c783e2", size = 227333, upload-time = "2026-10-03T14:56:28.811Z" }, - { url = "https://files.pythonhosted.org/packages/9a/2f/d3b6b876678ebb03017b7afd7111fe44d54b93f036a80ebb4b481dd1ab74/websockets-17.2-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:908d81d88bb16141613a6275059b5114656d5c2f0b5400b421d54fe6f1943507", size = 225082, upload-time = "2026-10-03T14:56:30.578Z" }, - { url = "https://files.pythonhosted.org/packages/32/b0/a69b573a5e56d2e7a5dcbb447466f442380cf81515e1cb1220cd626c8042/websockets-17.2-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:c6590e1eb624ff6b15b872421bc9a10bc6d2057635d69c6cd244ac3f928f85c6", size = 225945, upload-time = "2026-10-03T14:56:32.32Z" }, - { url = "https://files.pythonhosted.org/packages/70/be/a72911dc8e33f74c196012366ce4d99b1a803894a377a1ed0c8e66df9caa/websockets-17.2-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:61040f6f7da5a279d2f77496c69d51132aba75f701c52bded400d4c639277b18", size = 226241, upload-time = "2026-10-03T14:56:34.142Z" }, - { url = "https://files.pythonhosted.org/packages/7d/a9/02a68c1d8e5572918e0962d3aad881078f73ede43abd9b1336e4efaa8909/websockets-17.2-cp315-cp315t-win32.whl", hash = "sha256:f90bad2839c185a1edf8ee22a257cfc8a39e0e337a0490ab185dfa76ef04d1bd", size = 217847, upload-time = "2026-10-03T14:56:36.204Z" }, - { url = "https://files.pythonhosted.org/packages/2b/bf/3d7c33b8d5e7712a60e0149c017ed50394ec5e8cf72e5cb6a1ffaf11a42d/websockets-17.2-cp315-cp315t-win_amd64.whl", hash = "sha256:315551f4ccedbbf9fd4f7e8bf037a5948c976ade0e919ba5d8f581d465f6f725", size = 218169, upload-time = "2026-10-03T14:56:37.79Z" }, - { url = "https://files.pythonhosted.org/packages/27/57/ab34cc6460c5322e6932750fa5c6c64be89e6ee4e2707d13c4e9d3312b25/websockets-17.2-cp315-cp315t-win_arm64.whl", hash = "sha256:0a6220bdf8d5f11af71251a599092d89ac1d6bfac691c7f5951c5b07953947a0", size = 218089, upload-time = "2026-10-03T14:56:39.427Z" }, - { url = "https://files.pythonhosted.org/packages/8a/58/835cd51934d6780fa586f275b5d9901eead6d81569b4343b3767cdbaae4c/websockets-17.2-py3-none-any.whl", hash = "sha256:6aa59f0ef92e796b2db6f5f26550c4713c0e4036899fadf02f55e2ed4db0b7ae", size = 211883, upload-time = "2026-10-03T14:56:51.898Z" }, + { url = "https://files.pythonhosted.org/packages/27/ad/f65e365bbf38d61164b0c2f64776fc7714bfa59ac51e492b038a35f4ab1b/websockets-17.3-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:bc8fd230ef013951900c1fda100eebe4a4b1b407dde2a44186995042dda2c628", size = 218288, upload-time = "2026-10-19T16:55:53.387Z" }, + { url = "https://files.pythonhosted.org/packages/48/1c/33271eba53b4927a1b91d99d838a6d7b888e6f708425262c911450d73696/websockets-17.3-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:cdb8e4b0f6a41b857c74c03116e8ffbef59ef8e533e934b532843178990d01d5", size = 215977, upload-time = "2026-10-19T16:55:55.063Z" }, + { url = "https://files.pythonhosted.org/packages/3d/f3/6d982c60f6c7c95e5faed53d39263df273ecfda904c7d6920c37eee58dd1/websockets-17.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:018fab26140ce221c3853454a608e4e7004f4f10641cb97819dcd5b917bb2e99", size = 216193, upload-time = "2026-10-19T16:55:56.424Z" }, + { url = "https://files.pythonhosted.org/packages/ff/c6/4ff6b8d2a4b32719d4658bbd791ee9d19939a47a69b961c961fd9dc3a64c/websockets-17.3-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:dd0f3037d501178a761fd2814370aca547f79e99a200a61a1f6e7b79ec3d78b9", size = 225573, upload-time = "2026-10-19T16:55:58.252Z" }, + { url = "https://files.pythonhosted.org/packages/eb/8f/46e4030947f5590cb61832901f44a3edb07ef8d1e27f6c06b42c5f1c4d33/websockets-17.3-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8dbb3862aa649de95660b6337d97101977f160b5b354d07b82fe309874d2c8bc", size = 225927, upload-time = "2026-10-19T16:55:59.561Z" }, + { url = "https://files.pythonhosted.org/packages/f0/1b/3db30fdf4fbefa4ac463060d3f87e2d444f960aa1234028468c09b88bffc/websockets-17.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:af37a1e6b6bd506b597d5fb4cf111fb527d1b2223608a5d441d4738827cfd166", size = 227169, upload-time = "2026-10-19T16:56:01.029Z" }, + { url = "https://files.pythonhosted.org/packages/02/ed/849799cfc0e3e359b75ca1d50605376da52dc9ef0b2a53931e3f5991f96e/websockets-17.3-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:740bba4223fa5216d4d6c5dcd29e697f37229b8a61b69e1a50527b0ea993ab2c", size = 228542, upload-time = "2026-10-19T16:56:02.377Z" }, + { url = "https://files.pythonhosted.org/packages/19/53/dcc397e20ce792fffd9a8f79b6333275748ee7a23f807e576166a0b16edd/websockets-17.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4170556762c6fd2913d99dfdd8dbb4c927584899df7facb1034627e355263c45", size = 227772, upload-time = "2026-10-19T16:56:03.768Z" }, + { url = "https://files.pythonhosted.org/packages/c9/36/e5935782f71e9e22247b2a2e562da05e18d8945c49f2692ca1bdaceec178/websockets-17.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d355b22270be8062360b9df2ee0b10277590789a1059257916fb9e256a0cff80", size = 226518, upload-time = "2026-10-19T16:56:05.332Z" }, + { url = "https://files.pythonhosted.org/packages/27/76/bd5978a1e403fc646230cf35cf7c06b40b3a232d440881b9e89ede229b18/websockets-17.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4aa85b34c21cf97b3470793d0bf0763d32bfd1e67b83067a0a3ba8dc3474dacf", size = 223766, upload-time = "2026-10-19T16:56:06.703Z" }, + { url = "https://files.pythonhosted.org/packages/da/87/c66eb39bd851d5c14d9eb75aa64d03cfcf2ded77927ca067b5917291d19d/websockets-17.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a0e2e7c2c1a4f25db7b6596738d810c10732e77f852247300a1bc49e364d6dc9", size = 226644, upload-time = "2026-10-19T16:56:08.198Z" }, + { url = "https://files.pythonhosted.org/packages/fe/ae/b9a31d5497fa0b73629a097b173a26123da2ed656b562114644bc623414b/websockets-17.3-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:7fa69212ea60ea24e2d8aff23fe102b1f87f1b27a320f92d2e7b3468997a496a", size = 225107, upload-time = "2026-10-19T16:56:09.627Z" }, + { url = "https://files.pythonhosted.org/packages/61/0c/b773a102f84b60b2052b2cfde773d2f3cd3e8eaa93a145f54450460549cf/websockets-17.3-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:80a6e37e235780f189558ab804f3cbc01c18f5c578995739b49613d544d34ebf", size = 225891, upload-time = "2026-10-19T16:56:10.973Z" }, + { url = "https://files.pythonhosted.org/packages/b2/fa/e7e538575dd4912dea9ea4c9ae86519aa1420969d9cb5cf02fd10fd20ba7/websockets-17.3-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:628efe13630ac3397e7d1278b6be6f22f37a86d638a324334b4cd43c1daf9928", size = 227064, upload-time = "2026-10-19T16:56:12.295Z" }, + { url = "https://files.pythonhosted.org/packages/28/87/bf2b928c156deefd0c50285a48fc985e2ca85d284e7d9981cc5b79b0e696/websockets-17.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:443fa28652682d2a3ec3eca49af1b87cb2523561257683e62e03a98967b22138", size = 224618, upload-time = "2026-10-19T16:56:13.73Z" }, + { url = "https://files.pythonhosted.org/packages/8c/9a/d59817ba3535f787c1cf8990a01b3d2896a7d9d8f3b5d55bed22fce595a6/websockets-17.3-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:6705af9d1005127302c04d9f1533ba56e289336c465923ef878430e99295a0b9", size = 225571, upload-time = "2026-10-19T16:56:15.199Z" }, + { url = "https://files.pythonhosted.org/packages/a6/96/8210379dac889e16c75099aefffa200dc55dc8cb030010f309a7eb9c75c1/websockets-17.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:9bde5d82d7ac4859e0cb3a35e148f537a8d4e56aa43f42582b5f263982f095ae", size = 225869, upload-time = "2026-10-19T16:56:16.569Z" }, + { url = "https://files.pythonhosted.org/packages/97/0b/d4741c977fec9e6482beffece572d544d42d230f56fc3ef504f96521063a/websockets-17.3-cp314-cp314-win32.whl", hash = "sha256:be7be57097a65a8fab3d0f0a10d12931deb626a7d8082eb732b1ae41f3a202e3", size = 218232, upload-time = "2026-10-19T16:56:18.028Z" }, + { url = "https://files.pythonhosted.org/packages/33/69/2f9ce899776d0c522f7f44080cad4228a1c14ec13b0125a2efc12ebe41c9/websockets-17.3-cp314-cp314-win_amd64.whl", hash = "sha256:2b06213c8e1ce93b18d0476d7a5e647436c8f23d099888c9065e930e80f4bed5", size = 218582, upload-time = "2026-10-19T16:56:19.399Z" }, + { url = "https://files.pythonhosted.org/packages/4e/e9/11b885666d72add674440a3435c059bea325616959cec8be42605948714c/websockets-17.3-cp314-cp314-win_arm64.whl", hash = "sha256:6b7b540bb321f7f0a7b8f5d107f941bb361e3a154c0b594dae0394997b91b743", size = 218465, upload-time = "2026-10-19T16:56:21.005Z" }, + { url = "https://files.pythonhosted.org/packages/c4/0e/0c0f40ab8eefe9d5a6c78b4e267d8e37d70d3db8dd195574e0ab867e4f81/websockets-17.3-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:c9d06edfd974e918b40e0662a7e76dac13125529ee5ec9fbe42dc0d6faa209c3", size = 218614, upload-time = "2026-10-19T16:56:22.457Z" }, + { url = "https://files.pythonhosted.org/packages/1f/93/d22b3d63aa439bc01a9477880070d05e0b257b0548294786a33be3e842ec/websockets-17.3-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:0a0bcf66289b9d017c175562c35c5304b5cc3ed56aded994ae30b0a9654ba405", size = 216151, upload-time = "2026-10-19T16:56:23.95Z" }, + { url = "https://files.pythonhosted.org/packages/b9/92/f60a9672cdeb8cb4f7156e6e83c76d59fdc586737a97ac77129e9fc188a2/websockets-17.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f3ed747b7651668131029c720fbc32c7ee2d129bdad200e6c8f4037b20f7913a", size = 216354, upload-time = "2026-10-19T16:56:25.251Z" }, + { url = "https://files.pythonhosted.org/packages/9d/da/2be8c5f3c6612d4523f95b074e2352bc11bd090cf121d32bc11af1d52e70/websockets-17.3-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:5868a02c4f6d698ddd19b4c9742748c03dec045289915806845189d96bc3d314", size = 225937, upload-time = "2026-10-19T16:56:26.726Z" }, + { url = "https://files.pythonhosted.org/packages/85/1c/cb84aec871840acd66dfa84520bf7b8e48ce70bdb3f4baa4efd097fbe6b1/websockets-17.3-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fa87ea2ad64e59fe5622c1dc5716106b6d6e7e4b26b848cb5f5710a75c292ef5", size = 226215, upload-time = "2026-10-19T16:56:28.1Z" }, + { url = "https://files.pythonhosted.org/packages/9b/7b/cfb65557accec2eda2c80288e13eca6f1c797c579375906eb73f4053cfd1/websockets-17.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:65e937576141ac50fac32fa9736dbb3e0e4a7769f91be004011a09216c6ab05f", size = 227389, upload-time = "2026-10-19T16:56:29.518Z" }, + { url = "https://files.pythonhosted.org/packages/70/a3/7e08ec4b23e376ed228831aa44b2f2a3f2e7025cc66dda4b2e53dcc2619e/websockets-17.3-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:35381b575f70a08d4241e73c0cde4672bb9b2a9aae8767ca6b7fb61305a052c8", size = 229534, upload-time = "2026-10-19T16:56:31.072Z" }, + { url = "https://files.pythonhosted.org/packages/4b/7d/8a813e3e53956f876cfaad4f6358625bbd94168aa8046c44e2f3561f12af/websockets-17.3-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:5142798871a248b422f353e0684d51dfa119de3403a6fa0904b7cb806d9790ce", size = 227959, upload-time = "2026-10-19T16:56:32.656Z" }, + { url = "https://files.pythonhosted.org/packages/41/23/5f3dab6ab44f2f30042db83a658a57129d5e3c0425c798b3c3e727bc060a/websockets-17.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:e164c4a6868d11988c7f2386e3278ccff21c665cd7d08279b62709dd37a3167c", size = 226691, upload-time = "2026-10-19T16:56:34.101Z" }, + { url = "https://files.pythonhosted.org/packages/5b/49/99a3fb5a2e94d677fbd0d014a6f9a56a3d2b8e8007b7175727bc4fc4ae94/websockets-17.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:10b24e836ee0233ca078e1244f6ba40343a22127a9a736f46a7e7b18de2cc711", size = 224225, upload-time = "2026-10-19T16:56:35.452Z" }, + { url = "https://files.pythonhosted.org/packages/6f/e5/f503a4d757858d083c90ea85dd1fd636ae9eb63c9552c75a4c38514ee7ed/websockets-17.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:56a76556ab04ab54a675acce2cac8b4e74ea2cec558a7c9ed37e0367b04b5d50", size = 226984, upload-time = "2026-10-19T16:56:36.887Z" }, + { url = "https://files.pythonhosted.org/packages/4a/a6/8586844ad4b21988837624e1f4b331bc71919c85a6eb1ee3c8bd29893f27/websockets-17.3-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:33221e49be79d7bfe6d12bc52ef9458b256ab74fa8ac5660acfa3065de1c7870", size = 225436, upload-time = "2026-10-19T16:56:38.213Z" }, + { url = "https://files.pythonhosted.org/packages/a7/b9/f419506494d6ffd205a9c3258f56ff5aeae6976de26361057de74f4ab659/websockets-17.3-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:2fa79b9a48ae59744316c71a34cb9f76f30c93e117338522c5135cb7eac10351", size = 226325, upload-time = "2026-10-19T16:56:39.606Z" }, + { url = "https://files.pythonhosted.org/packages/28/c5/55a58faa67149e84cb98b33f9d8fda95242677ea07b78f3ecaf2e77877d6/websockets-17.3-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:f87f02afe05c1c246012e3de100a77cfbabd17db709fe93cf074271ab239b312", size = 227342, upload-time = "2026-10-19T16:56:41.041Z" }, + { url = "https://files.pythonhosted.org/packages/35/e0/fbe561dff24b6ef1b2034877f37b56eb9b81b3c103e2a8b03819bf2b3b6d/websockets-17.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:8135a549e27e9941a70e2d2156676ef2ffeff4b06e9d564dbaaae963482afdef", size = 225036, upload-time = "2026-10-19T16:56:42.433Z" }, + { url = "https://files.pythonhosted.org/packages/0d/37/d80753064f8370a24d3bec74807c3928d7b0ebef5381956e2a0108b365c2/websockets-17.3-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:ace8dd1efa5f44c482d091aa38322e20e3ffe5adcdba6b67ad9f4e7da58f3f68", size = 225807, upload-time = "2026-10-19T16:56:43.774Z" }, + { url = "https://files.pythonhosted.org/packages/f1/9d/a11d3168522a2959c4a5661728744d62b9eee344abdba5f9fbc1d359742f/websockets-17.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:2129c9a5fc05a230ce088745a790af073e582b584badc7b05db067ee326f56d7", size = 226241, upload-time = "2026-10-19T16:56:45.088Z" }, + { url = "https://files.pythonhosted.org/packages/06/28/16107b3b83fd2b5024bfecf8d000140f555204a0191202f9c4c981137810/websockets-17.3-cp314-cp314t-win32.whl", hash = "sha256:af7c474f0d8f4d2a196370e85033873c7b063a0b5da9a4f143aa023e3bd25407", size = 218365, upload-time = "2026-10-19T16:56:46.41Z" }, + { url = "https://files.pythonhosted.org/packages/77/dd/8b00e1dd552788d8c7762d7ef61de8f2f238d7357b1a02d4c4650b112196/websockets-17.3-cp314-cp314t-win_amd64.whl", hash = "sha256:0c63ad657695e8f1a48624754cb6b464118ac50d4c1d477c26461c98390f0a62", size = 218706, upload-time = "2026-10-19T16:56:47.754Z" }, + { url = "https://files.pythonhosted.org/packages/39/eb/5d90d57a5cbe614b2f13433aaf1c0b068c804589379061ba06812a185a89/websockets-17.3-cp314-cp314t-win_arm64.whl", hash = "sha256:3c4f631dc8d00e470cde3ec0a35d4b58a817bea592411956f1cf4099dbb306b3", size = 218622, upload-time = "2026-10-19T16:56:49.07Z" }, + { url = "https://files.pythonhosted.org/packages/6e/41/1efe5ecae1f32495dc75641704f959bd2d8e56cdc825f3acc5e010a51960/websockets-17.3-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:f300f28dff74ff4148c21e6971e4de6ca0c34e0086a3d95c017c5174ebf85ca5", size = 218211, upload-time = "2026-10-19T16:56:50.423Z" }, + { url = "https://files.pythonhosted.org/packages/01/ec/4edb653533241203db3f11522e122338d4fda50a41ce2a542029086f8eda/websockets-17.3-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:32029441af953e980b9940f042b337fe4ad9a97fa8b17a4ed878ec67b5b89ef4", size = 215944, upload-time = "2026-10-19T16:56:51.766Z" }, + { url = "https://files.pythonhosted.org/packages/d5/b9/ec450b9650925dd5cf27c1f36d0464786fe3ac702ed656879c4c0dd5c4df/websockets-17.3-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:09f007b2150eff015b2109f006e2ac2012d376f41486defa88ae78a776b00cff", size = 216164, upload-time = "2026-10-19T16:56:53.125Z" }, + { url = "https://files.pythonhosted.org/packages/15/1e/63f7fbff9f0540a0331f1a62424c9ff24faa6ba8aec264f55e9e31017eea/websockets-17.3-cp315-cp315-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:7bdc658c9474864893e0b6b94f1390bf6a912952df1870dd93bb7d1a902b5628", size = 225608, upload-time = "2026-10-19T16:56:54.442Z" }, + { url = "https://files.pythonhosted.org/packages/0c/1b/f6679ad61cb8c110033a1b30d561d5c1052983e202157251ff1778f27932/websockets-17.3-cp315-cp315-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:15decc0fe293d9e3f924bd483f3d76f3d0ff26845112a470e3f8842fec2c82a1", size = 226042, upload-time = "2026-10-19T16:56:55.727Z" }, + { url = "https://files.pythonhosted.org/packages/28/c6/ed993aaef8046cbbb939ae367ee3dd5383dbd055f0500a5b8aed101168b6/websockets-17.3-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:997617dd7856c5dad742bf6893b872319156dd43f47587541a59e4a852d52bbc", size = 227203, upload-time = "2026-10-19T16:56:57.076Z" }, + { url = "https://files.pythonhosted.org/packages/aa/96/7057dc71affe6c8252df8648f5a225642228a6fdd8d046a6b9aa1fb0e9eb/websockets-17.3-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a004c1fe5a72c496f28c6c1d8c2dc5cfe3278d99478e18f8b838c70a912b366f", size = 228900, upload-time = "2026-10-19T16:56:58.633Z" }, + { url = "https://files.pythonhosted.org/packages/43/e3/8b3268bb7e72b465e55172b86d55f58aa277fb6d9bac92aee615345874df/websockets-17.3-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:35fe35d20baa661cef4a12ee7bc42eaf2376d43ef9816ab29dd47ca957f7a73a", size = 227684, upload-time = "2026-10-19T16:56:59.975Z" }, + { url = "https://files.pythonhosted.org/packages/59/15/fa31c0a12240b1225c3712ff080d2f7ede4f695ca90e59fdb58e8affabba/websockets-17.3-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:8b3bede7d583bc9f47d8693a29984882e59077db1a5238ad977fc70ae8169741", size = 226580, upload-time = "2026-10-19T16:57:01.371Z" }, + { url = "https://files.pythonhosted.org/packages/22/b5/75d911c206a3035e2ecfad107878e62c06e574b2437fa2cfb4ecef436fa3/websockets-17.3-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dafbb58a149d1ebd5767b3f064f911c5ee0897608005e51c3904b0ada2a8d99c", size = 223967, upload-time = "2026-10-19T16:57:02.839Z" }, + { url = "https://files.pythonhosted.org/packages/0e/f8/f98aa36f611acd5770819f6690125c20d55f221fd6db8d00d284b5f78e89/websockets-17.3-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:80326bc571164bbbb6bead2e35d0c943b23b24fde629032197c58151529d24d8", size = 226689, upload-time = "2026-10-19T16:57:04.458Z" }, + { url = "https://files.pythonhosted.org/packages/1a/36/d52f5dff7597de6fbfc26c3d126abd668137915719aebf97abcc3be77eca/websockets-17.3-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:7d53fc4b90ef6e11de639dc18744d9be8cbd70ccb5ce80027aaffe7b1ab0215a", size = 225353, upload-time = "2026-10-19T16:57:05.857Z" }, + { url = "https://files.pythonhosted.org/packages/d4/10/dcd70ad9f9cd20e8af43284ac250af610e8bf9022da25f5b409153d033b4/websockets-17.3-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:fdd00937f15f5f93c8e53d8060a8c4c010431ab595a9b5b50ddd569a04603944", size = 225879, upload-time = "2026-10-19T16:57:07.249Z" }, + { url = "https://files.pythonhosted.org/packages/70/ab/f26c6bfa99d38ca3d1bf23145306d97c5d0c75e133a8d837373e4b3d4629/websockets-17.3-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:1dae5cd6c92d4440b15f20aa2987baa73c47ea14baa0fec8536a542e82e0ab33", size = 227031, upload-time = "2026-10-19T16:57:08.817Z" }, + { url = "https://files.pythonhosted.org/packages/d7/3f/c8fd74cdb526cc65295b7468f2f84ba16678646d8174288822f99f510274/websockets-17.3-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:b225c656efa2dedecc81651d2290170af25887b7de4f38c5efc7c5ae2b9442ab", size = 224842, upload-time = "2026-10-19T16:57:10.298Z" }, + { url = "https://files.pythonhosted.org/packages/c4/c5/b3a07ce1c54fc0f5441642f5352662b90d7e658c966730eaf6d78a5f20bc/websockets-17.3-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:bd645b5d2f810a181c44f4a71135e7c365b44f4be9ed3d1892635be13a857553", size = 225744, upload-time = "2026-10-19T16:57:11.615Z" }, + { url = "https://files.pythonhosted.org/packages/66/37/8321a8a6ab0215d4df0238e49ace5f539364cf337bb122b85c788e8c93ec/websockets-17.3-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:d2605a3b2af8d66f8b75312d223642c065ec773446c04566d9030ad5e2db13c9", size = 226006, upload-time = "2026-10-19T16:57:13.049Z" }, + { url = "https://files.pythonhosted.org/packages/f8/04/8c457a7e6a260b673bf4e6f2f422eb4af5f11c48505748359706f24f1bef/websockets-17.3-cp315-cp315-win32.whl", hash = "sha256:82e60c00cfb8493a5ba27cdf9b3695fdd153f666f73e11f18968f78ead1d8705", size = 218196, upload-time = "2026-10-19T16:57:14.41Z" }, + { url = "https://files.pythonhosted.org/packages/a0/7c/c2df129d9e344b99bb388bcac54fbbdfeec29e6c716c12dc9a3398757940/websockets-17.3-cp315-cp315-win_amd64.whl", hash = "sha256:4d5f2492cb2192aab2d42e57f2caa2159b4fd20fde06f859ec13354f524b176c", size = 218490, upload-time = "2026-10-19T16:57:16.234Z" }, + { url = "https://files.pythonhosted.org/packages/e6/4f/721b778c85559b2949e46246838e31a346dd815b3cdd8460fe517235675a/websockets-17.3-cp315-cp315-win_arm64.whl", hash = "sha256:55494475206113dbaf731dc8c5ffd5e1094175b753b7bbaad09fe3d7df61165c", size = 218403, upload-time = "2026-10-19T16:57:17.631Z" }, + { url = "https://files.pythonhosted.org/packages/b3/28/241d9ebb4dad79346b5a6abf9f14d7685b12ca358d250eea4920cdb1caf3/websockets-17.3-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:23cb6f6d64e3a389bb8ba150f947f9a6c8c0aca383e814ca3c5cffab738d5646", size = 218523, upload-time = "2026-10-19T16:57:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b0/bb/559fe539992102c66cbca38b488af6c7b184ab7736d819afc0f72268a2b4/websockets-17.3-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:f27e7e8a85512b6a4b490d5756edde8e6772883a5800e1476172e44eda3f7d7c", size = 216103, upload-time = "2026-10-19T16:57:20.412Z" }, + { url = "https://files.pythonhosted.org/packages/34/ee/e8ade4f13a3d0da68636fe44bb8ac1a3e36fc57070512788320dead42067/websockets-17.3-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:6342f79d6f50e680c8aa59e0615d745e867e51ffd59c77093be60b7c433067af", size = 216316, upload-time = "2026-10-19T16:57:21.761Z" }, + { url = "https://files.pythonhosted.org/packages/4d/06/4812b2a7a5e6571d9be425e76cf54ee69aaa15720caa02307785aa5b19a7/websockets-17.3-cp315-cp315t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:fc23e72bd7f8a8e540fed2244524d0a9aeb6ae3325f3d56f04a7a3debe5b04c2", size = 226467, upload-time = "2026-10-19T16:57:23.077Z" }, + { url = "https://files.pythonhosted.org/packages/d3/e8/adfc05d7f9b6b0cce6cc554426c50b7058609119b6b84a39171fe5ed4e50/websockets-17.3-cp315-cp315t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:722ab747ff5dc28070b1a8c020dba6466f13f2ab56bc251ad301fefa2454aa1e", size = 226736, upload-time = "2026-10-19T16:57:24.419Z" }, + { url = "https://files.pythonhosted.org/packages/96/b2/d5b93dd933552554507edae9e3a4890269a8ea8b46019450ea9c8a30e7cc/websockets-17.3-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0d20d6ea5aabbeba2e054d33e405192ef75dc12947bbd00ec97d23b14969d325", size = 228065, upload-time = "2026-10-19T16:57:25.908Z" }, + { url = "https://files.pythonhosted.org/packages/10/37/642f0f1cf1c4814147a4f0586f796a0b38e42b69e5a296bfc270fb660051/websockets-17.3-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:90cc9973c51811918247a78d9ab2f54871296e4e37e69a98aba4d3c9acedbf62", size = 229692, upload-time = "2026-10-19T16:57:27.63Z" }, + { url = "https://files.pythonhosted.org/packages/5b/33/eac81cf62ac394b61db6a27ccbe8fb9ae382016ef73ede1edf9242a6a43e/websockets-17.3-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:63b87c93e28957e6dc2bb92d31727a863c9bf4a4daa080f0995fdcb0242c9166", size = 228503, upload-time = "2026-10-19T16:57:29.031Z" }, + { url = "https://files.pythonhosted.org/packages/0d/84/e057788a9bd056f4f7d3a06b59efaaf724bbc6f2d1e1d316f8a93c951abc/websockets-17.3-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4f7431e71072962d3ed1768038200aa4f1e76d9c2d7bedd554d57a55faaf6644", size = 227379, upload-time = "2026-10-19T16:57:30.571Z" }, + { url = "https://files.pythonhosted.org/packages/fa/a7/cca1f9f45b893bb5b7545f2b234f910e07fcd2f8065724ac1a62a0961be2/websockets-17.3-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:534ed583fd4323de45d196535a444cebd1af2105e6e4e0cdcbf8cea266a8747a", size = 224719, upload-time = "2026-10-19T16:57:31.895Z" }, + { url = "https://files.pythonhosted.org/packages/1f/a0/5d07451027150c802ce4c9999e86994a3f0de5f4dbe3d1cce46995999269/websockets-17.3-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:8879b6a4040ee925c638548238ab7fe612cf5fa8752fe847346da0da8be81c3f", size = 227564, upload-time = "2026-10-19T16:57:33.236Z" }, + { url = "https://files.pythonhosted.org/packages/62/90/835f37e13fac0c615a700caae0d39deff1de2d2b9b341836a786f9b359e0/websockets-17.3-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:58e5d1e935a90878a935c7d2737a25c71cf4f1c13b6e6bef454ab722c4f338bb", size = 226218, upload-time = "2026-10-19T16:57:34.876Z" }, + { url = "https://files.pythonhosted.org/packages/16/8a/718639939b947fe715501e8073350fd3f8a42f8b5eba3c0c2389c5836513/websockets-17.3-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:0a657b51e4a5f4ac577c11e706ff9a884ea5aaf3e2666bae283a251c63e66acb", size = 226712, upload-time = "2026-10-19T16:57:36.248Z" }, + { url = "https://files.pythonhosted.org/packages/e8/45/6bef8572b9b0dee51369b69776af1e4c7cc3a807f199dc3c165bbcf1199a/websockets-17.3-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:411aa0fb234f42a1444dea1e177890ddfc577b36ffdfafec25cb0d2c818e1af8", size = 227819, upload-time = "2026-10-19T16:57:37.586Z" }, + { url = "https://files.pythonhosted.org/packages/f1/3f/3b726d0b46a132e641d2d8c0243ea8a7d39464dbaf28c7a206e51fbae681/websockets-17.3-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:623d4b44f64efdf235b1e8dce266057073d93cde6197e7d7684f20c297f67f30", size = 225573, upload-time = "2026-10-19T16:57:39.285Z" }, + { url = "https://files.pythonhosted.org/packages/90/96/cd972a12e1412d9b42c42ef42efbd81aa396ca9408d982d524012f011a8d/websockets-17.3-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:5d0c871efc696ef5eb8514df64b86ab28736db753ebb42a2101f3b562f4849c0", size = 226430, upload-time = "2026-10-19T16:57:40.915Z" }, + { url = "https://files.pythonhosted.org/packages/7d/2d/e11524771588188a39f6da234779d4942496b96c5a10b92f3df30db48608/websockets-17.3-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:1db2604e6aedb7a2cb2f6e0575d6c9a852ccac8239703a930ac83b63309177a5", size = 226728, upload-time = "2026-10-19T16:57:42.245Z" }, + { url = "https://files.pythonhosted.org/packages/17/9b/63722aed973f53ea579e1ce0c4563a2b8b536629e556feff26e4c6a2a60d/websockets-17.3-cp315-cp315t-win32.whl", hash = "sha256:07246d4f0293ef0b87eff3d7dd88c1f03dd9f678fa39f6de87b2af8504e7e0de", size = 218325, upload-time = "2026-10-19T16:57:43.585Z" }, + { url = "https://files.pythonhosted.org/packages/34/07/5e0c92e225546f1789c957c0383852797be7c899b07207e9b6eb4a93473f/websockets-17.3-cp315-cp315t-win_amd64.whl", hash = "sha256:856bbb24f2ee6436af7609267e73309e4bb0a0f9982f79f1c2e01019279c6671", size = 218641, upload-time = "2026-10-19T16:57:44.918Z" }, + { url = "https://files.pythonhosted.org/packages/d4/e4/339663de65c44ab3eb5209890e86eed26e1a131d6afab3df6f1c3e5b4d8c/websockets-17.3-cp315-cp315t-win_arm64.whl", hash = "sha256:a22c4956a4d858200dba83f0691bb692d77b0eeddfca80157820d20b85dea4f1", size = 218564, upload-time = "2026-10-19T16:57:46.332Z" }, + { url = "https://files.pythonhosted.org/packages/d7/b0/3f01fcb3406662f6deb1e3e72b34dd0979e993991c0b7435a9e12bc14f02/websockets-17.3-py3-none-any.whl", hash = "sha256:dc2b495a92b3971966bf102e73589f950db3ec1ca6713265355ce7771d97b199", size = 212375, upload-time = "2026-10-19T16:58:04.483Z" }, ]