services/dashboards
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 29_gateway_dashboard_page

Browse Source 
Phase 29 completion pass is done. Final state:

**Verification (all re-run this pass, no task files touched)**
- `uv run pytest` → **1746 passed, 16 skipped**; `uv run pytest --cov` → **99.05 %** (gate 94 %); `bash .agents/validate.sh` → **rc=0** (ruff clean, pyright `0 errors, 0 warnings`, `node --check` clean); `uv run pytest -m live --live` → **12 passed**.
- Live, read-only: dev server **10/10 sources `ok`** (incl. `gateway`); `data/logs/29-task06-live.cjs` → **49 PASS / 0 FAIL** with per-port inbound counts reconciled against a direct `/proc/net/tcp{,6}` read (divergence 0); `data/logs/verify-29-task06-predates.py` → 0 failed (1.1.0 arm renders `WHY_PREDATES`, no pipes, every tally `n/a`, job still `ok`); phase-27 mobile audit re-run over all nine served pages → **371 PASS / 0 FAIL / 0 FINDING** (`data/logs/29-final-mobile-audit.out`).
- Criteria: (1) unconfigured `GATEWAY_HOST` → `404 {configured:false}` with dial spies + `#gate-empty` sentence ✅; (2) 1.1.0/`--no-proxy` → `supported:false`, no `0`s, never red ✅; (3) live gateway 1.2.0 → pipes animate, totals reconcile ✅; (4) no `severity`/verdict colour anywhere, validate green ✅; (5) docs updated + gap 13 logged ✅; (6) the five existing dashboards, jobs and `/api/summary` unchanged (live control leg) ✅.

**Docs (consistency pass, incl. pre-existing drift)**
- `README.md`: architecture diagram now lists `sources/gateway.py`, the 30 s gateway job and the third `/snapshot` reader; `GATEWAY_HOST` row says *a name, never a URL*; the `POLL_MS_*` row states `/gateway`'s fixed 15 s browser poll; "nine pollers" → per-source cadences; phase-28 "until phase 29" wording → current state; new Testing paragraph for `tests/test_gateway_source.py` (join pinned from the captures, reconciliation identity, `getaddrinfo` spy).
- `AGENTS.md`: phase-28 stop-point wording replaced with the shipped phase-29 state; suite/coverage figures → 1746 / 16 skipped / 99.08 %→99.05 % band and `~99 %`; window-chips paragraph records that `/gateway` has **no** chips (fixed 24 h); new testing-policy bullet for the join; stale page counts fixed (seven → eight tabbed pages / `MARKUP_PAGES`, clock-order and badge-count transcripts annotated); gap 12 now records the phase-29 nine-page audit re-run.
- `tests/test_api.py`: comment/docstring-only fixes for the "seven pages" wording that phase 29 made eight.

**Deviations / notes**
- Nothing implemented: all six task files were already in `complete/`, so this run was the final verification + docs pass; no git index or phase file was touched.
- Coverage wobbles 99.05–99.08 % between runs on a pre-existing timing-dependent line (`collectors.py` `/slots`-alias `except`/`break`), not a phase-29 path (`sources/gateway.py` is 100 %).
- `uv pip list --outdated` still lists upgrades; dependencies did not move in this phase, so they were left alone rather than upgraded mid-verification.
- Historical browser-transcript counts (phase 18/20/25 script runs) were left as recorded history and annotated instead of rewritten.

**Next phase:** none — `.agents/phases/todo/` is empty after 29; further work needs a new phase authored first.
This commit is contained in:
ducoterra
2026-10-23 09:52:30 -04:00
parent 44d61d3ab9
commit fa4f38c694
68 changed files with 5714 additions and 200 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/29_gateway_dashboard_page/00_phase.md → .agents/phases/complete/29_gateway_dashboard_page/00_phase.md
View File
.agents/phases/todo/29_gateway_dashboard_page/01_config_and_source.md → .agents/phases/complete/29_gateway_dashboard_page/01_config_and_source.md
View File
.agents/phases/todo/29_gateway_dashboard_page/02_wiring.md → .agents/phases/complete/29_gateway_dashboard_page/02_wiring.md
View File
.agents/phases/todo/29_gateway_dashboard_page/03_api_routes.md → .agents/phases/complete/29_gateway_dashboard_page/03_api_routes.md
View File
.agents/phases/todo/29_gateway_dashboard_page/04_page.md → .agents/phases/complete/29_gateway_dashboard_page/04_page.md
View File
.agents/phases/todo/29_gateway_dashboard_page/05_nav_and_summary.md → .agents/phases/complete/29_gateway_dashboard_page/05_nav_and_summary.md
View File
.agents/phases/todo/29_gateway_dashboard_page/06_docs_and_verify.md → .agents/phases/complete/29_gateway_dashboard_page/06_docs_and_verify.md
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__00_phase.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__00_phase.a1.md
+19
View File
@@ -0,0 +1,19 @@
Phase 29 completion pass is done. Final state:
**Verification (all re-run this pass, no task files touched)**
- `uv run pytest` → **1746 passed, 16 skipped**; `uv run pytest --cov` → **99.05 %** (gate 94 %); `bash .agents/validate.sh` → **rc=0** (ruff clean, pyright `0 errors, 0 warnings`, `node --check` clean); `uv run pytest -m live --live` → **12 passed**.
- Live, read-only: dev server **10/10 sources `ok`** (incl. `gateway`); `data/logs/29-task06-live.cjs` → **49 PASS / 0 FAIL** with per-port inbound counts reconciled against a direct `/proc/net/tcp{,6}` read (divergence 0); `data/logs/verify-29-task06-predates.py` → 0 failed (1.1.0 arm renders `WHY_PREDATES`, no pipes, every tally `n/a`, job still `ok`); phase-27 mobile audit re-run over all nine served pages → **371 PASS / 0 FAIL / 0 FINDING** (`data/logs/29-final-mobile-audit.out`).
- Criteria: (1) unconfigured `GATEWAY_HOST` → `404 {configured:false}` with dial spies + `#gate-empty` sentence ✅; (2) 1.1.0/`--no-proxy` → `supported:false`, no `0`s, never red ✅; (3) live gateway 1.2.0 → pipes animate, totals reconcile ✅; (4) no `severity`/verdict colour anywhere, validate green ✅; (5) docs updated + gap 13 logged ✅; (6) the five existing dashboards, jobs and `/api/summary` unchanged (live control leg) ✅.
**Docs (consistency pass, incl. pre-existing drift)**
- `README.md`: architecture diagram now lists `sources/gateway.py`, the 30 s gateway job and the third `/snapshot` reader; `GATEWAY_HOST` row says *a name, never a URL*; the `POLL_MS_*` row states `/gateway`'s fixed 15 s browser poll; "nine pollers" → per-source cadences; phase-28 "until phase 29" wording → current state; new Testing paragraph for `tests/test_gateway_source.py` (join pinned from the captures, reconciliation identity, `getaddrinfo` spy).
- `AGENTS.md`: phase-28 stop-point wording replaced with the shipped phase-29 state; suite/coverage figures → 1746 / 16 skipped / 99.08 %→99.05 % band and `~99 %`; window-chips paragraph records that `/gateway` has **no** chips (fixed 24 h); new testing-policy bullet for the join; stale page counts fixed (seven → eight tabbed pages / `MARKUP_PAGES`, clock-order and badge-count transcripts annotated); gap 12 now records the phase-29 nine-page audit re-run.
- `tests/test_api.py`: comment/docstring-only fixes for the "seven pages" wording that phase 29 made eight.
**Deviations / notes**
- Nothing implemented: all six task files were already in `complete/`, so this run was the final verification + docs pass; no git index or phase file was touched.
- Coverage wobbles 99.05–99.08 % between runs on a pre-existing timing-dependent line (`collectors.py` `/slots`-alias `except`/`break`), not a phase-29 path (`sources/gateway.py` is 100 %).
- `uv pip list --outdated` still lists upgrades; dependencies did not move in this phase, so they were left alone rather than upgraded mid-verification.
- Historical browser-transcript counts (phase 18/20/25 script runs) were left as recorded history and annotated instead of rewritten.
**Next phase:** none — `.agents/phases/todo/` is empty after 29; further work needs a new phase authored first.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__00_phase.a1.validate
+74
View File
@@ -0,0 +1,74 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 24%]
........................................................................ [ 28%]
........................................................s............... [ 32%]
........................................................................ [ 36%]
........................................................................ [ 40%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 53%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
............................................................ssssssssssss [ 73%]
........................................................................ [ 77%]
........................................................................ [ 81%]
........................................................................ [ 85%]
........................................................................ [ 89%]
........................................................................ [ 93%]
................................................sss..................... [ 98%]
.................................. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 491 0 100%
src/dashboards/app.py 190 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 5 98% 323-324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 321 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3894 37 99%
Required test coverage of 94% reached. Total coverage: 99.05%
1746 passed, 16 skipped, 1 warning in 167.20s (0:02:47)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.10.19 2026.10.23 wheel
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__01_config_and_source.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__01_config_and_source.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Implemented**
- `config.py`: `gateway_host` (`GATEWAY_HOST`, default `gateway`), `gateway_interval` (30 s), `gateway_timeout` (10 s), `gateway_enabled` (name must be a `HOST_TARGETS` entry, `nut_enabled` shape) and it now counts in `any_source_enabled`.
- New `src/dashboards/sources/gateway.py`: `parse(snapshot, host)` joins agent 1.2.0's `haproxy`/`nginx`/`sockets` into inbound pipes (per listen port: domains, live count, distinct clients) and outbound pipes (haproxy `server`, concrete `proxy_pass`, `$var`→`map` fan-out), plus `unmatched` tallies, `samples()` gauges (`gateway.in_conns@<port>`, `gateway.out_conns@<host>:<port>`) and `parse_gateway_snapshot() -> Result`. Gauges only: no estimator/`Δ` window, no `severity`, `null` (never `0`) whenever the socket table or the join cannot answer; `supported:false` (1.1.0/`--no-proxy`) + per-section `why` + verbatim `section_errors`.
- Join never resolves a name: `socket.getaddrinfo` spy pins zero calls; exact (ip,port) match `by_port:false`, name matched on port only when that port has one configured host (`by_port:true`), several names on one port → nobody chosen (`connections:null`, sockets in `unmatched.outbound`, one `unmatched.groups` entry naming port/count/candidates). `join_sockets` attributes each socket once, so inbound+outbound+unmatched == the socket table.
- Tests: `tests/test_gateway_source.py` (35, incl. a fixture built by running the **real agent `Collector`** over the captured configs via new `gateway_snapshot` conftest fixture), 3 new `tests/test_config.py` cases, and a read-only `--live` test against the real gateway agent. Probe: `data/logs/verify-29-task01.py` → `29-task01-live.out`.
- Docs: `.env.example` + `.env.dev.example` gateway block, README Configuration rows, AGENTS.md (config bullet, new `sources/gateway.py` responsibility bullet, metric-naming/gauge family), `store.py` family docstring (`disks`/`gateway` added).
**Results** — `uv run pytest`: 1691 passed, 16 skipped; `uv run pytest --cov`: 99.05% total, `sources/gateway.py` and `config.py` 100%; `uv run ruff check src tests agent` + `uv run pyright`: clean; `bash .agents/validate.sh`: green; `uv run pytest -m live --live`: 12 passed; dev server restarted, all nine sources `ok`. Live join: 10 inbound / 31 outbound / 17 map entries, partition `19 + 1 + 8 = 28 == established` OK, zero `severity` keys.
**Deviations (documented in code + AGENTS.md)**
- Outbound series carry the port (`@host:port`, vs the phase's host-only sketch): this gateway's two haproxy Minecraft backends share `minecraft.reeselink.com` on 25565/25566, and one name would `INSERT OR REPLACE` one pipe's gauge over the other in silence (rule 2).
- The shared-port fan-out (16 `*.reeselink.com:443` map entries) yields `null` counts + `unmatched.groups` rather than a guessed or repeated attribution; `unmatched.other` is defined as live listeners no config claims (phase 28's "unclaimed socket").
- No live infra changes were made (read-only dial only). Note: `NOTES.md` is empty in the working tree (pre-existing, not this task); its two items are already shipped (phase 24 mTLS, phase 20 task 04 admin links), so I left it as-is.
**Next pending task:** `.agents/phases/todo/29_gateway_dashboard_page/02_wiring.md` (the `gateway` poller job, alerts rule, `/api/summary` block, `/api/config` entry).
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__01_config_and_source.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__01_config_and_source.a1.validate
+72
View File
@@ -0,0 +1,72 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 21%]
........................................................................ [ 25%]
........................................................................ [ 29%]
.......s................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 46%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 63%]
........................................................................ [ 67%]
........................................................................ [ 71%]
.....ssssssssssss....................................................... [ 75%]
........................................................................ [ 80%]
........................................................................ [ 84%]
........................................................................ [ 88%]
........................................................................ [ 92%]
.................................................................sss.... [ 97%]
................................................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 440 0 100%
src/dashboards/app.py 186 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 206 4 98% 304, 308-309, 326
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 307 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3809 36 99%
Required test coverage of 94% reached. Total coverage: 99.05%
1691 passed, 16 skipped, 1 warning in 163.88s (0:02:43)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__02_wiring.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__02_wiring.a1.md
+18
View File
@@ -0,0 +1,18 @@
Everything is green. Final state:
**Implementation** (`src/dashboards/`)
- `collectors.py`: `fetch_gateway_snapshot()` (third reader of the same agent `/snapshot`) + `gateway_job()` — **one** job (the `nut` shape), key/dashboard `gateway`, `label` = `GATEWAY_HOST`'s name, `GATEWAY_INTERVAL_SECONDS`/`GATEWAY_TIMEOUT_SECONDS`, **no** `RateEstimator`, `None` when the name isn't a target (invariant 3).
- `app.build_hub`: adds that job between `disks` and `llm` on the **same** `AgentTls` entry as `host:`/`disks:`, so one revoke fails all three without a dial.
- `alerts.py`: `HREF["/gateway"]`, staleness in the gateway's own 30 s interval, `AGENT_ECHO_DASHBOARDS = ("disks", "gateway")` with `_partner_host()` (gateway finds its host by **label**, not key) — dead agent = one line, wedged sections while the host answers = the only voice; **no** gateway threshold or payload argument.
- `api.py`: additive `/api/summary.gateway` (`configured, host, supported, unreachable, inbound, outbound, map_targets, unmatched, unmatched_listeners, partial, states`; `null` never `0` in the arms that didn't measure) and the `/api/config.gateway` row with redacted `display_url`.
**Tests/lint/coverage**
- 19 new tests (5 alerts, 11 api, 3 collectors); full suite **1712 passed, 16 skipped**, coverage **99.07%** (was 99.05%), `alerts.py`/`api.py`/`app.py` back to **100%**; `ruff` + `pyright` clean; `bash .agents/validate.sh` exit 0 (dep list is the pre-existing advisory).
- Mutation-checked: breaking the gateway label lookup fails 3 pins, so the echo rule isn't vacuous.
- Live (`--live`, `.env.dev`): **12 passed**; dev server restarted — `/api/health` **ten sources ok** incl. `gateway`, `/api/summary.gateway` live (in 12 / out 4 / map 17 / unmatched 10+2 / partial), `/api/config.gateway` `{enabled, gateway, https://10.1.1.27:9120, 30, 10}`.
**Docs**: AGENTS.md (poller job-of-its-own, collectors, alerts, api block, app, "ten sources" fleet fact) and README (alerts echo/interval rule, summary+config blocks).
**Decisions**: `states` as the repo-standard list of health records (not a `{gateway: str}` map); added `partial` so a sum that omitted an unattributable port isn't read as the total; no `severity` anywhere.
**Next pending task**: `29_gateway_dashboard_page/03_api_routes.md` — `/api/gateway/snapshot` + `/api/gateway/history`.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__02_wiring.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__02_wiring.a1.validate
+72
View File
@@ -0,0 +1,72 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 25%]
........................................................................ [ 29%]
.........................s.............................................. [ 33%]
........................................................................ [ 37%]
........................................................................ [ 41%]
........................................................................ [ 45%]
........................................................................ [ 50%]
........................................................................ [ 54%]
........................................................................ [ 58%]
........................................................................ [ 62%]
........................................................................ [ 66%]
........................................................................ [ 70%]
..........................ssssssssssss.................................. [ 75%]
........................................................................ [ 79%]
........................................................................ [ 83%]
........................................................................ [ 87%]
........................................................................ [ 91%]
........................................................................ [ 95%]
..............sss....................................................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 473 0 100%
src/dashboards/app.py 189 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 4 98% 324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 307 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3861 36 99%
Required test coverage of 94% reached. Total coverage: 99.07%
1712 passed, 16 skipped, 1 warning in 164.73s (0:02:44)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__03_api_routes.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__03_api_routes.a1.md
+13
View File
@@ -0,0 +1,13 @@
**Task 03 complete — `/api/gateway/snapshot` + `/api/gateway/history`.**
**Implementation** — `api.gateway_router()` (both GETs, included after `disks_router()` in `app.create_app`): `/snapshot` tests `settings.gateway_enabled` in its first line and raises `404 {"configured": false}` naming `GATEWAY_HOST` before any client/fetch/store read; answering → `sources/gateway.parse`'s payload verbatim + `states` (payload `ts` untouched); never-answered → new `gateway_source.stub(host, ts)` (`sources/gateway.py`: `_skeleton()` shared with `parse`, `WHY_NO_ANSWER`, `unreachable: true`, `supported: null`, every count `null`). `/history` goes through the one `_history_batch()` → `store.history_many()` door, `hosts=` defaulting to the ingress alone (no configured gate, since history dials nothing — the `/api/disks/history` rule).
**Tests** — 16 new tests in `tests/test_api.py` (both unconfigured arms — unset name and a name that is not a target — spy-pinned on `httpx.AsyncClient.get` + `collectors.fetch_snapshot`, with a control leg that names a real target; the stub's keys/counts/`states`; the payload↔`parse` key-set equality; inbound+outbound+unmatched == `sockets.established`; a recursive no-`severity`/no-`window_seconds` walk; history points re-read through `store.query`; unknown names → empty series; host default; family isolation in `/api/metrics?source=`; the two GETs, 405 on the rest, and the app's one upstream POST still `/api/ups/command`), plus the two paths added to `test_auth_api.py::PROTECTED_API`. Both controls were verified by mutation (gate moved after the fetch → refusal tests fail; batch source changed `gateway`→`host` → history tests fail).
**Gate** — `uv run pytest --cov`: **1728 passed, 16 skipped**, coverage **99.05%** (`api.py`, `app.py`, `sources/gateway.py` each 100%); `ruff` and `pyright` clean; `bash .agents/validate.sh` **exit 0** (only the pre-existing dependency advisory; no deps moved); `-m live --live` 12 passed. Dev server restarted: all ten sources `ok`, and live `/api/gateway/snapshot` against agent 1.2.0 answered 6 frontends / 17 map targets / 31 outbound pipes with the identity reconciling exactly (39 == 39), while `/history` served `gateway.in_conns@443` and empty for a name that never measured.
**Docs** — README and AGENTS `api.py` bullets now describe the pair (three arms, the gate-before-dial, the stub, the port-in-the-name gauges, no write), AGENTS records `gateway_source.stub()` and that all five `history` routes share the batched door.
**Deviations** — none. `tests/card_contract.py` untouched (task 04's `COLUMNS` device); task file left in `todo/`, no git commands run.
**Next task** — `.agents/phases/todo/29_gateway_dashboard_page/04_page.md`.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__03_api_routes.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__03_api_routes.a1.validate
+73
View File
@@ -0,0 +1,73 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 24%]
........................................................................ [ 28%]
.......................................s................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 41%]
........................................................................ [ 45%]
........................................................................ [ 49%]
........................................................................ [ 53%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 66%]
........................................................................ [ 70%]
..........................................ssssssssssss.................. [ 74%]
........................................................................ [ 78%]
........................................................................ [ 82%]
........................................................................ [ 86%]
........................................................................ [ 90%]
........................................................................ [ 94%]
..............................sss....................................... [ 99%]
................ [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 491 0 100%
src/dashboards/app.py 190 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 5 98% 323-324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 321 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3894 37 99%
Required test coverage of 94% reached. Total coverage: 99.05%
1728 passed, 16 skipped, 1 warning in 167.39s (0:02:47)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__04_page.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__04_page.a1.md
+10
View File
@@ -0,0 +1,10 @@
**Task 04 complete — `/gateway` page with animated pipes. Everything green.**
- **Page**: `web/pages/gateway.html` + `web/static/page-gateway.js` + a `/* ---- gateway pipes */` CSS section; route added in `app.py`. Four cards (`haproxy`, `nginx`, `map`, `unmatched`) keyed by each pipe's own `source`; hand-rolled SVG pipes (`<path>`, `vector-effect`), node label = `snapshot.host`, solo-side layouts for the stream map.
- **Honesty pins**: labels/counts/clients are `textContent` only; `null` → `n/a` (no `|| 0` on any count); a pipe animates **only** when `connections > 0`, `.still` otherwise; `prefers-reduced-motion` names **both** states; no `ok|warn|fail` token anywhere in the gateway CSS section; every plot carries `ariaLabel`; one `/api/gateway/history` request per card, plot built only while its `Charts` disclosure is open.
- **Degradation** (browser-proved, all from payload/route text): unconfigured → setup block naming `GATEWAY_HOST` + agent 1.2.0, zero red, all `n/a`; predates / never-answered → banner prints the payload's own `why`, nothing drawn, dimmed; silent socket table → config map stays, every count `n/a`.
- **Two defects found and fixed**: `LineChart._yExtent()` padded a flat series **below** its declared `yMin` (an idle pipe drew a `-1` connection axis) — shared fix, measured on `/system` `/disks` `/llm` `/ups` (no axis moved); and an all-unknown remainder/total rendered a card of `n/a`s or a `0` — now hidden/`n/a`.
- **Tests**: 14 new served-byte pins in `tests/test_api.py` (+3 existing audits widened to nine pages/twelve assets), `/gateway` added to `tests/test_auth_api.py`'s protected pages and served route table.
- **Browser (live fleet)**: `data/logs/29-task04-{pipes,degradation,unconfigured,charts-floor}.cjs` → all pass; 41 live pipes, `15 + 4 + 11 = 30 = established`; screenshots `29-task04-{gateway,charts-open,phone,predates,sockets_silent,unanswered,unconfigured}.png`, transcripts `.out`.
- **Gate**: `pytest` 1742 passed / 16 skipped, coverage 99.08 % (floor 94), `ruff` + `pyright` clean, `node --check` clean; dev server up with all ten sources `ok`.
- Docs (`AGENTS.md`/`README.md`) and the sixth masthead tab are task 05/06's deliverables and were deliberately left untouched; `PAGES` in `test_api.py` stays the five-tab table so task 05 moves it as one contract.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__04_page.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__04_page.a1.validate
+73
View File
@@ -0,0 +1,73 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 24%]
........................................................................ [ 28%]
....................................................s................... [ 32%]
........................................................................ [ 36%]
........................................................................ [ 40%]
........................................................................ [ 45%]
........................................................................ [ 49%]
........................................................................ [ 53%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
........................................................ssssssssssss.... [ 73%]
........................................................................ [ 77%]
........................................................................ [ 81%]
........................................................................ [ 86%]
........................................................................ [ 90%]
........................................................................ [ 94%]
............................................sss......................... [ 98%]
.............................. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 491 0 100%
src/dashboards/app.py 190 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 5 98% 323-324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 321 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3894 37 99%
Required test coverage of 94% reached. Total coverage: 99.05%
1742 passed, 16 skipped, 1 warning in 161.90s (0:02:41)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__05_nav_and_summary.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__05_nav_and_summary.a1.md
+18
View File
@@ -0,0 +1,18 @@
All gates green. Final state:
**Done (task 05)**
- `Gateway` tab added to all **eight** signed-in pages (six dashboards + `/settings` + `/certs`), `aria-current` on `/gateway` only; tab audit rewritten as `nav hrefs == TAB_ORDER` (`PAGES` is now the six dashboards) with per-page `aria-current` asserted in both directions; `test_the_certs_page_is_a_page_and_not_a_sixth_dashboard` renamed `…_not_a_dashboard` now that `/gateway` *is* the sixth.
- Overview ingress line: `#fleet-gate` in the fleet panel, rendered by `renderGateway()` from `/api/summary`'s block (`GATEWAY_FIELDS` resolved against the live payload — the `SLOT_FIELDS` device), `.stale` when unreachable, **hidden when `configured: false`**. `/api/summary`'s `gateway` block gained an additive `href` derived from `alerts.HREF`, so the row links with data instead of a page-side route literal (same rule the alert rows keep).
- Pins: `SUMMARY_CONTRACT` now names every gateway key; `NAV_STATUS_BADGES` gained `gateway`; new `test_the_overview_line_reads_the_gateway_block_it_claims` and `test_the_gateway_block_links_to_the_page_the_alert_row_links_to`.
- Docs: AGENTS.md (six-page/nav/badge counts, the ingress line, the `href` derivation, the wrap rule, the nine-page audit) and README.md updated.
**Deviations / defects found**
- Task assumption "the fifth `#nav-status` badge with **no JS change**" was false — `initChrome()`'s badge table is a literal, so app.js gained `{ key: 'gateway', label: 'gateway', href: '/gateway' }` (pinned).
- The fifth badge **broke phase 18's guarantee**: `#nav-status` was `nowrap` at ≤640px, so five chips (352px) widened the document to 372px — a sideways scroll on a phone. Fixed by letting the badge row wrap inside its own row (no label shrunk, nothing hidden); the served-CSS pin flipped to demand `wrap`, and the phase-27 mobile audit (extended to include `/gateway`) re-runs clean.
**Verification**
- `bash .agents/validate.sh`: **1744 passed, 16 skipped**, coverage 99.08%, ruff + pyright + `node --check` clean.
- `data/logs/29-task05-tabs.cjs` → `29-task05-tabs.out`: **97 PASS / 0 FAIL** (8 pages × 1280/360 six tabs + one-own-current + five populated badges + no sideways scroll; 360px dropdown lists all six; the tab navigates and draws the tallies; the `/` line reads the block's counts/href, and a wire-simulated `configured:false` renders nothing). Screenshots `data/logs/masthead-29-*.png`.
- `data/logs/27-task02-mobile-audit.cjs` (now including `/gateway`): **0 failures, 0 findings**.
- Dev server up; `/api/health` and `dev-server.sh status` report all ten sources `ok`.
- validate.sh's dependency advisory (coverage/fastapi/pycparser/ruff newer) is pre-existing; this task touched no dependencies.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__05_nav_and_summary.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__05_nav_and_summary.a1.validate
+73
View File
@@ -0,0 +1,73 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 24%]
........................................................................ [ 28%]
......................................................s................. [ 32%]
........................................................................ [ 36%]
........................................................................ [ 40%]
........................................................................ [ 45%]
........................................................................ [ 49%]
........................................................................ [ 53%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
..........................................................ssssssssssss.. [ 73%]
........................................................................ [ 77%]
........................................................................ [ 81%]
........................................................................ [ 85%]
........................................................................ [ 90%]
........................................................................ [ 94%]
..............................................sss....................... [ 98%]
................................ [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 491 0 100%
src/dashboards/app.py 190 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 5 98% 323-324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 321 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3894 37 99%
Required test coverage of 94% reached. Total coverage: 99.05%
1744 passed, 16 skipped, 1 warning in 164.35s (0:02:44)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__06_docs_and_verify.a1.err
View File
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__06_docs_and_verify.a1.md
+34
View File
@@ -0,0 +1,34 @@
Task 06 is complete. Final state:
## Done
**Docs**
- `README.md` — intro now *five* dashboards; `/gateway` page-table row (knob + **agent 1.2.0** requirement + the unconfigured-rather-than-red arm), the `GATEWAY_HOST` / `GATEWAY_INTERVAL_SECONDS` / `GATEWAY_TIMEOUT_SECONDS` config rows, and the user-facing paragraph on the topology question and both degraded arms.
- `AGENTS.md` — route-table row, the `/gateway` UI-conventions paragraph (pipes, `n/a`-not-`0`, no verdict colour, the label-line rule, one history request per card), the metric-naming paragraph's `gateway.in_conns@<listen-port>` / `gateway.out_conns@<host>:<port>`, the config bullet's `default \`gateway\``, and **gap 13** (the residual label width, with the rejected alternative and three costed options).
- New pin `tests/test_api.py::test_the_docs_name_the_page_its_knob_and_its_gauges` so prose can't rot: both page rows name `/gateway` + `GATEWAY_HOST` + 1.2.0, the bullet names the default, the gauges and `/api/gateway/*` stay named.
**Live read-only pass** — `data/logs/29-task06-live.cjs` → `29-task06-live.out`: **49 PASS / 0 FAIL**
- payload: `configured`/`supported`, agent `1.2.0`, **6 haproxy frontends** (each port→backend), nginx listening on **80/443/4443/8443**, **17 stream-map entries** (fan-out 443×16 + 4443×1), every entry's key/host/port complete.
- page: 4 cards, 41 aria-labelled pipes matching the payload's sentence shape, header tallies = payload figures, `in + out + unmatched == sockets.established` exact, `by port` chip on every `by_port` pipe, collapsed→open chart labelled with legend rows naming exactly that card's pipes.
- `/proc/net/tcp{,6}` via `ssh gateway-ops cat`: per-port inbound == ESTABLISHED-on-that-port (≤1 tolerance), `unmatched.inbound` == the port complement (`2222`, `9120`), `by_port` spot-check page 2 vs /proc 2, ambiguous `:443` → 16 null pipes + one `groups` sentence, and nulls never became `0`.
- control leg: nine pre-existing sources `ok`; `/` 3 fleet rows, `/system` 3 cards, `/disks` 2 hosts (gateway still hidden), `/llm` 2 engines, `/ups` 2 UPSes; `/`'s ingress line renders and disappears on `configured:false`; 360px has no sideways scroll.
- Screenshots: `29-task06-{gateway,card-haproxy,card-stream,charts-expanded,phone}.png`. Transcripts grepped clean of PEM/token/cookie.
**Pre-1.2.0 arm** — `data/logs/verify-29-task06-predates.py` (+ `.cjs`) → **PASS, 0 failed checks**: throwaway loopback dashboard + scratch SQLite + a stub agent from `tests/fixtures/agent_gateway.json` stamped `1.1.0` with the three sections stripped → `supported: false` with `WHY_PREDATES`, page prints that sentence, draws no cards, every tally `n/a`, banner `info` not red, job `ok`. Screenshot `29-task06-predates.png`.
## Defect found and fixed
The pass caught the page clipping its own answer: pipe labels showed **seven characters** of a 40-character target (`.pipe-label` was one `nowrap` flex row, so the name shrank to what the chip/footnote/count left). Wrapping the text inside that box was tried and measured worse (eleven lines of one word), so the fix gives the label the line: `flex-wrap: wrap` + `flex: 1 1 max-content` on `.pipe-name` → the name now gets the whole column and the chip/footnote/count wrap beneath. `test_a_clipped_pipe_label_keeps_its_full_text_on_the_element` pins both declarations plus the `title`/`aria-label` that keep the remaining ellipsis from being a lost fact; the residual (28 of 42 at 1440px, 16 of 42 at 360px) is gap 13 with the options ordered by cost.
## Gates
- `bash .agents/validate.sh` → **rc=0**: 1746 passed, 16 skipped, coverage **99.05 %**, ruff clean, pyright 0 errors, `node --check` clean.
- `./scripts/dev-server.sh status`: all **ten** sources `ok`.
- Regression control for the CSS change: phase 27's nine-page audit re-ran — **371 PASS, 0 failures, 0 findings**.
- validate.sh's dependency advisory (coverage/fastapi/pycparser/ruff newer) is pre-existing; no dependency moved.
## Notable decisions
- The kernel cross-check asserts the **payload's own partition** (`attributed + unmatched == established`) and reports the raw `/proc` total as a finding — two reads 0.8 s apart cannot be equal on a busy ingress, and pretending otherwise would make the pass flaky about a fact that isn't a bug. The per-port table is the real join proof.
- Payload and DOM are read payload → probe → payload, retrying if `ts` moved, so a 30 s poll can't compare a picture to a newer document.
- The 1.1.0 arm was proven on a loopback stub, never by rolling a fleet host back.
## Next pending task
This was phase 29's last file — `.agents/phases/todo/29_gateway_dashboard_page/` now holds only `00_phase.md` and `06_docs_and_verify.md`, so the phase is ready to move to `complete/`. No other phase is in `todo/`; gap 13 (pipe-label column width) is the one open item this task surfaced.
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__06_docs_and_verify.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/29_gateway_dashboard_page/29_gateway_dashboard_page__06_docs_and_verify.a1.validate
+74
View File
@@ -0,0 +1,74 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 8%]
........................................................................ [ 12%]
........................................................................ [ 16%]
........................................................................ [ 20%]
........................................................................ [ 24%]
........................................................................ [ 28%]
........................................................s............... [ 32%]
........................................................................ [ 36%]
........................................................................ [ 40%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 53%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 65%]
........................................................................ [ 69%]
............................................................ssssssssssss [ 73%]
........................................................................ [ 77%]
........................................................................ [ 81%]
........................................................................ [ 85%]
........................................................................ [ 89%]
........................................................................ [ 93%]
................................................sss..................... [ 98%]
.................................. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 182 0 100%
src/dashboards/api.py 491 0 100%
src/dashboards/app.py 190 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 216 4 98% 324, 328-329, 346
src/dashboards/config.py 126 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/gateway.py 321 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3894 36 99%
Required test coverage of 94% reached. Total coverage: 99.08%
1746 passed, 16 skipped, 1 warning in 165.41s (0:02:45)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.10.19 2026.10.23 wheel
coverage 7.16.2 7.17.0 wheel
fastapi 0.153.0 0.161.1 wheel
pycparser 3.11 3.12 wheel
ruff 0.17.1 0.17.2 wheel
.env.dev.example
+7
View File
@@ -44,6 +44,13 @@ export HOST_TARGETS="driveripper=https://driveripper.example.net:9120 deskwork=h
# export DISKS_INTERVAL_SECONDS=60
# export DISKS_TIMEOUT_SECONDS=30
# The /gateway source (phase 29) reads the one HOST_TARGETS agent that runs haproxy + nginx —
# agent 1.2.0's haproxy/nginx/sockets sections — on its own cadence. The default name already
# matches the HOST_TARGETS line above, so set these only to watch an ingress called something
# else, or to poll faster while developing the page.
# export GATEWAY_HOST=gateway
# export GATEWAY_INTERVAL_SECONDS=10
# The engines are dialled directly on their hostnames, like the agents.
export GUFO_BASE_URL=http://deskwork.example.net:8731
export VLLM_BASE_URL=http://driveripper.example.net:8000
.env.example
+16
View File
@@ -73,6 +73,22 @@ DISKS_INTERVAL_SECONDS=300
# fetch has to outlast HOST_TIMEOUT_SECONDS rather than inherit it.
DISKS_TIMEOUT_SECONDS=30
# --------------------------------------------------------------------------- #
# Gateway dashboard (/gateway): one HOST_TARGETS name is the ingress
# The `gateway` source reads the SAME agent /snapshot the system page reads, projected onto
# the agent's haproxy/nginx config sections and its live /proc socket table (agent 1.2.0).
# There is no URL here: `GATEWAY_HOST` must be a name in HOST_TARGETS, and a name that is not
# there simply means "this deployment has no ingress to watch" — the source is unconfigured,
# never a red panel for a host nobody installed (same rule as /disks, which has no targets of
# its own either).
# --------------------------------------------------------------------------- #
GATEWAY_HOST=gateway
# 30 s: a connection count is the shape of half a minute, not of ten seconds. The timeout is
# the plain agent fetch's own — these sections cost the agent two config reads and two /proc
# tables — with headroom because the same handler may be mid-smartctl pass for /disks.
GATEWAY_INTERVAL_SECONDS=30
GATEWAY_TIMEOUT_SECONDS=10
# --------------------------------------------------------------------------- #
# LLM dashboard: either engine is optional — unset means "not deployed here"
# and its half of the page disappears instead of showing a dead panel.
AGENTS.md
+293 -59
View File
@@ -19,16 +19,17 @@ keep its numbers honest, and the state of the fleet it watches.
## What this is
Four dashboards for one homelab, behind one domain, in one process, backed by one SQLite
file — plus an overview page that says which of the four you need:
Five dashboards for one homelab, behind one domain, in one process, backed by one SQLite
file — plus an overview page that says which of the five you need:
| Route | Question it answers | Source |
| --------- | ------------------------------------------------------------------------------ | ------------------------------------------------- |
| `/` | Is anything wrong, and where do I look first? | all four, in one request |
| `/system` | What are my machines doing — CPU, memory, disk, GPU, network, and the history | `agent/host_agent.py`, our own stdlib agent per host |
| `/disks` | Are any of my physical disks failing, and which one — health, temp, hours, wear | the same agents, `smartctl -i -A -H` per whole drive (never a partition) |
| `/llm` | Tokens/sec, active sessions, queue depth, KV pressure, latency percentiles | gufo `/metrics` + `/slots`, vLLM `/metrics` |
| `/ups` | Battery health, runtime left, power events, battery-test buttons | NUT `upsd` (our own wire-protocol client; 3051 here, not 3493) |
| Route | Question it answers | Source |
| ---------- | ------------------------------------------------------------------------------ | ------------------------------------------------- |
| `/` | Is anything wrong, and where do I look first? | all five, in one request |
| `/system` | What are my machines doing — CPU, memory, disk, GPU, network, and the history | `agent/host_agent.py`, our own stdlib agent per host |
| `/disks` | Are any of my physical disks failing, and which one — health, temp, hours, wear | the same agents, `smartctl -i -A -H` per whole drive (never a partition) |
| `/llm` | Tokens/sec, active sessions, queue depth, KV pressure, latency percentiles | gufo `/metrics` + `/slots`, vLLM `/metrics` |
| `/ups` | Battery health, runtime left, power events, battery-test buttons | NUT `upsd` (our own wire-protocol client; 3051 here, not 3493) |
| `/gateway` | What is coming in through the router, and where does it go — frontends, `server_name`s, the SNI map, live pipes | the same agent (1.2.0) on the one host `GATEWAY_HOST` names: its `haproxy` + `nginx` config sections joined to its `/proc/net/tcp{,6}` sockets |
History is a first-class feature, not a sparkline afterthought: every number worth graphing
is stored, rolled up (raw → 1m → 1h), and served from `/api/*/history`.
@@ -342,11 +343,12 @@ Responsibilities, one line each:
publish: the same rule that scrubs a drive's serial (invariant 6). Those two tables are volatile by
nature, so `tests/test_agent.py` asserts their **shape**, the ports the recorded configs claim, and
the absence of a globally routable endpoint — never a particular row, which a re-run may lawfully
change. What phase 28 stops at is the wire: `sources/host.py` passes **none** of the three into
`/api/system/snapshot` and `store.py` records no series for a proxy or a socket, so the sections
live in the agent's own document and reach no page until phase 29's. What the dashboard does read
today is their **error lines**, through `collectors.ADDON_ERROR_PREFIXES`, so a host with no proxy
config stays out of `/system`'s `degraded` (invariant 3).
change. What phase 28 stopped at was the wire: `sources/host.py` still passes **none** of the
three into `/api/system/snapshot`, and `store.py` still records no series for a config — the join
and the two gauges it publishes are phase 29's `sources/gateway.py`, which reads the same
`/snapshot` through its own `gateway` job. What the *system* page does read today is their
**error lines**, through `collectors.ADDON_ERROR_PREFIXES`, so a host with no proxy config stays
out of `/system`'s `degraded` (invariant 3).
- `config.py` — env only, no config file format, no CLI flags. `.env` is a *seed*; a real
env var always wins. Defaults are safe except the sources, which default to unset. The auth
block (phase 11) is `DASH_ADMIN_PASSWORD` (the **seed**, and `_env()` strips, so `VAR=` means
@@ -355,19 +357,32 @@ Responsibilities, one line each:
`http://127.0.0.1`; task 03 also treats `X-Forwarded-Proto: https` as secure), and
`DASH_PBKDF2_ITERATIONS` (default `auth.PBKDF2_ITERATIONS`, imported so the primitive and the
setting cannot drift) — that knob exists so the hermetic suite hashes in microseconds instead
of 0.3 s per client and is not a production tuning parameter.
of 0.3 s per client and is not a production tuning parameter. The gateway block (phase 29) is
`GATEWAY_HOST` (default `gateway`, and **a name, never a URL**: `gateway_enabled` is true only
when a `HOST_TARGETS` entry has it, so the page is unconfigured rather than red for a
deployment with no ingress — the `nut_enabled` shape, and the reason `any_source_enabled`
includes it), `GATEWAY_INTERVAL_SECONDS` (30 — a connection count is the shape of half a
minute, not of ten seconds) and `GATEWAY_TIMEOUT_SECONDS` (10, the plain agent fetch with
headroom for a handler that may be mid-smartctl pass for `/disks`).
- `poller.py` — job loop, health records, stale/last-good semantics, transition events.
`/disks` is a **source of its own** (`disks:<host>`, `DISKS_INTERVAL_SECONDS` 300 s,
`DISKS_TIMEOUT_SECONDS` 30 s, events under the dashboard name `disks`) reading the same
agents the `system` jobs read: the separate task is what keeps a SMART poll out of the
10-second utilisation loop and lets `/disks` age on its own instead of blinking `/system`.
`gateway` is a **job of its own** for the same reason (phase 29 task 02) — `GATEWAY_INTERVAL_SECONDS`
30 s, `GATEWAY_TIMEOUT_SECONDS` 10 s, health and events under the dashboard name `gateway` — but
**one** job rather than one per host (the `nut` shape), dialling the `HOST_TARGETS` entry
`GATEWAY_HOST` names with its `label` set to that host's name, because the page is about the single
ingress and a record per machine would ask four hosts a question only one can answer. It exists
only when that name is a target: an unconfigured source has no task that could look red
(invariant 3), and it is handed no `RateEstimator` because a count is not a rate (rule 2).
The job list is built once (`app.build_hub()`) because it is wired twice — production and
the injected-store test path — and two hand-written copies drifting is how a source ends up
polling in tests and not in the container. The
success path records (payload → samples → events) and only *then* moves `last_ok`, so a
refused SQLite write ages the source and alerts instead of leaving it current with charts
that have silently stopped drawing.
Events are stored under the **dashboard** name (`system`/`disks`/`llm`/`ups`), not the sample
Events are stored under the **dashboard** name (`system`/`disks`/`gateway`/`llm`/`ups`), not the sample
source (`host`) — a page filtering `/api/events` by `host` gets an empty feed forever
(found on `/system` during phase 03 browser verification).
- `collectors.py` — the fetch half. `AgentTls` (phase 15 task 03) is what makes an `https://`
@@ -390,8 +405,16 @@ Responsibilities, one line each:
problem. `agent_tls_map()` builds one entry per configured host (the CA and client pair are
shared — the dashboard has one identity — only the revocation answer is per name, and an absent
entry is *active*, never-issued being a different fact from revoked); `build_hub` takes that map
and hands the **same** entry to the `host:` and `disks:` jobs of one host, so a revocation
fails both sources with one sentence and neither of them dials. `fetch_snapshot()` keeps
and hands the **same** entry to the `host:`, `disks:` and `gateway` jobs of one host, so a
revocation fails every source reading that agent with one sentence and none of them dials
(`tests/test_api.py`'s revoked-gateway pin is the three-job version of the `host:`/`disks:` one).
`fetch_gateway_snapshot()` + `gateway_job()` (phase 29 task 02) are the third reader of one
`/snapshot`: the job is **one per deployment** (the `nut` shape, key `gateway`, `label` = the
`GATEWAY_HOST` name) on `GATEWAY_INTERVAL_SECONDS` with `GATEWAY_TIMEOUT_SECONDS`, it is handed
**no** `RateEstimator` (a connection count that goes through a Δ-window is a rate with a label and
no measurement behind it, rule 2), and `gateway_job` returns `None` when the name is not in
`HOST_TARGETS` — the `disks` configured gate, so an unconfigured source gets no task that could
look broken and `/api/health` says nothing about it. `fetch_snapshot()` keeps
`http://` targets on the shared client with their bearer token, unchanged from before the phase,
so a rollout is per host.
`NutMonitor` owns per-UPS transition bookkeeping (its
@@ -447,6 +470,41 @@ Responsibilities, one line each:
counts for the page header and drops a host with no payload from the sums while still
counting it in `hosts_total` — the `/llm` fleet-block rule, so the page never does
arithmetic and a dead agent never shrinks a fleet total.
- `sources/gateway.py` — the same snapshot projected onto the **connection topology** of the one
host `GATEWAY_HOST` names (phase 29): `parse(snapshot, host)` joins agent 1.2.0's `haproxy`
frontends and `nginx` `listen`s into **inbound** pipes (one per listen port, carrying the
`server_name` domains and the live ESTABLISHED count + distinct client addresses) and haproxy
`server` lines, concrete `proxy_pass` targets and the `$var`→`map` entries into **outbound**
pipes, plus the `unmatched` tallies (`inbound` = connections on a listening port no config
names, `outbound` = connections to something no target claims, `other` = live listeners no
config claims — phase 28's "an unclaimed socket stays visible as one"). `count` semantics are
the whole honesty of the module: **no DNS** (the spy test pins `socket.getaddrinfo` is never
called — `ipaddress` decides "is this already an address", and it never asks), an exact
`(address, port)` match for a target written as an address (`by_port: false`), a port-only
match for a **name** and only when that port has exactly one configured host
(`by_port: true`), and **no attribution at all** when several names share a port (the real
`stream.conf` map's sixteen `*.reeselink.com:443`): those pipes carry `connections: null`, the
sockets go to `unmatched.outbound`, and `unmatched.groups` says once per port how many and
among which candidates — picking one would be rule 2, and repeating the port's total on
sixteen pipes reads as sixteen times the traffic. A pipe is `null` (never `0`) whenever the
socket table did not answer or the join could not decide, and publishes **no sample** then, so
a chart cannot draw a zero for an unknown. The classification is one exclusive pass
(`join_sockets`), which is what makes "inbound + outbound + unmatched = the socket table"
hold — the identity task 06 checks against a direct `/proc/net/tcp` read. `supported: false`
(a 1.1.0 agent, or `--no-proxy`) keeps every key with empty lists and null counts plus
`WHY_PREDATES`, `available: false`/`null` per section becomes a `why` sentence and the agent's
own line in `section_errors`, and nothing here carries a `severity` (`net.*` precedent: a busy
proxy is the job) or a `window_seconds` (a count is not a rate). Gauges: `gateway.in_conns@<port>`
and `gateway.out_conns@<host>:<port>` — the port **is** in the outbound name, because this
fleet's two haproxy Minecraft backends share `minecraft.reeselink.com` on 25565 and 25566 and
two gauges on one name would overwrite each other in silence. `parse_gateway_snapshot()` is the
`Result` the poller consumes; there are **no events** (a connection count has no transition
worth a line — a proxy that stops routing shows up as its host's own poll failing). The
never-answered arm is `stub(host, ts)` — the **same skeleton** `parse` fills in, plus
`unreachable: true`, `supported: null` and `WHY_NO_ANSWER` — because the route that serves a host
with no payload yet must not hand the page a shape with keys missing, and `WHY_NO_ANSWER` is not
`WHY_PREDATES`: an agent that has not replied has not said it has no sections (invariant 3's two
facts staying two sentences).
- `sources/llm.py` — gufo + vLLM field maps (`GUFO_SERIES`, `VLLM_SERIES`,
`VLLM_HISTOGRAMS`), measured Δ rates **diffed per label set** (`rate_key()` keys the
estimator on the resolved name plus its labels; the cell is the Σ of the per-series
@@ -638,7 +696,18 @@ Responsibilities, one line each:
`severity.py` rates no sector counter and this module may not have a threshold of its own, so
grading a drive here would be the bug, not the fix (phase 09). Its `HREF` table does carry
`/disks`, so the one alert a disk job *can* raise — SMART failing while its host answers —
links to the page that names the drive.
links to the page that names the drive, and since phase 29 `/gateway` for the same reason: a
`gateway` job failing **while its host answers** (the proxy configs or the socket table went
quiet inside a healthy agent) has no other voice. The echo suppression is now a table
(`AGENT_ECHO_DASHBOARDS` = `disks`, `gateway`) because three jobs read one agent, and it finds the
partner differently for each: `disks:<host>` names the host in its key, while the single `gateway`
job names it in its **label** (its key is one word, and `GATEWAY_HOST` is what an alert prints).
A missing `host:<name>` record suppresses nothing — "I cannot find the host's job" is not evidence
the host is fine. The gateway source has **no verdict rule and none is wanted**, the same shape as
the disk rule: its payload publishes no `severity`, `severity.THRESHOLDS` has no `gateway.*` row,
`alerts.build` takes no gateway payload at all, so a `gateway` alert can only ever be
reachability — stale past 3× its own 30 s interval or stopped answering — and 400 connections on
an ingress is the job, not a fault.
- `store.py` — `sample(source,host,metric,res,ts)` + `event` + `meta`, rollups, pruning. The
`meta` table is the only durable key/value state in the app and is reached through three
named helpers — `get_meta`/`set_meta`/`delete_meta` (the last returns its rowcount, because
@@ -654,7 +723,7 @@ Responsibilities, one line each:
cannot remove a name, so `_note_series_write()` ends the cache on a `(source, host, metric)` the
store has never written and leaves a poll of an existing series alone — invalidating on every
sample was task 02's defect, and it put the 572 ms scan back on nearly every `/api/metrics` call
on a store nine pollers write to, behind a cache that never served; `delete_series()`/`maintain()`
on a store every poller writes to, behind a cache that never served; `delete_series()`/`maintain()`
invalidate unconditionally because a `DELETE` can take a name, `delete_series()` also forgetting
the triples it emptied so a re-created series counts as new again), through a generation counter so
a write racing a read cannot park a stale answer under a fresh timestamp, with the name cache
@@ -756,7 +825,7 @@ Responsibilities, one line each:
configured host **with drives to show** (disk rows + `smart` + `smartctl_available` +
per-row `severity` + a `fleet` block summed by the API, with a configured-but-never-answered
host as an `unreachable` stub whose counts are `None`, not `0`), and `/history` serves the
`disk.*@dev` series with unknown names answering as empty series like the other two
`disk.*@dev` series with unknown names answering as empty series like the other
history endpoints. The configured gate runs **before** anything that could reach the
network — `HOST_TARGETS` empty is `404 {"configured": false}` with no client, no fetch and
no `disks` job in the hub at all (invariant 3), and `tests/test_api.py` proves it with
@@ -778,11 +847,33 @@ Responsibilities, one line each:
only POST that **dials an upstream** stays `/api/ups/command` (invariant 1, pinned by enumerating
every served route's verbs and sorting the writes into fleet / session / certificate-authority
lists — the third is phase 15's, and it writes `meta` rows and nothing else).
All four `history` routes reach the store through one door, `api._history_batch()` →
`/api/gateway/*` (phase 29 task 03) is the ingress pair, and it is the same two rules on a
single-host source. `/snapshot` answers `sources/gateway.parse`'s payload for the one
`GATEWAY_HOST` entry — the haproxy frontends and the nginx `listen`/`map` echo, the inbound pipes
(port, domains, live count, up to `MAX_CLIENTS` client addresses), the outbound pipes (`by_port`
naming which half of the match is a socket fact), each pipe's own `metric`, the `unmatched`
tallies, and `supported`/`sections`/`why` — with `states` appended and the payload's own `ts`
left alone, because that is when the agent took the reading and it is what the page ages. Three
arms reach it and none of them is red: `404 {"configured": false}` **in the route's first line**
when `GATEWAY_HOST` names no `HOST_TARGETS` entry (unset and wrong-name are both spy-pinned to
`httpx.AsyncClient.get` and `collectors.fetch_snapshot`, and neither arm has a `gateway` job to
age out either), `supported: false` for a 1.1.0/`--no-proxy agent, and `gateway_source.stub()`
for a configured host that has never answered — the payload's key set plus `unreachable: true`
and minus only the `agent_version` nobody published, every count `null` because an agent that has
not spoken has not reported an idle gateway (rule 2). Nothing in the payload carries a `severity`
or a `window_seconds`, at any depth (pinned by a recursive walk: a busy proxy is the job and no
Δ was ever measured). `/history` serves the `gateway.in_conns@<port>` /
`gateway.out_conns@<host>:<port>` gauges with `hosts=` defaulting to the ingress **alone** (the
name the samples are stored under — defaulting to every target would draw empty plots for
machines nobody asked to route), and it carries no configured gate because it dials nothing:
like `/api/disks/history`, an unconfigured source answers no hosts and no series. The pair adds
no write (the verb enumeration pins the two GETs and that the app's one upstream POST is still
`/api/ups/command`), and `tests/test_auth_api.py` lists both paths as session-required.
All five `history` routes reach the store through one door, `api._history_batch()` →
`store.history_many()` (phase 14 task 02). That is a cost change and nothing else: the series
list is host-major and metric-minor in request order, each entry the same
`{source, host, metric, range_seconds, points}` document the per-series loop built, with the
same empty-series answer for a name that has no rows. `tests/test_api.py` pins it for all four
same empty-series answer for a name that has no rows. `tests/test_api.py` pins it for all five
routes against points read straight out of the store with `query()` — a third path, in neither
the loop nor the batch — so a future "optimisation" that moves a point, an order, or a key
fails a test instead of redrawing a chart wrong.
@@ -804,6 +895,25 @@ Responsibilities, one line each:
job with no payload yet is listed as a stub (`unreachable: true`, every value `null`) in
`system.hosts`/`llm.engines` exactly as `/api/system/snapshot` does, so `hosts_total`
counts the machines there are and `hosts_live` the ones answering.
The **`gateway` block** (phase 29 task 02) is additive in the same way: `{configured, host, href,
supported, unreachable, inbound, outbound, map_targets, unmatched, unmatched_listeners, partial,
states}`. Task 02 wrote it for programmatic consumers (the page itself reads
`/api/gateway/snapshot`, task 03); task 05 gave it one page-side reader, the single line `/`
renders under the fleet table, so `href` joined the block — derived from `alerts.HREF`, the one
table that writes a dashboard's route, so the overview's link and an alert about the same source
cannot spell the page differently (`test_the_gateway_block_links_to_the_page_the_alert_row_links_to`,
and the page half reads only block keys: `test_the_overview_line_reads_the_gateway_block_it_claims`
resolves the served `GATEWAY_FIELDS` against a live document). `inbound`/`outbound` are the sums of the pipes' own counts and `map_targets` counts the stream
map's routing entries; every figure is `null` — never `0` — in the arms that did not measure
(unconfigured, configured-but-never-answered, a 1.1.0/`--no-proxy agent, a silent socket table),
and `partial: true` says a total had to leave an unknown pipe out (a port whose configured targets
cannot be told apart) rather than pretend to be the whole number. `unmatched` counts
*connections* and `unmatched_listeners` counts *ports*, so they are two fields and never one sum,
and nothing in the block carries a `severity`. `/api/config` grew the matching row (`enabled`,
`host`, `url`, `interval_seconds`, `timeout_seconds`), where `url` is the named entry's
`Target.display_url` and `null` when no target carries the name — invariant 6's boundary applies
to a name-only source exactly as it does to a URL-bearing one, because this body is what a browser
renders.
- `app.py` — assembly, and the **session gate** (`session_gate` middleware, invariant 12) whose
`PUBLIC_PATHS`/`PUBLIC_PREFIXES` are the whole anonymous surface. It answers in this order and
the order is the design: no `AuthState` → `503` for everything; no session and no such route →
@@ -829,14 +939,18 @@ Responsibilities, one line each:
directory the lifespan's `finally` removes. What hands that material to the loops is
`agent_tls_map(material, store, targets)` → `build_hub(..., tls=...)`, called in **both**
branches the same way: no material (or no targets) means no TLS map, which is the plain
`http://`/token path rather than an error, and one map covers both of a host's jobs. A corrupt stored certificate raises out of the same
`http://`/token path rather than an error, and one map covers all of a host's jobs — including the
`gateway` job, which is why one revoke in `/certs` silences the pipes with no restart. A corrupt stored certificate raises out of the same
place — a dashboard that cannot present its own client certificate would otherwise spend the
fleet's TLS history reporting an upstream problem it caused itself.
`build_hub` also builds **two** `RateEstimator`s, because a window is only a claim about the
polls that fill it (phase 23, `TODO.md:8`'s "ramp up/ramp down"): the `host` jobs get
`RateEstimator(span=settings.rate_window)` (10 s — what a 10 s cadence can fill) and the `llm`
jobs `RateEstimator(span=settings.llm_interval)` (2 s, one poll interval, so a slot that starts
reads steady on the next poll instead of climbing one fifth of the true rate per poll), and
reads steady on the next poll instead of climbing one fifth of the true rate per poll), and the
`gateway` job (phase 29) is handed **neither** — it is added to the hub between the `disks` and
`llm` jobs, only when `settings.gateway_enabled`, because a gauge needs no window and one job
cannot be right about two cadences' worth of them.,
there is no third knob — the LLM window **is** `LLM_INTERVAL_SECONDS`, which is the only way
the pair cannot drift. One estimator for both could only be right about one of them: a 2 s span
over a 10 s poll sits the window-empty fallback against the `2 × span` ceiling and answers
@@ -844,9 +958,14 @@ Responsibilities, one line each:
The estimator's own gates (the `MIN_WINDOW_SECONDS` floor, the ceiling, the 8× jump
confirmation with its independent anchor) are unchanged and apply to the smaller window, whose
`Δ2.0s` label the payload carries with the value.
- `web/` — five dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`) plus the login page
(`/login`, `pages/login.html` + `static/page-login.js`), one stylesheet, `charts.js`. No
framework. `/login` renders the masthead's **brand link and theme toggle** and nothing else
- `web/` — six dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`, `/gateway`) plus the
login page (`/login`, `pages/login.html` + `static/page-login.js`), one stylesheet, `charts.js`.
No framework. The tab nav is those six and nothing else, in that order, on **all eight**
signed-in pages (phase 29 task 05): `tests/test_api.py::test_the_disks_tab_is_in_the_masthead_of_every_page`
compares the nav's hrefs to `TAB_ORDER` (which **is** `PAGES`) and asserts the `aria-current`
count per page — one, naming itself, on the six dashboards and zero on `/settings` and `/certs`.
`/gateway`'s own tab arrived with that task; until then the page existed behind a route with no
way to click to it. `/login` renders the masthead's **brand link and theme toggle** and nothing else
from it: no tabs (every tab bounces straight back here, and a nav of links that all redirect is
a lie about what is available), no status dots and no clock (there is nothing signed in for them
to say). It posts through `Dash.fetchJSON` like every other script, which is only safe because
@@ -864,8 +983,8 @@ Responsibilities, one line each:
login page is audited for orphan `id`s and CDN references too (`tests/test_auth_api.py`), fetched
unsigned because a signed-in `GET /login` is a `303` and would audit `/`'s markup under the wrong
name. `/settings` (`pages/settings.html` + `static/page-settings.js`) is a **page and not a
sixth dashboard**: same `<head>` and masthead, the five tabs with **no** `aria-current` (it is
not one of them), the same four `#nav-status` badges as every dashboard (phase 25 task 01 —
seventh dashboard**: same `<head>` and masthead, the six tabs with **no** `aria-current` (it is
not one of them), the same five `#nav-status` badges as every dashboard (phase 25 task 01 —
`TODO.md:3`'s "hosts disks engines UPS disappear when in the certs/account page" was the
missing container, and `Dash.initChrome()` fills it from `/api/health` with no JS change), and
it is reached from the `⚙ Account` link in
@@ -884,19 +1003,19 @@ Responsibilities, one line each:
`csrf` is now invalid — every later write, the UPS buttons included, 403s until a reload) and on
any `401` bounce.
`/certs` (`pages/certs.html` + `static/page-certs.js`) is the second of those admin pages, and it
is the same shape for the same reasons: five tabs with **no** `aria-current`, the four fleet
is the same shape for the same reasons: six tabs with **no** `aria-current`, the five fleet
badges in `#nav-status` like every other masthead (phase 25 task 01, same reasoning as
`/settings` above), and both masthead links — `⚷ Certs` beside `⚙ Account` — exactly as the five
`/settings` above), and both masthead links — `⚷ Certs` beside `⚙ Account` — exactly as the six
dashboards render them. **Both links are on both admin pages** (phase 20 task 04, `TODO.md:4`),
which retires phase 15's "no self-link" precedent: chrome that changes shape depending on where you already are reads
as a broken button, while a link to the page you are on is a no-op that answers with the page you
were looking at (`data/logs/20-task04-admin-links.cjs` measures exactly that — one document
response, no redirect, the same cluster afterwards). The two links therefore sit on all seven
signed-in pages (the five dashboards plus `/settings` and `/certs`) in one order
response, no redirect, the same cluster afterwards). The two links therefore sit on all eight
signed-in pages (the six dashboards plus `/settings` and `/certs`) in one order
(`⚷ Certs` → `⚙ Account` → theme toggle → clock — phase 25 task 03 moved the clock to the end so
it is the farthest-right item on a `flex-end` row), which is what lets
the pinned audit be one loop over `MARKUP_PAGES` instead of two over different tuples; `/login`
is the eighth page and the one with neither. Everything on it is
is the ninth page and the one with neither. Everything on it is
`GET /api/certs`: the Root CA block, the display-only Dashboard client line (that certificate is
minted with the root and has no route of its own), and one row per host with its status chip,
serial, expiry (local timestamp *and* `Dash.duration` left), issued-vs-deployed `bundle_rev`,
@@ -926,7 +1045,12 @@ Responsibilities, one line each:
`data/logs/27-task01-scroll.cjs`).
`app.js`'s masthead status dots are one per dashboard job —
including `disks`, because the same agent can answer the 10-second utilisation poll and
have a wedged SMART pass, and one dot for both would be right about neither. `app.js` owns the
have a wedged SMART pass, and one dot for both would be right about neither, and including
`gateway` (phase 29 task 05), which is one dot rather than one per host because the ingress is
one job and a dot for a source that stays `idle` when `GATEWAY_HOST` names nothing is how an
unconfigured source reads grey rather than red. The badge table is a literal in `initChrome()`,
so a new dashboard's job means a new entry there — `NAV_STATUS_BADGES` in `tests/test_api.py`
is the pin that says so. `app.js` owns the
colour vocabulary: `Dash.severityFrom(cellOrSeverity)` turns a payload verdict into an
`ok|warn|fail` class (`''` for anything unrated, so unknown never looks green) and
`Dash.hero(el, {label, value, unit, digits, severity, sub, bar})` renders/patches a
@@ -941,7 +1065,7 @@ Responsibilities, one line each:
**The masthead is three rows at every width, the two control rows flush right** (phase 20 task
01 made the rows, `TODO.md:5-6`; phase 25 task 02 re-justified them, `TODO.md:4` — "I made a
mistake earlier, I did want hosts disks engines UPS badges and certs/account/dark right
justified"): row 1 the brand, `#nav-toggle` and the five tabs; row 2 the `#nav-status`
justified"): row 1 the brand, `#nav-toggle` and the six tabs; row 2 the `#nav-status`
per-source dots; row 3
the `.masthead-right` controls (`⚷ Certs`, `⚙ Account`, theme, then the clock — phase 25 task 03
moved the clock last for `TODO.md:5`'s "the time should show up as the farthest right item";
@@ -960,11 +1084,20 @@ Responsibilities, one line each:
brand and the tabs on separate rows. `height: var(--header-h)` became **`min-height`** (three rows
do not fit in 58 px and a fixed height clips the controls) and is declared **once** — the ≤640px
block no longer restates `height`/`min-height`/`flex-wrap`, and that pair drifting is exactly what
would clip the bar on a phone only. Every page that renders the bar now renders three rows —
would clip the bar on a phone only — except that **`#nav-status` wraps at ≤640px since phase 29
task 05**: five badges are 352 px of chips against this bar's 320 px content box, and a `nowrap`
item that may not shrink does not get smaller, it widens the *document* (measured 372 px — the
sideways scroll phase 18 forbids). The fifth chip takes a second line of the same right-aligned
row and the bar grows by one line box; no label shrinks and nothing is hidden. Phase 18's
`nowrap` was right for four (267 px) and the pin now demands the opposite, with
`data/logs/29-task05-tabs.cjs` measuring 360 px documents on all eight pages.
Every page that renders the bar now renders three rows —
`/certs` and `/settings` grew the dots row in phase 25 task 01, where `#nav-status` is pinned as
the **first child** of `.masthead-right` on all seven signed-in pages (`app.js` is untouched:
the container's existence is what makes `initChrome` build the four badges, which is why the fix
was markup and not JS) — and `/login` remains the one page with no dots, rendering two rows
the **first child** of `.masthead-right` on all eight signed-in pages (`app.js`'s *badge list* is
a literal, and phase 25's fix was markup because the container was what was missing; phase 29
task 05 added the fifth entry — `{ key: 'gateway', label: 'gateway', href: '/gateway' }` — because
the ingress has a job of its own and a job with no badge is a source that blinks in silence) —
and `/login` remains the one page with no dots, rendering two rows
(brand, then the theme toggle). Pinned on the served bytes by phase 20's three tests in
`tests/test_api.py` — the row declarations (task 02 renamed that one to
`test_the_masthead_is_three_rows_right_justified_at_every_width`), the mobile block's silence
@@ -980,12 +1113,14 @@ Responsibilities, one line each:
(the row-break assertions are the same ones phase 20 wrote, deliberately: the re-justify must
not cost the rows), and the measured half is `data/logs/25-task02-justify.cjs` — seven pages ×
two widths, each demanding both rows compute `flex-end` and sit flush with the bar's right
padding edge, the four badges present, three rows intact, the dropdown still under the whole
padding edge, the four badges present (a four-badge claim — `29-task05-tabs.cjs` is the pin that
the fifth, `gateway`, joined them), three rows intact, the dropdown still under the whole
bar and no sideways scroll, plus `/login`'s two-row bar with no badges
(`data/logs/25-task02-justify.out`, screenshots `data/logs/25-task02-*.png`). **Phase 25 task 03
then put the clock at the end of the control row** (`TODO.md:5` — "The time should show up as the
farthest right item on the navbar"): the `<span class="clock" id="clock">` moved from the first
control to the last one on all seven templates, so the row reads `⚷ Certs` → `⚙ Account` → theme
control to the last one on all seven templates then (eight since `/gateway` joined the nav), so
the row reads `⚷ Certs` → `⚙ Account` → theme
toggle → clock. It is a **markup** move and deliberately not a CSS `order:` declaration, because
on a `flex-end` row the last child *is* the farthest-right item and an `order:` override would be
something the served-byte pin cannot see — `test_every_page_links_to_both_admin_pages` therefore
@@ -1015,7 +1150,8 @@ Responsibilities, one line each:
off it after a scroll; screenshots `data/logs/masthead-unsticky*.png`). Note that a browser cannot
tell `relative` from `relative; top: 0` — Chrome computes `top: 0px` for either — so the absence
of the offset is a served-CSS pin, not a measurement.
**The tab nav collapses into a hamburger at ≤640px** (phase 18): the seven tabbed pages each
**The tab nav collapses into a hamburger at ≤640px** (phase 18): the eight tabbed pages (the six
dashboards plus `/settings` and `/certs`; `/gateway` joined in phase 29 task 05) each
carry `#nav-toggle` (`class="ghost nav-toggle"`, `type="button"`, `aria-controls="nav-tabs"`,
`aria-label="Toggle navigation"`, `☰`) between `.brand` and `<nav class="tabs" id="nav-tabs">`,
and `/login` carries neither — it renders the brand link and the theme toggle only. Its base
@@ -1059,8 +1195,9 @@ Responsibilities, one line each:
The `if (navToggle && appNav)` guard is the whole `/login` story: the shared script loads there
and does nothing. The `640px` literal lives in two files — the stylesheet block and that media
query — and has to move in both, or the menu opens into a hidden nav. The contract is
pinned on the **served** bytes by three `tests/test_api.py` pins — the button on all seven pages
and nowhere on `/login`, and one test that **compares** the CSS media query with the JS
pinned on the **served** bytes by three `tests/test_api.py` pins — the button on every page in
`MARKUP_PAGES` (eight since `/gateway` joined the nav) and nowhere on `/login`, and one test that
**compares** the CSS media query with the JS
`matchMedia` instead of grepping for `640px` twice, so a drift fails with both numbers in the
message — and proven in a browser at 360px by `data/logs/nav-task04-verify.cjs` (the `/` +
`/system` interaction, 24 checks) and `data/logs/nav-final-pass.cjs` (the phase-completion
@@ -1081,9 +1218,12 @@ Responsibilities, one line each:
Metric naming: dotted (`cpu.usage_pct`, `mem.used_pct`, `disk.used_pct@/srv`,
`gpu.busy_pct@0`, `llm.decode_tokens_per_second`, `ups.charge_pct`), `@stream` for per-device
series — `/api/metrics` answers with exactly these names. The `sample.source` column is the
dashboard family — `host`, `llm`, `ups`, and `disks` for the SMART series
dashboard family — `host`, `llm`, `ups`, `disks` for the SMART series
(`disk.temp_c@nvme0n1`, `disk.power_on_hours@…`, `disk.reallocated@…`, `disk.pending@…`,
`disk.uncorrectable@…`, `disk.percentage_used@…`). Those keep their own family instead of
`disk.uncorrectable@…`, `disk.percentage_used@…`), and `gateway` for the phase-29 connection
gauges (`gateway.in_conns@<listen-port>`, `gateway.out_conns@<host>:<port>` — the port belongs
in the outbound name because two services on one host are two facts and one name would make the
second insert overwrite the first). Those keep their own family instead of
sitting beside `host`'s `disk.used_pct@/mount` because the cadences differ thirty-fold and
`GET /api/metrics?source=disks` has to answer "what does *this page* have history for".
That scheme is the history API's key — treat it as a public interface.
@@ -1134,9 +1274,11 @@ That scheme is the history API's key — treat it as a public interface.
`sudo deploy/install-agent.sh --bind 10.1.0.126 --port 9120 --as-root --caps CAP_SYS_RAWIO,CAP_SYS_ADMIN --force`
reproduces it and a re-run that would quietly replace a hand-fixed unit is now a refusal with a
diff attached rather than a clobber (gap 10).
`/api/health` today reads **all nine sources `ok`**: `host:driveripper`, `host:deskwork`,
`host:gateway`, `disks:driveripper`, `disks:deskwork`, `disks:gateway`, `nut`, `llm:gufo`,
`llm:vllm`.
`/api/health` today reads **all ten sources `ok`**: `host:driveripper`, `host:deskwork`,
`host:gateway`, `disks:driveripper`, `disks:deskwork`, `disks:gateway`, `gateway`, `nut`,
`llm:gufo`, `llm:vllm`. The tenth is phase 29's: one job for the ingress host, dialling the same
`https://` agent as `host:gateway` with the same certificate entry, and re-verified live on the
dev server with no change to any pre-existing source.
**The fleet is mTLS since 2026-10-21 (phase 24 task 03), so the bearer-token transport is now
history rather than the designed shape.** `gateway` runs `agent/host_agent.py` **1.2.0** (phase 28
task 05, 2026-10-22) and `deskwork`/`driveripper` still run **1.1.0**; the spread costs nothing,
@@ -1376,9 +1518,9 @@ uv run python scripts/dev-certs-export.py --url http://127.0.0.1:8090 \
# GET /api/certs/<name>/bundle and writes
# data/certs/<name>.bundle at 0600 — mints nothing
uv run pytest # hermetic: 1650 passed + 15 skipped (11 live,
uv run pytest # hermetic: 1746 passed + 16 skipped (12 live,
# 4 store_perf), no network
uv run pytest --cov # coverage gate (98.97%; floor 94%)
uv run pytest --cov # coverage gate (99.08%; floor 94%)
uv run pytest -m live --live # opt-in: real fleet / engines / upsd
uv run pytest -m store_perf --store-perf # opt-in: the store hot-path timings over the seeded
# store, priced against `store_perf.BUDGETS` (the cached
@@ -1497,6 +1639,14 @@ branch `main`); there is no CI on it, so `bash .agents/validate.sh` is the only
`test_the_real_drives_answer_every_number_the_disks_page_prints` asserts every number
`/disks` prints comes out of real drive output — that test is what caught the parser
reading the end of an attribute row instead of its annotated RAW_VALUE column.
- **A join is pinned by the identity it must satisfy and by the one question it may never ask**
(phase 29). `tests/test_gateway_source.py` runs the captured gateway configs and the recorded
`/proc/net/tcp{,6}` tables through the **real** `host_agent` parsers, so the join is compared with
the agent's document and not with a copy of its field names, and then demands the arithmetic:
inbound + outbound + unmatched equals the socket table's own ESTABLISHED count, a port no config
claims stays visible in `unmatched.other`, several names sharing one port answer `null` rather than
picking a favourite, and `socket.getaddrinfo` is spied so the day the join resolves a name the run
fails — deciding "already an address, or a name?" is all that module may ask (rule 2).
- Anything that needs a failure mode — upsd refusing a command, an engine 500ing, a host
rebooting mid-poll, a counter reset — is a fake in `tests/conftest.py`.
- **A verdict the operator sees is pinned on the surface that shows it, in both directions.**
@@ -1676,7 +1826,7 @@ branch `main`); there is no CI on it, so `bash .agents/validate.sh` is the only
existing series issues no `SELECT`, a write of a name nobody wrote ends the cache — and the live
version is `scripts/watch-store-perf.py`, which is where the defect showed up.
- Coverage gate: 80% is the repo floor, the phase gate in `.agents/validate.sh` enforces
94% (`--cov-fail-under`), and ~98% is where we are; keep it there. New/modified Python
94% (`--cov-fail-under`), and ~99% is where we are; keep it there. New/modified Python
still has to clear >90% on its own. API tests use `create_app()` with an in-memory DB,
never a real target, and never a real upstream: `/api/ups/command`'s send is proven against
a scripted `nut.NutClient` (`ScriptedNutClient` in `tests/test_api.py`) because the live
@@ -1693,7 +1843,10 @@ one — `/system`'s three per-host slots, `/disks`'s two per-host plots, `/llm`'
plots, `/ups`'s four battery plots — each pinned by `tests/test_api.py`, so a chart that ships
anonymous fails a test.
Window chips (5m…30d on `/system` and `/llm`, 1h…90d on `/ups` and `/disks`; the API accepts
5m…90d — SMART lands every 300 s, so a 5-minute window on `/disks` is one dot). A
5m…90d — SMART lands every 300 s, so a 5-minute window on `/disks` is one dot). `/gateway` has
**no** chips and one window: a connection trend is read as "the last day of this pipe", so each
card's plot is a fixed 24 h named by the `Charts` chip beside its toggle rather than a page control
that would re-fetch every card at once. A
panel that cannot get data says why in the panel — the operator should never have to open
devtools to learn that a source is unconfigured. Escape anything upstream-provided through `DashCharts.escapeHtml`.
@@ -1726,6 +1879,14 @@ the table never implies last good numbers are live and never silently loses a ma
No summary cards and no raw event feed survive on `/` — `tests/test_api.py` pins the
two-panel shape against the served HTML.
Under the fleet table sits the **ingress's one line** (phase 29 task 05): `#fleet-gate`, filled by
`renderGateway()` from `/api/summary`'s `gateway` block with the block's own `href`, so the page
invents neither a route nor a number (`gateway 14 inbound · 4 outbound`, `n/a` for a count the
source did not measure, the shared `.stale` dim when the agent stopped answering, and the reason in
its `title` when it is `partial`). `configured: false` renders **nothing at all** — the `disks`
block's invariant-3 rule, no row and no red one to learn to ignore — and the row is a `<p>` inside
the fleet panel because the page is two panels, not three (the pin counts `<section>`s).
Nothing waits for a writer that never came: `tests/test_api.py` fails if a page keeps an
`id` that no page script, no shared script, no stylesheet rule and no `aria-*`/`for`
attribute references — the orphan a deleted renderer leaves behind, caught on served bytes
@@ -1877,6 +2038,41 @@ repaints only when charge / verdict / state change (`drawGlyph()` keys a `data-d
charge float jitters every `NUT_INTERVAL_SECONDS`, and rebuilding the `<svg>` on that re-announces
an accessible name for a difference no eye can see.
**The `/gateway` page draws pipes, and a pipe's only claim is a count** (phase 29): four cards keyed
on the payload's own `source` word (`haproxy`, `nginx`, `map`) plus `unmatched`, each shipping
`hidden` and appearing on the poll that has pipes for it — a gateway with no nginx in it spends no
panel saying so, and a section that did not answer at all is the page's one `why` line rather than a
card repeating it in a smaller font. A row is inbound (`:<listen port>`, its `server_name`s as the
domains, up to `MAX_CLIENTS` client addresses as the footnote) or outbound (the target **as the
config wrote it**, with the SNI name the stream map matched on in front of it when exactly one key
routes there), and its count is `countText()` — a `null` from the join prints `n/a` and never `0`
(rule 2). The pipe is hand-rolled SVG (`role="img"` + its own `aria-label` naming port/target,
domains, clients and the count), it **animates only while the count is ≥ 1**, and it is **never
coloured by a verdict**: `severity.py` has no `gateway.*` row, the payload carries no `severity` at
any depth, and `tests/test_api.py::test_no_gateway_count_is_coloured_by_a_verdict` fails the day a
`severity`/`ok|warn|fail` token appears in the page or the script (a busy ingress is the job, the
`net.*` and GPU precedents). A target matched on its socket **port** alone carries a `by port` chip,
because that count is the port's and not the target's (`by_port`, and the reason a shared `:443`
fan-out prints `n/a` instead of repeating one number sixteen times). The page head's five tallies are
the socket table's own totals (`listeners`, `established`) beside the totals the pipes carry (`pipes
in`, `pipes out`) plus `unmatched`, so the page prints its reconciliation rather than asserting it,
and the `unmatched` card's three cells stay three (`other` counts **ports**, the other two
**connections**). A label gets its **line**: `.pipe-label` wraps and `.pipe-name` is based on `max-content`, so a short
`:22` sits beside its count and a long target takes the whole column with the `by port` chip, the
config's own `server` name and the count beneath it (task 06 measured the alternative — one `nowrap`
line, seven characters of a hostname — and `data/logs/29-task06-card-haproxy.png` is the result). What
still does not fit a ~150px column is ellipsised and not lost: `renderPipeRow` puts the whole string
on the element as `title` and `pipeAria` carries the whole endpoint, which is what
`test_a_clipped_pipe_label_keeps_its_full_text_on_the_element` pins along with the two declarations;
the residual and the options for it are gap 13. Charts follow `/system`: one
collapsed `Charts` `<details>` per card, built and
painted only while open, fed by **one** `/api/gateway/history` request per card per refresh naming
that card's own gauges, the legend naming pipes because the card heading already says which config
family it is. Every degradation arm uses the payload's own words — the `why` sentence for a
pre-1.2.0/`--no-proxy` agent or a silent section, the `unreachable` dim for a host that has never
answered, and `#gate-empty` (naming `GATEWAY_HOST` and agent **1.2.0**) for `404 {configured: false}`
— and a failed poll re-draws nothing, so the last topology stays on screen.
An event row names its subject **once**. Every feed (`/system`, `/disks`, `/llm`, `/ups`) renders the
message through `Dash.eventText(event)`, which drops the `UpstreamError: ` wrapper and a
*leading* duplicated subject the way `alerts._clean_error()` does for the overview — `poller.py`
@@ -2660,8 +2856,9 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
Task 02 wrote that audit: `data/logs/27-task02-mobile-audit.cjs`
(`NODE_PATH=<playwright node_modules> node …`, `DASH_BASE`/`DASH_PW`, transcript
`27-task02-mobile-audit.out`, screenshots `data/logs/mobile-27-<page>-<width>.png`) sweeps
**every page the app serves** — the seven signed-in ones plus `/login` unsigned, eight in
total, not the nine the phase file counted — at 360 px and 1280 px, and per page demands: no
**every page the app serves** — every signed-in page plus `/login` reached unsigned (nine of
them since phase 29 task 05 put `/gateway` in the list; the sweep is exactly the kind of page a
hand-written four-page loop would have missed) — at 360 px and 1280 px, and per page demands: no
sideways document scroll, no `table.data` clipped (its `overflow-x` must be `auto`/`scroll`
when its content is wider, and its tail column measured reachable), no element past the
viewport except inside a horizontal scroll container, the three-row masthead with its controls
@@ -2722,7 +2919,10 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
0 FAIL / 0 FINDING** (9 skips, each one a check that width cannot answer — the clock, hidden
below 900 px) on all eight served pages at both widths, exit 0
(`data/logs/27-task02-mobile-audit.out`, screenshots `mobile-27-*.png`; the first run is kept
as `27-task02-first-run.out`), and `data/logs/27-task03-control.cjs` proves the new check is
as `27-task02-first-run.out`). Phase 29's completion pass re-ran the same sweep now that
`/gateway` is in it — **371 PASS / 0 FAIL / 0 FINDING**, 10 skips, nine served pages, exit 0
(`data/logs/29-final-mobile-audit.out`) — so the two declarations above still hold on a page
this audit was written before, and `data/logs/27-task03-control.cjs` proves the new check is
not vacuous by neutralising both declarations and re-creating the whole first-run findings list
(3/4/2 header offenders at 360 px, 8 stranded disclosure tables at 1280 px). The served-byte
pins are `test_a_card_header_wraps_its_facts_instead_of_losing_its_tail` and
@@ -2734,7 +2934,7 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
beside it, `20_masthead_nav_polish` has landed in full (now complete): the bar is three
rows at every width (flush right on the two control rows since phase 25 task 02), it is no
longer sticky, the phone menu slides, and both
`⚙ Account` and `⚷ Certs` render on all seven signed-in pages including the two they name
`⚙ Account` and `⚷ Certs` render on all eight signed-in pages including the two they name
(task 04, reversing phase 15's "no self-link" rule at the operator's request). The other,
`24_dev_mtls_rollout` (never accepting a non-mTLS agent) is closed: its fleet half landed on
2026-10-21 and task 04 re-read the end state read-only the same day — gap 11. No `http://`
@@ -2742,6 +2942,40 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
seed one (`tests/test_deploy_scripts.py` keeps it that way). What remains of phase 24 is
ordinary rotation hygiene, not a missing step.
13. **`/gateway`'s long pipe labels are ellipsised at a desktop width — a column-width question,
not a lost fact** (found and half-fixed by phase 29 task 06's live pass,
`data/logs/29-task06-live.out`; the picture is `data/logs/29-task06-card-haproxy.png`).
What the pass found first: at 1440px the outbound name showed **seven characters**
(`reeseapps-…`) for a 40-character `server_name → target` string. `.pipe-label` was one `nowrap`
flex row, so the name shrank to whatever the `by port` chip, the config's own `server` name and
the `flex: none` count left. Wrapping the *text* inside that box was measured and rejected — it
tore one hostname over eleven lines of a word each. What shipped instead is the label getting
the line to itself: `flex-wrap: wrap` on `.pipe-label` beside `flex: 1 1 max-content` on
`.pipe-name`, so a short label (`:22`) keeps its count on the same line as before and a long one
is by itself wider than the column, takes the whole column, and lets the chip, the footnote and
the count wrap beneath it. That is ~22 characters instead of 7, the config's own `server` name
now reads in full, and nothing was truncated that was not truncated before.
**What is left** is arithmetic, not a bug: **28 of 42** labels still show an ellipsis because
`.pipe-columns` is `minmax(0, 1fr) auto minmax(0, 1fr)` inside one of three cards across a
1440px viewport, which is a ~150px column, and `reeseapps-brain-of-code.reeselink.com:443` is 41
characters. No fact is lost — `renderPipeRow` puts the whole string on the element as `title` and
`pipeAria` carries the whole endpoint, which is what
`tests/test_api.py::test_a_clipped_pipe_label_keeps_its_full_text_on_the_element` pins, and the
two declarations above are pinned beside it so the label cannot be squeezed back to seven
characters. In order of cost: (a) give the outbound column more of the card than the inbound one
(inbound labels are `:22`-shaped, so `minmax(0, .45fr) auto minmax(0, 1.55fr)` buys ~90px); (b)
two cards per row for the fan-out, which is the honest layout for a 17-row map but leaves the
third of the viewport empty; (c) accept it — the stream map's rows are read as a set, the
left-hand SNI name is the part that fits, and the count is the number the row is about. The
≤640px arm already stacks the two columns into one, where the label gets the whole card and
**16 of 42** clip instead of 28 (a stream-map string here runs to 70 characters, so even a 312px
column is not enough for the longest few) — measured in the same pass, which prints both counts
and demands a `title` on every clipped one. Whoever picks this up re-runs `data/logs/29-task06-live.cjs`: its
FINDING lines are the two counts, and the run fails if a clipped label ever stops carrying its
full text.
## Locked decisions (and why)
- **SQLite, not Prometheus/Grafana.** Handful of boxes, 1–10s resolution, one file the
README.md
+114 -39
View File
@@ -1,15 +1,16 @@
# Homelab Dashboards
Four dashboards and the overview page that points at them — one domain, one process, one SQLite
Five dashboards and the overview page that points at them — one domain, one process, one SQLite
file:
| Page | What it answers | Source |
| ----------- | ------------------------------------------------------------------------------- | -------------------------------------------------- |
| `/` | Is anything wrong, and which dashboard do I need? | all four, in one request (`/api/summary`) |
| `/system` | What are my machines doing — CPU, memory, disk, GPU, network, and the history of all of it | `agent/host_agent.py`, our own stdlib agent per host |
| `/disks` | Are any of my **physical** disks failing, and which one — health, temperature, power-on hours, wear, sector alarms | the same agents, running `smartctl -i -A -H` per whole drive (never a partition or a mount); a host whose agent has no `smartctl` is not a host on this page |
| `/llm` | Tokens/sec, active sessions, queue depth, KV-cache pressure, latency percentiles | llama.cpp-family (gufo) `/metrics` + `/slots`, vLLM `/metrics` — read-only |
| `/ups` | Battery health, runtime left, power events, and battery-test buttons | NUT `upsd`, on whatever port it really listens on (its own wire-protocol client) |
| Page | What it answers | Source |
| ------------ | ------------------------------------------------------------------------------- | -------------------------------------------------- |
| `/` | Is anything wrong, and which dashboard do I need? | all five, in one request (`/api/summary`) |
| `/system` | What are my machines doing — CPU, memory, disk, GPU, network, and the history of all of it | `agent/host_agent.py`, our own stdlib agent per host |
| `/disks` | Are any of my **physical** disks failing, and which one — health, temperature, power-on hours, wear, sector alarms | the same agents, running `smartctl -i -A -H` per whole drive (never a partition or a mount); a host whose agent has no `smartctl` is not a host on this page |
| `/llm` | Tokens/sec, active sessions, queue depth, KV-cache pressure, latency percentiles | llama.cpp-family (gufo) `/metrics` + `/slots`, vLLM `/metrics` — read-only |
| `/ups` | Battery health, runtime left, power events, and battery-test buttons | NUT `upsd`, on whatever port it really listens on (its own wire-protocol client) |
| `/gateway` | Which domains are coming in on which port, and where the traffic goes — the ingress's listeners, `server_name`s and SNI map joined to its live connections | the agent on **one** host, `GATEWAY_HOST` (default `gateway`), reading its `haproxy.cfg` + nginx tree and `/proc/net/tcp{,6}` — so it needs **agent 1.2.0**, and the whole page is *unconfigured* (never red) unless `GATEWAY_HOST` names a `HOST_TARGETS` entry |
No Prometheus, no Grafana, no CDN, no build step. Nothing is fetched from the internet at
runtime; every asset is served by this app.
@@ -155,8 +156,8 @@ what the config says: every `frontend` block with its name, each `bind` address:
written (`:::22` is the address `::` on port `22`) and the backend names it routes to
(`default_backend`, plus every named `use_backend`), and every `backend` block with each `server`'s
name, target and port — `port: null` when the line names none, never `0`. It interprets nothing:
the routing between the two lists is the **names**, so joining them is the page's job (the gateway
page is phase 29) and the agent never evaluates a rule. `available` answers three questions the way `smartctl_available` does —
the routing between the two lists is the **names**, so joining them is the gateway page's job
(`sources/gateway.py`) and the agent never evaluates a rule. `available` answers three questions the way `smartctl_available` does —
`true` read and parsed, `false` no config at that path (this host runs no haproxy), `null` a file
is there but could not be read or understood — and each non-`true` answer costs exactly **one**
`errors` line for the pass. Like the SMART lines, those lines never make the *system* page amber:
@@ -201,10 +202,13 @@ kernel without IPv6 has no `tcp6` file, which costs nothing and no line, while a
there and denied costs one. `available` is `true` when either table answered and `null` when
neither did, with one line saying why; there is no `false`, because a table that did not answer is
"no answer" and not "this host has no sockets". Nothing here is joined to a config: a socket does
not know its domain or its backend, so phase 29's gateway page will be what joins these ports to
the `haproxy` and `nginx` sections — where a socket no frontend claims stays visible as one. Until
that page exists the three sections live only in the agent's own `/snapshot` document: no `/api`
payload grows them, no `sample` row is stored for them, and nothing charts them.
not know its domain or its backend, so the **gateway page** is what joins these ports to the
`haproxy` and `nginx` sections, and a socket no config claims stays visible as one. The join lives
in `sources/gateway.py`, never in the agent: `/snapshot` keeps the three sections as the agent read
them, `/api/gateway/snapshot` serves the joined topology, and only the counts that join produces are
stored — as the `gateway.in_conns@<port>` / `gateway.out_conns@<host>:<port>` gauges the page
charts. Nothing in `/api/system/snapshot` grows a proxy or socket field, and no `sample` row is
written for a config.
All three parsers are pinned to captures rather than to descriptions, and
`./scripts/capture-gateway-config.sh` re-records them over the `GATEWAY_SSH` alias (default
@@ -459,8 +463,9 @@ cannot match against a denylist.
One admin, one password, and the server **refuses to start** when it has neither a stored
verifier nor `DASH_ADMIN_PASSWORD` — the domain is public and the fleet's hostnames and UPS
controls sit behind this login, so there is no way to run it open by accident. There is no
second account and no per-user view: the five pages (`/`, `/system`, `/disks`, `/llm`, `/ups`),
the two admin pages (`/settings`, `/certs`) and every data route belong to that one operator.
second account and no per-user view: the six dashboard pages (`/`, `/system`, `/disks`, `/llm`,
`/ups`, `/gateway`), the two admin pages (`/settings`, `/certs`) and every data route belong to
that one operator.
A session is an **HMAC-signed cookie** (`dash_session`), not a row in a table: it carries the
signed-in name and its own expiry, is valid for `DASH_SESSION_HOURS` (default `720` = 30 days),
@@ -671,10 +676,11 @@ documentation and is never loaded, and `scripts/dev-server.sh` exports `.env.dev
| `RATE_WINDOW_SECONDS` | the **host's** Δ window (default 10) — bytes/sec and jiffies→CPU %, measured over what a 10 s poll can fill; each value carries the window it actually got, and `/system`'s footnote prints that, not this default. A window the poll loop cannot honour — the ~0.2 s catch-up after a fetch slower than the interval, or a base older than 2× the window — answers one `n/a` instead of a spike, and a rate 8× above the last one is held until a window measured from a **different** sample confirms it (phase 13). The LLM rates do **not** use it: they are measured over `LLM_INTERVAL_SECONDS` (phase 23) |
| `LLM_INTERVAL_SECONDS` | the `/llm` poll cadence (default 2 s) **and** the Δ window behind every token rate — one estimator per measuring cadence, so a slot that starts reads steady within two polls instead of ramping over five. Slowing the poll slows the window with it; there is no separate LLM window knob, and a window narrower than the interval would answer `n/a` every poll |
| `DASH_HOST` / `DASH_PORT` | bind address for the dashboard itself (default `127.0.0.1:8090`; the container sets `0.0.0.0`) |
| `HOST_INTERVAL_SECONDS` / `NUT_INTERVAL_SECONDS` | how often the host agents and upsd are polled (10 / 5 s) |
| `HOST_INTERVAL_SECONDS` / `NUT_INTERVAL_SECONDS` | how often the host agents and upsd are polled (10 / 5 s). `GATEWAY_INTERVAL_SECONDS` (30) and `GATEWAY_TIMEOUT_SECONDS` (10) are the gateway source's own pair: a connection count is the shape of half a minute, and the fetch is the plain agent one with headroom for a handler that may be mid-smartctl pass |
| `DISKS_INTERVAL_SECONDS` | how often the disk-health source re-reads each agent for SMART (default 300 s — SMART is not a per-second signal). No targets of its own: the `disks:<host>` jobs exist exactly when `HOST_TARGETS` does |
| `HOST_TIMEOUT_SECONDS` / `LLM_TIMEOUT_SECONDS` / `NUT_TIMEOUT_SECONDS` | per-source fetch timeout — one slow source never holds up the others |
| `DISKS_TIMEOUT_SECONDS` | the disks fetch's own budget (default 30 s), because the agent runs `smartctl` inside its snapshot handler and a wedged SMART pass must not read as a dead host |
| `GATEWAY_HOST` | which `HOST_TARGETS` name is the ingress the `/gateway` page watches (default `gateway`). It has no URL of its own — the source reads the same agent `/snapshot` the system page reads, projected onto agent 1.2.0's `haproxy`/`nginx`/`sockets` sections — so a name that is not in `HOST_TARGETS` means "no ingress to watch", and the source is unconfigured rather than red. The page refetches itself every 15 s in the browser and has no `POLL_MS_*` knob, because the payload only moves when that one job re-reads the ingress |
| `GUFO_API_KEY` / `VLLM_API_KEY` | only if an engine is behind a bearer token; they never leave the server |
| `NUT_HOST` / `NUT_PORT` | upsd address — **whatever port it really listens on**, not always 3493. Reads are unauthenticated in stock NUT |
| `NUT_USER` / `NUT_PASSWORD` | only needed for instant commands (battery tests) |
@@ -686,7 +692,7 @@ documentation and is never loaded, and `scripts/dev-server.sh` exports `.env.dev
| `DASH_SESSION_HOURS` | session lifetime (default `720` = 30 days) |
| `DASH_COOKIE_SECURE` | `1` marks the session cookie `Secure`; off because dev is `http://127.0.0.1`, and a request that arrived over `X-Forwarded-Proto: https` is treated as secure either way |
| `DASH_PBKDF2_ITERATIONS` | verifier cost (default `600000`). The knob exists for the test suite; leave it alone in a deployment. Raising it on a real box does take effect: the next successful sign-in re-hashes a cheaper stored verifier and keeps the signing key |
| `POLL_MS_SYSTEM` / `POLL_MS_LLM` / `POLL_MS_UPS` | how often the *browser* refetches each page (10 s / 2 s / 5 s) — shown in the page footer. `/disks` has no `POLL_MS_*` of its own and polls at a fixed 30 s: its payload only moves when the `disks` job re-reads the agents. A backgrounded tab idles its polls and fires one on return; nothing an operator presses stops them (phase 21) |
| `POLL_MS_SYSTEM` / `POLL_MS_LLM` / `POLL_MS_UPS` | how often the *browser* refetches each page (10 s / 2 s / 5 s) — shown in the page footer. `/disks` and `/gateway` have no `POLL_MS_*` of their own and poll at a fixed 30 s / 15 s: the first payload only moves when the `disks` job re-reads the agents, the second when the one `gateway` job re-reads the ingress. A backgrounded tab idles its polls and fires one on return; nothing an operator presses stops them (phase 21) |
| `DASH_RELOAD` | `1` makes `uv run dashboards` watch the source and restart on a change — development only, and every poll loop restarts with it |
| `DEBUGPY` / `DEBUGPY_PORT` | `DEBUGPY=1` opens a debugpy listener (default 5678) without blocking startup |
| `RETAIN_RAW_HOURS` / `RETAIN_MINUTE_DAYS` / `RETAIN_HOUR_DAYS` | history rollup and pruning |
@@ -700,12 +706,13 @@ installed.
```
agent/host_agent.py (each host) gufo / vLLM upsd (NUT)
│ GET /snapshot │ GET /metrics │ LIST UPS / LIST VAR / LIST CMD
│ (twice: system 10s, disks 5m) │ + /slots (gufo) │
│ (three readers: system 10s, │ + /slots (gufo) │
│ disks 5m, gateway 30s) │ │
▼ ▼ ▼
src/dashboards/collectors.py ────────────────────────────────────────────┐
│ parse + Δcounter/Δt (rates.py) │
▼ ▼
sources/host.py · disks.py · llm.py · nut.py poller.py (one task per source)
sources/host.py · disks.py · gateway.py · llm.py · nut.py poller.py (one task per source)
│ samples events │
▼ ▼
store.py ── SQLite (WAL): sample(source,host,metric,res,ts) + event + meta ── api.py → web/
@@ -720,9 +727,14 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
`/api/health`, 5-minute cadence, its own fetch timeout) reading the *same* agent
`/snapshot` the system page reads — a separate task is what keeps a SMART poll out of the
10-second utilisation poll and lets drive history go stale without blinking `/system`, and
`alerts.py` is why one dead agent is still one line on the overview rather than two. And it never quietly *stops*: the task is that source's only
poller, so a SQLite write that gets refused is reported (`ok: false`, the real reason, the
numbers aging, one line in the event log) instead of freezing the source under a green dot.
`alerts.py` is why one dead agent is still one line on the overview rather than two. The gateway
page's job (phase 29) is the **third** reader of that same `/snapshot`, and one job for the whole
deployment rather than one per host — `GATEWAY_HOST` names the single ingress, and a job per host
would ask four machines a question only one can answer — on its own 30 s cadence with no
`RateEstimator`, because a connection count is not a rate. And it never quietly *stops*: the task
is that source's only poller, so a SQLite write that gets refused is reported (`ok: false`, the
real reason, the numbers aging, one line in the event log) instead of freezing the source under a
green dot.
* `store.py` — raw samples roll up to 1-minute and 1-hour averages and the raw rows are
pruned, so a year of history is a few hundred MB rather than a disk emergency. Nothing a
per-request path asks is answered by a whole-table scan **on every call** (phase 14): the two
@@ -730,8 +742,8 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
row counts behind `/api/health` — come from a 60 s / 30 s cache, so the scan is paid once per
window (its cost is the honest remainder in the known-gaps list) instead of once per request. `series_names`' is ended only by a write that can
change it: a `(source, host, metric)` the store has never written, or any `DELETE`. A poll of an
existing series leaves it alone, which is what makes the cache serve at all on a store nine pollers
write to (task 04's live watch is the proof: 535 ms per call before, 1.8 ms median after), and the
existing series leaves it alone, which is what makes the cache serve at all on a store every
poller writes to (task 04's live watch is the proof: 535 ms per call before, 1.8 ms median after), and the
counts age on the clock. A chart's resolution comes from one capped compound statement instead of
three full `COUNT`s per series. `last_maintain` and `size_bytes` are deliberately never cached —
the first is how an operator tells a stalled rollup loop from a healthy one.
@@ -801,12 +813,17 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
that stopped answering is `fail`; one that is stale but still serving its last good value
is `warn`; an **unconfigured** source never alerts; a dead source is one line rather than
one per stale metric; the list dedupes per (source, metric) and is capped at 12 with a
marker that says how many it hid.
marker that says how many it hid. A job that dials an agent another job already named adds no
line either (`AGENT_ECHO_DASHBOARDS`: `disks` and, since phase 29, `gateway` — three jobs read
one `/snapshot`, so one dead box is one overview row), while a job failing *while its host
answers* keeps its own voice and links to the page that explains it (`/disks` for a wedged SMART
pass, `/gateway` for a proxy or socket section that went quiet). Staleness is measured in each
source's own interval, so the gateway page goes amber after 90 quiet seconds and not after 30.
* `api.py` — the JSON every page reads: `/api/config`, `/api/health`, `/api/events`,
`/api/history`, `/api/metrics`, `/api/summary`, one `snapshot` + `history` pair per
dashboard (`/api/system/*`, `/api/disks/*`, `/api/llm/*`, `/api/ups/*` — including
`/api/ups/commands`, which answers what may be sent to each UPS right now and why not when it
may not), and the certificate authority's own surface (`/api/certs`,
dashboard (`/api/system/*`, `/api/disks/*`, `/api/gateway/*`, `/api/llm/*`, `/api/ups/*` —
including `/api/ups/commands`, which answers what may be sent to each UPS right now and why not
when it may not), and the certificate authority's own surface (`/api/certs`,
`/api/certs/<name>/bundle`, `POST …/rotate`, `POST
…/revoke`, `POST /api/certs/root/rotate`). GETs everywhere except `POST /api/ups/command`,
which stays the **only** route that dials anything: the `/api/certs/*` writes are `meta` row
@@ -818,7 +835,7 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
no env config answers `404 {"configured": false}` **without dialling** (so a page says
"not configured" instead of going red), and a history query for a series that does not
exist answers an empty series instead of an error (so a card asks for every plot it could
draw and reads `n/a` for the ones the drive never reported). The four `history` routes take
draw and reads `n/a` for the ones the drive never reported). The five `history` routes take
one batched door into the store (`api._history_batch` → `store.history_many`), so a page's
chart request is one grouped resolution query plus one index seek per series rather than four
queries per series — and the JSON it answers with is the same document the per-series loop
@@ -842,13 +859,35 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
drive the hardware called `FAILED` without a second call); it is additive, exactly as `alerts`
was, and the key paths of both are pinned with and without targets configured. Since phase 22
that block lists the **same hosts** `/api/disks/snapshot` does, and `fleet.hosts_total` is
that row count, so `/` and `/disks` can never describe two different fleets.
* `web/` — five static dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`) plus the three
admin pages (`/login`, `/settings`, `/certs`), one stylesheet, hand-rolled
that row count, so `/` and `/disks` can never describe two different fleets. `/api/summary` grew a
`gateway` block the same way in phase 29 (`configured`, `host`, `supported`, `unreachable`,
`inbound`, `outbound`, `map_targets`, `unmatched`, `unmatched_listeners`, `partial`, `states`):
the ingress's own tallies, `null` and never `0` in every arm that did not measure, `partial: true`
when a total had to leave an unattributable pipe out, and no `severity` anywhere — a connection
count is not a graded number. `/api/config` carries the matching row, including
`gateway.interval_seconds`/`timeout_seconds` and the named target's redacted `url`.
`/api/gateway/*` (phase 29 task 03) is that page's pair. `/snapshot` is one host's whole
topology — the haproxy frontends and the nginx `listen`/`map` echo the pipes were built from,
the inbound pipes (port, its domains, its live connection count, up to 32 client addresses), the
outbound pipes (target `host:port`, `by_port` naming which half of the match is a socket fact,
the count, the config that named it) each carrying the `metric` it is stored under, the
`unmatched` tallies, and `supported`/`sections`/`why` for the degradation arms — and the counts
reconcile: inbound + outbound + unmatched == `sockets.established`, so the page's numbers are
provable against a direct `/proc/net/tcp` read. `GATEWAY_HOST` naming no `HOST_TARGETS` entry is
a `404 {"configured": false}` answered before any client is built (both arms spy-pinned), a host
that has never answered is the `unreachable` stub whose every count is `null`, and a 1.1.0 agent
is `supported: false` plus its one `why` sentence — three arms, no red among them (invariant 3).
`/history` serves the `gateway.in_conns@<port>` and `gateway.out_conns@<host>:<port>` gauges —
the port in the outbound name because two services on one host are two facts — with `hosts=`
defaulting to the ingress alone, unknown names answering as empty series, and no dial either way
because history reads only SQLite.
* `web/` — six static dashboard pages (`/`, `/system`, `/disks`, `/llm`, `/ups`, `/gateway`) plus
the three admin pages (`/login`, `/settings`, `/certs`), one stylesheet, hand-rolled
SVG charts (`charts.js`), and a masthead status dot per dashboard job on **every** signed-in
page — the two admin pages included (`disks` has its own:
the same agent can answer the 10-second utilisation poll and have a wedged SMART pass).
The masthead is **three rows at every width, the two control rows flush right**: the brand and the five dashboard
the same agent can answer the 10-second utilisation poll and have a wedged SMART pass, and
`gateway` has one because its job is its own poll of one ingress).
The masthead is **three rows at every width, the two control rows flush right**: the brand and the six dashboard
tabs, then the per-source indicators on their own row, then the `⚷ Certs` / `⚙ Account` /
`☾ Dark` / clock controls on a row under those, in that order, so the **time is the farthest-right
thing on the bar** (phase 25: the badges and the cluster were re-justified right, then the clock
@@ -858,13 +897,13 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
indicators, and so two rows, and a control is never hidden to make a
narrower width work. There is no control that stops the polling (phase 21): the bar used to carry
a `⏸ Pause` button with a `Space` shortcut, and it left the numbers on screen looking current with
no `stale` label on them. Both admin links are on all seven
signed-in pages — the five dashboards plus `/settings` and `/certs`, the two they name included —
and not on `/login`, the eighth page and the only anonymous one: the cluster is the same wherever
no `stale` label on them. Both admin links are on all eight
signed-in pages — the six dashboards plus `/settings` and `/certs`, the two they name included —
and not on `/login`, the ninth page and the only anonymous one: the cluster is the same wherever
you are, and a link to the page you are already on is a harmless no-op rather than chrome that
vanishes. The bar is **not sticky**: it scrolls away
with the page, so every pixel of height goes to the data (it stays a positioned element, which is
what keeps the mobile dropdown anchored under it). At ≤640px the five tab links leave the bar
what keeps the mobile dropdown anchored under it). At ≤640px the six tab links leave the bar
and come back as a dropdown under the whole of it, opened by the `☰` button (`/login` renders no
tabs, so it has no button either), and that menu **slides**: 180 ms of fade plus an 8 px rise on
the way out and the same on the way back in, with nothing hidden from a keyboard or a screen
@@ -888,6 +927,9 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
list reads as an explicit "all clear", and a live problem moves the panel above the
table. A host that is not answering keeps its row — dimmed, every number `n/a` — so the
fleet never quietly gets smaller and last good values are never mistaken for live ones.
Under that table is the ingress's **one line** (`gateway · 14 inbound · 4 outbound`, linked to
`/gateway`): the counts are `/api/summary`'s own sums, a count the source did not measure prints
`n/a` rather than `0`, and a deployment with no `GATEWAY_HOST` renders no line at all.
A big colour-coded number is `Dash.hero(el, {...})` on the
`.hero` / `.hero-bar` styles, coloured by the `severity` in the payload via
`Dash.severityFrom()` — the only colour vocabulary is `ok` / `warn` / `fail`, and a value
@@ -994,6 +1036,26 @@ certs.py ─ root CA + client cert, minted once at startup (unreadable row → n
`No test initiated` or no variable at all — says the same "run a battery test" sentence as
`unknown`. Only a word the map has never seen stays a warning, printed verbatim so it can
be added.
`/gateway` answers the topology question `/system` cannot: which sockets on the ingress face
inbound, where each one's traffic goes, and how many connections are live on each. It is four
cards keyed on what the **config** said a pipe came from — `haproxy`, `nginx`, the `stream map`,
and `unmatched` — each hidden until a poll actually has pipes for it, so a router with no nginx
in it spends no panel saying so. An inbound row is a listen port with its `server_name`s and the
client addresses on it; an outbound row is the target **exactly as the config wrote it** (the SNI
name the stream map matched on in front of it when one key routes there), and a target the join
could only match on its socket **port** carries a `by port` chip, because that count belongs to
the port and not to that service. A pipe animates while it carries at least one live connection
and rests at zero, and **nothing on the page is coloured by a verdict** — a busy ingress is the
job, so `severity.py` rates no `gateway.*` metric and no number gets a green/amber/red tint. A
count the join did not decide prints `n/a` and never `0`, which is why the fan-out on a port many
domains share is `n/a` rather than one number repeated as if it were sixteen. The header's five
tallies print the reconciliation instead of asserting it: what the kernel's socket table reported
(`listeners`, `established`) beside what the pipes carry (`pipes in`, `pipes out`) and what no
config claims (`unmatched` — ports and connections kept as separate cells). Each card's trend is
behind a collapsed **Charts** toggle like `/system`'s, one `/api/gateway/history` request per
card per refresh. With no `GATEWAY_HOST` the page is the setup sentence rather than a red panel,
and an ingress running an agent older than **1.2.0** says so in the agent's own words with every
count `n/a` and nothing coloured.
The four event feeds (`/system`, `/disks`, `/llm`, `/ups`) are a **paged window** over the `event`
table rather than a printout of it: the list is a bounded box that scrolls, 50 events to a
page, with `1–50 of 200` between `‹ prev` and `next ›` (disabled at the ends) underneath.
@@ -1171,6 +1233,19 @@ quietly becoming a fallback chain. A busy
slot's `prompt` is the user's in-flight request text, so the capture script blanks it and
the page never reads it.
`tests/test_gateway_source.py` is where the ingress join's honesty lives, and it pins that module
from both ends. The input is the **captured** gateway — `gateway_haproxy.cfg`, the nginx main config
plus its included documents, and the two `/proc/net/tcp{,6}` captures — run through the real
`host_agent` parsers, so the test compares the join against the agent's own document rather than
against a copy of its field names. The assertions are the join's: the pipes carry exactly the ports,
`server_name`s and targets those documents name; `inbound + outbound + unmatched` equals the socket
table's own ESTABLISHED count, so no connection is counted twice or dropped; a listening port no
config claims stays visible as `unmatched.other`; several names sharing one port produce `null`
counts and a `unmatched.groups` line rather than one number repeated on sixteen pipes; and **no join
ever asks a resolver** — `socket.getaddrinfo` is spied on and fails the test, because deciding
"already an address, or a name?" is the only question that module may ask, and a DNS lookup in the
poll loop would be rule 2 with a round trip attached.
`tests/rate_window_contract.py` is the same device for one number rather than a field table: the Δ
window each source measures its rates over — `RATE_WINDOW_SECONDS` for the host agents,
`LLM_INTERVAL_SECONDS` for the engines, one estimator per measuring cadence (phase 23). It holds
src/dashboards/alerts.py
+50 -12
View File
@@ -26,7 +26,14 @@ The rules, and the reason each exists:
operator can see which line was crossed, and the numbers still live in one file.
* **A dead source is one alert.** Its last good payload is still in the hub, so its stale
threshold breaches would pile three more amber lines on top of the red one that already
names it. Payload-derived alerts are skipped for a source that is not answering.
names it. Payload-derived alerts are skipped for a source that is not answering. A job that
dials an agent another job already named (`AGENT_ECHO_DASHBOARDS`) adds no line either.
* **No verdict of its own, ever.** Nothing here compares a number to a threshold: the host and
LLM rules read `severity` dicts and the UPS rule reads `health.checks[]`. That is why a source
whose payload carries no verdicts can only ever alert on reachability — the `gateway` page's
connection counts are such a source (`severity.py` rates none, the `net.*` precedent), and a
busy proxy is the job rather than a fault. There is no `gateway` payload rule and none is
wanted: adding one would put a threshold in this module, which the rule above forbids.
* **Bounded and deduped.** One alert per (source, metric) — the `id` is that pair — and at
most :data:`MAX_ALERTS` of them, the last being a marker that says how many it hid. A
40-disk fleet must not turn the landing page into a log tail.
@@ -59,7 +66,8 @@ _RANK = {FAIL: 0, WARN: 1}
#: Where each dashboard lives, so every alert can say where to look. `disks` joined with the
#: `/disks` page itself (phase 09 task 04): before that route existed a disk alert pointed at
#: `/`, because a link to a page that answers 404 is worse than no link.
HREF = {"system": "/system", "disks": "/disks", "llm": "/llm", "ups": "/ups"}
HREF = {"system": "/system", "disks": "/disks", "gateway": "/gateway", "llm": "/llm",
"ups": "/ups"}
DEFAULT_HREF = "/"
#: A source whose last good value is older than this many of its own poll intervals is
@@ -167,20 +175,49 @@ def _interval(settings: Settings, dashboard: str) -> float:
# wrong interval would amber the landing page every 5 minutes for a source that is
# behaving exactly as configured.
return float(settings.disks_interval)
if dashboard == "gateway":
# 30 s by default: the pipes are drawn from a 30-second poll, so amber after 90 seconds of
# silence means "what you are looking at is two polls old" and nothing else. Judging it by
# the 10 s system interval would amber `/` for a source polling exactly as configured.
return float(settings.gateway_interval)
return 60.0
def _system_partner_outage(states: Sequence[Mapping[str, Any]], key: str) -> bool:
"""Is the `system` job for the same host down too? (used for `disks` job keys)
#: Dashboards whose job dials the **same** agent `/snapshot` the `system` job of that host dials.
#: Their failure is the consequence of an agent being down rather than a second fault, so while the
#: host's own job is also failing they add no line (`AGENTS.md`: one dead source, one line).
#: `gateway` joined in phase 29 for exactly the reason `disks` did in phase 09: the third reader of
#: one agent must not turn one outage into three overview rows.
AGENT_ECHO_DASHBOARDS = ("disks", "gateway")
The disk-health job reads the same agent `/snapshot` the system dashboard does, so its
own failure is the *consequence* of an agent being down, never a second fault. The
system alert already names the machine once (`AGENTS.md`: one dead source, one line),
so this suppresses the echo and nothing else: a `disks` job that fails while its host
answers — SMART wedged inside the agent's snapshot handler, say — is still one alert,
because that failure has no other voice.
def _partner_host(dashboard: str, key: str, label: str) -> str:
"""Which host's `system` job reads the same agent as this one — the echo rule's lookup.
Only called for :data:`AGENT_ECHO_DASHBOARDS`, which is why both arms answer a name. The
`disks:<host>` job carries its host in its key; the `gateway` job is one per deployment (the
`nut` shape) and carries its host in its **label**, because `GATEWAY_HOST`'s name is what the
alert prints and the key is the single word `gateway`, which says nothing about a machine.
"""
if dashboard == "gateway":
return label
return key.split(":", 1)[1] if ":" in key else key
def _system_partner_outage(states: Sequence[Mapping[str, Any]], host: str) -> bool:
"""Is the `system` job for that host down too?
The disk-health and gateway jobs read the same agent `/snapshot` the system dashboard does, so
their own failure is the *consequence* of an agent being down, never a second fault. The system
alert already names the machine once, so this suppresses the echo and nothing else: a `disks`
job that fails while its host answers — SMART wedged inside the agent's snapshot handler, say —
is still one alert, because that failure has no other voice. The same is true of a `gateway`
job whose host answers but whose proxy sections stopped parsing: only that job can say it.
A missing partner answers `False`, not `True`: "I cannot find the host's job" is not evidence
that the host is fine, and silencing this job on that guess would hide the one fault it has a
voice for.
"""
host = key.split(":", 1)[1] if ":" in key else key
partner = next((state for state in states
if state.get("dashboard") == "system"
and str(state.get("key") or "") == f"host:{host}"), None)
@@ -307,7 +344,8 @@ def _source_alerts(sink: _Sink, settings: Settings, states: Sequence[Mapping[str
href = HREF.get(dashboard, DEFAULT_HREF)
key = str(state.get("key") or label)
if not state.get("ok"):
if dashboard == "disks" and _system_partner_outage(states, key):
if dashboard in AGENT_ECHO_DASHBOARDS and _system_partner_outage(
states, _partner_host(dashboard, key, label)):
continue # the host agent is already named by its own alert; one line
# the alert starts with the name, so the name inside the message is removed —
# "driveripper: driveripper: ..." reads like two problems instead of one
src/dashboards/api.py
+204 -4
View File
@@ -33,10 +33,12 @@ from .collectors import NutMonitor
from .config import Settings, strip_userinfo
from .poller import Hub, SourceState
from .severity import classify
from .sources import gateway as gateway_source
from .sources import nut
from .sources.disks import DASHBOARD as DISKS_DASHBOARD
from .sources.disks import SOURCE as DISKS_SOURCE
from .sources.disks import fleet_disk_summary
from .sources.gateway import DASHBOARD as GATEWAY_DASHBOARD
from .sources.host import fleet_summary
from .sources.llm import fleet_totals
from .store import Event, SampleStore
@@ -196,6 +198,102 @@ def _ups_severity(card: dict[str, Any]) -> dict[str, str | None]:
}
def _gateway_url(settings: Settings) -> str | None:
"""The `display_url` of the `HOST_TARGETS` entry `GATEWAY_HOST` names, or `None`.
`display_url` rather than `url` for the reason every other row on `/api/config` has one: that
body goes to the browser, and a token written into a target's userinfo must stop at the backend
(`AGENTS.md` invariant 6). `None` when no target carries the name — which is the same fact
`enabled: false` states, printed as the URL the page would have dialled.
"""
return next((target.display_url for target in settings.host_targets
if target.name == settings.gateway_host), None)
def _pipe_totals(pipes: Any, counting: bool) -> tuple[int | None, bool]:
"""Σ of a pipe list's connection counts, and whether any pipe's count was unknown.
`counting` is the payload's own answer to "did the kernel's socket table reply". When it did
not, every count is `null` and so is the total, with nothing left to call partial: a `0` there
would print an idle gateway over a source that never measured one (`AGENTS.md` rule 2). When it
did, the sum covers the pipes that carry a number and `partial` says at least one other did not
(a port whose configured targets cannot be told apart, `sources/gateway.py`) — so a total is
never quietly a subtotal, and `partial` only ever qualifies a total that exists.
"""
rows = [pipe for pipe in (pipes or []) if isinstance(pipe, Mapping)]
if not counting:
return None, False
values = [pipe.get("connections") for pipe in rows]
known = [value for value in values
if isinstance(value, int) and not isinstance(value, bool)]
return sum(known), len(known) != len(values)
def _gateway_summary(payload: Mapping[str, Any] | None, settings: Settings) -> dict[str, Any]:
"""`/api/summary`'s gateway block: the ingress's totals, or a stub of unknowns.
Additive in phase 29. The page charts from `/api/gateway/snapshot`, which has one entry per
pipe; this block is the **tally** a programmatic consumer (and the overview's one line, task
05) reads without walking the pipes. Three arms, and they are three different sentences:
* **unconfigured** (`GATEWAY_HOST` is not a `HOST_TARGETS` name) — `configured: false`, every
count `null`, `states: []`. There is no job, so there is nothing to age (invariant 3).
* **configured, never answered** — the stub `system.hosts` and `/api/disks` use: the source is
in the deployment, it has no numbers yet, and `unreachable: true` says which of the two facts
it is. `host` still names the machine, because that is a config fact and not a measurement.
* **answering** — the payload's own tallies, summed without inventing anything: `inbound` /
`outbound` are the sums of the pipes that have a count, `partial` says one pipe did not, and
`map_targets` counts the stream `map`'s routing entries.
Nothing here carries a `severity`: a connection count is not a graded number (`severity.py`
rates none, the `net.*` precedent), and an `unmatched` socket is a report and not a fault.
`href` is the row's link, from `alerts.py`'s one table of dashboard routes, so the overview's
line and an alert about the same source cannot spell the page differently (the rule
`test_overview_alert_rows_link_to_the_dashboard_that_explains_them` holds for an alert row).
It travels in the unconfigured arm too — a link nobody renders costs nothing, and a block that
grew the key only when configured would make the page test for two shapes.
"""
block: dict[str, Any] = {
"configured": settings.gateway_enabled,
"host": settings.gateway_host if settings.gateway_enabled else None,
"href": alert_rules.HREF[GATEWAY_DASHBOARD],
"supported": None,
"unreachable": True,
"inbound": None,
"outbound": None,
"map_targets": None,
"unmatched": None,
# `unmatched` is connections no configured pipe claimed; `unmatched_listeners` is a
# *listener* count (a port this host serves that neither config names). Two units, so two
# fields: adding them into one number would be an aggregate of unlike things.
"unmatched_listeners": None,
"partial": False,
}
if payload is None:
return block
block["unreachable"] = False
block["supported"] = bool(payload.get("supported"))
block["host"] = str(payload.get("host") or block["host"] or settings.gateway_host)
if not block["supported"]:
# A 1.1.0 agent or `--no-proxy`: nothing was read, so "0 map targets" would be a claim
# about a config nobody looked at. `supported: false` plus nulls is the honest pair.
return block
sockets = payload.get("sockets") or {}
counting = (isinstance(sockets.get("established"), int)
and not isinstance(sockets.get("established"), bool))
block["map_targets"] = len((payload.get("nginx") or {}).get("map_targets") or [])
inbound, partial_in = _pipe_totals(payload.get("inbound"), counting)
outbound, partial_out = _pipe_totals(payload.get("outbound"), counting)
block["inbound"], block["outbound"] = inbound, outbound
block["partial"] = partial_in or partial_out
unmatched = payload.get("unmatched") or {}
block["unmatched"] = (int(unmatched.get("inbound") or 0) + int(unmatched.get("outbound") or 0)
if counting else None)
block["unmatched_listeners"] = unmatched.get("other") if counting else None
return block
# --------------------------------------------------------------------------- #
# The admin session: login, status, logout
# --------------------------------------------------------------------------- #
@@ -563,6 +661,20 @@ def common_router() -> APIRouter:
"interval_seconds": settings.disks_interval,
"timeout_seconds": settings.disks_timeout,
},
"gateway": {
# The ingress page has no upstream of its own either: it is configured exactly when
# the name `GATEWAY_HOST` holds is one of the agents being dialled, so an operator
# who monitors no host by that name gets `enabled: false` and a page that says "not
# configured" rather than a red panel (invariant 3). `url` is that entry's
# `display_url` — the same redaction boundary every other row keeps, because this
# route's body goes to the browser and a `HOST_TARGETS` userinfo token must not
# travel in it (invariant 6) — and `null` when no target carries the name.
"enabled": settings.gateway_enabled,
"host": settings.gateway_host,
"url": _gateway_url(settings),
"interval_seconds": settings.gateway_interval,
"timeout_seconds": settings.gateway_timeout,
},
"ups": {
"enabled": settings.nut_enabled,
"server": f"{settings.nut_host}:{settings.nut_port}" if settings.nut_enabled else None,
@@ -619,10 +731,10 @@ def common_router() -> APIRouter:
Everything else here is for **programmatic use**: this is the cheapest whole-fleet
probe that exists, so the live tests, an external check and a notifier later read
`llm.engines`, `ups.ups` and the `states`/`recent_events` no panel paints any more.
Keys are additive-only — `alerts` was added in phase 02 and `disks` in phase 09; the
`disks` block is for programmatic consumers too (which drives are down to `n/a`, which
ones the drive itself called `FAILED`), and nothing was renamed, dropped, or given a new
meaning. `tests/test_api.py` pins each path below against a future cleanup.
Keys are additive-only — `alerts` was added in phase 02, `disks` in phase 09 and `gateway`
in phase 29; the `disks` block is for programmatic consumers too (which drives are down to
`n/a`, which ones the drive itself called `FAILED`), and nothing was renamed, dropped, or
given a new meaning. `tests/test_api.py` pins each path below against a future cleanup.
"""
# A configured job that has never answered gets the same stub `/api/*/snapshot`
# hands out: a host in `HOST_TARGETS` with no payload yet is still in the fleet, and
@@ -652,6 +764,10 @@ def common_router() -> APIRouter:
# (`_disk_visible`). `disk_entries` stays unfiltered as the "are there disk jobs at all"
# question below, so an all-hidden fleet answers a zero fleet exactly as the snapshot does.
disks_hosts = [entry for entry in disk_entries if _disk_visible(entry)]
# the ingress's last good pipes, or `None` when the source is unconfigured (no job) or has
# never answered — `_gateway_summary` turns that second case into the stub
gateway_entry = ctx.hub.jobs.get(gateway_source.SOURCE)
gateway_payload = gateway_entry.state.payload if gateway_entry is not None else None
ups_payload = None
nut_job = ctx.hub.jobs.get("nut")
if nut_job and nut_job.state.payload:
@@ -716,6 +832,12 @@ def common_router() -> APIRouter:
"unreachable": bool(h.get("unreachable"))}
for h in disks_hosts],
"states": ctx.hub.states(DISKS_DASHBOARD)},
# additive in phase 29: the ingress's own totals, from the one `gateway` job. Same
# three-way shape as every block above — unconfigured, configured-but-never-answered
# (a stub of `null`s, never `0`), and answering — and no `severity` anywhere, because
# a connection count is not a graded number and an unclaimed socket is a report.
"gateway": {**_gateway_summary(gateway_payload, ctx.settings),
"states": ctx.hub.states(GATEWAY_DASHBOARD)},
"ups": {"enabled": ctx.settings.nut_enabled,
"simulated": bool(ups_payload and ups_payload.get("simulated")),
"ups": [{"name": card["name"], "overall": card["health"]["overall"],
@@ -902,6 +1024,84 @@ def disks_router() -> APIRouter:
return router
# --------------------------------------------------------------------------- #
# Gateway dashboard (the ingress's pipes)
# --------------------------------------------------------------------------- #
def gateway_router() -> APIRouter:
"""`/api/gateway/*` — GETs only, like every other dashboard's surface.
A fifth data page grows no write (invariant 1's shape): reading two config files and a kernel
socket table justifies no mutation, and the app's only route that dials an upstream stays
`POST /api/ups/command`. `tests/test_api.py` enumerates the served verbs to keep that true.
"""
router = APIRouter(prefix="/api/gateway", tags=[GATEWAY_DASHBOARD])
@router.get("/snapshot")
async def snapshot(ctx: AppContext = Depends(get_context)) -> dict[str, Any]:
"""The one ingress host's pipes: its configs, its live counts, and what nothing claimed.
The payload is `sources/gateway.parse`'s, verbatim and unread by this module: `haproxy`/
`nginx` are the config the agent read, `inbound` one entry per configured listen port (its
domains, its live connection count, up to `MAX_CLIENTS` client addresses), `outbound` one
entry per configured target (with `by_port` naming which half of the match is a socket
fact and which is the config's), `unmatched` the connections and listeners no config
names, and each pipe's own `metric` so a chart asks for the series it *is*. Three arms
reach the page and they are three different sentences: `404 {"configured": false}` when
`GATEWAY_HOST` names no target (answered below, before anything that could open a
socket — invariant 3), `supported: false` for an agent that predates the sections, and
`unreachable: true` for a host that has never answered — the stub, whose every count is
`null` because an agent that has not spoken has not reported an idle gateway (rule 2).
Nothing here carries a `severity` and nothing carries a `window_seconds`: a connection
count is neither a graded number nor a measurement over a window, so `/gateway` colours
nothing and prints no `Δ` (the `net.*` precedent).
"""
if not ctx.settings.gateway_enabled:
# First line of the route, before any client, fetch, or store read: a deployment that
# monitors no host by this name has nothing to dial, and a red panel for it would
# train the operator to ignore red on the pipes that do exist (invariant 3).
raise HTTPException(404, detail={"configured": False,
"detail": "no gateway host configured — set "
"GATEWAY_HOST to a HOST_TARGETS name"})
job = ctx.hub.jobs.get(gateway_source.SOURCE)
payload = job.state.payload if job is not None else None
if payload is None:
# configured and dialling, nothing in yet: the stub keeps the page's row (a dead agent
# must not read as "never configured") and every count unknown
payload = gateway_source.stub(ctx.settings.gateway_host,
ts=job.state.last_attempt if job is not None else None)
# `states` last and additive: the payload's own `ts` is when the *agent* took the reading,
# which is what the page ages, and a route-level timestamp would overwrite it.
return {**payload, "states": ctx.hub.states(GATEWAY_DASHBOARD)}
@router.get("/history")
def history(metrics: str = Query("gateway.in_conns"), hosts: str = Query(""),
range: str = Query("24h"), points: int = Query(600, ge=10, le=4000),
ctx: AppContext = Depends(get_context)) -> dict[str, Any]:
"""The `gateway.in_conns@<port>` / `gateway.out_conns@<host>:<port>` gauges.
`metrics` are full series names (comma-separated) and an unknown one answers as an
**empty** series, not a 404 — the contract every other `history` route has, so a card asks
for every pipe it could draw in one request and a target with no history draws an empty
plot. `hosts` is comma-separated or empty for the one host this source reads (the name
`GATEWAY_HOST` holds, which is also the `host` column the gauges are stored under). The
outbound name carries the **port** because two services on one host are two facts — one
name for `minecraft…:25565` and `…:25566` would make the second insert overwrite the
first. History reads only SQLite, so nothing dials here whichever way the config goes;
the answer comes through `_history_batch` → `store.history_many`, the one batched door the
other four `history` routes use (phase 14 task 02).
"""
span = parse_range(range)
wanted_metrics = [item.strip() for item in metrics.split(",") if item.strip()]
wanted_hosts = [item.strip() for item in hosts.split(",") if item.strip()] or (
[ctx.settings.gateway_host] if ctx.settings.gateway_enabled else [])
return {"hosts": wanted_hosts, "range_seconds": span,
"series": _history_batch(ctx.store, gateway_source.SOURCE, wanted_hosts,
wanted_metrics, span, points)}
return router
# --------------------------------------------------------------------------- #
# LLM dashboard
# --------------------------------------------------------------------------- #
src/dashboards/app.py
+31 -12
View File
@@ -39,6 +39,7 @@ from .api import (
certs_router,
common_router,
disks_router,
gateway_router,
llm_router,
refusal,
system_router,
@@ -52,6 +53,7 @@ from .collectors import (
UpstreamError,
agent_tls_map,
disks_jobs,
gateway_job,
host_jobs,
llm_jobs,
nut_job,
@@ -69,13 +71,18 @@ WEB = Path(__file__).resolve().parent / "web"
# `/settings` is a page and not a dashboard: it is in `PAGES` (so it is a route) and *not* in
# `PUBLIC_PATHS` (so the gate guards it with no extra code — the route-enumeration test in
# `tests/test_auth_api.py` proves that, and `tests/test_api.py` proves the anonymous refusal for
# this page specifically). It carries no `aria-current` tab because it is not one of the five
# dashboards (`/`, `/system`, `/disks`, `/llm`, `/ups`).
# this page specifically). It carries no `aria-current` tab because it is not one of the six
# dashboards (`/`, `/system`, `/disks`, `/llm`, `/ups`, `/gateway`).
# `/certs` (phase 15 task 05) is the second of those admin pages, for the same reason: the
# certificate authority is not a fifth dashboard, it is the thing you administer past them.
# certificate authority is not a dashboard, it is the thing you administer past them.
# `/gateway` (phase 29) is the sixth dashboard, so it is in the tab nav with `aria-current` of its
# own (task 05 — the pinned tab audit in `tests/test_api.py` walks this dict's six dashboards).
# Being in this dict is what puts it behind the session gate — `PUBLIC_PATHS` is untouched, and
# `tests/test_auth_api.py` proves the refusal from the served route table rather than from a list
# written here.
PAGES = {"/": "index.html", "/system": "system.html", "/disks": "disks.html",
"/llm": "llm.html", "/ups": "ups.html", "/login": "login.html",
"/settings": "settings.html", "/certs": "certs.html"}
"/llm": "llm.html", "/ups": "ups.html", "/gateway": "gateway.html",
"/login": "login.html", "/settings": "settings.html", "/certs": "certs.html"}
VERSION = "0.1.0"
# Everything else requires a signed-in admin. This list is the whole attack surface, so
@@ -110,14 +117,15 @@ def build_hub(settings: Settings, store: SampleStore, client: httpx.AsyncClient,
polling in tests and not in the container.
`tls` is the per-host certificate material (`agent_tls_map`, built from `build_certs`), and
it is a parameter rather than something this function derives because the two `host:`/`disks:`
jobs for one host must hold **the same** entry: the same identity on the wire and the same
revocation answer, so a revoked agent fails both of its sources with one sentence and neither
of them opens a socket.
it is a parameter rather than something this function derives because the `host:`, `disks:` and
`gateway` jobs of one host must hold **the same** entry: the same identity on the wire and the
same revocation answer, so a revoked agent fails every source that reads it with one sentence
and none of them opens a socket.
Order is host → disks → llm → ups so far only for readability: two jobs carry the same
host label (`host:gw`, `disks:gw`), so anything that looks a job up by label alone gets
the system record — `alerts.py` filters by dashboard instead and does not depend on it.
Order is host → disks → gateway → llm → ups so far only for readability: three jobs can carry
the same host label (`host:gw`, `disks:gw`, and `gateway` for the ingress), so anything that
looks a job up by label alone gets whichever registered first — `alerts.py` filters by
dashboard instead and does not depend on it.
There are **two** rate estimators, one per measuring cadence (phase 23). A window is only
a claim about the polls that fill it: the `host` jobs run every `HOST_INTERVAL_SECONDS`
@@ -130,6 +138,10 @@ def build_hub(settings: Settings, store: SampleStore, client: httpx.AsyncClient,
phase-13 gates (the `MIN_WINDOW_SECONDS` floor, the ceiling, the 8x jump confirmation) are
the estimator's own and therefore apply to the smaller window unchanged; the `Δ<window>`
label always prints the window actually used, so the smaller claim is legible.
Neither estimator reaches the `gateway` job. Its numbers are counts read at one instant, and a
count that goes through a Δ-window becomes a rate with a window label and no measurement
behind it (`sources/gateway.py`, AGENTS.md rule 2), so that job is handed no estimator at all.
"""
host_rates = RateEstimator(span=settings.rate_window)
llm_rates = RateEstimator(span=settings.llm_interval)
@@ -140,6 +152,12 @@ def build_hub(settings: Settings, store: SampleStore, client: httpx.AsyncClient,
for job in disks_jobs(list(settings.host_targets), client, DisksCollector(),
settings.disks_interval, timeout=settings.disks_timeout, tls=tls):
hub.add(job)
# A third reader of the same `/snapshot`, for the one host `GATEWAY_HOST` names (phase 29).
# `None` when that name is not a target: the source is unconfigured rather than red, so it
# gets no task at all (the `disks` gate, invariant 3) and `/api/gateway/*` 404s without a dial.
gateway = gateway_job(settings, client, tls=tls)
if gateway is not None:
hub.add(gateway)
for job in llm_jobs(settings, client, llm_rates):
hub.add(job)
nut_monitor: NutMonitor | None = None
@@ -359,6 +377,7 @@ def create_app(settings: Settings | None = None, store: SampleStore | None = Non
app.include_router(common_router())
app.include_router(system_router())
app.include_router(disks_router())
app.include_router(gateway_router())
app.include_router(llm_router())
app.include_router(ups_router())
src/dashboards/collectors.py
+48 -1
View File
@@ -17,10 +17,11 @@ from typing import Any
import httpx
from . import certs
from .config import Target, strip_userinfo
from .config import Settings, Target, strip_userinfo
from .poller import Job, Result
from .rates import RateEstimator
from .sources import disks as disks_source
from .sources import gateway as gateway_source
from .sources import llm as llm_source
from .sources import nut
from .sources.host import HostCollector, parse_host_snapshot
@@ -273,6 +274,25 @@ async def fetch_disk_snapshot(client: httpx.AsyncClient, target: Target,
return disks_source.parse_disks_snapshot(target.name, body, disks_collector)
# --------------------------------------------------------------------------- #
# Gateway topology (the /gateway dashboard)
# --------------------------------------------------------------------------- #
async def fetch_gateway_snapshot(client: httpx.AsyncClient, target: Target,
*, timeout: float | None = None,
tls: AgentTls | None = None) -> Result:
"""The same `/snapshot`, projected onto the connection topology of one ingress host.
A third read of an endpoint `system` already polls, for the same reason `disks` is a second
one: the cadences differ (a connection count is a 30 s shape, utilisation a 10 s one) and the
page has to age on its own instead of blinking `/system` (invariant 4). The fetch itself is
the plain one — the snapshot handler does no extra work for these sections beyond reading two
configs and two `/proc` tables — so `timeout` is headroom for a handler that may be mid-`smartctl`
pass for the other page, not for this one.
"""
body = await fetch_snapshot(client, target, timeout=timeout, tls=tls)
return gateway_source.parse_gateway_snapshot(target.name, body)
# --------------------------------------------------------------------------- #
# LLM engines
# --------------------------------------------------------------------------- #
@@ -493,6 +513,33 @@ def disks_jobs(targets: list[Target], client: httpx.AsyncClient,
]
def gateway_job(settings: Settings, client: httpx.AsyncClient, *,
tls: Mapping[str, AgentTls] | None = None) -> Job | None:
"""The **one** `gateway` job, for the host `GATEWAY_HOST` names — or `None`.
One job rather than one per host (the `nut` shape): the page is about the single ingress, and
a job per host would put a `gateway` record against every machine in the fleet for a question
only one of them answers. `label` is the host name, because that is what an alert and an event
print, and it is what `alerts.py` uses to find the `host:<name>` job dialling the same agent
(the echo rule: a dead agent is one line, not three).
`None` when the name is not in `HOST_TARGETS` — the `disks` job's configured gate, invariant
3: an unconfigured source gets **no task at all**, so `/api/health` carries no red record for a
page nobody installed and nothing dials. `tls` is the same per-host entry the `host:`/`disks:`
jobs of that host hold, so revoking the gateway's certificate fails all three with one sentence
and none of them opens a socket.
"""
target = next((item for item in settings.host_targets
if item.name == settings.gateway_host), None)
if target is None:
return None
entry = (tls or {}).get(target.name)
return Job(gateway_source.SOURCE, gateway_source.DASHBOARD, target.name,
settings.gateway_interval,
lambda: fetch_gateway_snapshot(client, target,
timeout=settings.gateway_timeout, tls=entry))
def llm_jobs(settings: Any, client: httpx.AsyncClient, rates: RateEstimator) -> list[Job]:
jobs: list[Job] = []
if settings.gufo_url:
src/dashboards/config.py
+30 -1
View File
@@ -189,6 +189,23 @@ class Settings:
disks_interval: float = _num("DISKS_INTERVAL_SECONDS", 300)
disks_timeout: float = _num("DISKS_TIMEOUT_SECONDS", 30)
# --- gateway dashboard -------------------------------------------------------
# Which `HOST_TARGETS` name is the ingress. It has no URL of its own: the `gateway` job
# dials the same `/snapshot` the `host:<name>` job dials, on its own cadence and with its
# own health record (the `disks` precedent) — so "not configured" is a *name* that is not
# in the target list, and the page 404s without dialling anything (invariant 3).
# The default is the fleet's real name, so an operator who monitors a host called `gateway`
# gets the page by installing agent 1.2.0 on it; anyone else sets this or leaves the
# host_targets list without that name.
gateway_host: str = _env("GATEWAY_HOST", "gateway")
# 30 s: a connection count is neither a per-second signal nor a drive's SMART — it is the
# shape of traffic over half a minute, which is what the pipes animate. The timeout is the
# plain agent fetch's (the snapshot handler does no work for these sections beyond reading
# two configs and two /proc tables), with headroom because the same handler may be mid-
# smartctl pass for a different page.
gateway_interval: float = _num("GATEWAY_INTERVAL_SECONDS", 30)
gateway_timeout: float = _num("GATEWAY_TIMEOUT_SECONDS", 10)
# --- llm dashboard ------------------------------------------------------------
gufo_url: str = _base_url("GUFO_BASE_URL")
gufo_api_key: str = _env("GUFO_API_KEY")
@@ -229,6 +246,17 @@ class Settings:
def nut_enabled(self) -> bool:
return bool(self.nut_host)
@property
def gateway_enabled(self) -> bool:
"""Is `GATEWAY_HOST` one of the hosts this deployment dials?
The `nut_enabled` shape: a name with no target behind it is *unconfigured*, not broken,
so `/gateway` says "not configured" and `/api/gateway/*` answers 404 without a dial.
Case-sensitive on purpose — `HOST_TARGETS` names key the history DB and `parse_targets`
restricts them to lowercase, so a capitalised setting is a typo and not a match.
"""
return self.gateway_host in {target.name for target in self.host_targets}
@property
def actions_allowed(self) -> bool:
"""Coarse switch. Whether a *specific* command can run also depends on the user
@@ -239,7 +267,8 @@ class Settings:
@property
def any_source_enabled(self) -> bool:
return bool(self.host_targets) or bool(self.gufo_url or self.vllm_url) or self.nut_enabled
return (bool(self.host_targets) or bool(self.gufo_url or self.vllm_url)
or self.nut_enabled or self.gateway_enabled)
def load() -> Settings:
src/dashboards/poller.py
+2 -2
View File
@@ -41,8 +41,8 @@ class Result:
@dataclass
class SourceState:
key: str # "host:driveripper", "disks:driveripper", "llm:gufo", "nut"
dashboard: str # "system" | "disks" | "llm" | "ups"
key: str # "host:driveripper", "disks:driveripper", "gateway", "llm:gufo", "nut"
dashboard: str # "system" | "disks" | "gateway" | "llm" | "ups"
label: str
configured: bool = True
ok: bool = False
src/dashboards/sources/gateway.py
+764
View File
@@ -0,0 +1,764 @@
"""Gateway source: the connection topology of the host `GATEWAY_HOST` names.
`/system` answers "what is this machine doing". This answers the one question the router
exists for: **what is coming in through it and where does it go out**. It reads the three
phase-28 sections of one agent `/snapshot` — `haproxy` (frontends + binds, backends +
servers), `nginx` (server blocks with `listen`/`server_name`/`proxy_pass`, `map` blocks) and
`sockets` (`/proc/net/tcp{,6}`'s LISTEN + ESTABLISHED) — and joins them: the **inbound**
pipes (one per configured listen port, with the domains that port serves and the live client
addresses) and the **outbound** pipes (one per configured target, with its live count).
What this module owns, and what it deliberately does not:
* **No measurement.** Every number here is a count read at one instant: there is no
`RateEstimator`, no `window_seconds`, no `Δ<window>` label (AGENTS.md rule 2 — a count is a
count). The gauges are `gateway.in_conns@<port>` and `gateway.out_conns@<host>:<port>`,
under their own `gateway` sample family so `GET /api/metrics?source=gateway` answers what
this page has history for and a 30-second connection count never mixes with the 10-second
system ones. They roll up raw → 1 m → 1 h like every other gauge (invariant 9).
* **No verdict.** `severity.py` rates no connection count and this payload carries no
`severity` dict on one — the `net.*` precedent: 400 connections on a reverse proxy is the
job, and 0 on an idle one is healthy. Pages colour nothing here.
* **No DNS, ever.** The agent never resolves a name and neither does this module
(`tests/test_gateway_source.py` spies `socket.getaddrinfo` and pins that it is not called
once). That single fact decides the whole outbound join, below.
The join, and where its honesty lives
------------------------------------
**Inbound** is unambiguous: a socket's **local port** is the port the config listens on, and
its **remote address** is the client. A socket whose local port no configured frontend names
is counted in `unmatched.inbound` — reported, never dropped, because "the gateway is carrying
traffic on a port no config mentions" is exactly the fact the operator needs.
**Outbound** cannot be unambiguous, and pretends otherwise nowhere. The config names targets
as **domains** (`nextcloud.reeselink.com:11000`) while `/proc` reports **addresses**
(`10.1.1.164:11000`), and resolving one to the other is the one thing this stack never does.
So a socket is attributed to at most one pipe, in this order:
1. an **exact** `(address, port)` match against a target written as an IP literal
(`by_port: false`) — a real, unambiguous join;
2. a **port-only** match against a target written as a **name**, and only when that port
belongs to exactly **one** configured host — the socket fact is real, the host is the
config's, and the pipe carries `by_port: true` to say which half is which. A target written
as an address never reaches this arm: a miss on the exact match above is a different
endpoint, not this one;
3. a port shared by **several** configured targets — the case the real `stream.conf` map is
(sixteen `*.reeselink.com:443` entries). Choosing one of them would be rule 2 with a
pipe's clothes on, so nothing is chosen: the socket lands in `unmatched.outbound` and the
group is reported once in `unmatched.groups` as `{port, connections, hosts, host_count}`,
which is the honest sentence — "N connections on port P, whose configured targets are
these M names, unresolved". Those pipes carry `connections: null`, not a share and not a
repeat of the port's total (a number drawn on sixteen pipes reads as sixteen times the
traffic).
A target that names **no port** (`server x host`) is never joinable at all, so its count is
`null` even when the socket table answered: `0` there would claim a measurement nobody took.
A pipe's count is therefore `null` — `n/a`, never `0` — whenever the socket table did not
answer or the join could not decide, and `0` only when the sockets did answer and this pipe
genuinely has none.
The `$var` → `map` fan-out
--------------------------
A stream `proxy_pass $targetBackend` has no target in its own right (nginx picks one per
connection from the map), so it contributes **no literal pipe**. The map whose `variable`
matches is what carries the fan-out: its entries become `nginx.map_targets`
(`{key: the SNI domain, host, port}`) *and* one outbound pipe per entry, so the page can draw
sixteen domains leaving the gateway and label each with the domain that reaches it. A map no
`proxy_pass` references (`$connection_upgrade`, whose values are the header words
`upgrade`/`close`) contributes nothing: it is not a routing table.
Series naming
-------------
Outbound gauges are keyed `gateway.out_conns@<host>:<port>`, with the port in the name. The
phase's first sketch keyed them on the host alone, and the live gateway proves that cannot
work: its two haproxy Minecraft backends are `minecraft.reeselink.com:25565` and
`minecraft.reeselink.com:25566`, two pipes whose counts are different facts about different
sockets. One name for both is one `INSERT OR REPLACE` overwriting the other in silence —
the aggregation failure rule 2 exists to catch. The port is also what makes the name stable
when the same host later gains or loses a second service.
"""
from __future__ import annotations
import ipaddress
import time
from collections.abc import Mapping, Sequence
from typing import Any
from ..poller import Result
from ..store import Sample
#: The sample family and the dashboard name — one word on purpose, so
#: `/api/events?source=gateway` and `hub.states("gateway")` agree (the phase-03 lesson).
SOURCE = "gateway"
DASHBOARD = "gateway"
#: The three phase-28 agent sections this projection reads. Their **presence** is what
#: distinguishes "this agent knows about gateway topology" from a 1.1.0 agent or one started
#: with `--no-proxy` (both of which publish none of the three keys), and that distinction is
#: the difference between "no pipes configured" and "this host cannot answer".
SECTIONS = ("haproxy", "nginx", "sockets")
#: The agent's own error-line prefixes for those sections. `collectors.ADDON_ERROR_PREFIXES`
#: keeps the same lines out of `/system`'s `degraded`; here they are the payload's `why`,
#: which is why they are carried through verbatim rather than paraphrased.
SECTION_ERROR_PREFIXES = ("haproxy", "nginx", "sockets")
#: How many distinct client addresses one inbound pipe carries. `connections` stays the true
#: socket count; the list is what a page lists beside the pipe, and a DDoS minute must not
#: turn the payload into a client dump.
MAX_CLIENTS = 32
#: Same bound for one ambiguous port group's candidate host list (the `stream.conf` map's 16
#: entries fit; `host_count` is the real total either way).
MAX_GROUP_HOSTS = 20
#: The sentence the unsupported arm prints. Two operator facts in one line — the agent is too
#: old, or it was told not to look — because both are fixed by reinstalling the agent, one
#: with and once without `--no-proxy`.
WHY_PREDATES = ("this agent reports no proxy or socket sections — it predates gateway support "
"(1.2.0) or runs with --no-proxy")
#: The sentence the never-answered arm prints, and the reason it is **not** `WHY_PREDATES`: an
#: agent that has not replied has not said it has no sections, and "install 1.2.0" is advice to
#: give a host that may already be running it (invariant 3 — the two facts stay two sentences).
WHY_NO_ANSWER = "the agent has not answered yet — no snapshot, so no config and no sockets"
# --------------------------------------------------------------------------- #
# the small readers: nothing here interprets, everything here tolerates
# --------------------------------------------------------------------------- #
def _text(value: Any) -> str | None:
"""A non-empty string, else ``None``."""
if isinstance(value, str) and value.strip():
return value.strip()
return None
def _port(value: Any) -> int | None:
"""A port number, or ``None``.
The agent publishes `null` for a bind or server that names no port and never a `0`, so a
non-number here is drift and gets the same answer. `0` is rejected rather than treated as
a port: "some port" and "port zero" are different claims, and joining on the latter would
match nothing real while looking like a fact.
"""
if isinstance(value, bool) or not isinstance(value, int | float):
return None
number = int(value)
return number if number > 0 else None
def _is_ip_literal(value: str | None) -> bool:
"""Is this target host already an address? `ipaddress` parses text and never dials.
That is the whole reason it is used instead of a resolver: the outbound join has to know
whether a name still *needs* resolving, and a module that asks the question must not
answer it (invariant 7's "never guess" applied to a join).
"""
if not value:
return False
try:
ipaddress.ip_address(value)
except ValueError:
return False
return True
def _mappings(value: Any) -> list[Mapping[str, Any]]:
"""The list of mappings at `value`, ignoring anything that is not one."""
if not isinstance(value, Sequence) or isinstance(value, str | bytes):
return []
return [item for item in value if isinstance(item, Mapping)]
def _section(snapshot: Mapping[str, Any], key: str) -> Mapping[str, Any] | None:
"""One of the three sections, or ``None`` when the agent did not publish it."""
value = snapshot.get(key)
return value if isinstance(value, Mapping) else None
def _available(section: Mapping[str, Any] | None) -> bool | None:
"""A section's own three-way `available`, or ``None`` when there was no section at all.
`False` (haproxy/nginx: no file at that path) and `None` (it could not be read, or the
sockets tables did not answer) are both invariants-3 arms: the host is not broken, this
one source of information is not there.
"""
if section is None:
return None
flag = section.get("available")
return flag if isinstance(flag, bool) else None
# --------------------------------------------------------------------------- #
# support / degradation: the arms that must never read as red
# --------------------------------------------------------------------------- #
def supported(snapshot: Mapping[str, Any]) -> bool:
"""Does this agent publish the proxy/socket sections at all?
Presence of **any** one of the three counts, so a hypothetical agent that publishes
`sockets` but not the configs is "supported, with a half missing" (which `sections` and
`why` then name) rather than "predates support". A 1.1.0 agent and a `--no-proxy` agent
both publish none, which is the single arm this answers `False` for.
"""
return any(key in snapshot for key in SECTIONS)
def section_status(snapshot: Mapping[str, Any]) -> dict[str, bool | None]:
"""One entry per section: the agent's `available`, or `None` when it published neither."""
return {key: _available(_section(snapshot, key)) for key in SECTIONS}
def section_errors(snapshot: Mapping[str, Any]) -> list[str]:
"""The agent's own section lines out of `errors` (verbatim, the `smart_errors` idiom)."""
errors = snapshot.get("errors")
lines: list[str] = []
for item in errors if isinstance(errors, Sequence) and not isinstance(errors, str | bytes) else ():
text = str(item)
head = text.strip().lower()
if any(head.startswith(f"{prefix}:") for prefix in SECTION_ERROR_PREFIXES):
lines.append(text)
return lines
def why_lines(snapshot: Mapping[str, Any]) -> list[str]:
"""Why this page has no pipes, one sentence per section that did not answer.
The unsupported arm gets one sentence (`WHY_PREDATES`) and nothing else — there is no
section-level detail to give when the agent has no idea the feature exists. Otherwise a
line names the section and its arm, in the agent's vocabulary of arms, and never a verdict
about the host: an absent haproxy config is "this host runs no haproxy", which is
invariant 3's unconfigured case printed in grey, not a fault.
"""
if not supported(snapshot):
return [WHY_PREDATES]
lines: list[str] = []
for key in SECTIONS:
flag = _available(_section(snapshot, key))
if flag is True:
continue
if _section(snapshot, key) is None:
lines.append(f"{key}: this agent published no {key} section")
elif flag is False:
lines.append(f"{key}: no config file at the path the agent reads — this host runs "
f"no {key} (or its config lives elsewhere)")
elif key == "sockets":
lines.append("sockets: neither /proc/net/tcp nor /proc/net/tcp6 answered")
else:
lines.append(f"{key}: its config is there but could not be read or parsed")
return lines
# --------------------------------------------------------------------------- #
# the config halves: haproxy, nginx, and the sockets table
# --------------------------------------------------------------------------- #
def _names(value: Any) -> list[str]:
"""The non-empty strings of a list, in order, deduplicated — backend and server names."""
if not isinstance(value, Sequence) or isinstance(value, str | bytes):
return []
found: list[str] = []
for item in value:
text = _text(item)
if text and text not in found:
found.append(text)
return found
def haproxy_frontends(section: Mapping[str, Any] | None) -> list[dict[str, Any]]:
"""The agent's frontend rows, projected to `{name, port, backend, routes}`.
One row **per bind port** — a frontend with `bind :::80` and `bind :::443` is two pipes,
because a pipe is a port. `backend` is the frontend's own `default_backend`; `routes`
names every backend it can reach (`default_backend` plus every `use_backend` name),
because a config that routes by a fetch expression picks per request and reporting only
the default would be wrong for it. A bind with no port (a unix socket) yields no row:
there is no port to draw or join, and inventing one is rule 2.
"""
rows: list[dict[str, Any]] = []
for frontend in _mappings((section or {}).get("frontends")):
name = _text(frontend.get("name"))
default = _text(frontend.get("default_backend"))
# a frontend's `backends` is a list of **names** (`default_backend` plus every
# `use_backend`, with a fetch expression dropped by the agent because it names nothing),
# so the join downstream stays name → backend block and interprets no rule.
routes = _names(frontend.get("backends"))
if default and default not in routes:
routes.insert(0, default)
for bind in _mappings(frontend.get("binds")):
port = _port(bind.get("port"))
if port is None:
continue
rows.append({"name": name, "port": port, "backend": default, "routes": routes})
return rows
def haproxy_targets(section: Mapping[str, Any] | None) -> list[dict[str, Any]]:
"""Every `server` line's `{name, host, port}` — the outbound half of a haproxy config."""
rows: list[dict[str, Any]] = []
for backend in _mappings((section or {}).get("backends")):
for server in _mappings(backend.get("servers")):
host = _text(server.get("target"))
if not host:
continue
rows.append({"name": _text(server.get("name")), "host": host,
"port": _port(server.get("port"))})
return rows
def nginx_listen_ports(section: Mapping[str, Any] | None) -> list[tuple[int, list[str]]]:
"""`(port, server_names)` per nginx `listen`, in config order.
`server_names` is kept **verbatim**, the catch-all `_` included — it is what the config
says, and a page that wants to call it "any name" can say so without this module editing
the file's own text.
"""
rows: list[tuple[int, list[str]]] = []
for server in _mappings((section or {}).get("servers")):
names = _names(server.get("server_names"))
for listen in _mappings(server.get("listen")):
port = _port(listen.get("port"))
if port is not None:
rows.append((port, names))
return rows
def nginx_targets(section: Mapping[str, Any] | None) -> list[dict[str, Any]]:
"""Every `proxy_pass` that names an address: `{name: null, host, port}`.
A `proxy_pass $var` has `target: null` by the agent's design and contributes no row here;
`nginx_map_targets()` is where its fan-out lives.
"""
rows: list[dict[str, Any]] = []
for server in _mappings((section or {}).get("servers")):
for proxy in _mappings(server.get("proxy_passes")):
target = _target_of(proxy.get("target"))
if target is None:
continue
rows.append({"name": None, "host": target[0], "port": target[1]})
return rows
def proxy_variables(section: Mapping[str, Any] | None) -> set[str]:
"""The variable names a `proxy_pass` passes to, without their `$` (the agent drops it once)."""
names: set[str] = set()
for server in _mappings((section or {}).get("servers")):
for proxy in _mappings(server.get("proxy_passes")):
variable = _text(proxy.get("variable"))
if variable:
names.add(variable.lstrip("$"))
return names
def nginx_map_targets(section: Mapping[str, Any] | None) -> list[dict[str, Any]]:
"""The `map` entries that route: `{key, host, port}`, for maps a `proxy_pass` actually uses.
`key` is the SNI domain the map matches on, so the page can say "this domain goes here".
Two refusals, both rule 2: a map no `proxy_pass` names contributes nothing (the
`$connection_upgrade` map's values are the header words `upgrade`/`close`, and calling one
a host would be inventing a fact to fill a field), and an entry whose target is `null`
contributes nothing either.
"""
if section is None:
return []
wanted = proxy_variables(section)
rows: list[dict[str, Any]] = []
for block in _mappings(section.get("maps")):
variable = _text(block.get("variable"))
if not variable or variable.lstrip("$") not in wanted:
continue
for entry in _mappings(block.get("entries")):
key = _text(entry.get("key"))
target = _target_of(entry.get("target"))
if not key or target is None:
continue
rows.append({"key": key, "host": target[0], "port": target[1]})
return rows
def _target_of(node: Any) -> tuple[str, int | None] | None:
"""A config `{host, port}` block as `(host, port)`, or ``None`` when it is not one.
One reader for the three places a target appears (a haproxy `server`, an nginx
`proxy_pass`, a `map` entry), so "no address written" is one answer everywhere: `proxy_pass
$var` has `target: null` by the agent's design and must produce no pipe in all of them.
"""
if not isinstance(node, Mapping):
return None
host = _text(node.get("host"))
if not host:
return None
return host, _port(node.get("port"))
def socket_rows(section: Mapping[str, Any] | None) -> tuple[list[Mapping[str, Any]],
list[Mapping[str, Any]]]:
"""`(listens, established)` from the sockets section — both empty when it did not answer."""
if section is None or _available(section) is not True:
return [], []
return _mappings(section.get("listens")), _mappings(section.get("established"))
def live_listen_ports(listening: Sequence[Mapping[str, Any]]) -> set[int]:
"""The ports this host is actually serving, from the kernel's LISTEN rows.
The set decides the direction of a socket nothing matched (a port here means this host is
the one serving it, which makes the connection an inbound one the configs do not name) and
is the denominator of `unmatched.other`. Duplicated addresses of one port collapse here:
a port is what the configs speak.
"""
return {port for port in (_port(item.get("port")) for item in listening) if port is not None}
def _remote(socket: Mapping[str, Any]) -> tuple[str | None, int | None]:
remote = socket.get("remote")
if not isinstance(remote, Mapping):
return None, None
return _text(remote.get("addr")), _port(remote.get("port"))
def _local_port(socket: Mapping[str, Any]) -> int | None:
"""The port this host is serving on. The local **address** is deliberately not read: the
join is by port, and a listen address (`0.0.0.0` vs the host's own) distinguishes nothing
a frontend config names anyway.
"""
local = socket.get("local")
if not isinstance(local, Mapping):
return None
return _port(local.get("port"))
# --------------------------------------------------------------------------- #
# the join
# --------------------------------------------------------------------------- #
def _outbound_key(host: str, port: int | None) -> tuple[str, int | None]:
return (host, port)
def _outbound_pipe(host: str, port: int | None, name: str | None, source: str | None,
*, counting: bool) -> dict[str, Any]:
"""One configured target, with the live count joined onto it later.
`metric` travels in the pipe so a page asks `/api/gateway/history` for the series this
pipe *is* instead of rebuilding the name (the same reason `/disks` cards and `/llm` cells
carry their own keys). `by_port` says how the join will have to work: a target written as
an address matches exactly, a target written as a name can only match on its port, and
the payload must never let the reader confuse the two.
"""
suffix = f"{host}:{port}" if port is not None else host
return {
"target": {"host": host, "port": port, "name": name,
"by_port": port is not None and not _is_ip_literal(host)},
# a target that names no port can never be joined at all, so its count stays unknown even
# when the socket table answered — `0` there would claim a measurement nobody took.
"connections": 0 if (counting and port is not None) else None,
"source": source,
"metric": f"{SOURCE}.out_conns@{suffix}",
}
def _bump(pipe: dict[str, Any]) -> None:
pipe["connections"] = int(pipe["connections"] or 0) + 1
def join_sockets(inbound: Mapping[int, dict[str, Any]],
outbound: Mapping[tuple[str, int | None], dict[str, Any]],
established: Sequence[Mapping[str, Any]],
live_listen_ports: set[int]) -> dict[str, Any]:
"""One pass over the ESTABLISHED table, attributing each socket **at most once**.
Mutates the pipes' `connections` and `clients` and returns
`{inbound, outbound, ambiguous}`: the connections no configured frontend names, the
connections no specific configured target claims (the ambiguous ones included), and the
per-port tally of the ambiguous fan-out. Decision order and why each arm is honest are in
the module docstring; the property that matters to the page is the partition —
inbound pipes + outbound pipes + `inbound` + `outbound` == the socket table, so the counts
on screen reconcile against a direct `/proc/net/tcp` read (phase 29 task 06 checks exactly
that), and no socket is ever drawn down two pipes.
The port-only arm counts **domain** targets only, and a port whose configured hosts are
more than one is never resolved by picking: `/proc` does not know which name the socket
was dialled to and neither does anything downstream of it.
"""
hosts_by_port: dict[int, set[str]] = {}
for host, port in outbound:
if port is not None:
hosts_by_port.setdefault(port, set()).add(host)
ambiguous: dict[int, int] = {}
clients: dict[int, list[str]] = {}
unmatched_in = 0
unmatched_out = 0
for socket in established:
local_port = _local_port(socket)
remote_addr, remote_port = _remote(socket)
pipe = inbound.get(local_port) if local_port is not None else None
if pipe is not None:
_bump(pipe)
if remote_addr is not None:
seen = clients.setdefault(int(pipe["port"]), [])
if remote_addr not in seen:
seen.append(remote_addr)
continue
exact = (outbound.get(_outbound_key(remote_addr, remote_port))
if remote_addr is not None else None)
if exact is not None:
_bump(exact)
continue
candidates = hosts_by_port.get(remote_port) if remote_port is not None else None
if candidates and remote_port is not None:
if len(candidates) > 1:
ambiguous[remote_port] = ambiguous.get(remote_port, 0) + 1
unmatched_out += 1
continue
sole = candidates.copy().pop()
# a target written as an address was already tried exactly above; a miss there is
# a different endpoint, not this one, so only a *name* may be matched by port.
named = (outbound.get(_outbound_key(sole, remote_port))
if not _is_ip_literal(sole) else None)
if named is not None:
_bump(named)
continue
if local_port is not None and local_port in live_listen_ports:
# this host is *serving* the socket, on a port neither config names
unmatched_in += 1
else:
unmatched_out += 1
for port, seen in clients.items():
inbound[port]["clients"] = sorted(seen)[:MAX_CLIENTS]
return {"inbound": unmatched_in, "outbound": unmatched_out, "ambiguous": ambiguous}
def _groups(ambiguous: Mapping[int, int],
pipes: Mapping[tuple[str, int | None], dict[str, Any]]) -> list[dict[str, Any]]:
"""The ambiguous port groups, one entry per port with live connections on it."""
groups: list[dict[str, Any]] = []
for port, count in sorted(ambiguous.items()):
hosts = sorted({host for host, target in pipes if target == port})
groups.append({"port": port, "connections": count, "hosts": hosts[:MAX_GROUP_HOSTS],
"host_count": len(hosts)})
return groups
def _skeleton(host: str) -> dict[str, Any]:
"""The payload with every list empty and every count `None`.
One declaration of the key set, used by `parse` (which then fills it in) and by `stub` (which
leaves it empty), so the two arms cannot drift apart in the one way that bites a page: a key
the page indexes existing in one arm and not the other.
"""
return {
"configured": True, # the route's gate decided this; the payload says so too
"supported": None, # the caller decides: `parse` a verdict, `stub` an unknown
"host": host,
"ts": None,
"sections": dict.fromkeys(SECTIONS, None),
"why": None,
"section_errors": [],
"haproxy": {"frontends": []},
"nginx": {"servers": [], "map_targets": []},
"inbound": [],
"outbound": [],
"unmatched": {"inbound": None, "outbound": None, "other": None, "groups": []},
"sockets": {"listens": None, "established": None},
}
def stub(host: str, *, ts: float | None = None) -> dict[str, Any]:
"""A configured gateway agent that has **never answered**, in this payload's own shape.
The `/api/system` and `/api/disks` stub rule (invariants 3 and 4): the source is in the
deployment, it has no numbers yet, and `unreachable: true` is what tells those two facts
apart. Every count is `None` rather than `0` — an agent that has not spoken has not reported
an idle ingress — and `supported` is `None` rather than `False`, because nobody has read its
version. The page keeps the row, dims it, and prints `n/a`.
"""
payload = _skeleton(host)
payload["ts"] = ts
payload["unreachable"] = True
payload["why"] = WHY_NO_ANSWER
return payload
def parse(snapshot: Mapping[str, Any], host: str) -> dict[str, Any]:
"""One agent snapshot -> the `/gateway` payload. Pure: no I/O, no clock, no resolver.
The shape is the page's whole world: `haproxy`/`nginx` are the config as the agent read it
(the echo the pipes were built from), `inbound`/`outbound` are the pipes with their live
counts, `unmatched` is everything no config claimed (plus the ambiguous port groups),
`sockets` is the raw size of what the agent saw, and `supported`/`sections`/`why` are the
three degradation arms. Counts are `null` — never `0` — when nothing answered, and no
count carries a `severity`.
`sockets.established` counts **kernel socket rows**, which is what the reconciliation
identity is stated against: a loopback connection is two rows, one per end, so a gateway
proxying to a service on itself appears once inbound and once outbound — the same traffic
seen from both sides, which is exactly what the two pipes are there to show.
"""
status = section_status(snapshot)
supports = supported(snapshot)
haproxy = _section(snapshot, "haproxy")
nginx = _section(snapshot, "nginx")
sockets = _section(snapshot, "sockets")
listening, established = socket_rows(sockets)
payload = _skeleton(host)
payload["supported"] = supports
payload["ts"] = snapshot.get("ts")
payload["sections"] = status
payload["why"] = " · ".join(why_lines(snapshot)) or None
payload["section_errors"] = section_errors(snapshot)
version = _text(snapshot.get("agent_version"))
if version:
payload["agent_version"] = version
if not supports:
# The 1.1.0 / --no-proxy arm: every key the page indexes is there and empty, every
# count is null, and `why` is the one sentence. Nothing here is red — an agent that
# cannot answer this question is unconfigured for it (invariant 3), and the job's own
# health record stays `ok` because the snapshot itself arrived fine.
return payload
payload["haproxy"]["frontends"] = haproxy_frontends(haproxy)
server_names: dict[int, list[str]] = {}
for port, names in nginx_listen_ports(nginx):
merged = server_names.setdefault(port, [])
for name in names:
if name not in merged:
merged.append(name)
payload["nginx"]["servers"] = [{"port": port, "server_names": names}
for port, names in server_names.items()]
payload["nginx"]["map_targets"] = nginx_map_targets(nginx)
# --- inbound pipes, one per configured listen port -------------------------
# A port belongs to one socket space whether haproxy or nginx names it (a host cannot
# bind it twice), so the port is the key and the domains of every config that names it
# are merged onto that one pipe. `source` is the first config to name it, which is the
# honest answer to "where did this pipe come from" and nothing more.
counting = _available(sockets) is True
inbound: dict[int, dict[str, Any]] = {}
def claim_port(port: int, source: str, domains: Sequence[str]) -> None:
pipe = inbound.get(port)
if pipe is None:
inbound[port] = {"port": port, "source": source, "domains": list(domains),
"connections": 0 if counting else None,
"clients": [], "metric": f"{SOURCE}.in_conns@{port}"}
return
merged = pipe["domains"]
for domain in domains:
if domain not in merged:
merged.append(domain)
for row in payload["haproxy"]["frontends"]:
claim_port(int(row["port"]), "haproxy", ())
for port, names in server_names.items():
claim_port(port, "nginx", names)
# --- outbound pipes, one per configured target -----------------------------
outbound: dict[tuple[str, int | None], dict[str, Any]] = {}
def claim_target(host_name: str, port: int | None, name: str | None,
source: str | None) -> None:
"""One pipe per configured `host:port`, first claimer wins.
haproxy is claimed before nginx before the map, which is the order that keeps
information rather than overwriting it: a haproxy `server` line carries a name and an
nginx `proxy_pass` does not, so a duplicate never displaces a name the config has.
The socket space is one pipe either way — a second pipe for the same endpoint would
draw the same connections twice.
"""
key = _outbound_key(host_name, port)
if key not in outbound:
outbound[key] = _outbound_pipe(host_name, port, name, source, counting=counting)
for row in haproxy_targets(haproxy):
claim_target(row["host"], row["port"], row["name"], "haproxy")
for row in nginx_targets(nginx):
claim_target(row["host"], row["port"], None, "nginx")
for row in payload["nginx"]["map_targets"]:
claim_target(row["host"], row["port"], None, "map")
if not counting:
# The configs are known and the live half is not: every pipe is drawn, every count is
# null, and the tallies say nothing rather than zero (rule 2 — the sockets section
# answers "no answer", never "no connections").
payload["inbound"] = list(inbound.values())
payload["outbound"] = list(outbound.values())
return payload
# --- the joins -------------------------------------------------------------
live_ports = live_listen_ports(listening)
tallies = join_sockets(inbound, outbound, established, live_ports)
# A port whose live sockets could not be attributed is unknown for **each** of its candidate
# targets — not `0` (there are connections; we cannot say whose) and not the port's total
# repeated on every pipe (a fan-out drawing reads that as M times the traffic). A port with
# no traffic at all keeps its honest `0` on every candidate, because that is a fact.
for ambiguous_port in tallies["ambiguous"]:
for (_target_host, target_port), pipe in outbound.items():
if target_port == ambiguous_port:
pipe["connections"] = None
payload["inbound"] = list(inbound.values())
payload["outbound"] = list(outbound.values())
payload["unmatched"] = {
"inbound": tallies["inbound"],
"outbound": tallies["outbound"],
# the listener-level fact, phase 28's "an unclaimed socket stays visible as one":
# a port this host listens on that neither config names.
"other": len([port for port in live_ports if port not in inbound]),
"groups": _groups(tallies["ambiguous"], outbound),
}
payload["sockets"] = {"listens": len(listening), "established": len(established)}
return payload
def samples(payload: Mapping[str, Any], ts: float | None = None) -> list[Sample]:
"""One gauge per pipe that has a number.
A `null` count publishes **no** sample — "unknown" is not a zero, and a chart of zeros for
a dead socket table is the failure mode rule 2 names. The timestamp is the poll's (when the
dashboard read the count), carried on `ts`; nothing here is a rate, so nothing here needs a
window.
"""
when = time.time() if ts is None else ts
rows: list[Sample] = []
for pipe in payload.get("inbound") or []:
if isinstance(pipe, Mapping):
rows.extend(_gauge(payload, pipe, when))
for pipe in payload.get("outbound") or []:
if isinstance(pipe, Mapping):
rows.extend(_gauge(payload, pipe, when))
return rows
def _gauge(payload: Mapping[str, Any], pipe: Mapping[str, Any],
when: float) -> list[Sample]:
value = pipe.get("connections")
metric = pipe.get("metric")
if not isinstance(metric, str) or isinstance(value, bool) or not isinstance(value, int | float):
return []
return [Sample(SOURCE, str(payload.get("host") or ""), metric, float(value), when)]
def parse_gateway_snapshot(name: str, snapshot: Mapping[str, Any],
*, now: float | None = None) -> Result:
"""One snapshot -> (`/gateway` payload, gauges, no events).
The `sources/disks.py` shape for the poller: payload first, samples from the same
snapshot, and the timestamp the store keys on is the snapshot's when it published one.
There are **no events** here on purpose: a connection count has no transition worth an
event line (a proxy that stops routing shows up as its host's own poll failing, which the
poller already reports), so this source adds nothing to the event feed of its own accord.
"""
payload = parse(snapshot, name)
stamp = payload.get("ts")
when = (float(stamp) if isinstance(stamp, int | float) and not isinstance(stamp, bool)
else (time.time() if now is None else now))
return Result(payload=payload, samples=samples(payload, when))
src/dashboards/store.py
+1 -1
View File
@@ -4,7 +4,7 @@ Why SQLite and not Prometheus: the fleet is a handful of boxes, the poll resolut
need is 1-10s, and a single file the container owns beats running a TSDB. Series are
addressed by `(source, host, metric, res, ts)`:
source the dashboard family — "host", "llm", "ups"
source the dashboard family — "host", "disks", "llm", "ups", "gateway"
host stable target key ("driveripper", "gufo", "apc-ups1")
metric dotted name, optionally `@stream` for per-device series
(cpu.usage_pct, cpu.usage_pct@core7, disk.used_pct@/srv, gpu.busy_pct@0)
src/dashboards/web/pages/certs.html
+1
View File
@@ -25,6 +25,7 @@
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/pages/disks.html
+1
View File
@@ -18,6 +18,7 @@
<a href="/disks" aria-current="page">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/pages/gateway.html
+222
View File
@@ -0,0 +1,222 @@
<!doctype html>
<html lang="en" data-theme="dark">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Homelab · gateway</title>
<link rel="stylesheet" href="/static/dash.css">
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><rect width='100' height='100' rx='22' fill='%23ff8a3d'/></svg>">
</head>
<body>
<header class="masthead">
<a class="brand" href="/"><span class="brand-mark">◆</span> Homelab</a>
<button class="ghost nav-toggle" id="nav-toggle" type="button"
aria-expanded="false" aria-controls="nav-tabs" aria-label="Toggle navigation">☰</button>
<nav class="tabs" id="nav-tabs" aria-label="Dashboards">
<a href="/">Overview</a>
<a href="/system">System</a>
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway" aria-current="page">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
<a class="ghost account" href="/certs" title="Agent certificates">⚷ Certs</a>
<a class="ghost account" href="/settings" title="Session and password">⚙ Account</a>
<button class="ghost" id="theme-toggle" type="button" aria-pressed="false">☾ Dark</button>
<span class="clock" id="clock">--:--:--</span>
</div>
</header>
<main>
<div class="page-head">
<div>
<h1>Gateway</h1>
<div class="sub" id="sub">reading the ingress…</div>
</div>
<div class="spacer"></div>
<!-- The socket table's own totals beside the totals the pipes carry, so the page prints its
reconciliation instead of asserting it: `listeners` and `established` are what the
kernel said, and the two rows under the cards add up to them only when every connection
belonged to a configured pipe. No cell is coloured — a connection count is not a graded
number (`net.*`'s rule, and `severity.py` has no `gateway.*` row). -->
<div class="gate-tallies" id="tallies">
<div class="metric">
<div class="label">listeners</div>
<div class="value sm" id="t-listeners">n/a</div>
</div>
<div class="metric">
<div class="label">established</div>
<div class="value sm" id="t-established">n/a</div>
</div>
<div class="metric">
<div class="label">pipes in</div>
<div class="value sm" id="t-inbound">n/a</div>
</div>
<div class="metric">
<div class="label">pipes out</div>
<div class="value sm" id="t-outbound">n/a</div>
</div>
<div class="metric">
<div class="label">unmatched</div>
<div class="value sm" id="t-unmatched">n/a</div>
</div>
</div>
</div>
<!-- The payload's own sentence about why it measured nothing, printed once for the page:
`sources/gateway.py` writes it for a pre-1.2.0 agent, for `--no-proxy`, for a host that
has never answered and for a config or socket table that did not — each with the agent's
own error line when it had one. A card does not repeat it. -->
<div class="banner info" id="gate-banner" role="status" hidden></div>
<!-- The other arm, and it is not an error: `/api/gateway/snapshot` answers
`404 {configured: false}` **before** dialling anything, so a deployment with no ingress
gets a setup instruction rather than a red panel (invariant 3). It is the same sentence the
route returns, which is why it names `GATEWAY_HOST` and the agent version that reads the
configs — the two things an operator has to change. -->
<div class="empty-state" id="gate-empty" hidden>
<h3>No gateway configured</h3>
<p>Set <code>GATEWAY_HOST</code> to the <code>HOST_TARGETS</code> name of the machine that
routes this fleet, and run <code>agent/host_agent.py</code> <b>1.2.0</b> on it — that is
the version that reads the proxy configs and the kernel's socket table.</p>
</div>
<!-- One card per source family the payload names, `data-gate` keyed on a pipe's own `source`
word plus `unmatched`: a pipe is filed by what the config said it came from, not by which
card the page would rather it sat on. Every card ships hidden and appears on the poll that
has pipes for it (or a config that answered and named none), so a gateway with no nginx in
it does not spend a panel saying so. -->
<section>
<div class="page-head"><h2 style="font-size:.95rem;margin:0">Connection map</h2></div>
<div class="grid auto items-start gate-grid" id="gate-cards">
<article class="card gate-card" data-gate="haproxy" hidden>
<header>
<span class="dot idle" data-k="dot"></span>
<h3>haproxy</h3>
<span class="chip" data-k="counts"></span>
<span class="spacer"></span>
<span class="faint mono updated"></span>
</header>
<div class="body stack">
<div class="pipe-columns">
<div class="pipe-col in" data-k="in"></div>
<div class="pipe-node" data-k="node" hidden></div>
<div class="pipe-col out" data-k="out"></div>
</div>
<div class="pipe-empty faint" data-k="empty" hidden></div>
<details class="charts-box" data-k="charts">
<summary>Charts<span class="chip charts-window" data-k="charts-window"></span></summary>
<div class="host-charts">
<div class="host-chart">
<div class="chart sm" data-chart="conns"></div>
<div class="legend" data-legend="conns"></div>
</div>
</div>
</details>
</div>
</article>
<article class="card gate-card" data-gate="nginx" hidden>
<header>
<span class="dot idle" data-k="dot"></span>
<h3>nginx</h3>
<span class="chip" data-k="counts"></span>
<span class="spacer"></span>
<span class="faint mono updated"></span>
</header>
<div class="body stack">
<div class="pipe-columns">
<div class="pipe-col in" data-k="in"></div>
<div class="pipe-node" data-k="node" hidden></div>
<div class="pipe-col out" data-k="out"></div>
</div>
<div class="pipe-empty faint" data-k="empty" hidden></div>
<details class="charts-box" data-k="charts">
<summary>Charts<span class="chip charts-window" data-k="charts-window"></span></summary>
<div class="host-charts">
<div class="host-chart">
<div class="chart sm" data-chart="conns"></div>
<div class="legend" data-legend="conns"></div>
</div>
</div>
</details>
</div>
</article>
<!-- The stream map's pipes are `map`-sourced while the listener they are reached through is
an `nginx` one, so this card is the fan-out alone. Joining the `443` listener into it
would be the page inventing a relationship the agent never reported. -->
<article class="card gate-card" data-gate="map" hidden>
<header>
<span class="dot idle" data-k="dot"></span>
<h3>stream map</h3>
<span class="chip" data-k="counts"></span>
<span class="spacer"></span>
<span class="faint mono updated"></span>
</header>
<div class="body stack">
<div class="pipe-columns">
<div class="pipe-col in" data-k="in"></div>
<div class="pipe-node" data-k="node" hidden></div>
<div class="pipe-col out" data-k="out"></div>
</div>
<div class="pipe-empty faint" data-k="empty" hidden></div>
<details class="charts-box" data-k="charts">
<summary>Charts<span class="chip charts-window" data-k="charts-window"></span></summary>
<div class="host-charts">
<div class="host-chart">
<div class="chart sm" data-chart="conns"></div>
<div class="legend" data-legend="conns"></div>
</div>
</div>
</details>
</div>
</article>
<!-- Not pipes: there is no config to draw these from, only the socket table's leftovers.
`listeners no config names` counts **ports** and the other two count **connections**,
which is why they are three cells and never one sum. -->
<article class="card gate-card" data-gate="unmatched" hidden>
<header>
<span class="dot idle" data-k="dot"></span>
<h3>unmatched</h3>
<span class="spacer"></span>
<span class="faint mono updated"></span>
</header>
<div class="body stack">
<div class="metrics">
<div class="metric">
<div class="label">listeners no config names</div>
<div class="value sm" data-k="um-other">n/a</div>
</div>
<div class="metric">
<div class="label">connections inbound</div>
<div class="value sm" data-k="um-in">n/a</div>
</div>
<div class="metric">
<div class="label">connections outbound</div>
<div class="value sm" data-k="um-out">n/a</div>
</div>
</div>
<ul class="gate-groups" data-k="groups" hidden></ul>
</div>
</article>
</div>
</section>
<footer class="pagefoot">
<span class="faint">a pipe animates while it carries at least one live connection and rests
at zero, and is never coloured by a verdict — a busy ingress is the job.</span>
<span class="faint">a <code>by port</code> pipe was matched on its socket port
alone: the target is a name, the agent resolves nothing, and that port carries more than one
configured host — so the count is the port's, not this target's.</span>
</footer>
</main>
<script src="/static/charts.js"></script>
<script src="/static/app.js"></script>
<script src="/static/page-gateway.js"></script>
</body>
</html>
src/dashboards/web/pages/index.html
+5
View File
@@ -18,6 +18,7 @@
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
@@ -67,6 +68,10 @@
<p>Run <code>agent/host_agent.py</code> on each machine and list it in
<code>HOST_TARGETS</code>.</p>
</div>
<!-- The ingress's one line, filled by page-index.js from `/api/summary`'s `gateway`
block and hidden while that source is unconfigured: a gateway nobody installed gets
no row here and no red one to learn to ignore (invariant 3). -->
<p class="faint mono" id="fleet-gate" style="margin:.5rem 0 0;font-size:.8rem" hidden></p>
</div>
</section>
</div>
src/dashboards/web/pages/llm.html
+1
View File
@@ -18,6 +18,7 @@
<a href="/disks">Disks</a>
<a href="/llm" aria-current="page">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/pages/settings.html
+1
View File
@@ -23,6 +23,7 @@
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/pages/system.html
+1
View File
@@ -18,6 +18,7 @@
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/pages/ups.html
+1
View File
@@ -18,6 +18,7 @@
<a href="/disks">Disks</a>
<a href="/llm">LLM</a>
<a href="/ups" aria-current="page">UPS</a>
<a href="/gateway">Gateway</a>
</nav>
<div class="masthead-right">
<div id="nav-status" class="gap" aria-live="polite"></div>
src/dashboards/web/static/app.js
+4
View File
@@ -658,6 +658,10 @@ const Dash = (() => {
{ key: 'disks', label: 'disks', href: '/disks' },
{ key: 'llm', label: 'engines', href: '/llm' },
{ key: 'ups', label: 'UPS', href: '/ups' },
// The fifth dashboard's job is a single job for the one ingress, so this badge is one
// dot rather than one per host — and it stays grey when `GATEWAY_HOST` names nothing,
// because `refreshStatus` only counts sources the health record calls `configured`.
{ key: 'gateway', label: 'gateway', href: '/gateway' },
];
nav.innerHTML = jobs.map((job) => `<span class="chip" title="${job.label}">
<span class="dot idle" data-dot="${job.key}"></span>${job.label}</span>`).join('');
src/dashboards/web/static/charts.js
+9 -1
View File
@@ -124,7 +124,15 @@ class LineChart {
}
}
if (!isFinite(lo)) return [0, 1];
if (lo === hi) { const pad = Math.abs(lo) * 0.1 || 1; return [lo - pad, hi + pad]; }
if (lo === hi) {
// A flat series still needs a band to draw in, but a declared floor is not negotiable:
// padding below `yMin: 0` put "-1 connections" on the axis of a gateway whose every pipe was
// idle, which is a scale describing a state that cannot happen (the same reason the page
// prints `n/a` instead of a made-up zero).
const pad = Math.abs(lo) * 0.1 || 1;
const floor = this.opts.yMin !== null ? Math.min(lo, this.opts.yMin) : lo - pad;
return [floor, hi + pad];
}
const margin = (hi - lo) * 0.12;
lo = this.opts.yMin !== null ? this.opts.yMin : lo - margin;
hi = this.opts.yMax !== null ? this.opts.yMax : hi + margin;
src/dashboards/web/static/dash.css
+121 -12
View File
@@ -284,20 +284,26 @@ button.ghost:disabled { opacity: .45; cursor: not-allowed; }
padding-top: calc(.5rem + env(safe-area-inset-top, 0px));
}
/* The same treatment one level down, and these three rules are still what a 360px phone
* actually requires — phase 18 measured both sides of them: without them the four status
* dots fell into a single-file column (`#nav-status` carries the shared `.gap` class, which
* is `flex-wrap: wrap`) and `⚷ Certs` broke over two lines, because every item shrank to a
* fraction of its content. So the cluster may wrap, its items may neither shrink nor break a
* label, and the dots stay the one line that makes them a readout instead of a stack. Row 3
* still wraps *within itself*: `⚷ Certs`, `⚙ Account` and the theme button do not fit in
* 320px once over on the narrowest phones, and nothing here hides a control to make
* the arithmetic work — a phone gets the same words every other width does. The row
* alignment is the base rule's `flex-end` (phase 25 task 02), restated nowhere: this block
* only tightens the gap, so the two rows cannot disagree about which edge they sit on. */
/* The same treatment one level down, and these rules are still what a 360px phone actually
* requires — phase 18 measured both sides of them: the status dots fell into a single-file
* column and `⚷ Certs` broke over two lines, because every item shrank to a fraction of its
* content. So the cluster may wrap, its items may neither shrink nor break a label, and the
* dots keep their labels. Row 3 still wraps *within itself*: `⚷ Certs`, `⚙ Account` and the
* theme button do not fit in 320px once over on the narrowest phones, and nothing here hides a
* control to make the arithmetic work — a phone gets the same words every other width does.
* The row alignment is the base rule's `flex-end` (phase 25 task 02), restated nowhere: this
* block only tightens the gap, so the two rows cannot disagree about which edge they sit on.
*
* The dots wrap too, since phase 29 task 05 gave them a fifth member. Phase 18's `nowrap` was
* right for four (measured then at 267px) and wrong for five: `hosts disks engines UPS
* gateway` is 352px of chips against this bar's 320px content box, and an item that may not
* shrink and may not wrap does not become smaller — it widens the *document* (372px, so a
* phone got a sideways scroll, which phase 18 exists to forbid). Wrapping inside a row that is
* `flex-basis: 100%` is not the single-file column phase 18 saw either: the fifth chip moves to
* a second line of the same right-aligned row and the bar grows by one line box. */
.masthead-right { flex-wrap: wrap; gap: .4rem; }
.masthead-right > * { flex: none; white-space: nowrap; }
#nav-status { flex-wrap: nowrap; }
#nav-status { flex-wrap: wrap; row-gap: .25rem; }
/* The reveal, through task 01's token: `#nav-toggle` reads its `display` from
* `--nav-toggle-display`, so declaring the token on the element here is the whole flip —
@@ -1253,6 +1259,102 @@ input[type="text"].confirm {
* and it is the one cell a table cannot nowrap: an address list. */
table.data td.san { white-space: normal; }
/* ---- gateway pipes ---------------------------------------------------------
* `/gateway` (phase 29 task 04): the connection map. One card per source family, and inside each
* card two columns of pipes joined by one node — the gateway itself. A pipe is one `<path>` with
* a travelling dash pattern, so the whole vocabulary here is ink and motion: a connection count
* is not a graded number, which is why this section draws no coloured verdict on any of it — the
* `net.*` precedent, and the API sends no verdict for a gateway figure at any depth.
*
* Three layout facts are load-bearing, all of them measured in `data/logs/29-task04-pipes.cjs`:
* the node is one rounded rect that does **not** stretch to the taller column (a 17-target
* fan-out would otherwise grow a 600 px wall beside a six-pipe side); an empty column collapses
* and the node moves to the edge it is joined to (`solo`); and at a phone width the three columns
* stack with the node between them, because 320 px of card has no room for a pipe *and* its
* label, and an unlabelled pipe is a line nobody can read.
*/
.gate-tallies { display: flex; flex-wrap: wrap; gap: .9rem; align-items: flex-end; }
/* "listeners no config names" is a fact, and `.metric .label` truncates with an ellipsis at a card
width that cannot fit it — a clipped label is a clipped fact (phase 27's rule), so these wrap.
The five header tallies keep the shared truncating rule: their labels are one word each. */
.gate-card .metric .label { white-space: normal; }
.gate-grid { min-width: 0; }
.pipe-columns {
display: grid; gap: .5rem 1rem; min-width: 0; align-items: center;
grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr);
}
/* A card with pipes on one side only. The stream map's fan-out is the case that ships: its
* inbound half is the nginx `443` listener and lives on the nginx card, because the payload never
* says which listener a map is reached through and the page does not guess one. Two classes, not
* one, because *which* side is empty decides which grid track the node belongs in. */
.pipe-columns.solo-out { grid-template-columns: auto minmax(0, 1fr); }
.pipe-columns.solo-in { grid-template-columns: minmax(0, 1fr) auto; }
.pipe-col { display: grid; gap: .5rem; min-width: 0; align-content: center; }
.pipe-node {
align-self: center; justify-self: center; text-align: center;
min-width: 6rem; max-width: 9rem; padding: .6rem .45rem;
border: 1px solid var(--border-strong); border-radius: var(--radius);
background: var(--bg-sunken); color: var(--text-muted);
font: 600 11.5px var(--mono); letter-spacing: .04em; overflow-wrap: anywhere;
}
.pipe-row { display: grid; gap: .12rem; min-width: 0; }
.pipe-label {
display: flex; flex-wrap: wrap; align-items: baseline; gap: .05rem .35rem; min-width: 0;
}
/* A pipe's label is the page's whole answer — which port, which domain, which target — and this row
is what makes the answer readable. Task 06 measured it on live data: as one `nowrap` flex line the
name shrank to whatever the `by port` chip, the config's own `server` name and the `flex: none`
count left, which drew seven characters of `reeseapps-gitea.reeselink.com:2222`; wrapping the text
inside that squeezed box was worse (eleven lines, a word at a time). So the name claims the line:
`max-content` as its base size means a name longer than the column is alone on its line and gets
the *whole* column (~22 characters), with the chip, the footnote and the count wrapping beneath it,
while a short label like `:22` still sits beside its count on one line exactly as before. What is
still clipped after that is a target longer than the column, which is a column-width fact
(AGENTS.md gap 13) and not a lost one: `renderPipeRow` puts the full string on the element as
`title`, `pipeAria` carries the whole endpoint, and
`test_a_clipped_pipe_label_keeps_its_full_text_on_the_element` pins both declarations and both
strings. The live pass re-counts the clipped labels on every run. */
.pipe-name {
flex: 1 1 max-content; font-size: 11.5px; min-width: 0;
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
}
.pipe-detail { font-size: 11px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
/* the count sits at the node end of the label line, so the eye goes from the number straight
into the pipe that carries it */
.pipe-count {
margin-inline-start: auto; padding-inline-start: .5rem; flex: none;
font: 600 12.5px var(--mono);
}
/* The pipe is a grid row of its own, the full width of its column — which is what makes it touch
* the node, since the inbound column is the node's left and the outbound column its right. */
.pipe { display: block; width: 100%; min-width: 0; height: 10px; }
.pipe-path { fill: none; stroke: var(--border-strong); stroke-width: 2; stroke-dasharray: 7 9; }
/* One connection is enough to move. The dash travels the pipe's own direction, which is
* *towards* the node in the inbound column and *away from* it in the outbound one — the node is
* the far end of one and the near end of the other, so one keyframe is the whole direction
* story and a reversed second one would only be a second thing to keep in step. */
.pipe.live .pipe-path { stroke: var(--accent); animation: pipe-flow 1.2s linear infinite; }
/* Zero is a static, faint line rather than an absent one: "this pipe exists and carries nothing"
* is a fact the drawing has to be able to say, and it is a different sentence from `n/a`. */
.pipe.still .pipe-path { stroke: var(--text-faint); stroke-dasharray: 3 7; opacity: .55; }
@keyframes pipe-flow {
/* one full dash+gap cycle (7 + 9), so the loop is seamless */
from { stroke-dashoffset: 0; }
to { stroke-dashoffset: -16; }
}
.pipe-clients { font-size: 10.5px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.pipe-empty { font-size: 12px; }
.gate-groups { list-style: none; margin: 0; padding: 0; display: grid; gap: .3rem; }
.gate-groups li { display: flex; gap: .45rem; align-items: baseline; min-width: 0; }
.gate-groups .pipe-name { flex: none; }
@media (max-width: 640px) {
.pipe-columns,
.pipe-columns.solo-out,
.pipe-columns.solo-in { grid-template-columns: minmax(0, 1fr); }
.pipe-node { max-width: none; width: 100%; }
}
/* ---- dialog ------------------------------------------------------------- */
dialog {
border: 1px solid var(--border-strong); border-radius: var(--radius); background: var(--bg-raised);
@@ -1313,4 +1415,11 @@ footer.pagefoot {
nav.tabs,
nav.tabs.is-open { transition: none; }
/* `/gateway`'s pipes, **both states**. The animated rule out-specifies the base one exactly
* the way `.is-open` out-specifies `nav.tabs`, so stilling the static state alone would leave
* a live pipe flowing for someone who asked for no motion — the same phase-46 lesson the menu
* pair above carries, applied the day a second animated component landed in this sheet. */
.pipe.live .pipe-path,
.pipe.still .pipe-path { animation: none; }
}
src/dashboards/web/static/page-gateway.js
+646
View File
@@ -0,0 +1,646 @@
/* Gateway page: what this ingress is connected to, and how much is on right now.
`/system` answers what the gateway box is doing. This answers the topology question it cannot:
which sockets face inbound, where each one's traffic goes, and how many connections are live on
each. The source is `/api/gateway/snapshot` — `sources/gateway.py`'s join of the agent's
`haproxy` and `nginx` config parses with its `/proc/net/tcp{,6}` tallies.
The page draws **pipes**, and a pipe's only claim is a count. It animates while that count is at
least one connection and rests at zero; it is never coloured by a verdict (`net.*`'s precedent:
a busy ingress is the job, and `severity.py` rates no `gateway.*` metric). `n/a` means the join
did not decide — a socket table that did not answer, or a port whose configured targets cannot be
told apart — and `n/a` never becomes `0` here (rule 2).
Two rules keep the drawing honest:
* **The label is the config's own text.** A bind keeps its port, a `server_name` keeps its
domain, a `proxy_pass` keeps the address it was written with, and a target the join matched
on its port alone says `by port` rather than pretending the socket named the host.
* **The unreconciled remainder is printed, not hidden.** `unmatched` is the connections no
configured pipe claims, and it is the operator's fact that the gateway talks to something
the configs do not name.
Charts follow the `/system` discipline: one collapsed `<details>` per card, painted only while
open, fed by **one** `/api/gateway/history` request per card per refresh naming every gauge that
card could draw.
*/
'use strict';
const { $, $$, fetchJSON, setTxt, setDot, num, duration, poll, markStale } = Dash;
const SVG_NS = 'http://www.w3.org/2000/svg';
const state = {
/* No window chips: the page's history is the count trend, and one window for every card is
* the honest default for a source that re-reads every 30 s. */
range: 86400,
rangeKey: '24h',
every: 30, // GATEWAY_INTERVAL_SECONDS, from /api/config — never guessed here
cards: new Map(), // card key -> its element (the skeletons the page ships)
rows: new Map(), // card key -> Map(pipe key -> row element): patched, not rebuilt
charts: new Map(), // card key -> LineChart, built on first expand only
history: new Map(), // card key -> { range, response } from the last snapshot pass
snapshot: null, // last payload, so a toggle paints without fetching
poller: null,
/* Half the job's interval: an age stamp cannot sit at "updated 29s" for a whole cycle, and
* nothing on the page claims a fresher reading than the agent took. */
pollMs: 15000,
};
// ---- the cards --------------------------------------------------------------
//
// One card per source family, keyed on the payload's own `source` word plus `unmatched`. That
// keying is not a layout preference: `source` is the only fact in the document that says which
// config a pipe came from, and the stream fan-out's pipes are `map`-sourced while the listener
// they are reached through is an `nginx` one — so the map card is outbound by construction, and
// joining the two would be the page inventing a relationship the agent never reported.
const CARDS = [
{ key: 'haproxy', title: 'haproxy', section: 'haproxy',
empty: 'haproxy names no listening port on this host' },
{ key: 'nginx', title: 'nginx', section: 'nginx',
empty: 'nginx names no listening port on this host' },
{ key: 'map', title: 'the stream map', section: 'nginx',
empty: 'the stream map routes to nothing' },
];
const CHART_HEIGHT = 120;
const COUNT_FORMAT = (v) => num(v, 0);
const CLIENTS_SHOWN = 2;
/** The measured count as text; the unknown is `null` so `setTxt` prints `n/a` and not `0`. */
function countText(value) {
return typeof value === 'number' && isFinite(value) ? num(value, 0) : null;
}
/** One pipe family's pipes, split on the payload's own `source` word. */
function familyPipes(snapshot, key) {
return {
inbound: (snapshot.inbound || []).filter((pipe) => pipe.source === key),
outbound: (snapshot.outbound || []).filter((pipe) => pipe.source === key),
};
}
/** `host:port` -> the stream-map keys that route to it, out of `nginx.map_targets`. */
function mapKeys(snapshot) {
const keys = new Map();
((snapshot.nginx || {}).map_targets || []).forEach((entry) => {
if (!entry || !entry.key) return;
const name = `${entry.host}:${entry.port}`;
const list = keys.get(name) || [];
list.push(entry.key);
keys.set(name, list);
});
return keys;
}
/**
* The label a pipe carries.
*
* Inbound is a port, because that is what the config binds. Outbound is the address the config
* was written with, and for a stream-map target the SNI name the map matched on rides in front of
* it — both of those strings are the document's own, and nothing here resolves or reverses one.
*/
function pipeName(pipe, keys) {
if (pipe.port != null) return `:${pipe.port}`;
const target = pipe.target || {};
const host = target.host || '?';
const endpoint = `${host}${target.port == null ? '' : `:${target.port}`}`;
const through = keys ? keys.get(`${target.host}:${target.port}`) : null;
return through && through.length === 1 ? `${through[0]} → ${endpoint}` : endpoint;
}
/** A pipe's stable identity: its own gauge name, unique per listen port and per target. */
function pipeKey(pipe) {
return pipe.metric || `${pipe.port}`;
}
/** The row's footnote: the haproxy `server` name, or the count of map keys on one endpoint. */
function pipeDetail(pipe, keys) {
const target = pipe.target || {};
if (target.name) return `server ${target.name}`;
const through = keys ? keys.get(`${target.host}:${target.port}`) : null;
if (through && through.length > 1) return `${through.length} map names`;
return null;
}
/** The accessible sentence for one pipe — every figure in it came from the payload. */
function pipeAria(direction, pipe, keys) {
const bits = [direction === 'in' ? 'inbound pipe' : 'outbound pipe'];
bits.push(direction === 'in' ? `port ${pipe.port}` : `to ${pipeName(pipe, keys)}`);
const domains = pipe.domains || [];
if (domains.length) bits.push(`names ${domains.join(', ')}`);
const clients = pipe.clients || [];
if (clients.length) bits.push(`clients ${clients.join(', ')}`);
bits.push(pipe.connections == null ? 'connections unknown'
: `${pipe.connections} connections`);
return bits.join(', ');
}
/** The pipe itself: hand-rolled SVG — one path, `role="img"`, an `aria-label` of its own. */
function buildPipe() {
const svg = document.createElementNS(SVG_NS, 'svg');
svg.setAttribute('class', 'pipe still');
svg.setAttribute('viewBox', '0 0 100 10');
svg.setAttribute('preserveAspectRatio', 'none');
svg.setAttribute('focusable', 'false');
const path = document.createElementNS(SVG_NS, 'path');
path.setAttribute('class', 'pipe-path');
// One geometry for both columns: the dash runs from x=0 to x=100, which is *towards* the
// gateway node in the inbound column and *away from* it in the outbound one. The node is the
// far end of one and the near end of the other, so a single keyframe is the whole direction
// story and a reversed second one would be a second thing to keep in step.
path.setAttribute('d', 'M0 5 H100');
path.setAttribute('vector-effect', 'non-scaling-stroke');
svg.appendChild(path);
return svg;
}
/**
* A pipe row: the label line (name, `by port`, the config's own name for the target, count),
* the SVG pipe under it, and the client addresses under that. Built once, patched afterwards.
*
* The pipe is the full width of its column, which is what makes it touch the node: the inbound
* column is the node's left and the outbound column its right, so one geometry pointing right
* runs *into* the gateway in one column and *away from* it in the other.
*/
function pipeRow(direction) {
const el = document.createElement('div');
el.className = `pipe-row ${direction}`;
el.innerHTML = `
<span class="pipe-label">
<span class="pipe-name mono"></span>
<span class="chip pipe-flag" hidden></span>
<span class="pipe-detail faint" hidden></span>
<span class="pipe-count"></span>
</span>
<span class="pipe-clients faint" hidden></span>`;
el.insertBefore(buildPipe(), el.querySelector('.pipe-clients'));
return el;
}
/**
* Patch one row from its pipe.
*
* Every string here is upstream-provided — a `server_name`, a `proxy_pass`, a client address —
* so it goes out through `textContent` and `title`, never through markup.
*/
function renderPipeRow(el, direction, pipe, keys) {
const name = el.querySelector('.pipe-name');
const text = pipeName(pipe, keys);
if (name.textContent !== text) name.textContent = text;
const domains = pipe.domains || [];
name.title = domains.length ? domains.join(', ') : text;
const flag = el.querySelector('.pipe-flag');
const byPort = !!(pipe.target && pipe.target.by_port);
flag.hidden = !byPort;
if (byPort) {
setTxt(flag, 'by port');
flag.title = 'matched on the port alone — this target is a name and the port carries more '
+ 'than one configured host, so the count is the port\'s, not this target\'s';
} else {
flag.removeAttribute('title');
}
// the config's own label for this target — the haproxy `server` name, or how many map names
// arrive at one endpoint
const detail = el.querySelector('.pipe-detail');
const note = pipeDetail(pipe, keys);
detail.hidden = note == null;
if (note != null) setTxt(detail, note);
const shown = countText(pipe.connections);
setTxt(el.querySelector('.pipe-count'), shown);
const svg = el.querySelector('.pipe');
const live = typeof pipe.connections === 'number' && isFinite(pipe.connections)
&& pipe.connections > 0;
svg.classList.toggle('live', live);
svg.classList.toggle('still', !live);
svg.setAttribute('role', 'img');
const aria = pipeAria(direction, pipe, keys);
if (svg.getAttribute('aria-label') !== aria) svg.setAttribute('aria-label', aria);
// The clients are the row's third line and its tooltip: the addresses are the operator's
// shortest path from "this port is busy" to "that machine is busy".
const clients = (pipe.clients || []).slice(0, CLIENTS_SHOWN);
const rest = (pipe.clients || []).length - clients.length;
const track = el.querySelector('.pipe-clients');
const shown2 = clients.length ? `${clients.join(' ')}${rest > 0 ? ` +${rest}` : ''}` : '';
track.hidden = shown2.length === 0;
setTxt(track, shown2);
}
/** Make a column's rows match `pipes`, patching the ones that stay and dropping the rest. */
function renderPipes(card, direction, key, pipes, keys) {
const box = card.querySelector(`[data-k="${direction}"]`);
const seen = state.rows.get(key) || new Map();
state.rows.set(key, seen);
const wanted = new Set();
pipes.forEach((pipe) => {
const id = `${direction}:${pipeKey(pipe)}`;
wanted.add(id);
let row = seen.get(id);
if (!row) {
row = pipeRow(direction);
seen.set(id, row);
box.appendChild(row);
}
renderPipeRow(row, direction, pipe, keys);
});
Array.from(seen.keys()).forEach((id) => {
if (wanted.has(id) || !id.startsWith(`${direction}:`)) return;
seen.get(id).remove();
seen.delete(id);
});
// An empty column collapses rather than holding half the card for nothing — which is what
// moves the node to the edge it is being joined to (the `solo` class below).
box.hidden = pipes.length === 0;
return pipes.length;
}
/** The gateway node: one rounded rect mid-card, labelled with the host the reading came from. */
function renderNode(card, snapshot, columns) {
const node = card.querySelector('[data-k="node"]');
const visible = columns.inCount + columns.outCount > 0;
node.hidden = !visible;
// One column of pipes only: the node moves to the edge it is joined to instead of floating in
// the middle of an empty half-card. Two classes, because *which* half is empty decides which
// grid track the node belongs in, and one shared class could only ever be right about one.
const frame = card.querySelector('.pipe-columns');
frame.classList.toggle('solo-out', visible && columns.inCount === 0);
frame.classList.toggle('solo-in', visible && columns.outCount === 0);
const label = snapshot.host || 'gateway';
if (node.dataset.text !== label) {
node.dataset.text = label;
node.textContent = label;
}
}
// ---- the unmatched card ------------------------------------------------------
//
// Not pipes: there is no config to draw these from, only the socket table's own leftovers. The
// two tallies are two different facts and never one sum — `other` counts *ports* no config
// claims, the other counts *connections* the join could not attribute to a target.
/** Does this payload leave anything unaccounted for? An all-zero family gets no card. */
/**
* Is there a remainder to report?
*
* A positive count or a group, yes. Three `null`s is not "unknown worth printing" — it is the join
* having measured nothing at all (a pre-1.2.0 agent, a silent socket table), and a card of three
* `n/a`s says less than the page's own `why` line above it already says (the dead-cell rule).
*/
function unmatchedVisible(snapshot) {
const un = snapshot.unmatched || {};
return [un.other, un.inbound, un.outbound].some(
(value) => typeof value === 'number' && value > 0) || (un.groups || []).length > 0;
}
/** One `<li>` of the "could belong to any of N" list — built, not templated: the names are data. */
function groupRow(group) {
const li = document.createElement('li');
const port = document.createElement('span');
port.className = 'pipe-name mono';
port.textContent = `:${group.port}`;
const why = document.createElement('span');
why.className = 'faint';
const hosts = (group.hosts || []).slice(0, CLIENTS_SHOWN);
const rest = (group.hosts || []).length - hosts.length;
why.textContent = `${countText(group.connections) || 'n/a'} connections · could be any of `
+ `${group.host_count} targets${hosts.length ? ` (${hosts.join(', ')}${rest > 0
? ` +${rest}` : ''})` : ''}`;
li.appendChild(port);
li.appendChild(why);
return li;
}
function renderUnmatched(card, snapshot, row) {
const un = snapshot.unmatched || {};
card.hidden = false;
setDot(card.querySelector('[data-k="dot"]'), dotFor(row, snapshot));
setTxt(card.querySelector('.updated'), stalenessText(row));
markStale(card, dimmed(snapshot));
setTxt(card.querySelector('[data-k="um-other"]'), countText(un.other));
setTxt(card.querySelector('[data-k="um-in"]'), countText(un.inbound));
setTxt(card.querySelector('[data-k="um-out"]'), countText(un.outbound));
const groups = un.groups || [];
const list = card.querySelector('[data-k="groups"]');
const wanted = groups.map((group) => `${group.port}|${group.connections}|${group.host_count}`);
const same = list.dataset.wanted === wanted.join(',') && list.children.length === groups.length;
list.hidden = groups.length === 0;
if (same) return;
list.dataset.wanted = wanted.join(',');
list.textContent = '';
groups.forEach((group) => list.appendChild(groupRow(group)));
}
// ---- card chrome ---------------------------------------------------------------
/** The reachability dot: the job's own health, never a number. */
function dotFor(row, snapshot) {
if (!row || !row.configured) return 'idle';
if (row.ok === false || snapshot.unreachable) return 'fail';
return Dash.ageSeconds(row.last_ok) > 3 * state.every ? 'warn' : 'ok';
}
function stalenessText(row) {
if (!row || !row.configured) return 'not configured';
if (row.last_ok == null) return row.error || 'never reached';
const seconds = Dash.ageSeconds(row.last_ok);
return seconds > state.every ? `stale ${duration(seconds)}`
: `updated ${seconds < 2 ? 'now' : duration(seconds)}`;
}
/** Did this payload fail to measure at all? Every arm of it dims; none of it reddens. */
function dimmed(snapshot) {
return !!snapshot.unreachable || snapshot.supported !== true;
}
/** Every card skeleton, wired once to the markup the page ships. */
function cardElements() {
const cards = new Map();
$$('#gate-cards [data-gate]').forEach((el) => {
cards.set(el.dataset.gate, el);
const box = el.querySelector('[data-k="charts"]');
if (box) box.addEventListener('toggle', () => chartsToggled(el.dataset.gate));
const chip = el.querySelector('[data-k="charts-window"]');
if (chip) chip.textContent = state.rangeKey;
});
return cards;
}
/**
* Paint one card from the snapshot.
*
* A card appears when it has a fact: pipes of its own. A family whose config section did not
* answer has none, and the page's own `why` line is the one sentence that says which section —
* so the card stays off the grid instead of repeating it in a smaller font.
*/
function renderCard(card, snapshot, row) {
const key = card.dataset.gate;
if (key === 'unmatched') {
if (unmatchedVisible(snapshot)) renderUnmatched(card, snapshot, row);
else card.hidden = true;
return;
}
const definition = CARDS.find((entry) => entry.key === key);
const { inbound, outbound } = familyPipes(snapshot, key);
if (inbound.length + outbound.length === 0) {
// Two different nothings. The section answered and named no pipe: that is a fact about this
// gateway's config, and the card says it. The section did not answer at all: the page's own
// `why` line is the one sentence that says which section, so no card repeats it.
const answered = (snapshot.sections || {})[definition.section] === true;
card.hidden = !answered;
if (answered) setTxt(card.querySelector('[data-k="empty"]'), definition.empty);
return;
}
card.hidden = false;
card.querySelector('[data-k="empty"]').hidden = true;
const keys = mapKeys(snapshot);
setDot(card.querySelector('[data-k="dot"]'), dotFor(row, snapshot));
setTxt(card.querySelector('.updated'), stalenessText(row));
markStale(card, dimmed(snapshot));
setTxt(card.querySelector('[data-k="counts"]'),
`${inbound.length} in · ${outbound.length} out`);
const columns = {
inCount: renderPipes(card, 'in', key, inbound, keys),
outCount: renderPipes(card, 'out', key, outbound, keys),
};
renderNode(card, snapshot, columns);
}
// ---- per-card charts ------------------------------------------------------------
//
// One plot per card, one request per card, every refresh: the query names every gauge that card
// could draw (`/api/gateway/history` answers an unknown name as an empty series on purpose) and
// the legend names **pipes**, not hosts — the card heading already says which config family this
// is, and the page is about one machine.
/** Every gauge this card's pipes publish. */
function cardStreams(snapshot, key) {
const keys = mapKeys(snapshot);
const { inbound, outbound } = familyPipes(snapshot, key);
return inbound.concat(outbound)
.filter((pipe) => !!pipe.metric)
.map((pipe) => ({ metric: pipe.metric, label: pipeName(pipe, keys) }));
}
/** The chart inside one card, built the first time that card's stack is open. */
function cardChart(key) {
const built = state.charts.get(key);
if (built) return built;
const card = state.cards.get(key);
if (!card) return null;
const definition = CARDS.find((entry) => entry.key === key);
const chart = new DashCharts.LineChart(card.querySelector('[data-chart="conns"]'), {
height: CHART_HEIGHT, xSpan: state.range, yMin: 0, yFormat: COUNT_FORMAT,
ariaLabel: `${(state.snapshot || {}).host || 'gateway'} — live connections on each `
+ `${definition.title} pipe over the last ${state.rangeKey}`,
});
state.charts.set(key, chart);
return chart;
}
function seriesFor(response, definitions) {
return definitions.map((definition) => {
const found = ((response || {}).series || []).find((s) => s.metric === definition.metric);
return { name: definition.label,
points: ((found || {}).points || []).map((p) => ({ ts: p.ts, value: p.value })) };
});
}
function chartsOpen(key) {
const card = state.cards.get(key);
const box = card && card.querySelector('[data-k="charts"]');
return !!(box && box.open);
}
/** Opening paints from the response this card already has, so the first expand needs no poll. */
function chartsToggled(key) {
if (!chartsOpen(key) || !state.snapshot) return;
const cached = state.history.get(key);
if (cached) drawCardCharts(key, cached);
else renderCardCharts(key, state.snapshot);
}
/** The paint pass, and the only place a `LineChart` here is built or drawn. */
function drawCardCharts(key, response) {
if (!chartsOpen(key) || !state.snapshot) return;
const card = state.cards.get(key);
if (!card) return;
const definitions = cardStreams(state.snapshot, key);
if (!definitions.length) return;
const chart = cardChart(key);
if (!chart) return;
const series = seriesFor(response, definitions);
chart.opts.xSpan = state.range;
chart.opts.area = false;
chart.setData(series);
// The legend names the lines the plot actually drew. A pipe whose join published no sample has
// no line, and a permanent `n/a` row for it is the dead-cell anti-pattern the vLLM slot grid
// avoids; `LineChart`'s own "no samples yet" says the same thing once when none of them did.
// `setData` stamps every series with the colour its path uses, so a swatch still matches its
// line after the filter instead of sliding to the next one in the palette.
const plotted = chart.series.filter((s) => s.points && s.points.length);
DashCharts.renderLegend(card.querySelector('[data-legend="conns"]'), plotted, COUNT_FORMAT,
false);
}
/**
* One card's history: one request, named by that card's own pipes.
*
* The request is per card per refresh whether or not the disclosure is open — the `/system`
* shape, whose cached response is what lets the first expand paint from data in hand instead of
* waiting a poll — while the *plot* is built only while it is open (`drawCardCharts`).
*/
async function renderCardCharts(key, snapshot) {
const definitions = cardStreams(snapshot, key);
if (!definitions.length) return;
const range = state.range;
const query = `/api/gateway/history?hosts=${encodeURIComponent(snapshot.host || '')}`
+ `&metrics=${encodeURIComponent(definitions.map((d) => d.metric).join(','))}`
+ `&range=${range}`;
const response = await fetchJSON(query).catch(() => null);
if (!response || !state.cards.has(key)) return; // the card went away mid-flight
state.history.set(key, response);
drawCardCharts(key, response);
}
// ---- the page header -------------------------------------------------------------
/**
* What the pipes on one side carry: the sum of the ones that answered, plus how many did not.
*
* This is the page's only arithmetic and it is a sum of published gauges — adding two counts is
* neither a rate nor a verdict. A pipe whose join could not decide contributes nothing and is
* counted in `unknown`, which is `/api/summary`'s `partial` flag drawn rather than a second
* opinion: the sockets behind it are already inside the `unmatched` column beside it, so
* `attributed + unmatched` stays the kernel's number and no figure on the row is invented. The
* marker is the whole point — a total that quietly left unknowns out reads as the whole number.
*/
function sumConnections(snapshot, side) {
const pipes = snapshot[side] || [];
if (!pipes.length) return null;
let attributed = 0;
let unknown = 0;
for (const pipe of pipes) {
if (typeof pipe.connections === 'number' && isFinite(pipe.connections)) {
attributed += pipe.connections;
} else {
unknown += 1;
}
}
// Nothing answered is not "nothing carrying": with every pipe unknown the side has no number at
// all, and `0` here would be a claim about traffic the join never measured (rule 2).
return unknown === pipes.length ? null : { attributed, unknown, total: pipes.length };
}
/**
* One reconciliation cell: its number, the shared `.stale` dim while it is a partial sum, and the
* tooltip naming what is missing — `/llm`'s fleet-block idiom for a total that left something out.
*/
function setTally(id, sum) {
const el = $(id);
setTxt(el, sum == null ? null : sum.attributed);
const metric = el.closest('.metric');
if (metric) metric.classList.toggle('stale', !!sum && sum.unknown > 0);
el.title = sum == null ? 'the join did not measure this side'
: `${sum.attributed} of ${sum.total} pipes answered${sum.unknown
? `; ${sum.unknown} could not be told apart, and their connections are under unmatched`
: ''}`;
}
function renderHeader(snapshot) {
const sockets = snapshot.sockets || {};
const un = snapshot.unmatched || {};
const unmatched = un.inbound == null && un.outbound == null
? null : (un.inbound || 0) + (un.outbound || 0);
setTxt($('#t-listeners'), countText(sockets.listens));
setTxt($('#t-established'), countText(sockets.established));
setTally('#t-inbound', sumConnections(snapshot, 'inbound'));
setTally('#t-outbound', sumConnections(snapshot, 'outbound'));
setTxt($('#t-unmatched'), countText(unmatched));
markStale($('#tallies'), dimmed(snapshot));
$('#sub').textContent = `${snapshot.host || 'gateway'} · agent `
+ `${snapshot.agent_version || 'n/a'} · `
+ `${(snapshot.inbound || []).length} inbound pipes · `
+ `${(snapshot.outbound || []).length} outbound`;
const banner = $('#gate-banner');
const why = snapshot.why || null;
banner.hidden = why == null;
if (why != null) setTxt(banner, why);
}
// ---- snapshot ----------------------------------------------------------------------
async function refresh() {
let snapshot;
try {
snapshot = await fetchJSON('/api/gateway/snapshot');
} catch (err) {
// Two different refusals. `404 {configured: false}` is the unconfigured arm, answered before
// anything was dialled, and the route's own sentence names the variable to set — so the setup
// block appears. Anything else is a failed read, and invariant 4's rule holds: the last good
// drawing stays on screen with the reason up top, rather than blanking the page.
$('#sub').textContent = err.message;
$('#gate-empty').hidden = err.status !== 404;
return;
}
$('#gate-empty').hidden = true;
state.snapshot = snapshot;
const states = {};
(snapshot.states || []).forEach((row) => { states[row.dashboard || row.key] = row; });
const row = states.gateway;
renderHeader(snapshot);
CARDS.forEach((definition) => renderCard(state.cards.get(definition.key), snapshot, row));
renderCard(state.cards.get('unmatched'), snapshot, row);
const visible = CARDS.filter((definition) => !state.cards.get(definition.key).hidden)
.map((definition) => definition.key);
pruneCards(visible);
await Promise.all(visible.map((key) => renderCardCharts(key, snapshot)));
}
/** Clear the row and chart state of every family that left the payload. */
function pruneCards(visible) {
CARDS.forEach((definition) => {
if (visible.includes(definition.key)) return;
const card = state.cards.get(definition.key);
if (card) {
card.hidden = true;
const rows = state.rows.get(definition.key);
if (rows) rows.clear();
}
state.rows.delete(definition.key);
state.history.delete(definition.key);
// The `LineChart` instance stays: a card is hidden rather than removed, so its plot is
// built once and repainted whenever the family comes back.
});
}
document.addEventListener('DOMContentLoaded', async () => {
state.cards = cardElements();
try {
const config = await fetchJSON('/api/config');
const gateway = config.gateway || {};
if (Number.isFinite(gateway.interval_seconds) && gateway.interval_seconds > 0) {
state.every = gateway.interval_seconds;
}
if (gateway.enabled === false) {
// The unconfigured arm before the first request: the setup block, no cards, and no poll
// loop dialling a route that answers 404 forever (invariant 3).
$('#sub').textContent = gateway.host
? `no host named ${gateway.host} in HOST_TARGETS — set GATEWAY_HOST`
: 'no gateway configured — set GATEWAY_HOST to a HOST_TARGETS name';
$('#gate-empty').hidden = false;
return;
}
} catch (err) { /* the defaults are the shipped ones */ }
state.poller = poll(async () => { await refresh(); }, state.pollMs);
Dash.bindShortcuts();
});
src/dashboards/web/static/page-index.js
+44
View File
@@ -96,6 +96,49 @@ function renderAlerts(alerts) {
}).join('');
}
/* The two figures the overview's ingress line prints, in the order it prints them. Each `key` is
* a key of `/api/summary`'s `gateway` block, and `tests/test_api.py` resolves this table against
* the payload the route actually serves (the `SLOT_FIELDS` device): a rename on either side fails
* there rather than printing `n/a` beside a busy ingress. */
const GATEWAY_FIELDS = [
{ key: 'inbound', word: 'inbound' },
{ key: 'outbound', word: 'outbound' },
];
/**
* The ingress's one line under the fleet table: the pipe totals `/api/summary` summed, linked to
* the page that draws them.
*
* Nothing here is counted in the browser and nothing here is a route. The link is the block's own
* `href` (`alerts.py`'s table is where a dashboard's page is written, the same rule the alert rows
* keep), the counts are that block's sums of the pipes, and `n/a` is what a `null` prints: an
* agent that has not answered, one that predates the proxy sections, and a socket table that did
* not reply all measured **nothing**, which is not the sentence "0 connections" (rule 2).
* `configured: false` renders no line at all — the `disks` block's invariant-3 rule: an ingress
* nobody installed gets no row here and no red one to learn to ignore.
*/
function renderGateway(gateway) {
const row = $('#fleet-gate');
const block = gateway || {};
if (block.configured !== true) {
row.hidden = true;
row.textContent = '';
return;
}
row.hidden = false;
// the shared dim for "these are the last good numbers" (invariant 4)
row.classList.toggle('stale', block.unreachable === true);
const why = block.unreachable === true ? `${block.host} is not answering`
: block.partial ? 'a pipe could not be told from another on its port, so the total leaves it out'
: '';
const counts = GATEWAY_FIELDS
.map(({ key, word }) => `${num(block[key], 0)} ${word}`).join(' · ');
row.innerHTML = `<a href="${DashCharts.escapeHtml(String(block.href || ''))}">`
+ `${DashCharts.escapeHtml(String(block.host || ''))}</a> `
+ `${DashCharts.escapeHtml(counts)}`;
if (why) row.title = why;
}
/**
* The fleet table: one row per host, columns unchanged.
*
@@ -161,6 +204,7 @@ async function refresh() {
`${live}/${total} hosts · ${engines} · ${ups} · ${problems}`;
renderAlerts(alerts);
renderFleet(hosts, system.states);
renderGateway(summary.gateway);
} catch (err) {
$('#summary-line').textContent = `summary failed: ${err.message}`;
}
tests/conftest.py
+70
View File
@@ -243,6 +243,76 @@ def disk_snapshot(agent_snapshots: dict[str, dict[str, Any]], smart_shape: Any)
return build
# ---- phase 29: the gateway page (the /gateway source, API and page) -------- #
#: The captured gateway tree, as the paths the agent's own `--root` resolution expects.
#: These are the verbatim phase-28 captures (`scripts/capture-gateway-config.sh`), so the
#: config half of a `/gateway` test is the file the real ingress runs, not a memory of it.
GATEWAY_TREE = {
"etc/haproxy/haproxy.cfg": "gateway_haproxy.cfg",
"etc/nginx/nginx.conf": "gateway_nginx_main.conf",
"etc/nginx/conf.d/http.conf": "gateway_nginx_conf_d_http.conf",
"etc/nginx/stream.d/stream.conf": "gateway_nginx_stream_d_stream.conf",
"proc/net/tcp": "gateway_proc_net_tcp.txt",
"proc/net/tcp6": "gateway_proc_net_tcp6.txt",
}
GATEWAY_MODES = ("full", "predates", "no_haproxy", "no_nginx", "sockets_silent")
@pytest.fixture
def gateway_snapshot(host_agent_module: Any, tmp_path: Path) -> Any:
"""`build(mode="full")` -> a 1.2.0-style agent snapshot carrying the three proxy sections.
Not a hand-written dict: the captured configs and `/proc` tables are laid into a fake
`--root` tree and read by the **real agent `Collector`**, so the sections handed to
`sources/gateway.py` are the bytes agent 1.2.0 serves. A field rename on either side of the
wire therefore fails a source test instead of drawing an empty pipe on the page (rule 7).
Modes are the degradation arms the page has to render:
* `full` — all three sections `available: true`, live sockets from the capture
* `predates` — none of the three keys at all (an agent older than 1.2.0, or `--no-proxy`)
* `no_haproxy` / `no_nginx` — that one config file is absent: `available: false` plus the
agent's own **not found** line, the other half of the page untouched
* `sockets_silent` — neither `/proc` table readable: `available: null`, so every live count
must come back `None` and never `0`
"""
def build(mode: str = "full") -> dict[str, Any]:
if mode not in GATEWAY_MODES:
raise AssertionError(f"unknown gateway snapshot mode {mode!r}")
root = tmp_path / f"root-{mode}"
for relative, fixture in GATEWAY_TREE.items():
if not fixture:
continue
target = root / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text((FIXTURES / fixture).read_text())
if mode == "no_haproxy":
(root / "etc/haproxy/haproxy.cfg").unlink()
if mode == "no_nginx":
for path in (root / "etc/nginx/nginx.conf", root / "etc/nginx/conf.d/http.conf",
root / "etc/nginx/stream.d/stream.conf"):
path.unlink()
if mode == "sockets_silent":
(root / "proc/net/tcp").unlink()
(root / "proc/net/tcp6").unlink()
errors: list[str] = []
collector = host_agent_module.Collector(root)
snapshot: dict[str, Any] = {
"ok": True, "schema": 1, "ts": 1_760_000_000.0, "host": "gateway",
"agent_version": host_agent_module.AGENT_VERSION, "errors": errors,
}
if mode != "predates":
snapshot["haproxy"] = collector.haproxy(errors)
snapshot["nginx"] = collector.nginx(errors)
snapshot["sockets"] = collector.sockets(errors)
return snapshot
return build
@pytest.fixture
def metrics_text() -> Any:
"""metrics_text('gufo') / metrics_text('vllm') -> the captured exposition verbatim."""
tests/test_alerts.py
+79 -1
View File
@@ -14,9 +14,10 @@ are the ones that make the landing page trustworthy rather than merely busy:
from __future__ import annotations
import inspect
from typing import Any
from dashboards import alerts
from dashboards import alerts, severity
from dashboards.config import Settings
from dashboards.sources import nut
from dashboards.sources.nut_sim import NutSimulator
@@ -481,6 +482,83 @@ def test_a_dead_upsd_is_one_alert_and_not_its_stale_cards_too() -> None:
assert [item["source"] for item in out] == ["NUT server"]
# --------------------------------------------------------------------------- #
# the gateway page: reachability is the only voice, and a dead agent is still one line
# --------------------------------------------------------------------------- #
def gateway_state(**kwargs: Any) -> dict[str, Any]:
"""The health record of the one `gateway` job (the `nut` shape: key `gateway`, label = host)."""
return state("gateway", "gateway", "gateway", **kwargs)
def test_a_stale_gateway_job_is_a_warn_measured_in_its_own_thirty_second_poll() -> None:
"""The pipes are drawn from a 30 s poll, so 60 s of silence is two healthy polls.
Judging this source by the 10 s system interval would amber the landing page every minute for
a job behaving exactly as configured — the same mistake the `disks` rule exists to avoid, one
cadence later.
"""
settings = Settings(gateway_interval=30.0)
assert alerts.build(settings, [gateway_state(age=60.0)]) == []
out = alerts.build(settings, [gateway_state(age=100.0)])
assert out[0]["severity"] == "warn"
assert out[0]["href"] == "/gateway", "the overview's row must link to the page it describes"
assert "every 30s" in out[0]["message"], "the message prints the gateway's own interval"
def test_a_gateway_job_that_fails_while_its_host_answers_is_the_only_voice_it_has() -> None:
"""A wedged proxy/socket section is a fault nothing else can report (the `disks` rule, once more)."""
out = alerts.build(SETTINGS, [
gateway_state(ok=False, error="UpstreamError: gateway: timed out"),
state("host:gateway", "system", "gateway"),
], now=2_000.0)
assert len(out) == 1
assert out[0]["source"] == "gateway"
assert out[0]["href"] == "/gateway"
assert "timed out" in out[0]["message"]
def test_a_gateway_job_that_failed_because_its_agent_is_down_is_one_line_not_two() -> None:
"""Three jobs dial one agent; the operator has one problem, so the overview says it once."""
down = {"ok": False,
"error": "UpstreamError: gateway: http://h:9120/snapshot unreachable (ConnectError)"}
out = alerts.build(SETTINGS, [
gateway_state(**down),
state("disks:gateway", "disks", "gateway", **down),
state("host:gateway", "system", "gateway", **down),
], now=2_000.0)
assert len(out) == 1, "the echo of one outage must not read as three problems"
assert out[0]["id"].startswith("gateway:host:"), out[0]["id"]
def test_an_unconfigured_gateway_source_says_nothing_even_when_its_job_says_down() -> None:
"""No `HOST_TARGETS` entry named `GATEWAY_HOST` means no job — and a reported one stays quiet."""
out = alerts.build(Settings(gateway_interval=30.0),
[gateway_state(ok=False, configured=False, error="not configured")],
now=2_000.0)
assert out == []
def test_a_gateway_payload_with_a_fabricated_verdict_alerts_nothing() -> None:
"""The phase's no-verdict rule, asserted as a rule and not only as an absence.
Three ways, because deleting a rule satisfies an absence-only assertion:
* `alerts.build` takes no gateway payload at all — its signature is the whole surface, so a
gateway count can reach an alert only through a threshold added to this module;
* `severity.THRESHOLDS` has no `gateway.*` row, so no such threshold could be written without
editing the one file that owns thresholds (and a busy proxy is the job, the `net.*` case);
* the module's `HREF` does carry `/gateway`, so reachability still reaches the overview.
"""
assert not {"gateway", "gateway_payload"} & set(inspect.signature(alerts.build).parameters)
assert not [name for name in severity.THRESHOLDS if name.startswith("gateway.")], (
"a connection count is not a graded number")
assert alerts.HREF["gateway"] == "/gateway"
assert alerts.AGENT_ECHO_DASHBOARDS == ("disks", "gateway")
# and an answering gateway with nothing else going on is the all clear
assert alerts.build(Settings(gateway_interval=30.0), [gateway_state()], [], [], None, [],
now=2_000.0) == []
# --------------------------------------------------------------------------- #
# configuration
# --------------------------------------------------------------------------- #
tests/test_api.py
+1406 -49
View File
File diff suppressed because it is too large Load Diff
tests/test_auth_api.py
+13 -4
View File
@@ -56,7 +56,12 @@ CURL_HEADERS = {"accept": "*/*", "user-agent": "curl/8.5.0"}
PROTECTED_API = (
"/api/config", "/api/events", "/api/metrics", "/api/history", "/api/summary",
"/api/system/snapshot", "/api/system/history", "/api/disks/snapshot",
"/api/disks/history", "/api/llm/snapshot", "/api/llm/history", "/api/ups/snapshot",
"/api/disks/history",
# Phase 29: the ingress's pipes. `…/snapshot` carries the gateway's whole routing map — every
# domain it serves and every address it dials — so it is listed here for the same reason the
# bundle download is, whatever the payload happens to hold today.
"/api/gateway/snapshot", "/api/gateway/history",
"/api/llm/snapshot", "/api/llm/history", "/api/ups/snapshot",
"/api/ups/history", "/api/ups/commands", "/api/auth/status",
# Phase 15: the certificate authority's own surface. `/api/certs/<name>/bundle` is every
# agent's private key, so "it needs a session" is the whole point of listing it here.
@@ -68,7 +73,11 @@ PROTECTED_API = (
#: thing that must never regress is a new route answering anonymously. `/certs` is the sharper
#: case of the two — an anonymous caller reaching it would see the CA's serials and every host
#: the dashboard dials.
PROTECTED_PAGES = ("/", "/system", "/disks", "/llm", "/ups", "/settings", "/certs")
#: `/gateway` (phase 29 task 04) is the sixth dashboard: it prints this fleet's ingress map —
#: every listening port, every `server_name`, every internal target — so it is gated by being a
#: page and nothing else.
PROTECTED_PAGES = ("/", "/system", "/disks", "/llm", "/ups", "/gateway", "/settings",
"/certs")
#: The one write this app has, with the fields the route needs (the UPS is unconfigured, so a
@@ -1598,8 +1607,8 @@ def test_the_served_route_table_is_the_surface_the_gate_describes(anon: TestClie
pages = sorted(str(route.path) for route in served_routes(anon)
if isinstance(getattr(route, "path", None), str)
and not str(route.path).startswith(("/api", "/static")))
assert pages == ["/", "/certs", "/disks", "/llm", "/login", "/settings", "/system",
"/ups"]
assert pages == ["/", "/certs", "/disks", "/gateway", "/llm", "/login", "/settings",
"/system", "/ups"]
for path in pages:
response = anon.get(path, headers=BROWSER_HEADERS)
if path == "/login":
tests/test_collectors.py
+77 -1
View File
@@ -28,11 +28,13 @@ from dashboards.collectors import (
disks_jobs,
fetch_disk_snapshot,
fetch_engine,
fetch_gateway_snapshot,
fetch_host_snapshot,
fetch_snapshot,
gateway_job,
host_jobs,
)
from dashboards.config import Target
from dashboards.config import Settings, Target
from dashboards.poller import Result
from dashboards.rates import RateEstimator
from dashboards.sources import nut
@@ -409,6 +411,80 @@ async def test_smart_addon_errors_reach_the_disk_payload_verbatim(
assert "degraded" not in system.payload, "and /system stays unbothered by it"
# --------------------------------------------------------------------------- #
# gateway topology (the /gateway source's fetch half, phase 29)
# --------------------------------------------------------------------------- #
async def test_the_gateway_fetch_reads_the_same_agent_snapshot_as_the_other_two(
gateway_snapshot: Any) -> None:
"""One endpoint, three readers: the pipes come out of `/snapshot` and nothing else.
The gateway job has no URL of its own (`GATEWAY_HOST` is a *name*), so this is the pin that
keeps it that way — and that the projection's own honesty rules survive the trip through the
collector: gauges under the `gateway` family, **no** events (a count has no transition worth a
line), and no `window_seconds` for a number that was never diffed (rule 2).
"""
body = gateway_snapshot("full")
seen: list[str] = []
def handler(request: httpx.Request) -> httpx.Response:
assert request.url.path == "/snapshot", "the gateway page has no endpoint of its own"
seen.append(request.url.path)
return httpx.Response(200, json=body)
target = Target("gateway", "http://agent.test")
async with client_for(handler) as client:
result = await fetch_gateway_snapshot(client, target, timeout=10)
assert seen == ["/snapshot"]
assert result.payload["host"] == "gateway"
assert result.payload["supported"] is True
assert result.payload["inbound"], "the configured pipes arrive with the snapshot"
assert result.samples and all(sample.source == "gateway" for sample in result.samples), (
"the gauges are their own sample family, not `host`'s")
assert result.events == [], "a connection count has no transition to report"
assert "window_seconds" not in result.payload, "a count is not a rate"
async def test_the_gateway_poll_carries_its_own_timeout_and_never_disables_one() -> None:
"""The third budget on one endpoint: the client default until the job asks for its own.
Same rule the `disks` poll was pinned with — a literal `None` means *no timeout at all* to
httpx, which would hang the only poller this source has (invariant 4) — and the reason
`GATEWAY_TIMEOUT_SECONDS` is a setting rather than a constant inherited from `system`.
"""
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(200, json={"ok": True, "ts": 1.0, "errors": []})
client = TimeoutRecordingClient(transport=httpx.MockTransport(handler),
timeout=4.0, base_url="http://agent.test")
target = Target("gateway", "http://agent.test")
async with client:
await fetch_gateway_snapshot(client, target)
assert client.timeouts[-1] is httpx.USE_CLIENT_DEFAULT
await fetch_gateway_snapshot(client, target, timeout=10)
assert client.timeouts[-1] == 10, "the gateway poll waits on its own budget"
assert all(timeout is not None for timeout in client.timeouts)
async def test_the_gateway_job_is_one_job_named_after_the_host_it_reads() -> None:
"""`key`/`dashboard`/`label`/`interval` at the factory, and `None` for a name nobody dials.
The label is what an alert and an event print and what `alerts.py` uses to find the `host:` job
reading the same agent, so it is the host's name and not the source's — while the key stays the
single word `gateway` because there is exactly one such job per deployment.
"""
settings = Settings(host_targets=(Target("gateway", "https://gw.test"),),
gateway_interval=45.0, gateway_timeout=7.0)
entry = AgentTls(verify="/ca.pem", cert=("/c.pem", "/k.pem"), status=lambda: certs.ACTIVE)
async with httpx.AsyncClient() as client:
job = gateway_job(settings, client, tls={"gateway": entry})
assert job is not None
assert (job.key, job.dashboard, job.label) == ("gateway", "gateway", "gateway")
assert job.interval == 45.0
assert job.enabled is True, "a job that exists is a job that runs"
missing = Settings(host_targets=(Target("deskwork", "http://desk.test"),))
assert gateway_job(missing, httpx.AsyncClient()) is None, "unconfigured is no task at all"
# --------------------------------------------------------------------------- #
# TLS: the dashboard dials a protected exporter with its own client certificate
# --------------------------------------------------------------------------- #
tests/test_config.py
+48
View File
@@ -87,6 +87,54 @@ def test_settings_default_to_no_sources_configured() -> None:
assert settings.any_source_enabled is False
def test_the_gateway_source_names_the_ingress_host_and_its_own_cadence() -> None:
"""Phase 29: one `HOST_TARGETS` name is the gateway, and it is polled on its own clock.
`gateway_host` defaults to the fleet's real name so installing agent 1.2.0 on a host called
`gateway` is the whole configuration — but the **default is not a target**: with no
`HOST_TARGETS` entry of that name the source is unconfigured, `/api/gateway/*` 404s without
dialling, and the page says so (invariant 3). The interval sits between the system poll and
SMART because a connection count is the shape of half a minute, not of ten seconds.
"""
settings = Settings()
assert settings.gateway_host == "gateway"
assert settings.gateway_interval == 30.0
assert settings.gateway_timeout == 10.0
assert settings.gateway_enabled is False
assert settings.any_source_enabled is False
def test_gateway_enabled_is_true_only_for_a_name_that_is_a_target() -> None:
"""A name with no agent behind it is unconfigured, not broken — the `nut_enabled` shape."""
def with_targets(raw: str, name: str = "gateway") -> bool:
targets, _problems = parse_targets(raw)
return Settings(host_targets=tuple(targets), gateway_host=name).gateway_enabled
assert with_targets("gateway=https://gw.example.net:9120") is True
assert with_targets(
"deskwork=https://d.example.net:9120 gateway=https://gw.example.net:9120") is True
assert with_targets("deskwork=https://d.example.net:9120") is False
assert with_targets("router=https://gw.example.net:9120", name="router") is True
assert with_targets("Gateway=https://gw.example.net:9120") is False, ("names are lowercase")
assert with_targets("") is False
def test_the_gateway_settings_come_from_the_environment(monkeypatch: pytest.MonkeyPatch) -> None:
"""`GATEWAY_HOST=router` is how a deployment whose ingress is called something else opts in."""
monkeypatch.setenv("GATEWAY_HOST", "router")
monkeypatch.setenv("GATEWAY_INTERVAL_SECONDS", "15")
monkeypatch.setenv("GATEWAY_TIMEOUT_SECONDS", "20")
reloaded = importlib.reload(config)
try:
assert reloaded.SETTINGS.gateway_host == "router"
assert reloaded.SETTINGS.gateway_interval == 15.0
assert reloaded.SETTINGS.gateway_timeout == 20.0
finally:
for name in ("GATEWAY_HOST", "GATEWAY_INTERVAL_SECONDS", "GATEWAY_TIMEOUT_SECONDS"):
monkeypatch.delenv(name)
importlib.reload(config)
def test_the_disk_source_defaults_to_a_five_minute_cadence() -> None:
"""SMART is not a per-second signal (phase 09), and the fetch waits for the agent's
own 20 s smartctl budget. Both are env-only, like every other interval."""
tests/test_gateway_source.py
+718
View File
@@ -0,0 +1,718 @@
"""`sources/gateway.py`: the config + socket join that becomes the `/gateway` pipes.
The shape half of these tests runs the **captured** gateway configs and `/proc` tables through
the real agent `Collector` (the `gateway_snapshot` fixture), so a rename of a section key on
either side of the wire fails here rather than drawing an empty pipe on a live page. The
arithmetic half uses synthetic sections, because the point of those tests is which socket lands
in which pipe and what the source refuses to decide — facts a capture states once and a
re-capture may lawfully change (the same rule `tests/test_agent.py` holds for the socket tables).
"""
from __future__ import annotations
import json
import socket as socket_module
from collections.abc import Mapping, Sequence
from typing import Any
from dashboards.sources import gateway
# --------------------------------------------------------------------------- #
# synthetic builders: a section exactly as the agent serves it, minus the file
# --------------------------------------------------------------------------- #
def bind(port: int | None, addr: str = "::") -> dict[str, Any]:
return {"addr": addr, "port": port}
def haproxy_section(frontends: Sequence[dict[str, Any]],
backends: Sequence[dict[str, Any]] = (), *,
available: bool | None = True) -> dict[str, Any]:
return {"available": available, "config": "/etc/haproxy/haproxy.cfg",
"frontends": list(frontends), "backends": list(backends)}
def frontend(name: str, ports: Sequence[int | None], *, default: str | None = None,
use: Sequence[str] = ()) -> dict[str, Any]:
return {"name": name, "binds": [bind(port) for port in ports],
"default_backend": default, "backends": list(use)}
def backend(name: str, servers: Sequence[dict[str, Any]]) -> dict[str, Any]:
return {"name": name, "servers": list(servers)}
def server(name: str, target: str | None, port: int | None = None) -> dict[str, Any]:
return {"name": name, "target": target, "port": port}
def nginx_section(servers: Sequence[dict[str, Any]] = (), maps: Sequence[dict[str, Any]] = (), *,
available: bool | None = True) -> dict[str, Any]:
return {"available": available, "config": "/etc/nginx/nginx.conf", "files": [],
"servers": list(servers), "upstreams": [], "maps": list(maps)}
def nginx_server(ports: Sequence[int | None], names: Sequence[str] = (),
passes: Sequence[dict[str, Any]] = ()) -> dict[str, Any]:
return {"listen": [bind(port, addr="") for port in ports], "server_names": list(names),
"proxy_passes": list(passes)}
def proxy_pass(target: tuple[str, int] | None, *, variable: str | None = None,
location: str | None = "/") -> dict[str, Any]:
host, port = target if target else (None, None)
return {"location": location, "scheme": "http" if target else None,
"target": None if host is None else {"host": host, "port": port},
"variable": variable}
def mapping_block(variable: str,
entries: Sequence[tuple[str, tuple[str, int | None] | None]]) -> dict[str, Any]:
return {"variable": variable,
"entries": [{"key": key,
"target": None if target is None else {"host": target[0],
"port": target[1]}}
for key, target in entries]}
def sockets_section(listens: Sequence[dict[str, Any]] = (),
established: Sequence[dict[str, Any]] = (), *,
available: bool | None = True) -> dict[str, Any]:
return {"available": available, "listens": list(listens), "established": list(established)}
def listen(port: int, addr: str = "0.0.0.0", family: str = "v4") -> dict[str, Any]:
return {"family": family, "addr": addr, "port": port}
def conn(local: tuple[str, int], remote: tuple[str, int], *,
family: str = "v4") -> dict[str, Any]:
return {"family": family, "local": {"addr": local[0], "port": local[1]},
"remote": {"addr": remote[0], "port": remote[1]}}
def snapshot(*, haproxy: dict[str, Any] | None = None, nginx: dict[str, Any] | None = None,
sockets: dict[str, Any] | None = None, errors: Sequence[str] = (),
agent_version: str | None = "1.2.0") -> dict[str, Any]:
"""The document one agent `/snapshot` would answer, with only what a test names."""
doc: dict[str, Any] = {"ok": True, "schema": 1, "ts": 1_760_000_000.0, "host": "gateway",
"errors": list(errors)}
if agent_version is not None:
doc["agent_version"] = agent_version
if haproxy is not None:
doc["haproxy"] = haproxy
if nginx is not None:
doc["nginx"] = nginx
if sockets is not None:
doc["sockets"] = sockets
return doc
def pipe_by_port(payload: Mapping[str, Any], port: int) -> dict[str, Any]:
for pipe in payload["inbound"]:
if pipe["port"] == port:
return pipe
raise AssertionError(f"no inbound pipe on port {port}: {[p['port'] for p in payload['inbound']]}")
def pipe_to(payload: Mapping[str, Any], host: str, port: int | None) -> dict[str, Any]:
for pipe in payload["outbound"]:
if pipe["target"]["host"] == host and pipe["target"]["port"] == port:
return pipe
raise AssertionError(f"no outbound pipe to {host}:{port}")
def walk_keys(node: Any) -> set[str]:
"""Every key name anywhere in a payload — the device the "no severity / no window" pins use."""
if isinstance(node, Mapping):
found = set(node.keys())
for value in node.values():
found |= walk_keys(value)
return found
if isinstance(node, Sequence) and not isinstance(node, str | bytes):
return set().union(*(walk_keys(item) for item in node)) if node else set()
return set()
# --------------------------------------------------------------------------- #
# the captured gateway: every configured pipe, from the real agent code path
# --------------------------------------------------------------------------- #
def test_the_captured_configs_yield_every_configured_pipe(gateway_snapshot: Any) -> None:
"""Six haproxy frontends, four nginx listen ports, the stream map's fan-out — all present."""
payload = gateway.parse(gateway_snapshot("full"), "gateway")
assert payload["configured"] is True
assert payload["supported"] is True
assert payload["why"] is None, "a gateway whose every section answered gets no why-line"
assert payload["sections"] == {"haproxy": True, "nginx": True, "sockets": True}
haproxy_ports = [row["port"] for row in payload["haproxy"]["frontends"]]
assert haproxy_ports == [22, 2000, 5201, 8080, 25565, 25566], "config order, one row per bind"
assert {row["backend"] for row in payload["haproxy"]["frontends"]} == {
"gitea-2222", "sec-gitea-2000", "iperf3-5201", "unifi-external-8080",
"testcraft-25565", "nimcraft-25566"}
assert [row["port"] for row in payload["nginx"]["servers"]] == [80, 443, 4443, 8443]
names = {row["port"]: row["server_names"] for row in payload["nginx"]["servers"]}
assert names[80] == ["_"], "the catch-all is the config's own text, kept verbatim"
assert names[443] == [], "the stream server routes by SNI, so it names no domain of its own"
assert "nextcloud.reeseapps.com" in names[4443] and "gitea.reeseapps.com" in names[4443], (
"five vhosts share 4443 and their domains belong to the one pipe for that port")
assert payload["nginx"]["map_targets"], "the stream map is the fan-out the page draws"
assert len(payload["nginx"]["map_targets"]) == 17
homepage = next(row for row in payload["nginx"]["map_targets"]
if row["key"] == "homepage.reeseapps.com")
assert homepage == {"key": "homepage.reeseapps.com",
"host": "reeseapps-homepage.reeselink.com", "port": 443}
def test_the_source_reads_the_keys_the_agent_actually_publishes(gateway_snapshot: Any,
host_agent_module: Any) -> None:
"""The coupling is pinned: `binds`/`port`, `servers`/`target`, `listen`/`server_names`.
The fixture runs the real parser, so this compares the source's output against the agent's
own document rather than against a copy of its field names — which is what makes a rename
fail here instead of printing `n/a` beside a healthy gateway (AGENTS.md rule 7).
"""
snap = gateway_snapshot("full")
payload = gateway.parse(snap, "gateway")
agent_bind_ports = {bind["port"] for frontend in snap["haproxy"]["frontends"]
for bind in frontend["binds"]}
assert {row["port"] for row in payload["haproxy"]["frontends"]} == agent_bind_ports
agent_servers = {(row["name"], row["target"], row["port"])
for block in snap["haproxy"]["backends"] for row in block["servers"]}
assert {(pipe["target"]["name"], pipe["target"]["host"], pipe["target"]["port"])
for pipe in payload["outbound"] if pipe["source"] == "haproxy"} == agent_servers
agent_listen_ports = {listen["port"] for server in snap["nginx"]["servers"]
for listen in server["listen"]}
assert {row["port"] for row in payload["nginx"]["servers"]} == agent_listen_ports
# and the three section names themselves, as the snapshot publishes them
assert {"haproxy", "nginx", "sockets"} <= set(snap)
assert host_agent_module.AGENT_VERSION == "1.2.0", "the sections these keys belong to"
def test_the_live_counts_reconcile_with_the_socket_table(gateway_snapshot: Any) -> None:
"""Every ESTABLISHED row is counted exactly once — in a pipe, or in `unmatched`.
This is the property phase 29 task 06 checks against a direct `/proc/net/tcp` read; pinning
it on the capture means the arithmetic is already right before anything is dialled.
"""
snap = gateway_snapshot("full")
payload = gateway.parse(snap, "gateway")
established = len(snap["sockets"]["established"])
in_pipes = sum(pipe["connections"] for pipe in payload["inbound"])
out_pipes = sum(pipe["connections"] or 0 for pipe in payload["outbound"])
unmatched = payload["unmatched"]
assert payload["sockets"] == {"listens": len(snap["sockets"]["listens"]),
"established": established}
assert in_pipes + out_pipes + unmatched["inbound"] + unmatched["outbound"] == established
# the ambiguous fan-out is inside `unmatched.outbound`, not on top of it
assert sum(group["connections"] for group in unmatched["groups"]) <= unmatched["outbound"]
def test_a_listening_port_no_config_claims_stays_visible(gateway_snapshot: Any) -> None:
"""Phase 28's "an unclaimed socket stays visible as one", counted, not silently dropped."""
snap = gateway_snapshot("full")
payload = gateway.parse(snap, "gateway")
live = {row["port"] for row in snap["sockets"]["listens"]}
configured = {pipe["port"] for pipe in payload["inbound"]}
assert payload["unmatched"]["other"] == len(live - configured)
# the agent's own port is dialled by nobody in either config, so it is always the example
assert 9120 in live - configured
def test_the_map_fanout_becomes_pipes_labelled_by_domain(gateway_snapshot: Any) -> None:
"""Every map entry is a pipe the page can draw, and `$var` never becomes a host."""
payload = gateway.parse(gateway_snapshot("full"), "gateway")
map_pipes = [pipe for pipe in payload["outbound"] if pipe["source"] == "map"]
assert len(map_pipes) == len(payload["nginx"]["map_targets"])
assert not [pipe for pipe in payload["outbound"]
if pipe["target"]["host"] is None or "$" in str(pipe["target"]["host"])], (
"a proxy_pass $var carries no address, so it must not become a pipe of its own")
brain = pipe_to(payload, "reeseapps-brain-of-reese.reeselink.com", 443)
assert brain["target"]["by_port"] is True, "a name can only ever be matched on its port"
# --------------------------------------------------------------------------- #
# the inbound join
# --------------------------------------------------------------------------- #
def test_a_socket_lands_in_the_frontend_that_owns_its_local_port() -> None:
snap = snapshot(
haproxy=haproxy_section([frontend("ssh", [22], default="ssh-be")]),
nginx=nginx_section([nginx_server([443], ["a.example.test"])]),
sockets=sockets_section(
listens=[listen(22), listen(443)],
established=[conn(("10.1.1.27", 443), ("203.0.113.7", 51000)),
conn(("10.1.1.27", 443), ("203.0.113.9", 51001)),
conn(("10.1.1.27", 22), ("198.51.100.4", 33000))]))
payload = gateway.parse(snap, "gateway")
https = pipe_by_port(payload, 443)
assert https["source"] == "nginx"
assert https["connections"] == 2
assert https["clients"] == ["203.0.113.7", "203.0.113.9"], "the client is the remote end"
assert https["domains"] == ["a.example.test"]
assert https["metric"] == "gateway.in_conns@443"
assert pipe_by_port(payload, 22)["source"] == "haproxy"
assert payload["unmatched"] == {"inbound": 0, "outbound": 0, "other": 0, "groups": []}
def test_a_frontend_that_is_idle_is_still_a_pipe_with_zero() -> None:
"""A configured port with no connections is `0`, not `n/a` — the socket table did answer."""
snap = snapshot(haproxy=haproxy_section([frontend("ssh", [22])]),
sockets=sockets_section(listens=[listen(22)]))
payload = gateway.parse(snap, "gateway")
assert pipe_by_port(payload, 22)["connections"] == 0
def test_a_connection_on_a_port_no_config_names_is_reported_not_dropped() -> None:
"""`unmatched.inbound`: this host is serving something neither file mentions."""
snap = snapshot(
nginx=nginx_section([nginx_server([443], ["a.example.test"])]),
sockets=sockets_section(
listens=[listen(443), listen(8081)],
established=[conn(("10.1.1.27", 8081), ("10.1.0.9", 41000)),
conn(("10.1.1.27", 8081), ("10.1.0.9", 41001))]))
payload = gateway.parse(snap, "gateway")
assert pipe_by_port(payload, 443)["connections"] == 0
assert payload["unmatched"]["inbound"] == 2, "the page's 'N other connections'"
assert payload["unmatched"]["other"] == 1, "8081 is a listener no frontend claims"
def test_one_port_named_by_both_configs_is_one_pipe_carrying_both_domains() -> None:
"""A host cannot bind a port twice, so the socket space is one pipe and the facts merge."""
snap = snapshot(
haproxy=haproxy_section([frontend("http", [80])]),
nginx=nginx_section([nginx_server([80], ["a.example.test"])]),
sockets=sockets_section(listens=[listen(80)],
established=[conn(("10.1.1.27", 80), ("203.0.113.1", 1234))]))
payload = gateway.parse(snap, "gateway")
ports = [pipe["port"] for pipe in payload["inbound"]]
assert ports == [80], "no second pipe, and no socket counted down both"
assert pipe_by_port(payload, 80)["connections"] == 1
assert pipe_by_port(payload, 80)["domains"] == ["a.example.test"]
def test_the_client_list_is_unique_sorted_and_bounded() -> None:
ports = [80]
many = [conn(("10.1.1.27", 80), (f"203.0.113.{i % 256}", 40000 + i))
for i in range(gateway.MAX_CLIENTS + 12)]
many.append(conn(("10.1.1.27", 80), ("203.0.113.1", 45678)))
snap = snapshot(nginx=nginx_section([nginx_server(ports, ["a.example.test"])]),
sockets=sockets_section(listens=[listen(80)], established=many))
pipe = pipe_by_port(gateway.parse(snap, "gateway"), 80)
assert len(pipe["clients"]) == gateway.MAX_CLIENTS
assert pipe["clients"] == sorted(pipe["clients"])
assert pipe["connections"] == len(many), "the count stays the truth when the list is capped"
# --------------------------------------------------------------------------- #
# the outbound join
# --------------------------------------------------------------------------- #
def test_a_target_written_as_an_address_matches_exactly_and_says_by_port_false() -> None:
snap = snapshot(
nginx=nginx_section([nginx_server([8443], ["app.example.test"],
[proxy_pass(("127.0.0.1", 8081))])]),
sockets=sockets_section(
listens=[listen(8443)],
established=[conn(("10.1.1.27", 39000), ("127.0.0.1", 8081)),
conn(("10.1.1.27", 39002), ("10.9.9.9", 8081))]))
payload = gateway.parse(snap, "gateway")
exact = pipe_to(payload, "127.0.0.1", 8081)
assert exact["target"]["by_port"] is False
assert exact["connections"] == 1, "the one socket dialled to that address"
assert exact["metric"] == "gateway.out_conns@127.0.0.1:8081"
assert payload["unmatched"]["outbound"] == 1, (
"the same port on a different address is not this target's connection")
def test_a_domain_target_is_matched_on_port_alone_and_labelled_by_port() -> None:
"""The address half is the config's, the socket half is the kernel's, and `by_port` says so."""
snap = snapshot(
nginx=nginx_section([nginx_server([4443], ["cloud.example.test"],
[proxy_pass(("nextcloud.example.test", 11000))])]),
sockets=sockets_section(
listens=[listen(4443)],
established=[conn(("10.1.1.27", 4443), ("10.1.0.4", 55555)),
conn(("10.1.1.27", 54340), ("10.1.0.254", 11000))]))
payload = gateway.parse(snap, "gateway")
pipe = pipe_to(payload, "nextcloud.example.test", 11000)
assert pipe["target"]["by_port"] is True
assert pipe["connections"] == 1
assert pipe["source"] == "nginx"
assert payload["unmatched"]["groups"] == []
def test_a_port_shared_by_several_names_resolves_none_of_them() -> None:
"""The real `stream.conf` shape: sixteen names on 443, and no basis for picking one.
The sockets go to `unmatched.outbound` with a group naming the candidates; each candidate's
own count stays `null` — a repeated port total drawn on sixteen pipes reads as sixteen times
the traffic, and a split of it is arithmetic nobody measured (rule 2).
"""
entries = [(f"site{i}.example.test", (f"backend{i}.example.test", 443)) for i in range(4)]
snap = snapshot(
nginx=nginx_section([nginx_server([443], [], [proxy_pass(None, variable="targetBackend")])],
maps=[mapping_block("$targetBackend", entries)]),
sockets=sockets_section(
listens=[listen(443)],
established=[conn(("10.1.1.27", 443), ("203.0.113.5", 4000)),
conn(("10.1.1.27", 50000), ("10.2.2.2", 443)),
conn(("10.1.1.27", 50002), ("10.2.2.3", 443))]))
payload = gateway.parse(snap, "gateway")
assert len(payload["nginx"]["map_targets"]) == 4
for _key, (host, port) in entries:
assert pipe_to(payload, host, port)["connections"] is None, "unknown, never 0 and never 2"
group = payload["unmatched"]["groups"][0]
assert group == {"port": 443, "connections": 2,
"hosts": ["backend0.example.test", "backend1.example.test",
"backend2.example.test", "backend3.example.test"],
"host_count": 4}
assert payload["unmatched"]["outbound"] == 2
assert payload["unmatched"]["inbound"] == 0
def test_the_group_host_list_is_bounded_and_reports_the_real_total() -> None:
entries = [(f"site{i}.example.test", (f"backend{i}.example.test", 443))
for i in range(gateway.MAX_GROUP_HOSTS + 6)]
snap = snapshot(
nginx=nginx_section([nginx_server([443], [], [proxy_pass(None, variable="be")])],
maps=[mapping_block("$be", entries)]),
sockets=sockets_section(established=[conn(("10.1.1.27", 50000), ("10.2.2.2", 443))]))
group = gateway.parse(snap, "gateway")["unmatched"]["groups"][0]
assert len(group["hosts"]) == gateway.MAX_GROUP_HOSTS
assert group["host_count"] == len(entries)
def test_no_join_ever_asks_a_resolver() -> None:
"""The whole point of `by_port`: this module must not turn a name into an address.
Spied on the live `socket` module the stdlib resolvers use, and run over the captured
gateway as well as a synthetic one, so a "helpful" future `socket.getaddrinfo` call inside
the join fails here rather than making 33 DNS queries every 30 seconds from the poll loop.
"""
calls: list[tuple[Any, ...]] = []
def spy(*args: Any, **kwargs: Any) -> Any: # pragma: no cover - must never run
calls.append((args, kwargs))
raise AssertionError("the gateway join must never resolve a name")
original = socket_module.getaddrinfo
socket_module.getaddrinfo = spy
try:
gateway.parse(snapshot(
haproxy=haproxy_section([frontend("mc", [25565])],
[backend("mc-be", [server("mc1", "minecraft.example.test",
25565)])]),
nginx=nginx_section([nginx_server([443], ["a.example.test"],
[proxy_pass(("app.example.test", 8080))])]),
sockets=sockets_section(listens=[listen(25565), listen(443)],
established=[conn(("10.1.1.27", 443), ("203.0.113.5", 900)),
conn(("10.1.1.27", 1), ("10.2.2.2", 8080))])),
"gateway")
finally:
socket_module.getaddrinfo = original
assert calls == []
def test_two_services_on_one_host_are_two_pipes_not_one_sum() -> None:
"""The reason the outbound series name carries its port (`minecraft` 25565 and 25566)."""
snap = snapshot(
haproxy=haproxy_section([frontend("mc1", [25565]), frontend("mc2", [25566])],
[backend("testcraft", [server("mc1", "mc.example.test", 25565)]),
backend("nimcraft", [server("mc2", "mc.example.test", 25566)])]),
sockets=sockets_section(
established=[conn(("10.1.1.27", 40000), ("10.5.5.5", 25565)),
conn(("10.1.1.27", 40002), ("10.5.5.5", 25566)),
conn(("10.1.1.27", 40004), ("10.5.5.5", 25566))]))
payload = gateway.parse(snap, "gateway")
assert pipe_to(payload, "mc.example.test", 25565)["connections"] == 1
assert pipe_to(payload, "mc.example.test", 25566)["connections"] == 2
assert [pipe["metric"] for pipe in payload["outbound"]] == [
"gateway.out_conns@mc.example.test:25565", "gateway.out_conns@mc.example.test:25566"]
names = {pipe["target"]["name"] for pipe in payload["outbound"]}
assert names == {"mc1", "mc2"}, "the config's own server names survive the dedup"
def test_a_target_with_no_port_is_a_pipe_with_an_unknown_count() -> None:
"""`server x host` names no port, so there is nothing to join on and nothing to invent."""
snap = snapshot(
haproxy=haproxy_section([], [backend("be", [server("solo", "solo.example.test")])]),
sockets=sockets_section(established=[conn(("10.1.1.27", 40000), ("10.5.5.5", 8080))]))
pipe = pipe_to(gateway.parse(snap, "gateway"), "solo.example.test", None)
assert pipe["target"]["by_port"] is False
assert pipe["connections"] is None
assert pipe["metric"] == "gateway.out_conns@solo.example.test"
def test_the_same_target_from_two_configs_is_one_pipe() -> None:
snap = snapshot(
nginx=nginx_section([nginx_server([4443], ["a.example.test"],
[proxy_pass(("keycloak.example.test", 8080))]),
nginx_server([8443], ["admin.a.example.test"],
[proxy_pass(("keycloak.example.test", 8080))])]),
sockets=sockets_section(established=[conn(("10.1.1.27", 40000), ("10.5.5.5", 8080))]))
payload = gateway.parse(snap, "gateway")
targets = [pipe for pipe in payload["outbound"]
if pipe["target"]["host"] == "keycloak.example.test"]
assert len(targets) == 1
assert targets[0]["connections"] == 1, "one pipe, so one count, so no double draw"
# --------------------------------------------------------------------------- #
# the map: which maps route, and which are just header words
# --------------------------------------------------------------------------- #
def test_a_map_no_proxy_pass_references_contributes_nothing() -> None:
"""`$connection_upgrade`'s values are `upgrade`/`close` — header words, never a host."""
snap = snapshot(
nginx=nginx_section([nginx_server([443], [], [proxy_pass(None, variable="targetBackend")])],
maps=[mapping_block("connection_upgrade",
[("default", ("upgrade", None)),
("", ("close", None))]),
mapping_block("targetBackend",
[("a.example.test", ("be.example.test", 443))])]),
sockets=sockets_section(listens=[listen(443)]))
payload = gateway.parse(snap, "gateway")
assert [row["key"] for row in payload["nginx"]["map_targets"]] == ["a.example.test"]
assert [pipe["target"]["host"] for pipe in payload["outbound"]] == ["be.example.test"]
def test_a_map_entry_with_no_address_is_skipped() -> None:
snap = snapshot(
nginx=nginx_section([nginx_server([443], [], [proxy_pass(None, variable="be")])],
maps=[mapping_block("be", [("a.example.test", None),
("b.example.test", ("be2.example.test", 443))])]))
payload = gateway.parse(snap, "gateway")
assert [row["key"] for row in payload["nginx"]["map_targets"]] == ["b.example.test"]
def test_a_duplicate_endpoint_keeps_the_name_the_config_has() -> None:
"""haproxy is claimed first, so nginx writing the same endpoint loses nothing."""
snap = snapshot(
haproxy=haproxy_section([frontend("app", [80])],
[backend("app-be", [server("app1", "app.example.test", 9000)])]),
nginx=nginx_section([nginx_server([8443], ["app.example.test"],
[proxy_pass(("app.example.test", 9000))])]),
sockets=sockets_section(established=[conn(("10.1.1.27", 40000), ("10.5.5.5", 9000))]))
payload = gateway.parse(snap, "gateway")
app_pipes = [pipe for pipe in payload["outbound"]
if pipe["target"]["host"] == "app.example.test"]
assert len(app_pipes) == 1
assert app_pipes[0]["target"]["name"] == "app1"
assert app_pipes[0]["source"] == "haproxy"
assert app_pipes[0]["connections"] == 1
def test_a_proxy_pass_naming_no_host_is_no_pipe() -> None:
"""`target: {host: ""}` is drift, and drift yields no pipe rather than a pipe to nowhere."""
snap = snapshot(nginx=nginx_section([nginx_server(
[443], [], [{"location": "/", "scheme": "http", "target": {"host": " ", "port": 80},
"variable": None}])]))
payload = gateway.parse(snap, "gateway")
assert payload["outbound"] == []
assert payload["nginx"]["servers"] == [{"port": 443, "server_names": []}]
def test_the_readers_refuse_the_shapes_that_are_not_values() -> None:
"""The three guards every section read goes through, on the inputs that are not data.
Pinned directly (the way `tests/test_config.py` pins `_flag`/`_num`) because each one is the
difference between a pipe and no pipe: an empty string is not an address, a bare string is
not a list of names, and whitespace is not a host.
"""
assert gateway._is_ip_literal("") is False
assert gateway._is_ip_literal(None) is False
assert gateway._is_ip_literal("203.0.113.7") is True
assert gateway._is_ip_literal("[::1]") is False, "/proc spells addresses without brackets"
assert gateway._is_ip_literal("app.example.test") is False
assert gateway._names("not a list") == []
assert gateway._names(None) == []
assert gateway._names([None, "a-be", "a-be", " ", "b-be"]) == ["a-be", "b-be"]
assert gateway._target_of({"host": " ", "port": 80}) is None
assert gateway._target_of(["host", 80]) is None
assert gateway._target_of({"host": "be.example.test", "port": "nope"}) == (
"be.example.test", None)
# --------------------------------------------------------------------------- #
# the degradation arms: three ways to have nothing, none of them red
# --------------------------------------------------------------------------- #
def test_an_agent_without_the_sections_predates_gateway_support(gateway_snapshot: Any) -> None:
"""The 1.1.0 / `--no-proxy` arm: same keys, empty and null, one sentence, no verdict."""
payload = gateway.parse(gateway_snapshot("predates"), "gateway")
assert payload["supported"] is False
assert payload["why"] == gateway.WHY_PREDATES
assert "predates gateway support" in payload["why"]
assert payload["haproxy"] == {"frontends": []}
assert payload["nginx"] == {"servers": [], "map_targets": []}
assert payload["inbound"] == [] and payload["outbound"] == []
assert payload["unmatched"] == {"inbound": None, "outbound": None, "other": None,
"groups": []}
assert payload["sockets"] == {"listens": None, "established": None}
assert payload["sections"] == {"haproxy": None, "nginx": None, "sockets": None}
assert gateway.samples(payload, 1.0) == [], "an unknown count publishes no gauge"
def test_a_host_running_no_haproxy_says_so_and_keeps_the_nginx_pipes(gateway_snapshot: Any) -> None:
payload = gateway.parse(gateway_snapshot("no_haproxy"), "gateway")
assert payload["supported"] is True, "the agent knows the feature; the host has no haproxy"
assert payload["sections"]["haproxy"] is False
assert "no haproxy" in (payload["why"] or "")
assert "not found" not in (payload["why"] or ""), "the page's sentence, not the agent's log"
assert payload["section_errors"], "the agent's own line still travels, verbatim"
assert any("not found" in line for line in payload["section_errors"])
assert [row["port"] for row in payload["nginx"]["servers"]] == [80, 443, 4443, 8443]
assert [pipe["port"] for pipe in payload["inbound"] if pipe["source"] == "nginx"] == \
[80, 443, 4443, 8443]
assert payload["haproxy"]["frontends"] == []
def test_an_unreadable_config_is_a_different_sentence_from_no_config() -> None:
snap = snapshot(haproxy=haproxy_section([], available=None),
nginx=nginx_section([nginx_server([80], ["a.example.test"])]),
sockets=sockets_section(listens=[listen(80)]))
payload = gateway.parse(snap, "gateway")
assert payload["sections"]["haproxy"] is None
assert "could not be read or parsed" in (payload["why"] or "")
assert [pipe["port"] for pipe in payload["inbound"]] == [80]
def test_a_socket_table_that_did_not_answer_leaves_every_count_null(
gateway_snapshot: Any) -> None:
"""`available: null` is "no answer", and the only honest counts are `null` (rule 2)."""
payload = gateway.parse(gateway_snapshot("sockets_silent"), "gateway")
assert payload["sections"]["sockets"] is None
assert "sockets:" in (payload["why"] or "")
assert payload["sockets"] == {"listens": None, "established": None}
assert payload["inbound"], "the configured pipes are still drawn — the config did answer"
assert all(pipe["connections"] is None for pipe in payload["inbound"])
assert all(pipe["connections"] is None for pipe in payload["outbound"])
assert payload["unmatched"] == {"inbound": None, "outbound": None, "other": None,
"groups": []}
assert gateway.samples(payload, 1.0) == []
def test_a_partial_snapshot_names_the_missing_section_only() -> None:
"""A hypothetical agent publishing `sockets` but no configs: supported, with a gap named."""
snap = snapshot(sockets=sockets_section(listens=[listen(9999)]))
payload = gateway.parse(snap, "gateway")
assert payload["supported"] is True
assert payload["sections"] == {"haproxy": None, "nginx": None, "sockets": True}
assert payload["unmatched"]["other"] == 1, "9999 is a listener nothing claims"
assert "haproxy" in (payload["why"] or "") and "nginx" in (payload["why"] or "")
# --------------------------------------------------------------------------- #
# what the payload must never contain, and the gauges it must
# --------------------------------------------------------------------------- #
def test_no_count_carries_a_verdict_or_a_window(gateway_snapshot: Any) -> None:
"""`net.*` precedent for colour, rule 2 for the `Δ` label: a count is a count.
Both arms are walked, so a `severity` bolted onto a pipe or a `window_seconds` copied from
`sources/host.py` fails here — the first would be a judgement `severity.py` never made, the
second would claim a measurement this source does not take.
"""
for mode in ("full", "predates", "sockets_silent"):
keys = walk_keys(gateway.parse(gateway_snapshot(mode), "gateway"))
assert "severity" not in keys, mode
assert not {key for key in keys if "window" in key or key.startswith("Δ")}, mode
def test_the_payload_is_json_serialisable_end_to_end(gateway_snapshot: Any) -> None:
"""It goes out over `/api/gateway/snapshot` verbatim, so nothing in it may be exotic."""
json.dumps(gateway.parse(gateway_snapshot("full"), "gateway"))
def test_samples_are_one_gauge_per_pipe_under_the_gateway_family(gateway_snapshot: Any) -> None:
snap = gateway_snapshot("full")
payload = gateway.parse(snap, "gateway")
rows = gateway.samples(payload, 1_760_000_000.0)
assert rows, "a live gateway publishes gauges"
assert {row.source for row in rows} == {gateway.SOURCE}
assert {row.host for row in rows} == {"gateway"}
named = {row.metric: row.value for row in rows}
assert named["gateway.in_conns@443"] == float(pipe_by_port(payload, 443)["connections"])
counted = [pipe for pipe in payload["inbound"] + payload["outbound"]
if pipe["connections"] is not None]
assert len(named) == len(counted), "one gauge per pipe with a number, and none without"
for pipe in counted:
assert named[pipe["metric"]] == float(pipe["connections"]), (
"the stored gauge is the number the page prints, never a second reading of it")
def test_parse_gateway_snapshot_stamps_its_samples_from_the_snapshot() -> None:
"""The x-axis is when the agent took the snapshot, the way `sources/disks.py` does it."""
result = gateway.parse_gateway_snapshot(
"gateway", snapshot(haproxy=haproxy_section([frontend("ssh", [22])]),
sockets=sockets_section(listens=[listen(22)])), now=999.0)
assert result.events == [], "a connection count has no transition worth an event line"
assert [sample.ts for sample in result.samples] == [1_760_000_000.0]
assert [sample.metric for sample in result.samples] == ["gateway.in_conns@22"]
def test_a_snapshot_with_no_timestamp_falls_back_to_the_callers_clock() -> None:
doc = snapshot(haproxy=haproxy_section([frontend("ssh", [22])]),
sockets=sockets_section(listens=[listen(22)]))
doc.pop("ts")
result = gateway.parse_gateway_snapshot("gateway", doc, now=42.0)
assert [sample.ts for sample in result.samples] == [42.0]
def test_the_gauges_skip_a_pipe_with_no_number() -> None:
"""No sample for an unknown count — a chart of zeros for a dead socket table is the
failure mode rule 2 names, and `sources/disks.py` already refuses it for unreadable drives."""
payload = gateway.parse(snapshot(haproxy=haproxy_section([frontend("ssh", [22])])), "gateway")
assert payload["inbound"][0]["connections"] is None
assert gateway.samples(payload, 1.0) == []
def test_a_snapshot_whose_sections_are_empty_is_an_honest_empty_page() -> None:
"""A 1.2.0 agent on a box with two empty configs: supported, and nothing drawn."""
snap = snapshot(haproxy=haproxy_section([]), nginx=nginx_section(),
sockets=sockets_section())
payload = gateway.parse(snap, "gateway")
assert payload["supported"] is True and payload["why"] is None
assert payload["inbound"] == [] and payload["outbound"] == []
assert payload["sockets"] == {"listens": 0, "established": 0}
assert payload["unmatched"] == {"inbound": 0, "outbound": 0, "other": 0, "groups": []}
def test_junk_in_the_sections_yields_no_pipes_and_no_exception() -> None:
"""The source is a projection of a document it does not control — drift is `n/a`, not a crash."""
snap: dict[str, Any] = {"ts": 1.0, "host": "gateway",
"haproxy": {"available": True, "frontends": [3, None],
"backends": {"nope": 1}},
"nginx": {"available": True, "servers": "not a list", "maps": None},
"sockets": {"available": True, "listens": [None],
"established": [{"local": 5, "remote": None}]}}
payload = gateway.parse(snap, "gateway")
assert payload["inbound"] == [] and payload["outbound"] == []
assert payload["unmatched"]["inbound"] == 0
assert payload["sockets"] == {"listens": 0, "established": 1}
def test_a_port_of_zero_or_a_name_of_nothing_is_no_match() -> None:
"""`0` is not a port and an empty target is not a host, so neither becomes a pipe (rule 2)."""
snap = snapshot(
haproxy=haproxy_section([frontend("u", [None, 0]), frontend("ok", [22])],
[backend("b", [server("?", "", 22), server("z", "z.example.test",
0)])]),
sockets=sockets_section(listens=[listen(0), listen(22)]))
payload = gateway.parse(snap, "gateway")
assert [row["port"] for row in payload["haproxy"]["frontends"]] == [22]
assert [pipe["target"]["host"] for pipe in payload["outbound"]] == ["z.example.test"]
assert payload["outbound"][0]["target"]["port"] is None
assert payload["unmatched"]["other"] == 0, "a 0 is not a port, claimed or otherwise"
def test_a_use_backend_map_is_reported_as_routes_not_one_chosen_backend() -> None:
"""haproxy picks per request here; the payload names every candidate and chooses none."""
snap = snapshot(haproxy=haproxy_section(
[frontend("fe", [80], default="base-be", use=("blue-be", "green-be"))]))
row = gateway.parse(snap, "gateway")["haproxy"]["frontends"][0]
assert row["backend"] == "base-be"
assert row["routes"] == ["base-be", "blue-be", "green-be"]
tests/test_live_fleet.py
+41 -1
View File
@@ -33,7 +33,7 @@ from dashboards import certs, collectors
from dashboards.config import Settings, load
from dashboards.rates import RateEstimator
from dashboards.severity import classify
from dashboards.sources import nut
from dashboards.sources import gateway, nut
from dashboards.sources.disks import SMART_SERIES, DisksCollector
from dashboards.sources.host import HostCollector
from slots_contract import assert_slot_shape
@@ -255,6 +255,46 @@ async def test_disks_source_charts_the_physical_drives_the_agent_reports(
pytest.skip("no host agent answered")
async def test_the_gateway_source_joins_the_live_ingress(settings: Settings,
agent_tls: dict[str, collectors.AgentTls]) -> None:
"""Phase 29: `sources/gateway.py` over the **real** gateway agent, read-only.
The production path — the same `fetch_snapshot` the poller will run, through the stored
mTLS authority — so what fails here is a drift between the agent's sections and the join,
not a typo in a fixture. The assertions are the ones that hold whatever the traffic is:
the agent answers the three sections, every configured frontend is a pipe, and the
inbound/outbound/unmatched tallies **partition** the kernel's ESTABLISHED table, which is
what lets the page's numbers reconcile against a direct `/proc/net/tcp` read.
"""
target = next((item for item in settings.host_targets if item.name == settings.gateway_host),
None)
if target is None:
pytest.skip(f"GATEWAY_HOST={settings.gateway_host!r} is not in HOST_TARGETS")
async with httpx.AsyncClient(timeout=settings.gateway_timeout) as client:
try:
snapshot = await collectors.fetch_snapshot(client, target,
timeout=settings.gateway_timeout,
tls=agent_tls.get(target.name))
except collectors.UpstreamError as exc:
pytest.skip(f"{target.name} did not answer: {exc}")
payload = gateway.parse(snapshot, target.name)
if not payload["supported"]:
pytest.fail(f"{target.name} runs agent {payload.get('agent_version')} — the gateway page "
"needs 1.2.0's haproxy/nginx/sockets sections (AGENTS.md gap 11's rollout)")
assert payload["host"] == target.name
assert payload["sockets"]["established"] == len(snapshot["sockets"]["established"])
assert payload["inbound"], "no frontend on either config would leave the page empty"
for pipe in payload["inbound"]:
assert isinstance(pipe["connections"], int), "the socket table answered, so a count is known"
for group in payload["unmatched"]["groups"]:
assert group["connections"] > 0 and group["host_count"] > 1, (
"a group is only ever the unresolved fan-out of several names on one port")
attributed = sum(pipe["connections"] or 0 for pipe in payload["inbound"] + payload["outbound"])
assert attributed + payload["unmatched"]["inbound"] + payload["unmatched"]["outbound"] == \
payload["sockets"]["established"], "every socket counted exactly once"
assert all(sample.source == gateway.SOURCE for sample in gateway.samples(payload, 0.0))
@pytest.mark.parametrize("engine", ["gufo", "vllm"])
async def test_engine_still_publishes_the_series_we_parse(engine: str,
settings: Settings) -> None: