phase: 06_system_page_gpu_verdicts

All criteria verified and the project is fully green.

**Verified (final pass — all 3 phase-06 tasks were already in `complete/`)**
- `classify("gpu.busy_pct", 99)` / `gpu.mem_used_pct` → `None`; `THRESHOLDS` has no `gpu.*` row; `alerts.py`/`api.py` carry no GPU verdict; live browser: deskwork GPU at **91 %** and **98 %** renders `class="hero"`, `color=rgb(22,25,31)` (plain text), no bar, no `.sev-*`.
- GPU temp on the hero `.sub` from live agents: `Radeon 8060S Graphics · VRAM 165.1 MiB / 512.0 MiB · pool 78% · 48 °C`, R9700 `· 40 °C`, gateway prints none (no `n/a`, no `0 °C`).
- Charts collapsed by default: fresh load = 3 panels / 3 toggles / **0 open / 0 `<svg>`**; expand → 3 SVGs (510 px wide), repaints across polls (legend 3.3 %→5.6 %), collapsed siblings keep their own height (803 vs 358/339).
- `gpu.busy_pct@0` still answers from `/api/system/history` (345 points) — verdict gone, data intact.

**Tests / lint / coverage**
- `bash .agents/validate.sh` → **EXIT=0**; `uv run pytest --cov --cov-fail-under=94` → **550 passed, 8 skipped, 97.91 %**; `uv run ruff check src tests agent` → clean; `uv run pyright` → 0 errors; `node --check` all scripts clean; `uv pip list --outdated` empty.
- `uv run pytest -m live --live` → **8 passed** (real agents/engines/upsd); dev server restarted, `/api/health` all sources `ok`.
- New screenshots: `data/logs/phase06-final-collapsed-default.png`, `data/logs/phase06-final-expanded-gpu-busy-uncoloured.png`.

**Completion criteria:** classify→None ✅ · pytest+94 % gate ✅ · ruff+pyright ✅ · validate.sh ✅ · screenshots of the three behaviours on live data ✅ · AGENTS.md severity section claims no GPU rows ✅

**Docs:** fixed 3 items — (1) `AGENTS.md` pytest count stale (546 → 550); (2) `app.js` `severityFor` docstring still claimed "`page-ups.js` still calls it (phase 05 converts it)", contradicting the code/tests/AGENTS.md (comment-only); rest of README/AGENTS reviewed in full (env table vs `config.py`+`.env.example`, agent routes incl. `/schema`, alerts rules, page descriptions, contracts) — consistent.

**Notable:** fixed a real pre-existing defect in `scripts/dev-server.sh` — `stop` slept 1 s and deleted the pidfile without waiting for exit, so `restart` with a browser open overlapped two servers on one SQLite file, producing a `database is locked` store refusal that `/system` rendered as a red event. Now waits (bounded 20 s) then SIGKILLs; verified a restart with live polling logs no such error/event. No live infra config was changed.

**Next pending phase:** `.agents/phases/todo/07_llm_monitor_style_pages`
This commit is contained in:
2026-10-17 14:04:52 -04:00
parent 9f3bf3d503
commit de325caa49
37 changed files with 792 additions and 128 deletions
+40 -13
View File
@@ -160,10 +160,14 @@ Responsibilities, one line each:
`classify()` → `ok`/`warn`/`fail`/`None`, plus `attach()`/`with_severity()` to put the verdict
beside a value. No measurement, no verdict: `None` in → `None` out, never a green pass for
an unknown. The verdicts travel **additively** in payloads — `cpu.severity`,
`memory.severity`, `disks.severity`, `gpus[].severity`, each UPS card's `severity`
(charge/load/runtime), `severity` inside rated LLM cells (KV pressure, classified on a
`memory.severity`, `disks.severity`, each UPS card's `severity` (charge/load/runtime),
`severity` inside rated LLM cells (KV pressure, classified on a
percent scale because the engines publish a 0-1 ratio), and `severity` per host/fleet/UPS
row in `/api/summary`. Pages render the class; they never compare a number to a threshold.
There is **no `gpu.*` row and no `gpus[].severity`** (phase 06): these are LLM servers,
a busy GPU is the job at every level, so the GPU card and the fleet GPU column render
uncoloured — like `net.*`, whose colourlessness predates it. Removing the verdict is not
removing the data: `gpu.busy_pct@N` / `gpu.mem_used_pct@N` are still stored and charted.
- `alerts.py` — `build(settings, job_states, host_payloads, engine_payloads, ups_payload,
recent_events)` → the "what is wrong right now" list `/api/summary` serves as `alerts`
(`{id, severity, source, message, href, at}`, fail first then warn, then by source). A
@@ -250,7 +254,7 @@ cp .env.dev.example .env.dev # dev-only: ssh tunnels + live targets
./scripts/dev-server.sh status # per-source ok/DOWN
./scripts/dev-server.sh logs
uv run pytest # hermetic: 543 passed + 8 live-skipped, no network
uv run pytest # hermetic: 550 passed + 8 live-skipped, no network
uv run pytest --cov # coverage gate (97.9%; floor 94%)
uv run pytest -m live --live # opt-in: real fleet / engines / upsd
uv run ruff check src tests agent && uv run pyright
@@ -287,8 +291,9 @@ branch `main`); there is no CI on it, so `bash .agents/validate.sh` is the only
asserts it against the three captures (discrete GPU, APU, no GPU) and over HTTP, and
`test_every_host_serves_the_hero_card_contract` asserts the same table against the live
agents — so a rename fails a test, and a card that shows `n/a` cannot be "fixed" by
inventing a field. `network` carries no `severity` on purpose (throughput has no good/bad),
and the contract pins its absence. `tests/ups_contract.py` is the same device for the `/ups`
inventing a field. `network` carries no `severity` on purpose (throughput has no good/bad)
and neither does a GPU row (phase 06: a busy GPU is the job on an LLM server) — the
contract pins both absences. `tests/ups_contract.py` is the same device for the `/ups`
hero row and battery glyph (`HERO_FIELDS`, `UPS_SHAPES`, `assert_ups_hero_contract`):
`tests/test_ups_contract.py` drives it from `tests/fixtures/nut_<ups>.vars` — verbatim
`LIST UPS`/`LIST VAR`/`LIST CMD` replies from the two real UPSes — and
@@ -414,10 +419,18 @@ Colour comes from the payload's `severity` dicts and nowhere else; a missing num
verdict — gateway's CPU temp, any host's absent GPU — prints `n/a` **uncoloured**. NET is
one slot with two lines (`.hero-lines`, `↓` in / `↑` out) and is deliberately colourless:
throughput has no good/bad, so `network` carries no severity. The GPU slot holds one hero
per card (model, VRAM used/total and the honest APU `pool` view in the `.sub`), and
everything not glance-worthy — load averages, clock, disk read/write/busy, uptime, GPU
power/temp, filesystems, processes — folds into a collapsed "More detail" `<details>`: a
simpler surface, not a less complete page.
per card whose `.sub` carries model, VRAM used/total, the honest APU `pool` view and the
card's temperature (`gpus[].temp_c` — the agent's max over that card's hwmon labels, the
same value stored as `gpu.temp_c@N`, so the hero and the history cannot disagree), and it is
colourless for the same reason (phase 06): a 95 %-busy card on an LLM server is the job, so
the hero carries no `severity` and no utilisation bar — a bar under a verdictless number
would be a verdict by shape. A card whose driver exposes no temperature input prints **no**
temperature in that footnote rather than `n/a` or `0 °C`: "where you can" is the operator's
wording, and `n/a` is reserved for numbers the card is supposed to have. Everything not
glance-worthy — load averages, clock, disk read/write/busy, uptime, GPU power and the
per-label temp breakdown, filesystems, processes — folds into a collapsed "More detail"
`<details>`: a simpler surface, not a less complete page (the table keeps its Temp column;
this was a promotion, not a de-duplication).
A `Δ<window>` label is a measurement, never a config value: `sources/host.py` sends the window
the estimator actually used as the payload's `window_seconds`, and `/system`'s footnote prints
@@ -453,14 +466,28 @@ panel and the host/metric detail picker are deleted, along with the `#chart-*` /
the `select` / `label.field` styling only those pickers used. Each panel stacks its own
`.host-charts` — utilisation (CPU, memory, worst disk), network (in/out), then GPU busy — where
the GPU slot is created and destroyed with `host.gpus`, so gateway shows neither a phantom card
nor a hole. **One `/api/system/history` request per host per refresh**
nor a hole. **That stack is collapsed behind a per-host `Charts` `<details>`** (phase 06: "only
visible if expanded… there will be lots of hosts"): the default `/system` view is the hero row,
the summary carries the window label the stack will draw (the chips themselves stay in the page
head, one control per page), and `drawHostCharts()` is the single place that decides whether to
paint — so nothing is built or drawn while the stack is shut (`LineChart` inside a
`display:none` container measures 0 px, and rebuilding three charts per host every poll for
plots nobody sees is the waste this removes). The **history request is deliberately unchanged**:
one per host per refresh, naming every series the panel could draw, and its response is cached
per host so the first expand paints from data in hand rather than waiting a poll; a cached
response from another window is re-fetched instead of being drawn under the wrong label.
Expanding one panel must not stretch its collapsed siblings into blank cards, so `#hosts`
carries the existing `.grid.items-start`. **One `/api/system/history` request per host per refresh**
(`hosts=<that host>&metrics=<that panel's series>&range=…`, pinned by
`test_every_system_chart_is_drawn_inside_its_own_host_panel`): unknown metrics come back as
`test_every_system_chart_is_drawn_inside_its_own_host_panel` and the collapse by
`test_system_history_charts_are_collapsed_behind_a_toggle`): unknown metrics come back as
empty series on purpose, so a panel asks for every series it could draw and asks once. A legend
here names metrics (or one GPU's model), never hosts — the panel heading already says who this
is, and a legend that grew a second host is the bug this replaced. A host that stops reporting
loses its panel *and* its charts (`pruneHosts`), never a chart of nothing. Per-core visuals are
gone from `/system` and `renderCores` is **deleted** from `charts.js` with its last caller.
loses its panel, its `Charts` toggle *and* its charts (`pruneHosts` deletes the card plus the
payload/chart/history maps), never a chart of nothing or an orphan control in the grid. Per-core
visuals are gone from `/system` and `renderCores` is **deleted** from `charts.js` with its last
caller.
`drawHostChart()` shades a chart only when one series has points — two or three filled areas
stack into one block and bury the quieter line, which is how a idle CPU would hide behind a busy
memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryable from