phase: 06_system_page_gpu_verdicts
All criteria verified and the project is fully green.
**Verified (final pass — all 3 phase-06 tasks were already in `complete/`)**
- `classify("gpu.busy_pct", 99)` / `gpu.mem_used_pct` → `None`; `THRESHOLDS` has no `gpu.*` row; `alerts.py`/`api.py` carry no GPU verdict; live browser: deskwork GPU at **91 %** and **98 %** renders `class="hero"`, `color=rgb(22,25,31)` (plain text), no bar, no `.sev-*`.
- GPU temp on the hero `.sub` from live agents: `Radeon 8060S Graphics · VRAM 165.1 MiB / 512.0 MiB · pool 78% · 48 °C`, R9700 `· 40 °C`, gateway prints none (no `n/a`, no `0 °C`).
- Charts collapsed by default: fresh load = 3 panels / 3 toggles / **0 open / 0 `<svg>`**; expand → 3 SVGs (510 px wide), repaints across polls (legend 3.3 %→5.6 %), collapsed siblings keep their own height (803 vs 358/339).
- `gpu.busy_pct@0` still answers from `/api/system/history` (345 points) — verdict gone, data intact.
**Tests / lint / coverage**
- `bash .agents/validate.sh` → **EXIT=0**; `uv run pytest --cov --cov-fail-under=94` → **550 passed, 8 skipped, 97.91 %**; `uv run ruff check src tests agent` → clean; `uv run pyright` → 0 errors; `node --check` all scripts clean; `uv pip list --outdated` empty.
- `uv run pytest -m live --live` → **8 passed** (real agents/engines/upsd); dev server restarted, `/api/health` all sources `ok`.
- New screenshots: `data/logs/phase06-final-collapsed-default.png`, `data/logs/phase06-final-expanded-gpu-busy-uncoloured.png`.
**Completion criteria:** classify→None ✅ · pytest+94 % gate ✅ · ruff+pyright ✅ · validate.sh ✅ · screenshots of the three behaviours on live data ✅ · AGENTS.md severity section claims no GPU rows ✅
**Docs:** fixed 3 items — (1) `AGENTS.md` pytest count stale (546 → 550); (2) `app.js` `severityFor` docstring still claimed "`page-ups.js` still calls it (phase 05 converts it)", contradicting the code/tests/AGENTS.md (comment-only); rest of README/AGENTS reviewed in full (env table vs `config.py`+`.env.example`, agent routes incl. `/schema`, alerts rules, page descriptions, contracts) — consistent.
**Notable:** fixed a real pre-existing defect in `scripts/dev-server.sh` — `stop` slept 1 s and deleted the pidfile without waiting for exit, so `restart` with a browser open overlapped two servers on one SQLite file, producing a `database is locked` store refusal that `/system` rendered as a red event. Now waits (bounded 20 s) then SIGKILLs; verified a restart with live polling logs no such error/event. No live infra config was changed.
**Next pending phase:** `.agents/phases/todo/07_llm_monitor_style_pages`
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
# Phase 06 — The system page stops grading GPU usage, shows GPU temp, and tucks charts away
|
||||
|
||||
**Source:** `TODO.md` L3, L4, L9 — "GPU verdicts + system-page cleanup": "High gpu usage is not a warning or error, these are LLM servers and that's expected" · "The historical charts on the system resources page should only be visible if expanded" · "Pull GPU temp for the system resources tab where you can."
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/severity.py` (`THRESHOLDS` — the only place thresholds live), `src/dashboards/sources/host.py` (per-GPU `severity` dict + `gpu.*` samples), `tests/card_contract.py` / `tests/test_system_contract.py` (the pinned card contract), `src/dashboards/web/static/page-system.js` (hero row, `gpuSub()`, the per-host `.host-charts` stack), `src/dashboards/web/static/dash.css`
|
||||
|
||||
## Objective
|
||||
|
||||
`/system` watches LLM servers, where a busy GPU is the job, not a fault: the GPU card stops
|
||||
carrying any verdict (muted, never green and never red), the GPU temperature the agent
|
||||
already reports moves up onto the hero row, and the per-host history charts collapse behind
|
||||
a "Charts" toggle so the default view is current stats only.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `05_ups_hero_battery_glyph` (complete) — the shared `.hero` vocabulary and the per-host
|
||||
chart stack this phase reworks; nothing in this phase may fight its conventions.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_drop_gpu_severity_verdicts.md` — remove `gpu.busy_pct` / `gpu.mem_used_pct` from the verdict table; the GPU card renders uncoloured; the contract pins the *absence*.
|
||||
2. `02_gpu_temp_on_hero_row.md` — surface `gpus[].temp_c` (already in the payload) on the GPU hero's `.sub` line.
|
||||
3. `03_collapse_host_history_charts.md` — wrap each host's `.host-charts` in a collapsed-by-default toggle; charts draw lazily on expand.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_severity.py`, `tests/test_host_source.py`, `tests/test_system_contract.py`
|
||||
(all three captures — discrete GPU, APU, no GPU — plus over HTTP) must pin the new
|
||||
no-verdict shape; `tests/test_api.py` pins the chart-toggle markup on served bytes.
|
||||
- Coverage: **>90%** on new/modified code, ≥94% overall gate maintained.
|
||||
- Browser (DoD): dev server live at `127.0.0.1:8090`, screenshot per change in `data/logs/` —
|
||||
a busy GPU (driveripper's R9700 under load) renders uncoloured, the temp is on the hero
|
||||
row, charts are collapsed by default and draw on expand.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `classify("gpu.busy_pct", 99)` is `None` and no page script colours a GPU number from a threshold
|
||||
- [ ] `uv run pytest` green; coverage gate (94%) holds
|
||||
- [ ] `uv run ruff check src tests agent && uv run pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] screenshots in `data/logs/` show the three behaviours on live data
|
||||
- [ ] `AGENTS.md` severity section no longer claims GPU rows exist
|
||||
@@ -0,0 +1,51 @@
|
||||
# Task 01 — GPU usage and VRAM stop carrying a verdict
|
||||
|
||||
**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:3` — "High gpu usage is not a warning or error, these are LLM servers and that's expected"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Remove the GPU verdict entirely: a 95 %-busy card on an LLM server is the job, not a
|
||||
warning, so the GPU card prints its numbers uncoloured at every utilisation.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/severity.py` — delete the `gpu.busy_pct` and `gpu.mem_used_pct` rows
|
||||
from `THRESHOLDS`. No verdict at all — not a raised threshold: "high usage is expected"
|
||||
applies to every level, and the module's own rule is that a metric the table does not
|
||||
know classifies to `None`, never `ok`.
|
||||
2. `src/dashboards/sources/host.py` — stop classifying the GPU rows: drop the `busy_pct` /
|
||||
`mem_used_pct` classifications from the per-GPU `severity` dict (remove the dict when it
|
||||
becomes empty), leaving the `gpu.*` **samples** untouched — the history series stay
|
||||
stored and queryable; removing the verdict is not removing the data.
|
||||
3. `tests/card_contract.py` — pin the absence, the way it already pins `network`'s
|
||||
colourlessness: the GPU card paths (`gpus[].model/busy_pct/vram_*/mem_used_pct`) stay,
|
||||
the verdict paths go. `tests/test_system_contract.py` follows (all three captures + HTTP).
|
||||
4. `tests/test_severity.py` — assert `classify("gpu.busy_pct", 99)` /
|
||||
`classify("gpu.mem_used_pct", 99)` return `None`; keep the boundary-ordering test
|
||||
consistent with the smaller table. `tests/test_host_source.py` — the GPU row no longer
|
||||
carries a `severity` dict.
|
||||
5. `src/dashboards/web/static/page-system.js` — the GPU hero renders without `severity` and
|
||||
without the utilisation bar (a bar under a verdictless number is a verdict by shape);
|
||||
drop the `data-sev="gpus[].severity.busy_pct"` hook. `n/a` stays uncoloured as before.
|
||||
6. `AGENTS.md` — the `severity.py` section lists `gpus[].severity` among the rated cells;
|
||||
update it so the file and the code agree (DoD: rules and fleet facts travel with the
|
||||
change).
|
||||
|
||||
- ASSUMPTION: "no verdict at all (muted)" was the operator's confirmed reading of the item
|
||||
— the card shows busy % / VRAM with no colour class at any level, rather than "only fail
|
||||
above a new ceiling". If the operator later wants a ceiling, it is one table row back.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_severity.py` (no GPU row in the table; `None` verdicts),
|
||||
`tests/test_host_source.py` (payload shape), `tests/test_system_contract.py` (the pinned
|
||||
contract over the three captures + over HTTP).
|
||||
- Coverage: **>90%** on this task's modified code; overall gate (94%) maintained.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest` green with the updated contract
|
||||
- [ ] a live busy GPU (driveripper, R9700) renders its numbers with no colour class and no bar
|
||||
- [ ] `gpu.busy_pct@N` still answers from `/api/system/history` (data intact, verdict gone)
|
||||
- [ ] `uv run ruff check src tests agent && uv run pyright` clean
|
||||
@@ -0,0 +1,44 @@
|
||||
# Task 02 — GPU temperature on the hero row
|
||||
|
||||
**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:9` — "Pull GPU temp for the system resources tab where you can."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The GPU temperature is already collected end-to-end (agent `gpus[].temps` → payload
|
||||
`gpus[].temp_c`, max over the card's hwmon labels, and a `gpu.temp_c@N` sample) but the
|
||||
hero row never shows it — it sits in the collapsed "More detail" table. Move it up so the
|
||||
one number that matters while a card is busy is glanceable.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-system.js` — `gpuSub()`: append the temperature to the
|
||||
GPU hero's `.sub` line when `gpu.temp_c != null` (e.g.
|
||||
`VRAM 31.2 GB / 32 GB · pool 98% · 63 °C`); when the agent cannot read it (the APU
|
||||
build on deskwork or a card with no hwmon input), the sub line simply does not print a
|
||||
temperature — `n/a` is reserved for numbers the card is supposed to have, and "where you
|
||||
can" is the item's own wording.
|
||||
2. Keep the "More detail" table exactly as-is (it carries per-label temps, power, and both
|
||||
memory pools); this is a surface change, not a dedup.
|
||||
3. No backend changes: `tests/fixtures/agent_deskwork.json` / `agent_driveripper.json`
|
||||
already carry `gpus[].temps`, so the hermetic page tests can exercise both the
|
||||
with-temp and without-temp paths.
|
||||
|
||||
- ASSUMPTION: the payload's `temp_c` (agent's max over hwmon labels) is "the" GPU temp —
|
||||
the same max the agent already computes — so no new field, no new metric name, and the
|
||||
`gpu.temp_c@N` history series keeps its existing name.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `node --check` on `page-system.js`; served-page assertions in `tests/test_api.py`
|
||||
if the markup structure changes.
|
||||
- Coverage: **>90%** on modified code (mostly JS; the Python side is unchanged).
|
||||
- Browser (DoD): screenshot in `data/logs/` with live data — driveripper's R9700 shows a
|
||||
temperature on the hero row; deskwork's APU row does not print a bogus one.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] GPU hero `.sub` includes `· <n> °C` on hosts whose agent reports temps
|
||||
- [ ] hosts without a readable sensor show no temperature (not `0 °C`, not red)
|
||||
- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean
|
||||
- [ ] screenshot in `data/logs/` against live data
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Task 03 — Per-host history charts collapse behind a toggle
|
||||
|
||||
**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:4` — "The historical charts on the system resources page should only be visible if expanded. I only want to see current stats to keep things clean. There will be lots of hosts and historical charts make things look messy."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The default `/system` view is the hero row (current stats) only. Each host's `.host-charts`
|
||||
stack (utilisation, network, GPU busy) moves behind a collapsed "Charts" toggle, so a
|
||||
fleet that grows to many hosts reads clean at a glance — and a host that stops reporting
|
||||
still loses its panel and charts (`pruneHosts`) exactly as today.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-system.js` — wrap each host's `.host-charts` in a
|
||||
native `<details>` (collapsed by default; `summary` reads "Charts" and carries the
|
||||
window chip so the expanded state still says which window is drawn). Create the
|
||||
`LineChart` instances **lazily on first expand** (the `state.charts` map already keys
|
||||
them) and re-run the draw pass on refresh **only for hosts whose charts are open** —
|
||||
drawing into a collapsed container is wasted work, and the page's one-
|
||||
`/api/system/history`-request-per-host discipline is unchanged (the request still names
|
||||
every series the panel could draw; only the paint is deferred).
|
||||
2. `pruneHosts()` keeps destroying chart instances with the panel; add the `details`
|
||||
element to the removal so a dead host leaves no orphan toggle.
|
||||
3. `src/dashboards/web/static/dash.css` — style `summary` as a quiet toggle in the existing
|
||||
vocabulary (no new palette, no `.sev-*` family); the collapsed stack takes no height.
|
||||
4. `tests/test_api.py` — pin the served markup: one `<details>` per host panel, collapsed
|
||||
by default (no `open` attribute), summary text "Charts"; the orphan-`id` rule keeps
|
||||
passing (delete no rule that a live selector still uses).
|
||||
5. `node --check` on both scripts.
|
||||
|
||||
- ASSUMPTION: the toggle is **per host** (not one global switch) and **collapsed by
|
||||
default** — "only visible if expanded" is the item's own wording, and per-host matches
|
||||
the page's one-panel-per-host discipline; the window chips (5 m … 30 d) stay where they
|
||||
are and keep driving the same single history request.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_api.py` served-bytes assertions (details present, closed by default).
|
||||
- Coverage: **>90%** on modified code; overall gate (94%) maintained.
|
||||
- Browser (DoD): screenshot in `data/logs/` — collapsed by default on a multi-host page,
|
||||
one host expanded with its three charts drawn, a second refresh while expanded repaints.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] default load shows hero rows only; no chart pixels before a toggle is opened
|
||||
- [ ] expanding a host draws its charts on the first open and on every later refresh while open
|
||||
- [ ] a pruned (dead) host removes its toggle along with its panel and charts
|
||||
- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean
|
||||
Reference in New Issue
Block a user