diff --git a/.agents/phases/todo/06_system_page_gpu_verdicts/00_phase.md b/.agents/phases/todo/06_system_page_gpu_verdicts/00_phase.md new file mode 100644 index 0000000..df6294f --- /dev/null +++ b/.agents/phases/todo/06_system_page_gpu_verdicts/00_phase.md @@ -0,0 +1,42 @@ +# Phase 06 — The system page stops grading GPU usage, shows GPU temp, and tucks charts away + +**Source:** `TODO.md` L3, L4, L9 — "GPU verdicts + system-page cleanup": "High gpu usage is not a warning or error, these are LLM servers and that's expected" · "The historical charts on the system resources page should only be visible if expanded" · "Pull GPU temp for the system resources tab where you can." +**Story:** n/a +**Context:** `src/dashboards/severity.py` (`THRESHOLDS` — the only place thresholds live), `src/dashboards/sources/host.py` (per-GPU `severity` dict + `gpu.*` samples), `tests/card_contract.py` / `tests/test_system_contract.py` (the pinned card contract), `src/dashboards/web/static/page-system.js` (hero row, `gpuSub()`, the per-host `.host-charts` stack), `src/dashboards/web/static/dash.css` + +## Objective + +`/system` watches LLM servers, where a busy GPU is the job, not a fault: the GPU card stops +carrying any verdict (muted, never green and never red), the GPU temperature the agent +already reports moves up onto the hero row, and the per-host history charts collapse behind +a "Charts" toggle so the default view is current stats only. + +## Dependencies + +- `05_ups_hero_battery_glyph` (complete) — the shared `.hero` vocabulary and the per-host + chart stack this phase reworks; nothing in this phase may fight its conventions. + +## Tasks + +1. `01_drop_gpu_severity_verdicts.md` — remove `gpu.busy_pct` / `gpu.mem_used_pct` from the verdict table; the GPU card renders uncoloured; the contract pins the *absence*. +2. `02_gpu_temp_on_hero_row.md` — surface `gpus[].temp_c` (already in the payload) on the GPU hero's `.sub` line. +3. `03_collapse_host_history_charts.md` — wrap each host's `.host-charts` in a collapsed-by-default toggle; charts draw lazily on expand. + +## Testing & Quality + +- Unit: `tests/test_severity.py`, `tests/test_host_source.py`, `tests/test_system_contract.py` + (all three captures — discrete GPU, APU, no GPU — plus over HTTP) must pin the new + no-verdict shape; `tests/test_api.py` pins the chart-toggle markup on served bytes. +- Coverage: **>90%** on new/modified code, ≥94% overall gate maintained. +- Browser (DoD): dev server live at `127.0.0.1:8090`, screenshot per change in `data/logs/` — + a busy GPU (driveripper's R9700 under load) renders uncoloured, the temp is on the hero + row, charts are collapsed by default and draw on expand. + +## Completion Criteria + +- [ ] `classify("gpu.busy_pct", 99)` is `None` and no page script colours a GPU number from a threshold +- [ ] `uv run pytest` green; coverage gate (94%) holds +- [ ] `uv run ruff check src tests agent && uv run pyright` clean +- [ ] `bash .agents/validate.sh` green +- [ ] screenshots in `data/logs/` show the three behaviours on live data +- [ ] `AGENTS.md` severity section no longer claims GPU rows exist diff --git a/.agents/phases/todo/06_system_page_gpu_verdicts/01_drop_gpu_severity_verdicts.md b/.agents/phases/todo/06_system_page_gpu_verdicts/01_drop_gpu_severity_verdicts.md new file mode 100644 index 0000000..a526738 --- /dev/null +++ b/.agents/phases/todo/06_system_page_gpu_verdicts/01_drop_gpu_severity_verdicts.md @@ -0,0 +1,51 @@ +# Task 01 — GPU usage and VRAM stop carrying a verdict + +**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:3` — "High gpu usage is not a warning or error, these are LLM servers and that's expected" +**Story:** n/a + +## Objective + +Remove the GPU verdict entirely: a 95 %-busy card on an LLM server is the job, not a +warning, so the GPU card prints its numbers uncoloured at every utilisation. + +## Work + +1. `src/dashboards/severity.py` — delete the `gpu.busy_pct` and `gpu.mem_used_pct` rows + from `THRESHOLDS`. No verdict at all — not a raised threshold: "high usage is expected" + applies to every level, and the module's own rule is that a metric the table does not + know classifies to `None`, never `ok`. +2. `src/dashboards/sources/host.py` — stop classifying the GPU rows: drop the `busy_pct` / + `mem_used_pct` classifications from the per-GPU `severity` dict (remove the dict when it + becomes empty), leaving the `gpu.*` **samples** untouched — the history series stay + stored and queryable; removing the verdict is not removing the data. +3. `tests/card_contract.py` — pin the absence, the way it already pins `network`'s + colourlessness: the GPU card paths (`gpus[].model/busy_pct/vram_*/mem_used_pct`) stay, + the verdict paths go. `tests/test_system_contract.py` follows (all three captures + HTTP). +4. `tests/test_severity.py` — assert `classify("gpu.busy_pct", 99)` / + `classify("gpu.mem_used_pct", 99)` return `None`; keep the boundary-ordering test + consistent with the smaller table. `tests/test_host_source.py` — the GPU row no longer + carries a `severity` dict. +5. `src/dashboards/web/static/page-system.js` — the GPU hero renders without `severity` and + without the utilisation bar (a bar under a verdictless number is a verdict by shape); + drop the `data-sev="gpus[].severity.busy_pct"` hook. `n/a` stays uncoloured as before. +6. `AGENTS.md` — the `severity.py` section lists `gpus[].severity` among the rated cells; + update it so the file and the code agree (DoD: rules and fleet facts travel with the + change). + +- ASSUMPTION: "no verdict at all (muted)" was the operator's confirmed reading of the item + — the card shows busy % / VRAM with no colour class at any level, rather than "only fail + above a new ceiling". If the operator later wants a ceiling, it is one table row back. + +## Testing & Quality + +- Unit: `tests/test_severity.py` (no GPU row in the table; `None` verdicts), + `tests/test_host_source.py` (payload shape), `tests/test_system_contract.py` (the pinned + contract over the three captures + over HTTP). +- Coverage: **>90%** on this task's modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] `uv run pytest` green with the updated contract +- [ ] a live busy GPU (driveripper, R9700) renders its numbers with no colour class and no bar +- [ ] `gpu.busy_pct@N` still answers from `/api/system/history` (data intact, verdict gone) +- [ ] `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/06_system_page_gpu_verdicts/02_gpu_temp_on_hero_row.md b/.agents/phases/todo/06_system_page_gpu_verdicts/02_gpu_temp_on_hero_row.md new file mode 100644 index 0000000..55488f7 --- /dev/null +++ b/.agents/phases/todo/06_system_page_gpu_verdicts/02_gpu_temp_on_hero_row.md @@ -0,0 +1,44 @@ +# Task 02 — GPU temperature on the hero row + +**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:9` — "Pull GPU temp for the system resources tab where you can." +**Story:** n/a + +## Objective + +The GPU temperature is already collected end-to-end (agent `gpus[].temps` → payload +`gpus[].temp_c`, max over the card's hwmon labels, and a `gpu.temp_c@N` sample) but the +hero row never shows it — it sits in the collapsed "More detail" table. Move it up so the +one number that matters while a card is busy is glanceable. + +## Work + +1. `src/dashboards/web/static/page-system.js` — `gpuSub()`: append the temperature to the + GPU hero's `.sub` line when `gpu.temp_c != null` (e.g. + `VRAM 31.2 GB / 32 GB · pool 98% · 63 °C`); when the agent cannot read it (the APU + build on deskwork or a card with no hwmon input), the sub line simply does not print a + temperature — `n/a` is reserved for numbers the card is supposed to have, and "where you + can" is the item's own wording. +2. Keep the "More detail" table exactly as-is (it carries per-label temps, power, and both + memory pools); this is a surface change, not a dedup. +3. No backend changes: `tests/fixtures/agent_deskwork.json` / `agent_driveripper.json` + already carry `gpus[].temps`, so the hermetic page tests can exercise both the + with-temp and without-temp paths. + +- ASSUMPTION: the payload's `temp_c` (agent's max over hwmon labels) is "the" GPU temp — + the same max the agent already computes — so no new field, no new metric name, and the + `gpu.temp_c@N` history series keeps its existing name. + +## Testing & Quality + +- Unit: `node --check` on `page-system.js`; served-page assertions in `tests/test_api.py` + if the markup structure changes. +- Coverage: **>90%** on modified code (mostly JS; the Python side is unchanged). +- Browser (DoD): screenshot in `data/logs/` with live data — driveripper's R9700 shows a + temperature on the hero row; deskwork's APU row does not print a bogus one. + +## Completion Criteria + +- [ ] GPU hero `.sub` includes `· °C` on hosts whose agent reports temps +- [ ] hosts without a readable sensor show no temperature (not `0 °C`, not red) +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean +- [ ] screenshot in `data/logs/` against live data diff --git a/.agents/phases/todo/06_system_page_gpu_verdicts/03_collapse_host_history_charts.md b/.agents/phases/todo/06_system_page_gpu_verdicts/03_collapse_host_history_charts.md new file mode 100644 index 0000000..27bcfd5 --- /dev/null +++ b/.agents/phases/todo/06_system_page_gpu_verdicts/03_collapse_host_history_charts.md @@ -0,0 +1,49 @@ +# Task 03 — Per-host history charts collapse behind a toggle + +**Phase:** `06_system_page_gpu_verdicts` · **Source:** `TODO.md:4` — "The historical charts on the system resources page should only be visible if expanded. I only want to see current stats to keep things clean. There will be lots of hosts and historical charts make things look messy." +**Story:** n/a + +## Objective + +The default `/system` view is the hero row (current stats) only. Each host's `.host-charts` +stack (utilisation, network, GPU busy) moves behind a collapsed "Charts" toggle, so a +fleet that grows to many hosts reads clean at a glance — and a host that stops reporting +still loses its panel and charts (`pruneHosts`) exactly as today. + +## Work + +1. `src/dashboards/web/static/page-system.js` — wrap each host's `.host-charts` in a + native `
` (collapsed by default; `summary` reads "Charts" and carries the + window chip so the expanded state still says which window is drawn). Create the + `LineChart` instances **lazily on first expand** (the `state.charts` map already keys + them) and re-run the draw pass on refresh **only for hosts whose charts are open** — + drawing into a collapsed container is wasted work, and the page's one- + `/api/system/history`-request-per-host discipline is unchanged (the request still names + every series the panel could draw; only the paint is deferred). +2. `pruneHosts()` keeps destroying chart instances with the panel; add the `details` + element to the removal so a dead host leaves no orphan toggle. +3. `src/dashboards/web/static/dash.css` — style `summary` as a quiet toggle in the existing + vocabulary (no new palette, no `.sev-*` family); the collapsed stack takes no height. +4. `tests/test_api.py` — pin the served markup: one `
` per host panel, collapsed + by default (no `open` attribute), summary text "Charts"; the orphan-`id` rule keeps + passing (delete no rule that a live selector still uses). +5. `node --check` on both scripts. + +- ASSUMPTION: the toggle is **per host** (not one global switch) and **collapsed by + default** — "only visible if expanded" is the item's own wording, and per-host matches + the page's one-panel-per-host discipline; the window chips (5 m … 30 d) stay where they + are and keep driving the same single history request. + +## Testing & Quality + +- Unit: `tests/test_api.py` served-bytes assertions (details present, closed by default). +- Coverage: **>90%** on modified code; overall gate (94%) maintained. +- Browser (DoD): screenshot in `data/logs/` — collapsed by default on a multi-host page, + one host expanded with its three charts drawn, a second refresh while expanded repaints. + +## Completion Criteria + +- [ ] default load shows hero rows only; no chart pixels before a toggle is opened +- [ ] expanding a host draws its charts on the first open and on every later refresh while open +- [ ] a pruned (dead) host removes its toggle along with its panel and charts +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/07_llm_monitor_style_pages/00_phase.md b/.agents/phases/todo/07_llm_monitor_style_pages/00_phase.md new file mode 100644 index 0000000..0a01e92 --- /dev/null +++ b/.agents/phases/todo/07_llm_monitor_style_pages/00_phase.md @@ -0,0 +1,55 @@ +# Phase 07 — The LLM page looks like llm_monitor again + +**Source:** `TODO.md` L5 — "make this look like the llm_monitor project": "The LLM dashboard is nothing like ~/Experimental/llm_monitor. I'm missing the 'slots' tables (I really liked that). I'm missing the aggregate throughput at the top. Basically, make this look like the llm_monitor project. The gufo slots table that exists isn't even working and I want it gone anyway." +**Story:** n/a +**Context:** `~/Projects/Experimental/llm_monitor/static/index.html` (the reference: fleet-totals section ~L710, shared stats strip ~L743, gufo slot cards `buildSlotCard`/`renderSlots` ~L1179, `/slots` passthrough with `/v1/slots` fallback ~L750), `src/dashboards/web/pages/llm.html` + `src/dashboards/web/static/page-llm.js` (phase-04 layout: per-engine heroes, `CELLS` sparkline list, alarm chips, four trend charts), `src/dashboards/sources/llm.py` (`fleet_totals()` ~L327 — already sums decode/prompt tok/s, running/waiting, token totals; `_sessions()` passes gufo's raw `/slots` rows through as `sessions.list`), `scripts/capture-fixtures.sh` (today captures engine `/metrics` only — **no `/slots` capture exists**, which is why the slot table never matched the live shape) + +## Objective + +Port `llm_monitor`'s visual structure onto the LLM page — the aggregate fleet block at the +top, the per-engine stats strip, the slot-card grid — while keeping what phase 04 and the +locked UI conventions own: the per-engine panel (two throughput heroes, compact sparkline +cells, alarm chips, the four half-height trend charts) and every provenance/severity rule. +The broken gufo slot table is gone and its replacement is fed by a `/slots` shape pinned +from the live server, never from memory. + +`llm_monitor` is the design reference, not a dependency: a separate app, nothing imported. + +- ASSUMPTION (phase, confirmed with the operator): "full restyle" = port the three + llm_monitor structures above (aggregate block, stats strip, slot grid). The per-engine + panel and chart set stay; `llm_monitor`'s peak-ctx cell does **not** come over because + the payload publishes no peak-context number — a shape is not a verdict, and a missing + number prints `n/a`, never a borrowed one. + +## Dependencies + +- `06_system_page_gpu_verdicts` (todo) — same web assets (`dash.css`, `app.js`, page + scripts); land in that order so the CSS token vocabulary is settled first. +- `04_llm_chart_readability` (complete) — the layout this phase reworks; its NOTES + (`NOTES-chart-set.md`) and provenance/severity conventions bind. + +## Tasks + +1. `01_slots_schema_and_table_removal.md` — delete the broken slot table; extend `capture-fixtures.sh` to record gufo's live `/slots` and pin the shape in a fixture + contract test. +2. `02_fleet_totals_block.md` — render the aggregate block at the top from the existing `fleet_totals()` payload (hidden below two engines, as today). +3. `03_engine_stats_strip.md` — port the shared stats strip: one card per engine (KV bar, tok/s, sessions; gufo extras; alarm chips), fed only by payload fields that already exist. +4. `04_slot_grid.md` — the llm_monitor slot grid: one keyed card per gufo slot, updated in place, from `sessions.list`. + +## Testing & Quality + +- Hermetic: `tests/test_api.py` key-path pins (fleet block, strip, grid markup on served + bytes), `node --check` on every touched script, fixture-driven `sessions.list` shape test. +- Live: `uv run pytest -m live --live` — the `/slots` fixture test re-asserts against the + running gufo (`deskwork:8731`), and the page renders with both engines live and with a + single engine (one clean half, not a red panel). +- Browser (DoD): side-by-side with `llm_monitor` on its own port against the same live + engines; screenshots in `data/logs/` (two-engine, single-engine, gufo-busy states). +- Coverage: **>90%** on new/modified code, ≥94% overall gate maintained. + +## Completion Criteria + +- [ ] the old `#slots-card` table is gone from HTML, CSS and JS (and from any test that pinned it) +- [ ] `tests/fixtures/slots_gufo.json` exists, is produced by `./scripts/capture-fixtures.sh`, and the contract test fails if the live shape drifts +- [ ] top aggregate block shows summed decode/prompt tok/s, running/waiting, token totals — hidden below two engines +- [ ] stats strip + slot grid render on live data; single-engine page is one clean half +- [ ] `bash .agents/validate.sh` green; screenshots in `data/logs/` diff --git a/.agents/phases/todo/07_llm_monitor_style_pages/01_slots_schema_and_table_removal.md b/.agents/phases/todo/07_llm_monitor_style_pages/01_slots_schema_and_table_removal.md new file mode 100644 index 0000000..66224e2 --- /dev/null +++ b/.agents/phases/todo/07_llm_monitor_style_pages/01_slots_schema_and_table_removal.md @@ -0,0 +1,52 @@ +# Task 01 — Remove the broken slot table; pin the /slots schema from the live server + +**Phase:** `07_llm_monitor_style_pages` · **Source:** `TODO.md:5` — "The gufo slots table that exists isn't even working and I want it gone anyway." +**Story:** n/a + +## Objective + +Delete the slot table that has never matched gufo's live `/slots` shape, and replace +"trust a memory of an upstream" (rule 7) with a recorded ground truth: a fixture captured +from the running gufo, pinned by a contract test. + +## Work + +1. `scripts/capture-fixtures.sh` — add a gufo `/slots` capture: dial + `${GUFO_BASE_URL%/}/slots` with the `/v1/slots` fallback (the same order + `llm_monitor`'s passthrough at ~L750 uses — a server that only exposes the alias must + still be capturable), save verbatim as `tests/fixtures/slots_gufo.json`. Skip cleanly + (with a note) when `GUFO_BASE_URL` is unset, matching the script's existing engine + handling. +2. Delete the broken table end-to-end: `#slots-card` / `#slots-empty` / `#slots-table` + markup in `src/dashboards/web/pages/llm.html`, `renderSlots()` and the dead helpers in + `src/dashboards/web/static/page-llm.js`, and the CSS rules in `dash.css` that only that + table used (the orphan rule for both `id`s and selectors — `tests/test_api.py` enforces + both). Any test that pinned the table's markup or its guessed field names + (`slot.state`, `slot.n_prompt`, `slot.decode_per_second`, …) gets the pin removed or + turned into the new fixture-based assertion. +3. `tests/` — add the shape contract: a table of the `/slots` field paths the slot grid + (task 04) will read, asserted against `tests/fixtures/slots_gufo.json` hermetically and, + in a `live`-marked test, against the running gufo — the same device as + `tests/card_contract.py`. `tests/test_llm_source.py` keeps asserting that + `sessions.list` is the passthrough (raw rows, no re-mapping in `sources/llm.py`'s + `_sessions()` — the grid, not the source, decides what a slot field means). +4. `./scripts/capture-fixtures.sh` — run it against the live fleet and commit the new + fixture (serials/PII scrubbed, same as the existing captures). + +- ASSUMPTION: the fixture is the *live* shape — whatever the running gufo answers today — + not a README's; if the live server is idle (zero busy slots) at capture time, the + capture still records the idle slot rows, which is the shape the grid must handle. + +## Testing & Quality + +- Unit: fixture-based shape test (hermetic), `node --check` on `page-llm.js`, + `tests/test_api.py` orphan-`id`/orphan-selector rules still pass after the deletion. +- Live: the `live`-marked shape test against `deskwork:8731`. +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] no `#slots-table` / `#slots-card` / `renderSlots` remains in HTML, JS or CSS +- [ ] `tests/fixtures/slots_gufo.json` is in the repo and reproducible via `./scripts/capture-fixtures.sh` +- [ ] the shape contract fails when the fixture (or the live server) drifts +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/07_llm_monitor_style_pages/02_fleet_totals_block.md b/.agents/phases/todo/07_llm_monitor_style_pages/02_fleet_totals_block.md new file mode 100644 index 0000000..04a492a --- /dev/null +++ b/.agents/phases/todo/07_llm_monitor_style_pages/02_fleet_totals_block.md @@ -0,0 +1,54 @@ +# Task 02 — Aggregate throughput block at the top + +**Phase:** `07_llm_monitor_style_pages` · **Source:** `TODO.md:5` — "I'm missing the aggregate throughput at the top." +**Story:** n/a + +## Objective + +The page leads with the fleet: one block above the engine panels with the summed numbers +llm_monitor's `#fleet` section shows — decode/prompt tok/s, running/waiting, token totals, +engines answering — which `fleet_totals()` in `sources/llm.py` already computes over the +engines that answered. Phase 04 retired the summed display to counts only; this task brings +the sums back in llm_monitor's shape. + +## Work + +1. `src/dashboards/web/pages/llm.html` + `page-llm.js` — replace the current `#totals` + card (counts only) with the llm_monitor-style fleet block: status dot, "all engines", + an `N engines · …` tag, the note "sums only engines answering now", and the + `running / waiting / prompt tok/s / decode tok/s / total tokens` row, rendered from + `fleet_totals()` via the existing payload (`/api/llm/snapshot` → `fleet`). + Provenance discipline stays: the sums carry no severity (throughput has no + good/bad — the same rule phase 04 wrote), and a dead engine's last-good values + contribute nothing to a total presented as current (`engines_live` vs `engines_total` + makes that visible, exactly as `fleet_totals()`'s docstring promises). +2. Keep the existing visibility rule — hidden below two engines (a cross-engine sum of one + engine duplicates that engine's own panel; the page comment at `#totals` says so and + `llm_monitor` hides the same way). +3. `tests/test_api.py` — pin the `fleet` key paths the block reads + (`engines_total`, `engines_live`, `decode_tokens_per_second`, + `prompt_tokens_per_second`, `requests_active`, `requests_waiting`, `tokens_total`) with + and without targets configured, and the served-markup presence/absence with one vs two + engines. +4. `dash.css` — block styling from the existing token vocabulary (`.metrics`/`.metric` + patterns already in the file; no new palette). + +- ASSUMPTION: no backend change — `fleet_totals()` already sums exactly the numbers the + block prints; if a wanted number turns out to be missing from the payload, it stays + `n/a` rather than inventing a new metric (rule 2). + +## Testing & Quality + +- Unit: `tests/test_api.py` (key paths + markup, one vs two engines), `node --check`. +- Live: `uv run pytest -m live --live` with both engines; the block sums the two live + panels' numbers (a hand-check against the per-engine values on the same screen). +- Browser (DoD): screenshots in `data/logs/` — two-engine (block visible, sums correct), + single-engine (block absent, one clean half). +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] the top block prints summed decode/prompt tok/s, running/waiting, token totals from `fleet_totals()` +- [ ] hidden below two engines; a dead engine dims the tag and contributes nothing to the sums +- [ ] no severity colour anywhere on the block +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/07_llm_monitor_style_pages/03_engine_stats_strip.md b/.agents/phases/todo/07_llm_monitor_style_pages/03_engine_stats_strip.md new file mode 100644 index 0000000..a789087 --- /dev/null +++ b/.agents/phases/todo/07_llm_monitor_style_pages/03_engine_stats_strip.md @@ -0,0 +1,54 @@ +# Task 03 — Per-engine stats strip + +**Phase:** `07_llm_monitor_style_pages` · **Source:** `TODO.md:5` — "Basically, make this look like the llm_monitor project." (the shared stats strip, `index.html` ~L743) +**Story:** n/a + +## Objective + +Port llm_monitor's shared stats strip — one card per engine between the fleet block and +the engine panels — so the read-across-the-room numbers (throughput, KV pressure, +sessions, the gufo extras) sit in one row instead of being scattered inside the panels. +The panels keep their detail; the strip is the summary. + +## Work + +1. `src/dashboards/web/pages/llm.html` + `page-llm.js` — one stat card per configured + engine: status dot + name + timestamp, the KV-cache bar (the one rated cell — colour + from the payload's `severity`, the existing `scale: 100` ratio→percent conversion), + prompt tok/s and decode tok/s (with the provenance tags, uncoloured), total tokens, + and the session counts (gufo: active slots via `sessions.list` length/`active`; vLLM: + running/waiting from its count metrics). +2. Gufo-only extras on the gufo card, vLLM-only rows on the vLLM card — the existing + `only` scoping in `CELLS` is the model: prefix-cache hit %, spec-accept % + (gufo), queue time (vLLM). **No peak-ctx cell** — the payload publishes no + peak-context number (phase assumption; a missing number prints `n/a` or the row is + absent, never a borrowed one). +3. `preemptions` / `device_lost` remain **alarm chips that exist only while non-zero** + (phase-04 rule, `.chip.fail` pattern) — on the strip card, not in a permanent cell. +4. `dash.css` — the strip is a new layout of existing vocabulary (`.card`, `.metrics`, + the KV bar already has its tokens); no new palette, no `.sev-*` family. +5. `tests/test_api.py` — pin the strip's served markup (one card per engine, gufo/vLLM + scoping) and the payload field paths it reads; `node --check`. +6. The engine panels keep their hero row and the sparkline cells the strip *does not* + show (latency means + p95s, queue mean); cells that end up rendered in **both** strip + and panel get demoted from the panel to the strip — one value, one home (the + "one value drawn twice" rule from the UPS hero row). + +- ASSUMPTION: the strip renders whatever the payload already sends — no new field in + `sources/llm.py` — so a single-engine deployment shows one card and the hidden + fleet block, and nothing on the strip is ever a number the upstream did not publish. + +## Testing & Quality + +- Unit: `tests/test_api.py` (markup + field paths), `node --check`. +- Live: both engines, then single-engine (one clean card, no red half). +- Browser (DoD): screenshot in `data/logs/` — strip visible, KV bar coloured from the + payload verdict, alarm chip present only in a non-zero state. +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] one card per configured engine, between the fleet block and the panels, with KV bar + tok/s + sessions +- [ ] no value printed twice (strip vs panel); gufo/vLLM rows scoped correctly +- [ ] alarm chips absent at zero, present and red while non-zero +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/07_llm_monitor_style_pages/04_slot_grid.md b/.agents/phases/todo/07_llm_monitor_style_pages/04_slot_grid.md new file mode 100644 index 0000000..5dcc757 --- /dev/null +++ b/.agents/phases/todo/07_llm_monitor_style_pages/04_slot_grid.md @@ -0,0 +1,53 @@ +# Task 04 — The gufo slot grid + +**Phase:** `07_llm_monitor_style_pages` · **Source:** `TODO.md:5` — "I'm missing the 'slots' tables (I really liked that)." +**Story:** n/a + +## Objective + +Bring back what the operator liked about llm_monitor: one card per gufo slot, keyed and +created once, updated in place on refresh — fed by `sessions.list` (the raw `/slots` +passthrough whose shape task 01 pinned in `tests/fixtures/slots_gufo.json`). + +## Work + +1. `src/dashboards/web/pages/llm.html` + `page-llm.js` — implement the grid with + llm_monitor's `buildSlotCard`/`renderSlots` idiom (~L1179/L1209): a `.slot` card per + slot id (keyed `Map`, created once, patched in place — a busy slot does not get + rebuilt and re-announced on every poll), the card top shows the slot id + model, and + the per-slot fields come **only from the pinned fixture's shape** (state/busy + marker, prompt + generated token counts, cache share, per-slot speed — the exact keys + the fixture records; a field the live shape does not carry prints `n/a`, never a + guessed fallback chain like the old table's). +2. The grid lives in the gufo section under the "sessions (GET /slots)" label; the + empty state is the honest one (no slots answered → the panel says so, it does not + render a card of nothing). A queued (deferred) request is **not** in `/slots` — the + existing note under the grid stays (it is information, not flavour). +3. vLLM section: keep the one-line note that vLLM exposes no per-slot JSON and its + metrics are the equivalent of the session grid — no grid, no fake rows. +4. `dash.css` — `.slot` / `.slot-top` / `.slot.busy` styling ported to the existing token + vocabulary (busy state is a border/glyph, not a new palette entry); `node --check`. +5. `tests/test_api.py` — pin the grid markup on served bytes (container, empty state, + label); the fixture-driven render path is covered by the shape contract from task 01. + +- ASSUMPTION: the slot card reads the fixture's fields one-to-one (no `a ?? b` fallback + chains — the old table's `slot.n_prompt ?? slot.prompt_tokens` style is exactly the + guessing that made it "not even working"); if a live capture shows two spellings of one + fact, the contract names the canonical one and the other is not rendered. + +## Testing & Quality + +- Unit: fixture-driven render test (the pinned shape through `renderSlots`-equivalent + logic, via served-bytes assertions), `node --check`. +- Live: gufo with at least one active request (a card with real numbers) and an idle + server (cards show idle, no invented speed). +- Browser (DoD): screenshots in `data/logs/` — idle, one busy slot, multiple busy slots; + refresh in place (cards do not flicker/rebuild). +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] one keyed card per gufo slot, created once and updated in place across refreshes +- [ ] every printed field traces to a key in `tests/fixtures/slots_gufo.json` (no fallback chains) +- [ ] idle server: honest empty/idle state; vLLM: note only, no grid +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/08_events_paging_and_copy/00_phase.md b/.agents/phases/todo/08_events_paging_and_copy/00_phase.md new file mode 100644 index 0000000..abe2745 --- /dev/null +++ b/.agents/phases/todo/08_events_paging_and_copy/00_phase.md @@ -0,0 +1,44 @@ +# Phase 08 — Event feeds page; the flavour text goes + +**Source:** `TODO.md` L6, L7 — "page the event tables; strip the flavour text": "All the events tables need to be scrollable and paginated. They are causing the page to be massive." · "Get rid of most of the flavor text and add a note to AGENTS.md to not add it unless necessary. It just comes off as LLM generated slop, not helpful information." +**Story:** n/a +**Context:** the three `ul.events` feeds (`src/dashboards/web/pages/system.html` L58, `llm.html` L103, `ups.html` L70-72), `Dash.eventText()` in `src/dashboards/web/static/app.js`, the page-head sub-lines with the quoted flavour text (`system.html` L45, `index.html` L73), `AGENTS.md` → *UI conventions*, `tests/test_api.py` (served-bytes pins, orphan-`id` rule) + +## Objective + +A dead host's 500-event history must not make `/system` take a page to scroll: every +event feed becomes a bounded, scrollable, paginated window. The descriptive copy that +explains the app to the operator ("agent-reported values; rates are measured over the Δ +window…") goes away page by page, and `AGENTS.md` records the rule so the slop does not +crawl back in. + +## Dependencies + +- `07_llm_monitor_style_pages` (todo) — same page files and `app.js`/`dash.css`; this + phase lands on the restructured `/llm` page, not the phase-04 one. +- `06_system_page_gpu_verdicts` (todo) — rewrites `/system`'s panel structure; event-feed + work must not fight it for the same markup. + +## Tasks + +1. `01_paginate_event_feeds.md` — one shared pager: the three `ul.events` become scrollable containers with a 50-per-page window and a prev/next + count footer; no API change. +2. `02_strip_flavor_text.md` — audit all four pages and the page scripts for descriptive copy, delete what is not functional, and pin the rule in `AGENTS.md`'s UI conventions. + +## Testing & Quality + +- Hermetic: `tests/test_api.py` served-bytes pins (pager markup, no removed flavour + strings still present), `node --check` on touched scripts, the orphan-`id` rule + (the pager introduces `id`s — every one must be referenced). +- Live: `uv run pytest -m live --live` — the event endpoints still answer; a feed with + more than 50 stored events (the real event table has history) paginates instead of + dumping. +- Browser (DoD): screenshots in `data/logs/` — a long feed bounded and scrollable, page + 2 reachable, the removed sub-lines gone from the page heads. +- Coverage: **>90%** on modified code, ≥94% overall gate maintained. + +## Completion Criteria + +- [ ] all three feeds scroll in a bounded container and paginate 50-per-page with a count footer +- [ ] no removed flavour string survives in served HTML or JS +- [ ] `AGENTS.md` carries the flavour-text rule under UI conventions +- [ ] `bash .agents/validate.sh` green; screenshots in `data/logs/` diff --git a/.agents/phases/todo/08_events_paging_and_copy/01_paginate_event_feeds.md b/.agents/phases/todo/08_events_paging_and_copy/01_paginate_event_feeds.md new file mode 100644 index 0000000..34d6832 --- /dev/null +++ b/.agents/phases/todo/08_events_paging_and_copy/01_paginate_event_feeds.md @@ -0,0 +1,56 @@ +# Task 01 — Event feeds: scrollable and paginated + +**Phase:** `08_events_paging_and_copy` · **Source:** `TODO.md:6` — "All the events tables need to be scrollable and paginated. They are causing the page to be massive." +**Story:** n/a + +## Objective + +Every event feed (`/system`, `/llm`, `/ups`) renders a bounded, scrollable window over the +events instead of a growing `ul` that makes the page take a scroll to read. One shared +helper, three call sites — a pager that only works on one page is the bug this shape +prevents. + +## Work + +1. `src/dashboards/web/static/app.js` — add `Dash.eventsPager(el, events, opts)`: renders + the visible page of `events` into the feed's `ul` (keeping the existing + `Dash.eventText()` row rendering and per-row subject-once rule untouched), with a + footer: `1–50 of N` and prev/next controls (disabled at the ends, `aria`-labelled). + Page size defaults to 50; the feed container is the scrollable element, the page + stays short. +2. Call sites — `page-system.js`, `page-llm.js`, `page-upts.js` → `page-ups.js`: each + replaces its direct `ul#events` population with the pager (the "loading…" / empty / + not-configured states keep their current behaviour; pagination only applies when there + is more than one page of data). On a refresh that brings new events, the pager resets + to the first page (an operator reading page 3 of stale history should not silently land + on page 3 of the new set). +3. `src/dashboards/web/static/dash.css` — `.events` gets a bounded height (scrollable + container, existing token vocabulary); the pager footer styled as a quiet `.faint` + strip. No new palette. +4. No API change: `/api/events` keeps answering exactly what it does (server-side list, + no `page`/`limit` params added — the feeds are page-local and the endpoint's + contract is pinned by `tests/test_api.py`). +5. `tests/test_api.py` — pin the served markup (the three feeds keep their `id`s; the + pager's new `id`s/`aria-*` are all referenced — the orphan rule stays green), and + `node --check` on all touched scripts. + +- ASSUMPTION: **client-side** pagination (50-per-page, scrollable container) is the + confirmed reading — the event tables grow with history and the server already serves + the list; a server-side `page`/`limit` API change would re-open a pinned endpoint + contract for a page-local problem. + +## Testing & Quality + +- Unit: `tests/test_api.py` (markup pins), `node --check`; a fixture-events render check + (page 1 of 3 pages, footer counts, disabled prev on page 1). +- Live: `uv run pytest -m live --live` — event endpoints unchanged. +- Browser (DoD): screenshot in `data/logs/` — a feed with >50 events bounded and + scrollable, page 2 showing rows 51–100. +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] all three feeds paginate 50-per-page with a `1–50 of N` footer and prev/next +- [ ] the page height no longer grows with event history; the container scrolls +- [ ] refresh resets to page 1; loading/empty/not-configured states unchanged +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/08_events_paging_and_copy/02_strip_flavor_text.md b/.agents/phases/todo/08_events_paging_and_copy/02_strip_flavor_text.md new file mode 100644 index 0000000..42981ac --- /dev/null +++ b/.agents/phases/todo/08_events_paging_and_copy/02_strip_flavor_text.md @@ -0,0 +1,56 @@ +# Task 02 — Strip the flavour text; pin the rule in AGENTS.md + +**Phase:** `08_events_paging_and_copy` · **Source:** `TODO.md:7` — "Get rid of most of the flavor text and add a note to AGENTS.md to not add it unless necessary. It just comes off as LLM generated slop, not helpful information. Like 'agent-reported values; rates are measured over the Δ window in the footer, and each panel's charts cover only that host' or 'every number is scraped from the box it belongs to · null renders as n/a'" +**Story:** n/a + +## Objective + +The pages stop narrating themselves. Copy that explains the app's own design to the +operator ("agent-reported values; rates are measured over the Δ window…", "every number is +scraped from the box it belongs to · null renders as n/a") is deleted; copy that is +*functional* — a source tag, a `Δ10.0s` provenance, a "not configured" explanation, the +SIMULATED badge, an error that says why a panel is empty — stays. `AGENTS.md` records +the rule so it is a convention, not a mood. + +## Work + +1. Audit, page by page: `src/dashboards/web/pages/index.html`, `system.html`, `llm.html`, + `ups.html` and the `src/dashboards/web/static/page-*.js` / `app.js` strings. The two + lines the TODO quotes (the `system.html` L45 sub-line, the `index.html` L73 sub-line) + are confirmed deletions; every other candidate is judged on one test: *would the + operator act differently without it?* No → delete. Kept by default: the `Δwindow` + footnotes (measurements, rule: "a Δ label is a measurement, never a config value"), + the per-rate provenance tags ("scraped / measured Δcounter/Δt / estimated" — dropping a + tag is a regression, not a simplification), `n/a`-and-why states, the "queued (deferred) + requests are not in /slots" note, and any string a test pins as functional. +2. Delete the confirmed flavour from markup *and* from JS that injects the same copy; + a sub-line that becomes empty removes its container, not an empty ``. +3. `tests/test_api.py` — where a deleted string was pinned, update the pin; add an + assertion that the two quoted strings (and the other confirmed deletions) are **absent** + from served bytes, so the slop cannot creep back in silently. +4. `AGENTS.md` → *UI conventions* — add the rule: **Flavour text is not helpful + information.** Pages do not narrate their own data pipeline or design; copy exists + only when the operator needs it to act (a why-this-panel-is-empty, a provenance tag, + a measurement label). The two deleted lines are the cited examples. (DoD: AGENTS.md + updates with rule changes — this is the task that does it, not the conversion.) +5. `node --check` on every touched script. + +- ASSUMPTION: "most of the flavour text" = the audit rule above (act-differently test), + with the two quoted lines as the floor of deletions — the operator's examples, not the + ceiling. If a borderline string survives the audit, it survives because it is a + measurement or a why, and the audit notes say which. + +## Testing & Quality + +- Unit: `tests/test_api.py` (absence pins for the deleted strings; served-bytes shape + unchanged otherwise), `node --check`. +- Browser (DoD): screenshot in `data/logs/` — the four page heads without the removed + sub-lines; no empty containers. +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] the two quoted lines (and the rest of the audit's deletions) are gone from served HTML and JS +- [ ] functional copy (provenance tags, Δ footnotes, empty-state explanations) intact +- [ ] `AGENTS.md` carries the flavour-text rule with the deleted lines as cited examples +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/09_disks_smartctl/00_phase.md b/.agents/phases/todo/09_disks_smartctl/00_phase.md new file mode 100644 index 0000000..713e42d --- /dev/null +++ b/.agents/phases/todo/09_disks_smartctl/00_phase.md @@ -0,0 +1,52 @@ +# Phase 09 — A /disks page for physical disk health (smartctl) + +**Source:** `TODO.md` L8 — "a truly physical disk monitoring page": "Add a 'disks' tab dedicated to smartctl and per-disk metrics. Note: not per-partition, but truly a physical disk monitoring page that collects data about all my physical disks across my fleet (should only be deskwork and driveripper right now)." +**Story:** n/a +**Context:** `agent/host_agent.py` (physical whole disks via `/sys/block`, ~L400 — already partition-free; stdlib-only, `ProtectSystem=strict`), `src/dashboards/config.py` (env only, sources default to unset), `src/dashboards/poller.py` (one task per source, stale/last-good, events under the dashboard name), `src/dashboards/store.py` (WAL, rollups raw→1m→1h, retention — rule 9), `src/dashboards/api.py` (GETs, 404-when-unconfigured without dialling), the three existing dashboards + `/` overview (a new tab joins the nav and the overview's "which of the three" table) + +## Objective + +A fourth dashboard: `/disks` — one card per **physical** disk (the agent's whole- +`/sys/block` enumeration, never partitions), each carrying `smartctl` health: overall +health, temperature, power-on hours, and the sector/wear alarms. Its own `disks` source +and poller task at a slow 5-minute cadence (SMART is not a per-second signal), its own +API (`/api/disks/snapshot`, `/api/disks/history`), rollups and retention, and a page that +reads `n/a` for what `smartctl` cannot see — never red for a missing binary. + +- ASSUMPTION (phase, confirmed): the agent reads SMART **directly** (`smartctl` as the + agent user, e.g. `disk` group) — no sudo — and `DISKS_INTERVAL_SECONDS` defaults to + **300** s. `smartctl` absent or a disk unreadable → that disk's `smart` is `null` + with an error entry; the page says "smartctl not available" dimmed (invariants 3/4: + unconfigured is not broken; a dead source must not blink another). + +## Dependencies + +- `08_events_paging_and_copy` (todo) — sequence only: it settles `app.js`/`dash.css` + (the pager + copy audit) that the new page builds on; no hard code dependency. + +## Tasks + +1. `01_agent_smartctl_collection.md` — `host_agent.py` gains a `smartctl` pass over its existing whole-disk list (per-device `smartctl -i -A -H`, stdlib `subprocess`, timeout-bounded); fixture from the real disks; deploy docs. +2. `02_disks_source_poller_store.md` — the `disks` source (payload + `disk.*@dev` samples), its own poller job at `DISKS_INTERVAL_SECONDS` (default 300), rollups + retention in `store.py`. +3. `03_disks_api_and_config.md` — `/api/disks/snapshot` + `/api/disks/history`, config wiring, `404 {"configured": false}` without dialling, severity rows for the health/temperature metrics. +4. `04_disks_page_and_overview.md` — the `/disks` page (per-disk cards, alarm chips, per-host history charts), nav tab + overview entry + header status dot. + +## Testing & Quality + +- Hermetic: fixture-driven agent tests (`tests/test_agent.py` + recorded `smartctl` + output), `tests/test_disks_source.py` (new), `tests/test_api.py` key-path pins (with + and without config), `node --check`. +- Live: `uv run pytest -m live --live` — the agent on deskwork/driveripper answers + `smart` fields for its real disks; **live tests must not write** (no `smartctl -t`, no + self-test on hardware something is plugged into). +- Browser (DoD): screenshots in `data/logs/` — per-disk cards on live data, the + "smartctl not available" state, the overview with the new tab. +- Coverage: **>90%** on new/modified code, ≥94% overall gate maintained. + +## Completion Criteria + +- [ ] `GET /api/disks/snapshot` answers per-disk `smart` data for every physical disk the agents report; `404 {"configured": false}` with no agents configured, without dialling +- [ ] `/api/disks/history` serves the `disk.*@dev` series through the existing rollups +- [ ] the `/disks` page renders one card per physical disk (never per-partition) with health, temp, power-on hours and non-zero alarm chips +- [ ] a host without `smartctl` (or a denied read) shows dimmed `n/a` cards, never a red page +- [ ] `bash .agents/validate.sh` green; screenshots in `data/logs/` diff --git a/.agents/phases/todo/09_disks_smartctl/01_agent_smartctl_collection.md b/.agents/phases/todo/09_disks_smartctl/01_agent_smartctl_collection.md new file mode 100644 index 0000000..e92247d --- /dev/null +++ b/.agents/phases/todo/09_disks_smartctl/01_agent_smartctl_collection.md @@ -0,0 +1,71 @@ +# Task 01 — The agent collects smartctl per physical disk + +**Phase:** `09_disks_smartctl` · **Source:** `TODO.md:8` — "…collects data about all my physical disks across my fleet" (the collection half) +**Story:** n/a + +## Objective + +`agent/host_agent.py` answers SMART data for every physical disk it already enumerates +(whole `/sys/block` devices — the existing partition-free list), via a direct, +timeout-bounded `smartctl` call, without ever breaking the agent: no binary, a denied +read, or a wedgy device degrades that disk's `smart` to `null` with an `errors[]` +entry and leaves the rest of the snapshot intact. + +## Work + +1. `agent/host_agent.py` — a `smart()` pass inside the existing disk collection: for each + physical whole disk (the existing `/sys/block` enumeration, skipping the same + virtual/`dm-*` entries it already skips), run `smartctl -i -A -H ` via stdlib + `subprocess` with a per-call timeout (ASSUMPTION: **5 s per device**; the whole pass is + bounded so a wedgy disk cannot stall the snapshot). Parse, from the recorded output: + `model`/`serial` (from `-i`, falling back to the agent's existing disk identity when + smartctl's identity block is absent), `overall_health` (OK/FAILED), `temp_c` + (air-temperature line, absent on many SSDs → `None`), `power_on_hours`, + `reallocated_sector_ct`, `current_pending_sector_ct`, `offline_uncorrectable` (media + errors), `percentage_used` (SSD wear). Attach as `smart: {…}` per disk row in the + `GET /snapshot` docstring and payload. +2. Failure semantics: `smartctl` missing → **one** `errors[]` entry per snapshot (not one + per disk), every disk's `smart` is `null`; a single denied/wedgy device → only that + disk's `smart` is `null` with its own error entry. The agent's `/health` stays up: + SMART is a best-effort add-on, exactly like the existing per-section `_safe()` + handling. +3. Stdlib-only is kept (no new agent dependency; `subprocess` is stdlib, consistent with + the locked "our own stdlib host agent" decision). `GET /info` is unchanged (smartctl + is a per-snapshot capability, not identity). +4. Fixtures (rule 7 — never trust a memory of an upstream): record `smartctl -i -A -H` + output from the **real** disks on deskwork and driveripper (one representative SSD and + one HDD if the fleet has both) into `tests/fixtures/smartctl_.txt`, captured by + an extension of `scripts/capture-fixtures.sh` (same script, same PII/serial scrubbing + as the existing captures — a serial scrubbed in a fixture is not a serial in the + browser). The parse tests assert against these verbatim captures; a `smartctl` version + drift that breaks a field fails the fixture test before a dashboard does. +5. `tests/test_agent.py` — a fixture-rooted collector case: happy path (fields present), + binary-missing (one error entry, all `smart: null`), single-device failure (only that + disk null), non-physical device skipped. +6. Deployment docs: `deploy/install-agent.sh` (or its README section) and `README.md` note + the requirement — `smartmontools` installed and the agent user able to read SMART + (e.g. `disk` group); absence is a documented graceful degradation, not an install + failure (DoD: fleet facts travel with the change). + +- ASSUMPTION: **direct read, no sudo** — `smartctl` runs as the agent user (confirmed + with the operator); if a host's permissions deny it, that host's disks degrade to the + "smartctl not available" state, and the fix is a group grant, not a sudoers entry + (command-scoped sudo exists for NUT, but SMART reads do not need root and the agent's + `ProtectSystem=strict` unit stays as-is). + +## Testing & Quality + +- Unit: `tests/test_agent.py` against the recorded fixtures (all four failure/happy modes + above); the real-`/proc` test keeps touching only the box running pytest (existing + policy — a box without `smartctl` exercises the binary-missing path, which is itself + a valid assertion). +- Live: `uv run pytest -m live --live` — the live agents' snapshots carry `smart` fields + for the real disks (read-only; no `smartctl -t` anywhere in the suite). +- Coverage: **>90%** on the agent's new code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] `GET /snapshot` disk rows carry `smart` (model/serial/health/temp/power-on/sectors/wear) on a host with a readable `smartctl` +- [ ] missing binary → one error entry + `smart: null` everywhere; one bad device → only that disk null; agent `/health` stays up +- [ ] `tests/fixtures/smartctl_*.txt` in the repo, reproducible via `./scripts/capture-fixtures.sh`, serials scrubbed +- [ ] `uv run pytest` green; `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/09_disks_smartctl/02_disks_source_poller_store.md b/.agents/phases/todo/09_disks_smartctl/02_disks_source_poller_store.md new file mode 100644 index 0000000..059bde4 --- /dev/null +++ b/.agents/phases/todo/09_disks_smartctl/02_disks_source_poller_store.md @@ -0,0 +1,66 @@ +# Task 02 — The `disks` source, its slow poller job, and the store's rollups + +**Phase:** `09_disks_smartctl` · **Source:** `TODO.md:8` — the storage + polling half of the physical-disk dashboard +**Story:** n/a + +## Objective + +`disks` becomes a first-class source: one payload per host (the physical disks and their +`smart` fields), one `disk.*@dev` sample set per poll, its **own** asyncio poller task at +a slow 5-minute cadence — SMART is not a per-second signal, and a dead `disks` job must +age its numbers in the UI exactly like any other source (invariant 4), with the store's +existing rollups and retention doing the history work (rule 9). + +## Work + +1. `src/dashboards/config.py` — `DISKS_INTERVAL_SECONDS` (default **300**; env-only, the + existing pattern). The `disks` source is *configured* when at least one host agent + target is configured — it reads the same `/snapshot` the `system` source does, so it + has no targets of its own; with no host targets it is unconfigured and 404s without + dialling (invariant 3). +2. `src/dashboards/sources/disks.py` (new) — snapshot → payload per host: the physical + disk rows (device, model, size, rotation from the agent's existing `disks` fields) + plus `smart` (task 01's fields, `None` where the agent had nothing — the page prints + `n/a`, never `0`), plus a `smartctl_available` flag per host so the page can say + *why* a card is `n/a` instead of leaving it a mystery. Samples (dotted, `@dev` stream + suffix, the existing naming scheme — the history API's key): + `disk.health_ok@` is **not** a series (it is a verdict, not a number); the + series are `disk.temp_c@`, `disk.power_on_hours@`, + `disk.reallocated@`, `disk.pending@`, `disk.uncorrectable@`, + `disk.percentage_used@` (a count/percent metric, not a rate — no `Δ` window + applies). Events under the **dashboard** name `disks` (the phase-03 lesson: a page + filtering `/api/events` by source gets an empty feed otherwise). +3. `src/dashboards/poller.py` — register the job (one task per source, its own timeout + and health record; the 300 s cadence means the stale threshold — 3× its own interval — + is 15 min, and `alerts.py`'s existing ok-but-stale `warn` rule covers it with no + change: the alert reads the job's own interval). +4. `src/dashboards/store.py` — the new series follow the existing rollup path + (raw → 1m → 1h) and retention; at a 300 s cadence the raw rows are 288/day/host/disk — + the retention story is the existing one, but the task must verify the rollup window + arithmetic handles a 300 s raw resolution (a 1-minute rollup bucket straddles two + polls; the existing "latest wins" semantics must keep holding, asserted by a test). +5. `tests/test_disks_source.py` (new) — fixture-snapshot → payload/samples (fields present, + `null` where the agent had none, `smartctl_available` false → all `n/a`), the 300 s + cadence in the job registration, the unconfigured-without-dialling 404 path, the + rollup-bucket test. + +- ASSUMPTION: `power_on_hours` and the sector counters are stored as **counters-as + reported** (monotonic by nature) — no `Δ` rate is derived; a rate the upstream does not + publish is `n/a` (rule 2). `disk.health_ok` (OK/FAILED) lives in the payload only; + colour comes from `severity.classify` on the numeric rows (task 03). + +## Testing & Quality + +- Unit: `tests/test_disks_source.py` (the full matrix above), `tests/test_poller.py` + additions (the job's own timeout/health, a failed poll keeps the last good payload and + marks stale — the invariant-4 behaviour on the new job), `tests/test_store.py` + (rollup/retention on the new series at 300 s resolution). +- Coverage: **>90%** on new/modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] `uv run pytest` green with the new source/job/store tests +- [ ] with no host targets, the `disks` job is unconfigured and 404s without dialling +- [ ] a failed `disks` poll ages the payload (stale in the UI) instead of zeroing it +- [ ] 1-minute rollup buckets are correct at 300 s raw resolution (asserted, not assumed) +- [ ] `uv run ruff check src tests agent && uv run pyright` clean diff --git a/.agents/phases/todo/09_disks_smartctl/03_disks_api_and_config.md b/.agents/phases/todo/09_disks_smartctl/03_disks_api_and_config.md new file mode 100644 index 0000000..c1e03eb --- /dev/null +++ b/.agents/phases/todo/09_disks_smartctl/03_disks_api_and_config.md @@ -0,0 +1,62 @@ +# Task 03 — The disks API, config wiring, and the severity rows + +**Phase:** `09_disks_smartctl` · **Source:** `TODO.md:8` — the API half of the physical-disk dashboard +**Story:** n/a + +## Objective + +`/api/disks/snapshot` and `/api/disks/history` answer like every other dashboard's +endpoints (GETs only, 404-when-unconfigured, redacted error strings, the health record in +`/api/health`), and the numbers that *are* worth a verdict get one row in +`severity.py` — the only place thresholds live — while the alarm counters get none +(a non-zero sector count is an incident, not a graded number: presence, like the +`/llm` preemption chips, never a threshold). + +## Work + +1. `src/dashboards/api.py` — `GET /api/disks/snapshot` (per-host: disk rows + `smart` + + `smartctl_available`) and `GET /api/disks/history` (the `disk.*@dev` series through + the existing history mechanism; unknown names answer as empty series on purpose, the + same contract `/api/system/history` and `/api/llm/history` have). No POST anywhere on + this surface (invariant 1 — the one write in the app stays `POST /api/ups/command`). +2. `404 {"configured": false}` with **no dialling** when no host agent is configured + (invariant 3), asserted with a test that would catch a client being constructed before + the gate. `UpstreamError` messages carry redacted URLs only (rule 6 — the existing + `Target.display_url` path; the disks source reuses the host targets' redaction, so + nothing new leaves the process with a token). +3. `src/dashboards/severity.py` — the new rows (ASSUMPTION — proposed bounds, pinned by + test, adjustable in one place without touching the page): + - `disk.temp_c`: warn **50**, fail **70** (a HDD spinning at 70 °C is a fire; an SSD + without a sensor simply has no value and gets no verdict — no measurement, no + verdict); + - `disk.percentage_used`: warn **80**, fail **100** (a 100 %-used SSD is a + replacement, not a warning). + The sector counters (`disk.reallocated`, `disk.pending`, + `disk.uncorrectable`) get **no row**: non-zero is an alarm chip, zero is the + absence of one (never green). +4. `src/dashboards/alerts.py` — no change expected: the ok-but-stale `warn` and the + dead-source `fail` rules read the job's own interval and reachability, and the + unconfigured-never-alerts rule covers the unconfigured case. **If** a test shows the + new job needs nothing, say so in the report; an alert with its own threshold is a + bug (the module is a pure projection). +5. `src/dashboards/app.py` — assembly wiring (settings/store injection, so tests build an + app with an in-memory DB and no targets, the existing pattern). +6. `tests/test_api.py` — key-path pins for `/api/disks/snapshot` (with and without + targets; a configured job with no payload yet lists the hosts as stubs, `n/a` + everywhere — the same stub shape `/api/system/snapshot` has), `/api/disks/history` + (known + unknown series), the 404-without-dialling case, and `/api/health` carrying + the new job's record. + +## Testing & Quality + +- Unit: `tests/test_api.py` (the pins above), `tests/test_severity.py` (the two new rows, + boundary ordering, the counter metrics deliberately absent from `THRESHOLDS`), + `tests/test_alerts.py` (the new job's stale/dead/unconfigured states). +- Coverage: **>90%** on modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] `curl /api/disks/snapshot` answers per-host disk + smart data (or `404 {"configured": false}` with no dial) +- [ ] `/api/disks/history` serves the `disk.*@dev` series; unknown names → empty series +- [ ] `classify("disk.temp_c", 75)` is `fail`, `classify("disk.reallocated", 1)` is `None` (presence, not a verdict) +- [ ] `bash .agents/validate.sh` green diff --git a/.agents/phases/todo/09_disks_smartctl/04_disks_page_and_overview.md b/.agents/phases/todo/09_disks_smartctl/04_disks_page_and_overview.md new file mode 100644 index 0000000..7881151 --- /dev/null +++ b/.agents/phases/todo/09_disks_smartctl/04_disks_page_and_overview.md @@ -0,0 +1,74 @@ +# Task 04 — The /disks page, the nav tab, and the overview entry + +**Phase:** `09_disks_smartctl` · **Source:** `TODO.md:8` — "Add a 'disks' tab dedicated to smartctl and per-disk metrics" (the page half) +**Story:** n/a + +## Objective + +The fourth dashboard: `/disks` — one card per **physical** disk (the agent's whole- +`/sys/block` enumeration; a partition is never a card), led by what the operator needs +while a disk is acting up: health, temperature, power-on hours, and the sector alarms — +plus per-host history charts under the same discipline as `/system` (collapsed by +default, phase 06; one history request per host per refresh; charts belong to the host +they chart). + +## Work + +1. `src/dashboards/web/pages/disks.html` + `src/dashboards/web/static/page-disks.js` + (new) — the page, in the existing vocabulary (masthead nav + a `Disks` tab with + `aria-current` on itself, `.card`, `.heroes`-free layout — the disk card leads with + the disk identity, not a big number): + - **per-disk card**: host + device + model (+ size / rotation from the agent's + existing fields), a health line (OK/FAILED from `smart.overall_health`, coloured + from the payload's severity verdict — a FAILED health is `fail` by the existing + `disk.percentage_used`/`temp_c` rows or the explicit health field; **never** a + verdict the API did not send), temperature, power-on hours, percentage-used where + the device has it (HDDs may not — `n/a`, uncoloured); + - **alarm chips that exist only while non-zero** (the `/llm` `.chip.fail` idiom): + reallocated, pending, uncorrectable — a disk at zero shows no chip row at all, + because a permanent "0 pending sectors" row trains the eye to ignore it; + - **`smartctl` unavailable** (per host, `smartctl_available: false`): the host's + cards render dimmed (`.stale`) with `n/a` and a one-line why — "smartctl not + available on " — never a red panel (invariant 3); + - **per-host chart section** (phase-06 discipline): collapsed `
` "Charts" by + default, lazy-drawn `LineChart`s for `disk.temp_c@` (and + `disk.percentage_used@` where present) through **one** + `/api/disks/history` request per host per refresh naming every series it could draw; + a host that stops reporting loses its cards *and* charts (`pruneHosts` pattern). +2. `src/dashboards/web/pages/*.html` — the `Disks` tab joins the masthead nav on all + four pages (the orphan-`id` rule: every new `id` is referenced); the overview's + "which dashboard do you need" table gains the `/disks` row + ("are any of my physical disks failing, and which one") and the header's + per-source status dot picks up the `disks` job. +3. `src/dashboards/web/pages/index.html` + `page-index.js` — `/api/summary` is the + overview's single call; add the `disks` payload to its contract **additively** + (the existing key paths stay pinned — `tests/test_api.py` fails if a summary key + retires), and the Fleet/Warnings panels treat the `disks` source like the others + (an alert from `alerts.py` needs nothing new; a dead `disks` job is one line). +4. `src/dashboards/web/static/dash.css` — the disk-card + chip styling from existing + tokens (no new palette, no `.sev-*` family); `node --check` on every touched script. +5. `tests/test_api.py` — pin the served markup (nav tab on all four pages, overview row, + the page's card/empty/stale states) and the `/api/summary` additive key path. + +- ASSUMPTION: the page's hero per card is **identity + health**, not a single big number + (a disk has no one-number story the way a UPS has charge %); the numbers that matter + (temp, hours, wear) are equal-size `.metric` cells, and the verdicts come only from + task 03's `severity` rows — `n/a` where the device does not publish the field. + +## Testing & Quality + +- Hermetic: `tests/test_api.py` (markup + summary contract), `node --check`, the + orphan-`id`/orphan-selector rules. +- Live: `uv run pytest -m live --live` (read-only); the real fleet today has physical + disks on deskwork and driveripper — the page must show both hosts' cards. +- Browser (DoD): screenshots in `data/logs/` — live cards on both hosts, the + "smartctl not available" state (a host with the binary removed, or a `smartctl_available: + false` fixture), collapsed charts expanded, the overview with the new tab + row. +- Coverage: **>90%** on new/modified code; overall gate (94%) maintained. + +## Completion Criteria + +- [ ] `/disks` renders one card per physical disk on live data (never per-partition), with health/ temp/hours/wear and non-zero-only alarm chips +- [ ] a `smartctl`-less host is dimmed with a one-line why — never red +- [ ] nav tab on all four pages; the overview's table lists `/disks`; the header dot tracks the job +- [ ] `bash .agents/validate.sh` green; screenshots in `data/logs/`