phase: 03_system_per_host_panels
**Phase 03 final verification pass — all 4 tasks already in `complete/`; verified criteria, fixed 3 pre-existing defects.**
- **Verified live** (headless Chromium + curl against `:8090`, all 6 sources `ok`): 3 panels × six heroes in order `cpu,mem,temp,disk,gpus,net`; charts per panel `util,net[,gpu]` (gateway: no GPU chart/hole); legends name metrics only — no legend names a foreign host; every `<svg>` `role="img"` + host-scoped `aria-label`; 2 sections, 0 `<select>`; `grep -rn "renderCores|core_usage" src/dashboards/web/` → nothing; console error-free; new shot `data/logs/final-system-panels.png`.
- **Defect fixed (credential hygiene, invariant 6)**: `collectors.py` put the *dialed* URL into every `UpstreamError`, and that string lands in the health record (`/api/health`) and the `event` table (`/api/events` → page); `/api/config` also echoed `GUFO_BASE_URL`/`VLLM_BASE_URL` verbatim. Added `config.strip_userinfo()` (now what `Target.display_url` uses) and redacted all message/config URLs. Tests: `test_a_failing_host_never_echoes_its_token_in_the_message` (×3 paths), `test_a_failing_engine_never_echoes_credentials_in_its_url`, `test_a_dead_host_never_writes_its_token_into_the_feed`, `test_engine_urls_are_redacted_the_same_way_hosts_are`, `test_strip_userinfo_is_the_one_redaction_every_url_leaves_by`. (`parse_targets` already moves `HOST_TARGETS` userinfo into a header, so this is a backstop for hosts and a real fix for engine URLs.)
- **Defect fixed (invariant 2)**: `/system` footnote printed `Δ{HOST_INTERVAL}` as the measurement; it now prints each host's measured `window_seconds` (range when they disagree) — `rates over Δ10.0s · poll 10s`. Pinned by `test_system_footnote_prints_the_window_it_actually_measured`. Also fixed the page's stale "window in each header" note and the empty-state's `--token` (argv) advice → `--token-file`.
- `bash .agents/validate.sh` → **EXIT=0**: **472 passed, 7 skipped**, coverage **96.87%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` clean · `uv run pytest -m live --live` → **7 passed** · `uv pip list --outdated` empty.
- Criteria: no multi-host legend ✅ · no per-core visual ✅ · six heroes/panel incl. no-GPU `n/a` uncoloured ✅ · per-core history still queryable ✅ (354 pts) · ≥3 shape screenshots ✅ (`system-driveripper/deskwork/gateway.png` + dark/light/stale) · pytest+coverage+lint ✅ · phases 01/02 unchanged (their pin tests + `/api/summary` shape tests pass; live UPS gap in AGENTS.md re-confirmed read-only against real upsd) ✅.
- **Docs**: AGENTS.md — invariant 6 rewritten to cover message/event redaction, `collectors.py` line, new "Δ is a measurement" rule, "hosts answering total" → "nine-figure fleet-totals panel" (the subtitle still counts hosts), test count 452→472, coverage 96.8→96.9, rewrapped an over-long line. README.md — same fleet-totals wording, `RATE_WINDOW_SECONDS` now covers every measured rate (not just tokens), agent route list completed (`/`, `/schema`, `?top=N`), `LIST CMD` in the diagram, new paragraph on URL redaction.
- **Deviation**: the criterion's literal `curl …/api/system/history?host=…&metric=cpu.usage_pct@core7` uses singular params that endpoint has never accepted (it takes `hosts=`/`metrics=` and ignores unknown ones). Verified 354 points via `…system/history?hosts=driveripper&metrics=cpu.usage_pct@core7&range=1h` and `/api/history?source=host&host=…&metric=…`; I did **not** add singular param aliases (unauthorized API surface). No live infra changes made — reads only.
- **Next pending phase**: `04_llm_chart_readability`.
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
# Phase 03 — System page is per-host panels
|
||||
|
||||
**Source:** `TODO.md` L4 — "The system page doesn't need shared charts for anything. It should only have per-host charts." · L5 — "Per host charts need to be simpler. Larger numbers, proper green/yellow/red color coding. Don't need per-core. Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Anchors* (dropped visuals are intentional, series stay in SQLite), `01_severity_language`, `src/dashboards/web/pages/system.html`, `src/dashboards/web/static/page-system.js`, `src/dashboards/sources/host.py`
|
||||
|
||||
## Objective
|
||||
|
||||
`/system` stops drawing seven fleet-wide charts and a "pick a host, pick a metric" detail
|
||||
chart. It becomes one panel per host: six big colour-coded numbers — cpu %, mem %, cpu temp,
|
||||
disk usage %, GPU usage + VRAM, network up/down — and that host's own chart underneath.
|
||||
|
||||
Nothing per-core anywhere on the page, and nothing that requires a selection to make sense.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — the cards render `severity` from the payload; they do not
|
||||
get to keep their own thresholds.
|
||||
- `02_overview_fleet_and_alerts` (todo) — the fleet-wide "what stands out" role moved to the
|
||||
overview's alerts list, which is what makes removing the shared charts safe.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_host_card_payload_contract.md` — prove the six card metrics already exist in `/api/system/snapshot` and pin them with tests (add fields only if genuinely missing).
|
||||
2. `02_hero_cards_without_per_core.md` — rewrite `hostCard()`/`renderHostCard()` into big colour-coded numbers, no `renderCores`.
|
||||
3. `03_per_host_charts_replace_shared.md` — delete the fleet-chart and series-detail sections; give each host its own compact charts.
|
||||
4. `04_verify_every_host_shape.md` — browser verification across a discrete GPU, an APU, and a box with no GPU.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Python: contract tests for the payload fields the cards read; coverage **>90%** on any
|
||||
changed Python (ideally none — this phase is frontend plus tests).
|
||||
- `uv run pytest -m live --live` must stay green: `test_every_host_agent_answers_and_parses`
|
||||
is what proves the fields exist on all three real hosts, including the ones that report
|
||||
`n/a`.
|
||||
- Browser, live data, every host shape: driveripper (discrete R9700), deskwork (Strix Halo
|
||||
APU — VRAM *and* GTT, so "3% VRAM" and "78% memory" are both true), gateway (no GPU at all
|
||||
→ the honest "no GPU reported by this agent" state).
|
||||
- The `cpu.usage_pct@coreN` series must still be stored and still be queryable from
|
||||
`/api/system/history` — removing the plot is not removing the data.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/system` contains no chart whose legend lists more than one host
|
||||
- [ ] no per-core visualisation anywhere on `/system` (`grep -rn "renderCores\|core_usage" src/dashboards/web/` → nothing, or charts.js-only if kept unused on purpose)
|
||||
- [ ] each host panel shows exactly: cpu %, mem %, cpu temp, disk %, GPU busy + VRAM (or the no-GPU state), network ↓/↑
|
||||
- [ ] `curl -s "localhost:8090/api/system/history?host=driveripper&metric=cpu.usage_pct@core7&range=1h"` still returns points
|
||||
- [ ] three screenshots in `data/logs/`, one per host shape
|
||||
- [ ] `uv run pytest` green, coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,50 @@
|
||||
# Task 01 — Host card payload contract
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:5` — "Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Establish, with tests, that everything the new cards need is already served — so the card
|
||||
rewrite is a rendering change and nobody "fixes" a field that was never missing.
|
||||
|
||||
## Work
|
||||
|
||||
1. Read the current payload in `parse_host_snapshot()` (`src/dashboards/sources/host.py`)
|
||||
and confirm the six card metrics map to:
|
||||
- cpu % → `cpu.usage_pct`
|
||||
- mem % → `memory.used_pct`
|
||||
- cpu temp → `temperatures.cpu_c` **and** `cpu.temp_c` (both exist; pick one and say which
|
||||
in the card code comment — `cpu.temp_c` is the one `/api/summary` already uses)
|
||||
- disk usage % → `disks.worst_used_pct` + `disks.worst_mount` (the mount is what makes the
|
||||
number actionable)
|
||||
- GPU → `gpus[].busy_pct`, `gpus[].vram_used_bytes`, `gpus[].vram_total_bytes`,
|
||||
`gpus[].vram_used_pct`, `gpus[].mem_used_pct` (the honest APU view), `gpus[].model`
|
||||
- network ↓/↑ → `network.rx_bytes_per_s`, `network.tx_bytes_per_s`
|
||||
plus the phase-01 `severity` sub-dicts (`cpu.severity`, `memory.severity`,
|
||||
`disks.severity`, `gpus[].severity`).
|
||||
2. `tests/test_system_contract.py` (new) — one test that runs a real fixture snapshot
|
||||
(`tests/fixtures/agent_driveripper.json`) through `parse_host_snapshot()` and asserts each
|
||||
of the paths above exists (keys present, values may be `None`), and one that runs the
|
||||
`agent_gateway.json` fixture (no GPU) asserting `gpus == []` rather than a missing key.
|
||||
3. Add the **only** payload change this phase is allowed, if the card needs it:
|
||||
`disks.worst.severity` must be present (it comes from phase 01) and
|
||||
`network` gets no severity (throughput has no good/bad — say so in a comment so nobody
|
||||
adds a meaningless green later).
|
||||
4. Note in the test docstring what is *deliberately* not on the card: per-core usage
|
||||
(still stored as `cpu.usage_pct@coreN`), swap, processes, sensors, per-interface rows.
|
||||
- ASSUMPTION: the card reads `cpu.temp_c`; if the executor finds it absent where
|
||||
`temperatures.cpu_c` is present on a real host, use `temperatures.cpu_c` and record the
|
||||
choice in the commit message rather than changing the payload.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- New tests green; they must fail if a field is renamed, which is the point.
|
||||
- `uv run pytest -m live --live` for the same assertions against the real fleet.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest tests/test_system_contract.py -v` green
|
||||
- [ ] no payload field renamed or removed (`git diff src/dashboards/sources/host.py` is empty or adds only a comment)
|
||||
- [ ] `uv run pytest`, `ruff`, `pyright` clean
|
||||
- [ ] no visual change yet — this task changes no page code
|
||||
@@ -0,0 +1,60 @@
|
||||
# Task 02 — Hero cards, no per-core
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:5` — "Per host charts need to be simpler. Larger numbers, proper green (good), yellow (warn), red (danger) color coding. Don't need per-core. Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Each host panel leads with six big colour-coded numbers and nothing else in the way.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-system.js` — `hostCard()`:
|
||||
- replace the two `.metrics` blocks with a single hero row built from the phase-01 `.hero`
|
||||
markup: CPU %, MEM %, TEMP °C, DISK %, GPU %, NET ↓/↑ (network is one hero with two
|
||||
lines — in and out — rather than two competing numbers);
|
||||
- **delete the per-core block** (`<div class="label faint">Per-core</div>` and
|
||||
`data-k="cores"`);
|
||||
- drop from the top row: Load 1/15, Clock GHz, Disk read/write/busy, Up — into the
|
||||
existing `<details>` (see below), not deleted from the page's reach;
|
||||
- keep `data-k` names for whatever survives so `renderHostCard()` keeps working, and give
|
||||
each hero a `data-sev` hook for the severity class.
|
||||
2. `renderHostCard()`:
|
||||
- drive colour from `host.cpu.severity.usage_pct`, `host.memory.severity.used_pct`,
|
||||
`host.disks.severity.worst_used_pct`, `gpu.severity.busy_pct` — via
|
||||
`Dash.severityFrom(...)`; delete the `severityFor(value, 75, 90)` calls on
|
||||
`page-system.js:129/139/144`;
|
||||
- value `null` → `n/a` with **no** severity class;
|
||||
- memory hero shows `%` big, `200 / 253 GiB` as the `.sub` line (the percentage is the
|
||||
verdict, the bytes are the detail);
|
||||
- disk hero shows `%` big, the mount (`disks.worst_mount`) as `.sub` — "80 % of what?"
|
||||
must be answerable without a click;
|
||||
- GPU block: one line per card — model, `busy %` with severity, `VRAM used / total` and
|
||||
the `.sub` pool view (`mem_used_pct`) for APUs. No GPU → the existing
|
||||
"No GPU reported by this agent" line;
|
||||
- network hero uses `Dash.rate()` on `network.rx_bytes_per_s` / `tx_bytes_per_s`, and
|
||||
renders `n/a` when `any_net` produced nulls rather than a confident `0 B/s`;
|
||||
- the existing `<details>` becomes "More detail": load averages, clock, disk read/write/
|
||||
busy, uptime, filesystems, top processes. Collapsed by default.
|
||||
3. `src/dashboards/web/static/dash.css` — add only what the hero row needs if phase 01 did
|
||||
not already (grid of 6 heroes, wraps to 2×3 and 3×2 at narrow widths; `.cores` rule can
|
||||
stay until task 03 confirms nothing else uses it).
|
||||
- ASSUMPTION: `temp` uses `cpu.temp_c` (see task 01). Hosts with no CPU sensor show `n/a`
|
||||
uncoloured — `gateway` in this fleet.
|
||||
- ASSUMPTION: the `.details`/`<details>` disclosure keeps the information that is not
|
||||
glance-worthy. The TODO asks for a simpler surface, not a less complete page.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- No JS tests (locked anchor). Contract is covered by `tests/test_system_contract.py`.
|
||||
- Browser: verify against live data on driveripper (GPU), and confirm the severity colours
|
||||
change when a value crosses a threshold — e.g. temporarily override with
|
||||
`THRESHOLDS` in a REPL or by pointing at a host under load. One screenshot per state.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] each host card shows exactly six heroes in this order: CPU, MEM, TEMP, DISK, GPU, NET
|
||||
- [ ] `grep -n "cores\|renderCores" src/dashboards/web/static/page-system.js` → no matches in the card code
|
||||
- [ ] no `severityFor(` call remains in `page-system.js`
|
||||
- [ ] a host with no GPU and no CPU temp renders `n/a` with no colour (screenshot)
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# Task 03 — Per-host charts replace the shared ones
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:4` — "The system page doesn't need shared charts for anything. It should only have per-host charts." and `TODO.md:5` — "Per host charts need to be simpler"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Delete the six fleet charts and the "pick a host / pick a metric" detail chart. Every chart
|
||||
that survives belongs to exactly one host and is inside that host's panel.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/system.html` — delete:
|
||||
- the whole `<section class="grid cols-2" aria-label="Fleet charts">` (L65–91):
|
||||
`#chart-cpu`, `#legend-cpu`, `#chart-mem`, `#legend-mem`, `#chart-disk`, `#legend-disk`,
|
||||
`#chart-net`, `#legend-net`, `#chart-gpu`, `#legend-gpu`, `#chart-temp`, `#legend-temp`;
|
||||
- the whole "Series detail" `<section>` (L93–107): `#detail-host`, `#detail-metric`,
|
||||
`#chart-detail`;
|
||||
- the `<section class="card flat" aria-label="Fleet totals">` (L40–63) — its job moved to
|
||||
the overview's fleet table + alerts in phase 02;
|
||||
- inside each host panel, add a chart slot: `<div class="chart sm" data-chart="util"></div>`
|
||||
and `<div class="chart sm" data-chart="net"></div>` (plus `data-chart="gpu"` rendered
|
||||
only when the host reports GPUs).
|
||||
2. `src/dashboards/web/static/page-system.js` —
|
||||
- delete `makeCharts()`'s seven `state.charts.*` constructions, `refreshCharts()`,
|
||||
`refreshGpuChart()`, and the `#detail-*` select wiring;
|
||||
- add `renderHostCharts(host)`: per host, **one** utilisation chart (cpu %, mem %,
|
||||
disk-worst %, shared 0–100 y-axis, three lines, one legend) and **one** network chart
|
||||
(↓ and ↑ on the same axis), plus a GPU utilisation chart when `host.gpus.length`;
|
||||
- source them from `/api/system/history` — the endpoint already returns per-host series
|
||||
(`{"hosts": [...], "series": [{"host", "metric", "points"}]}`), so this is a query per
|
||||
host-metric set, not a new endpoint; keep one request per host per refresh window, not
|
||||
one per metric, if the endpoint supports a metric list (it accepts
|
||||
`host=`/`metric=`/`range=` — reuse `seriesFor()`);
|
||||
- respect the existing range control (`state.range`) and `Dash.bindPause`;
|
||||
- height ~120px: these are supporting evidence under a number, not the main event.
|
||||
3. `src/dashboards/web/static/charts.js` — if `renderCores` now has no callers
|
||||
(`grep -rn "renderCores" src/dashboards/web/`), delete it and its CSS. Do not otherwise
|
||||
touch `LineChart`; phase 04 depends on it.
|
||||
4. `src/dashboards/web/static/dash.css` — remove `.cores` rules and any now-unused chart
|
||||
grid classes, verified by grep before deletion.
|
||||
- ASSUMPTION: three charts per host (utilisation, network, GPU-if-present) rather than
|
||||
one chart per metric. Six metrics × three hosts = eighteen charts on screen is the
|
||||
unreadable version of "per-host charts".
|
||||
- ASSUMPTION: no cross-host chart stays anywhere. Comparing hosts is the overview fleet
|
||||
table's job; `/system` answers "what is this machine doing".
|
||||
- ASSUMPTION: history retention and the stored series are untouched —
|
||||
`cpu.usage_pct@coreN` keeps being polled and stored, and remains queryable.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `tests/test_api.py`: if it asserts the removed chart endpoints, update it; the endpoints
|
||||
themselves stay (`/api/system/history` is used by the per-host charts and the live tests).
|
||||
- Browser, live data: chart per host, no legend with two host names in it, and the range
|
||||
chips (1h/6h/24h/7d/30d/90d) still re-fetch. Watch for the old bug class: a chart that
|
||||
keeps drawing after its host disappears from the payload.
|
||||
- Performance sanity: with three hosts the page must still poll smoothly — count the XHRs per
|
||||
refresh in devtools and keep it under ~6.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/system | grep -c 'id="chart-cpu"\|id="chart-detail"'` → 0
|
||||
- [ ] every `LineChart` on `/system` is inside a `[data-host]` panel
|
||||
- [ ] `curl -s "localhost:8090/api/system/history?host=driveripper&metric=cpu.usage_pct@core7&range=1h"` still returns points
|
||||
- [ ] `uv run pytest` green (including `-m live --live`), coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] screenshot of `/system` with all three hosts in `data/logs/`
|
||||
@@ -0,0 +1,49 @@
|
||||
# Task 04 — Verify every host shape in a browser
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:4–5` — the page must be right for *every* host, not just the one that was convenient to develop against
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the new `/system` page on all three real hosts — including the two shapes that usually
|
||||
get missed: an APU with two memory pools, and a box with no GPU and no CPU temperature.
|
||||
|
||||
## Work
|
||||
|
||||
1. Confirm the dev loop is live: `./scripts/dev-tunnels.sh status` (three agents + the upsd
|
||||
forward OK), `./scripts/dev-server.sh restart`, `/api/health` all `ok`.
|
||||
2. Verify and screenshot each host panel on `http://127.0.0.1:8090/system`:
|
||||
- **driveripper** — discrete Radeon AI PRO R9700: busy %, VRAM used/total (~32 GiB),
|
||||
power/temp where reported;
|
||||
- **deskwork** — Strix Halo APU: `vram_used_pct` low while `mem_used_pct` is high is
|
||||
*correct*, and the `.sub` pool line must make that legible rather than contradictory;
|
||||
- **gateway** — no GPU: the "No GPU reported by this agent" state, GPU hero `n/a` with
|
||||
**no** colour; CPU temp `n/a` likewise.
|
||||
3. Check the failure path, not just the happy path: stop one agent
|
||||
(`./scripts/dev-tunnels.sh stop` for a single host, or `ssh <host> "pkill -f '[h]ost_agent.py'"`)
|
||||
and confirm the panel keeps its last good numbers, marks them stale
|
||||
(`stalenessText`), turns its dot amber/red, and does **not** blank or zero the heroes.
|
||||
Screenshot it. Restore afterwards (`./scripts/dev-tunnels.sh start`).
|
||||
4. Accessibility pass on the new markup: hero values are real text (selectable, not SVG
|
||||
paths), each chart container has a label or `aria-label`, colour is never the *only*
|
||||
signal — the number itself must carry the meaning.
|
||||
5. Light and dark theme both checked (the header toggle), since severity colours come from
|
||||
theme tokens.
|
||||
6. Write a short verification note under this phase's directory as
|
||||
`NOTES-verify.md`: date, what was checked, what looked wrong (if anything), screenshots
|
||||
referenced by filename.
|
||||
- ASSUMPTION: the note lives next to the phase rather than in git history alone, so the
|
||||
next reader can tell what was actually seen on screen.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- This task produces evidence, not code. If verification finds a defect, fix it in this task
|
||||
and re-shoot; do not leave it as a note.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/system-driveripper.png`, `data/logs/system-deskwork.png`, `data/logs/system-gateway.png` exist and were opened/read before declaring success
|
||||
- [ ] a stale-host screenshot shows last-good values marked stale, not zeros
|
||||
- [ ] both themes checked; no colour-only signalling
|
||||
- [ ] `NOTES-verify.md` written in this phase directory
|
||||
- [ ] `uv run pytest` still green after any fix made during verification
|
||||
Reference in New Issue
Block a user