phase: 04_llm_chart_readability

**Phase 04 final verification pass — all tasks already in `complete/`; verified + fixed 3 defects found in verification.** - Verified `/llm` live (both engines, gufo-only instance on :8099, `llm_monitor` on :8095) in a real browser; dev server left running/healthy. - Completed task-04's truncated layout fix: `#engines` now carries `.items-start` (the CSS rule existed but nothing used it) — gufo 544 px vs vLLM 715 px, no blank card; pinned by `test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling`. - Fixed: event rows said the subject twice ("gufo gufo is answering again"), and `/ups` mangled mid-message names ("sent to apc: OK" → "sent to OK") → shared `Dash.eventText()` (strips `UpstreamError: ` + a *leading* name only, mirroring `alerts._clean_error`) used by `/system`, `/llm`, `/ups`; new test. - Fixed: count sparklines autoscaled rollup averages (queue max 0.1379) into full-height spikes → `sparkline({minSpan})`, `minSpan: 1` on sessions/queued (spike 30 px → 4 px); assertions added. - Tests: `bash .agents/validate.sh` → **EXIT=0** (`uv run pytest --cov --cov-fail-under=94`: **485 passed, 7 skipped**, coverage **96.87%**; `ruff check src tests agent`: All checks passed; `pyright`: 0 errors; `node --check` all static JS clean; repo hygiene clean; `uv pip list --outdated` empty). `uv run pytest -m live --live` → **7 passed**. - Completion criteria: heroes largest per engine ✔ (38.4 px vs 16.8 px row values); sparkline per cell + selected window ✔ (5/5 gufo, 7/7 vLLM; 6 h chip moved KV max 53 %→58 %); mean+p95 without hover ✔ (243 / 48.8 / 285 ms feet); provenance tag ✔ (`Δ10.0s`/`Δ8.0s` + `title`=exact source); single-engine ✔ (1 panel, totals hidden, zero red); screenshots ✔ `data/logs/llm-verify-{both-engines,charts-and-slots,single-engine,monitor-reference}.png` + `NOTES-chart-set.md`; pytest/cov/ruff/pyright ✔; earlier phases unchanged-behaviour ✔ (all 01–03 tests pass). - Deviation (documented, not code): throughput heroes stay **neutral ink** — `severity.py` has no throughput row and 0 tok/s idle is healthy; colour comes from rated cells (KV) + reachability. Kept per `NOTES-chart-set`. - Deviation: could not append the task-04 written verdict to `NOTES-chart-set.md` (it lives in `todo/`, which I must not edit). Verdict: ours reads better than `:8095` — bigger throughput numbers, per-cell shape over the selected window, percentiles inline; `llm_monitor` only sparklines decode+KV and has no window control. - Docs: AGENTS.md updated (485 test count, `sparkline` `minSpan` rule, `Dash.eventText` event-row rule, `/llm` panel `items-start`); README reviewed in full — commands, config table (all `config.py` vars present), architecture bullets, page descriptions verified accurate; no other inconsistencies found. - Note for the harness: `.agents/phases/todo/03…/NOTES-verify.md` and `todo/04…/NOTES-chart-set.md` are untracked notes left behind by their phases (task files moved, notes did not). - Next pending phase: `05_ups_hero_battery_glyph`.
2026-10-17 07:03:58 -04:00
parent 9757e3c947
commit f44a617ef9
42 changed files with 1428 additions and 194 deletions
@@ -0,0 +1,206 @@
+# NOTES-chart-set — what `/llm` is for, and what it shows
+
+**Phase 04, task 01 deliverable (no code changes).** Decides the cell set, the colour rules,
+and the contracts tasks 02–03 implement. Both implementations were read side by side on
+2026-10-17; every line number below was verified against the file it cites.
+
+## The two sources
+
+**Reference — `~/Projects/Experimental/llm_monitor/static/index.html`** (separate app on
+`:8095`; design reference only, never a dependency — ASSUMPTION kept: it is not ours to own,
+and importing it would break the no-build single-app structure):
+
+| What | Where |
+|---|---|
+| sparkline CSS vocabulary (`.sparks`, `.spark`, `.spark-now`, `.spark-axis`, `.spark-line`, `.spark-area`) | L398–495 |
+| `UI.historyLen = 60` (client ring buffer; `pushHist` L1542–1545; fed per engine L1570–1571, L1582–1583) | L1003 |
+| `drawSpark(name, values, format, fixedMax)` | L1092 (`SPARK_W = 100`/`SPARK_H = 30` L1090–1091) |
+| right-aligned growth — `const step = SPARK_W / (UI.historyLen - 1)`, `x = SPARK_W - (values.length - 1 - i) * step` | L1158–1163 |
+| empty state: `< 2` values → `empty` class, blank paths (never a zero line) | L1112–1118 |
+| flat-window honesty: pad keeps the line mid-plot, axis reports the one real value, not the pad (`const flat` L1128) | L1126–1133 |
+| top/middle/bottom scale marks (`axis()` helper, `levels`, `fmtAxis` with enough decimals that the three labels stay distinguishable; axis writes L1155–1157) | L1101–1104, L1135, L1138–1154 |
+| `fixedMax` ratio axes: KV sparks called with `fixedMax = 1` → 0–100% always | L1356 (gufo), L1420 (vLLM) |
+| per-engine throughput `dd`s: `#gufo-ptps` / `#gufo-dtps`, `#vllm-ptps` / `#vllm-dtps` | L757 / L761, L802 / L806 |
+| `setThroughput(el, cell)` — value + honest tag (`Δ<w>` for measured, `lifetime avg`, `from TPOT`, `derived`; `title` = exact `source`) | L1303–1320 |
+| percentile foot lines: `#vllm-ttft-p95`, `#vllm-tpot-p95`, `#vllm-queue-p95` under their mean cards | L950, L955, L960 |
+| token counters as cells whose foot line names the upstream series (`#vllm-ptok`, `#vllm-gtok`) | L964, L969 |
+| slot cards: `SLOT_FIELDS` L1170–1177, `buildSlotCard` L1179 (built once, patched in place), `renderSlots` L1209 (DOM order == slot order, vanished slots removed) | — |
+| the only thresholds in that app: KV bar warn ≥ 0.75, crit ≥ 0.9 | L1085–1086 |
+| single-engine case: fleet block hidden when one engine — "a cross-engine sum of one engine just duplicates that engine's own card" | L1500–1504 |
+| dead engine holds last good data + `stale` class + error banner, never zeroes | L1514–1535 |
+
+**Current page** — `src/dashboards/web/pages/llm.html` + `src/dashboards/web/static/page-llm.js`:
+
+| What | Where |
+|---|---|
+| engine-totals card (`#f-decode` L45, `#f-prompt` L47, KV bar L54–56, tokens lifetime L52–53) | `llm.html` L39–60 |
+| four full-height `LineChart`s `#chart-decode/prompt/kv/queue` | `llm.html` L62–81 (chart divs L66, L71, L75, L79) |
+| per-engine panels grid `#engines` + honest "No engines configured" empty state | `llm.html` L83–91 |
+| gufo slots table `#slots-card` L94–102; events feed L103–106; provenance footer L109–113 | `llm.html` |
+| `CELLS` — the 14-equal-size grid this phase dissolves | `page-llm.js` L16–33 |
+| `cellValue()` — per-key payload mapping | L35–93 |
+| `provenance()` — `Δ10.0s` / formula tag from `source`, nothing for scraped gauges | L96–105 |
+| `enginePanel()` / `renderPanel()` | L112–136 / L138–214 |
+| **JS thresholds to remove**: `severityFor(value.value, 75, 90)` on the KV bar | L194–195 |
+| device_lost non-zero special case (the alarm-chip seed) | L196–200 |
+| `makeCharts()` — `height: 165` for all four | L241–249 (165 at L242) |
+| `refreshCharts()` — three `/api/llm/history` calls; the third already carries `requests_running,requests_waiting,kv_cache_usage` | L256–292 (L264–265) |
+| **JS thresholds to remove**: `severityFor(kv, 75, 90)` on fleet KV | L307 |
+| window chips `Dash.rangeControl` (affects charts today, sparklines after this phase) | L374 |
+
+Shared helpers this builds on: `charts.js` `LineChart` L43, `renderLegend` L329 (with
+`showPeak`), `sparkline` L348–373 (the function task 02 upgrades); `app.js` `severityFor`
+L118 (documented FALLBACK ONLY — phases 04–05 exist to retire its remaining call sites),
+`severityFrom` L139, `hero()` L187.
+
+## Decision
+
+### KEEP (unchanged, do not regress)
+
+- **Provenance tags on every throughput number** — `provenance()` (`page-llm.js` L96–105)
+  and the footer legend (`llm.html` L109–113). AGENTS.md invariant 2; the `llm_monitor`
+  equivalent is `setThroughput` L1303–1320. Ours is strictly better (it prints the measured
+  Δ window); it ports as-is into the hero cells.
+- **History window chips** (`page-llm.js` L374, `Dash.rangeControl`). Divergence from the
+  reference on purpose: `llm_monitor` sparks use a fixed 60-sample client buffer (L1003);
+  our sparklines read `/api/llm/history` with the *selected* window, so chips and sparks
+  always agree. Do not copy `historyLen`.
+- **Per-engine panels** (`llm.html` L84) with per-source health, stale/last-good dimming,
+  and the upstream chip. Unconfigured engines render no panel at all (L85–90) — invariant 3.
+- **gufo slot list** (`llm.html` L94–102, `renderSlots` L216–239). Adopt from
+  `llm_monitor` only the *in-place patch* idiom (`buildSlotCard` L1179) if rows churn; the
+  table itself stays.
+- **Events card** (`llm.html` L103–106). Phase 02 deleted the raw feed from `/` only;
+  per-engine up/down transitions stay here.
+
+### HERO — the two largest numbers, per engine panel
+
+- `llm.decode_tokens_per_second` and `llm.prompt_tokens_per_second` (payload cells built in
+  `sources/llm.py`: gufo measured Δcounters, vLLM native gauge → measured → lifetime
+  estimate, each labelled with its `source`).
+- Rendered with `Dash.hero()` (`app.js` L187) with the `Δ<window>` provenance in `.sub`.
+- **Throughput has NO severity.** `severity.py` `THRESHOLDS` (L46–57) has no throughput row
+  and gets none: 0 tok/s on an idle server is not red, and a page-invented threshold is
+  exactly what invariant "a threshold exists nowhere else" forbids. The number renders in
+  neutral ink; colour in the hero row comes from the engine's reachability (status dot +
+  `.stale` dimming, already implemented in `renderPanel`), mirroring the `/system` NET slot
+  precedent ("throughput has no good/bad", AGENTS.md UI conventions). *Note: the phase
+  checklist says the heroes are "colour-coded" — the honest reading is the page's colour
+  language (neutral numbers, verdict colour on rated cells, reachability colour on the
+  panel). Task 04's side-by-side screenshots should show this; if the operator wants the
+  digits themselves tinted, that needs a ratified `severity.py` row first, not a JS guess.*
+
+### SPARKLINE CELLS — compact rows, one per metric, each with its window's history
+
+| Cell | Payload source (`cellValue`, L35–93) | History series for the spark | Colour |
+|---|---|---|---|
+| Sessions active | `sessions.active` (gufo `/slots`, vLLM count) | `llm.requests_running` ✔ stored | none |
+| Queued | `requests_deferred` / `num_requests_waiting` | `llm.requests_waiting` ✔ stored | none |
+| KV cache % | `kv_cache_usage_ratio` / `gpu_cache_usage` (×100) | `llm.kv_cache_usage` ✔ stored | **payload `severity`** |
+| TTFT | `time_to_first_token_seconds__mean`, foot: `__p95` | `llm.ttft_seconds` ✔ stored | none |
+| TPOT | `time_per_output_token_seconds__mean`, foot: `__p95` | `llm.tpot_seconds` — **new, see Python delta** | none |
+| Queue mean | `request_queue_time_seconds__mean` | `llm.queue_seconds` ✔ stored | none |
+| Prefix cache hit % | `prefix_cache_hit_ratio` / `prompt_cache_hit_ratio` | `llm.prefix_cache_hit_ratio` ✔ stored | fixed max 100 |
+| Spec accept % | `spec_accept_rate` (gufo only) | `llm.spec_accept_rate` — **new, see Python delta** | fixed max 100 |
+
+- Percentiles print as **foot lines, no hover** (reference: L950/L955/L960), from the live
+  snapshot cells — p95 needs no stored series.
+- TTFT/TPOT/queue are vLLM-only; spec accept is gufo-only. The other engine's row renders
+  `n/a` uncoloured (rule 2), or the row is engine-scoped — task 03's choice, but a
+  gufo panel must never show a red/empty latency mystery.
+- `/api/llm/history` (`api.py` L353–365) answers *any* metric name; unknown ones come back
+  as empty series — so the page can request the full cell list in one call. `refreshCharts`
+  (L256–292) already makes three calls; extend the third (L264–265) with the cell series
+  instead of adding requests, and feed both charts and sparks from the same responses.
+
+### CONDITIONAL / DEMOTED
+
+- **`tokens_total` (lifetime counter)** → quiet footnote line in each engine panel (the
+  existing `.sessions-note`/`.debug` strip, `page-llm.js` L203–213), like `llm_monitor`'s
+  counter cards whose foot line just names the series (L964, L969). A lifetime sum is not a
+  glance metric.
+- **`preemptions` and `device_lost`** → **alarm chips rendered only when non-zero, absent
+  otherwise** (seed: the `device_lost` special case L196–200). A permanent cell for
+  something that is always 0 trains the eye to ignore the row. Chip class is `.chip.fail`
+  (existing vocabulary, `dash.css` L336); the rule is *existence of an incident*, not a
+  graded threshold — `sources/llm.py` already treats any non-zero `gufo_device_lost_total`
+  as an incident (comment at `GUFO_SERIES`), and zero/null simply draws no chip.
+- **Engine-totals card (L39–60)**: keep engines-live, decode/prompt sums, active, queued —
+  demote tokens to footnote, and **remove the fleet KV bar** (L54–56 + `severityFor` at
+  L307): `fleet_totals` (`sources/llm.py` L325–368, `kv_cache_usage_max` L360–363) carries
+  no `severity`, and the rated per-engine KV cells plus the KV chart say it better. Hide the
+  totals card when fewer than two engines are configured — copy `llm_monitor` L1500–1504
+  ("a cross-engine sum of one engine just duplicates that engine's own card"); this also
+  satisfies the single-engine completion criterion.
+
+### CHARTS
+
+- All four `LineChart`s stay: decode, prompt, KV, queue — one engine per series (queue chart:
+  running+waiting per engine), existing legend `renderLegend` (`charts.js` L329) with
+  `showPeak` on the bursty throughput plots, units-bearing y-format (already passed).
+- **Drop `height: 165` → ~110** (`page-llm.js` L242) **and move the chart section below the
+  engine panels** so the cells read first (the task says half-height "so the cells read
+  first"; keeping full-height charts on top would contradict that).
+- Task 03 must pass `ariaLabel` to all four charts (AGENTS.md: every chart names itself;
+  `/system` did this in phase 03, `/llm` never adopted it).
+
+### COLOUR RULES (from `severity.py` and nowhere else)
+
+- **KV pressure**: rated. `THRESHOLDS["llm.kv_cache_usage_pct"] = (75, 90)`
+  (`severity.py` L53), attached per engine cell by `SEVERITY_CELLS` (`sources/llm.py`
+  L99–103) with `scale=100` because both engines publish a 0–1 ratio. The page reads it via
+  `Dash.severityFrom` (`app.js` L139) and **both `severityFor(..., 75, 90)` call sites
+  (L194–195, L307) are deleted** — the `app.js` L112–117 fallback comment stays true only
+  if phase 04 does this.
+- **Queue depth**: **not rated** — `THRESHOLDS` has no queue row (L46–57). `severityFrom`
+  returns `''` → no colour. The queue chart and sparkline carry the signal as *shape*. If
+  the operator wants a verdict, that is a future `severity.py` row (ratified numbers,
+  tests), never a JS threshold.
+- **Throughput, sessions, latency (TTFT/TPOT/queue mean)**: not rated → colourless numbers.
+  Explicit statement per the task: **0 tok/s on an idle server is not red.**
+- **Alarm chips**: `.chip.fail` when the counter is non-zero; absent otherwise (above).
+
+### SPARKLINE CONTRACT (input for task 02, mirroring `drawSpark` L1092–1176)
+
+`DashCharts.sparkline()` (currently L348–373: full-width autoscale, no labels) becomes:
+
+1. **Right-aligned growth**: newest sample pinned right, x mapped over the *selected time
+   window* (`[now − range, now]`, like `LineChart._xExtent`) rather than over sample index —
+   a partly-filled window grows in from the right exactly like `drawSpark`'s fixed-step
+   buffer (L1158–1163) but stays honest about *when* samples happened.
+2. **Fixed max** for ratio metrics (KV, prefix hit, spec accept: 0–100) so a squiggle means
+   the same thing on every screen — reference `fixedMax` calls L1356/L1420; autoscale
+   otherwise.
+3. **Current/max labels**: current value in the cell header (reference `.spark-now`, L419)
+   and window max as the label; where the reference prints three axis marks (L1135,
+   L1155–1157) our compact rows print `now` + `max` — adopt its formatting discipline:
+   enough decimals that labels stay distinguishable (L1137–1152), a flat window reports the
+   one real value, never the pad artifact (L1126–1133).
+4. **Severity colour**: stroke/area from the payload verdict (`ok`/`warn`/`fail` tokens from
+   `severityFrom`, unrated → `--series-1`), not `llm_monitor`'s fixed per-engine hue.
+5. **Empty**: < 2 finite points → "no history", never a zero line (reference L1112–1118;
+   current code already does this).
+
+## Python delta (the phase's only one, flagged for task 03)
+
+`HISTORY_METRICS` (`sources/llm.py` L113–123; sampled at L292–297) does **not** store TPOT
+or spec-accept, so those two cells would show permanent "no history" slots — the dead-cell
+anti-pattern this note exists to prevent. Decision: extend `HISTORY_METRICS` additively —
+
+- `tpot_seconds` ← `("time_per_output_token_seconds__mean",)` (vLLM only)
+- `spec_accept_rate` ← `("spec_accept_rate",)` (gufo only)
+
+New *names* in the public metric namespace (allowed: "phases may stop plotting a series,
+they may not rename or stop storing it" — adding is the safe direction), consumed by the
+generic `Sample("llm", engine, f"llm.{metric}", …)` loop (L292–297) and the store's generic
+rollups; retention needs nothing new. The phase overview says "Python: none *expected*" —
+this is a small, additive, tested exception; `/api/llm/*` fields keep their names/meanings
+(additive-only anchor holds), and task 03 adds a unit test per new series plus an
+`/api/llm/history` round-trip. Everything else in this phase is HTML/JS/CSS.
+
+## Non-goals
+
+- No changes to `/api/llm/snapshot` consumers outside this page, no `/ups`-style hero glyph
+  work (phase 05), no per-slot sparklines, no touching `llm_monitor` itself.
+- The stored-metric set is never pruned by this phase: cells leave the *page*, series stay
+  in SQLite (PLAN.md, dropped-visuals anchor).
@@ -0,0 +1,13 @@
+**Phase 04 final verification pass — all tasks already in `complete/`; verified + fixed 3 defects found in verification.**
+
+- Verified `/llm` live (both engines, gufo-only instance on :8099, `llm_monitor` on :8095) in a real browser; dev server left running/healthy.
+- Completed task-04's truncated layout fix: `#engines` now carries `.items-start` (the CSS rule existed but nothing used it) — gufo 544 px vs vLLM 715 px, no blank card; pinned by `test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling`.
+- Fixed: event rows said the subject twice ("gufo gufo is answering again"), and `/ups` mangled mid-message names ("sent to apc: OK" → "sent to  OK") → shared `Dash.eventText()` (strips `UpstreamError: ` + a *leading* name only, mirroring `alerts._clean_error`) used by `/system`, `/llm`, `/ups`; new test.
+- Fixed: count sparklines autoscaled rollup averages (queue max 0.1379) into full-height spikes → `sparkline({minSpan})`, `minSpan: 1` on sessions/queued (spike 30 px → 4 px); assertions added.
+- Tests: `bash .agents/validate.sh` → **EXIT=0** (`uv run pytest --cov --cov-fail-under=94`: **485 passed, 7 skipped**, coverage **96.87%**; `ruff check src tests agent`: All checks passed; `pyright`: 0 errors; `node --check` all static JS clean; repo hygiene clean; `uv pip list --outdated` empty). `uv run pytest -m live --live` → **7 passed**.
+- Completion criteria: heroes largest per engine ✔ (38.4 px vs 16.8 px row values); sparkline per cell + selected window ✔ (5/5 gufo, 7/7 vLLM; 6 h chip moved KV max 53 %→58 %); mean+p95 without hover ✔ (243 / 48.8 / 285 ms feet); provenance tag ✔ (`Δ10.0s`/`Δ8.0s` + `title`=exact source); single-engine ✔ (1 panel, totals hidden, zero red); screenshots ✔ `data/logs/llm-verify-{both-engines,charts-and-slots,single-engine,monitor-reference}.png` + `NOTES-chart-set.md`; pytest/cov/ruff/pyright ✔; earlier phases unchanged-behaviour ✔ (all 01–03 tests pass).
+- Deviation (documented, not code): throughput heroes stay **neutral ink** — `severity.py` has no throughput row and 0 tok/s idle is healthy; colour comes from rated cells (KV) + reachability. Kept per `NOTES-chart-set`.
+- Deviation: could not append the task-04 written verdict to `NOTES-chart-set.md` (it lives in `todo/`, which I must not edit). Verdict: ours reads better than `:8095` — bigger throughput numbers, per-cell shape over the selected window, percentiles inline; `llm_monitor` only sparklines decode+KV and has no window control.
+- Docs: AGENTS.md updated (485 test count, `sparkline` `minSpan` rule, `Dash.eventText` event-row rule, `/llm` panel `items-start`); README reviewed in full — commands, config table (all `config.py` vars present), architecture bullets, page descriptions verified accurate; no other inconsistencies found.
+- Note for the harness: `.agents/phases/todo/03…/NOTES-verify.md` and `todo/04…/NOTES-chart-set.md` are untracked notes left behind by their phases (task files moved, notes did not).
+- Next pending phase: `05_ups_hero_battery_glyph`.
@@ -0,0 +1 @@
+ok
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 14%]
+........................................................................ [ 29%]
+........................................................................ [ 43%]
+......................................sssssss........................... [ 58%]
+........................................................................ [ 73%]
+........................................................................ [ 87%]
+............................................................             [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   228
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+485 passed, 7 skipped, 1 warning in 44.38s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1 @@
+Now I have everything verified. Writing the decision note:
@@ -0,0 +1 @@
+error: assistant error: litellm.APIConnectionError: APIConnectionError: OpenAIException - Response payload is not completed: <TransferEncodingError: 400, message='Not…
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 15%]
+........................................................................ [ 30%]
+........................................................................ [ 45%]
+...........................sssssss...................................... [ 60%]
+........................................................................ [ 75%]
+........................................................................ [ 90%]
+...............................................                          [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   226
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+472 passed, 7 skipped, 1 warning in 43.29s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,7 @@
+**Task 01 complete — chart-set decision note written.**
+
+- **Implemented:** `.agents/phases/todo/04_llm_chart_readability/NOTES-chart-set.md` — keep/hero/sparkline-cell/conditional-demote/charts decisions with every claim cited by path+line from both `llm_monitor/static/index.html` (drawSpark L1092, right-aligned growth L1158–1163, fixedMax L1356/L1420, setThroughput L1303, historyLen L1003, percentile feet L950/955/960, slot cards L1179/L1209, single-engine hide L1500–1504) and this repo (`page-llm.js` CELLS L16–33 + `severityFor` sites L194–195/L307, `charts.js` sparkline L348, `llm.py` HISTORY_METRICS L113/SEVERITY_CELLS L99, `severity.py` L46–57); colour rules = KV rated via payload severity (75/90), queue depth & throughput explicitly unrated → colourless; sparkline contract for task 02; flagged one additive Python delta (store `llm.tpot_seconds`, `llm.spec_accept_rate` so no cell is a permanent "no history" slot).
+- **Tests/lint:** `uv run pytest` exit 0 (all passed, 7 live-skipped) · `--cov --cov-fail-under=94` → TOTAL 97% · `uv run ruff check src tests agent` clean · `uv run pyright` 0 errors · `node --check` all `web/static/*.js` clean.
+- **Docs:** no doc impact — note-only task, no user-visible change yet; `git status` shows only the new note (plus pre-existing untracked harness files).
+- **Notable decisions:** throughput heroes stay neutral-ink (phase checklist's "colour-coded" reconciled with the no-invented-threshold rule via the `/system` NET precedent; noted for task 04); fleet-totals KV bar removed and the card hidden below two engines; `HISTORY_METRICS` extension marked as the phase's single Python exception for task 03 to implement with tests.
+- **Next pending task:** `02_sparkline_upgrade.md`.
@@ -0,0 +1 @@
+ok
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 15%]
+........................................................................ [ 30%]
+........................................................................ [ 45%]
+...........................sssssss...................................... [ 60%]
+........................................................................ [ 75%]
+........................................................................ [ 90%]
+...............................................                          [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   226
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+472 passed, 7 skipped, 1 warning in 43.32s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,10 @@
+All green. Task complete.
+
+**Report**
+
+- **Implemented** — `DashCharts.sparkline()` upgraded to the `drawSpark` contract (task 02) in `src/dashboards/web/static/charts.js`: new opts `fixedMax` (data clamps to a fixed `[0, ceiling]` axis + top/bottom scale marks, never axis-to-data rescale), `align: 'right'` + `span` (newest pinned right, growth leftwards over real time), `labels: {current, max}` foot via caller's `format`, `severity` (stroke from `--ok/--warn/--fail` tokens; null → faint `--text-faint` stroke, no fill), `minPoints` (default 2 → "no history"), `ariaLabel`; dependency-free SVG, no animation/transition; `LineChart` untouched (single diff hunk in the sparkline region).
+- **Tests** — added `test_sparkline_ships_the_fixed_scale_and_verdict_options` to `tests/test_api.py` (served-bytes pin: option semantics, clamp expression, `x1 - o.span`, token colours, `minPoints: 2`, no `.sev-*`, no transitions).
+- **Verification** — `uv run pytest`: **473 passed, 7 skipped**; `--cov-fail-under=94`: **96.87%**; `ruff` + `pyright` clean; `node --check` clean; `bash .agents/validate.sh` passes. Live suite `uv run pytest -m live --live`: **7 passed**. Browser synthetic check on live `/llm`: 10/10 PASS incl. `fixedMax: 100` keeps data 5–10 at y≈29/34 (autoscaled control hits y≈2), 0/1-point → "no history", align-right first x=160.8/190 — screenshot at `data/logs/sparkline-upgrade-task02.png`. Criterion grep on served file: 14 (≥3).
+- **Docs** — AGENTS.md UI conventions gained the shared sparkline contract paragraph; no README impact (no setup/command/API change).
+- **Decisions** — unrated spark defaults to faint grey (task-file rule) over NOTES-chart-set's `--series-1`; callers wanting orange pass `color`/`severity`. `align:'right'` anchors at the newest sample's ts (task's "newest pinned right"), with `span` setting the window. No new Python, no CSS/HTML changes.
+- **Next pending task**: `03_llm_layout_simplified.md`.
@@ -0,0 +1 @@
+ok
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 15%]
+........................................................................ [ 30%]
+........................................................................ [ 45%]
+............................sssssss..................................... [ 60%]
+........................................................................ [ 75%]
+........................................................................ [ 90%]
+................................................                         [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   226
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+473 passed, 7 skipped, 1 warning in 43.62s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,18 @@
+All green. Final state verified; dev server stays up on http://127.0.0.1:8090 with the new page live.
+
+## Report — Task 03 `llm_layout_simplified` (phase 04)
+
+**Implemented**
+- `web/pages/llm.html`: totals card shrunk to engines-answering/active/queued and hidden below 2 engines; summed decode/prompt/KV/tokens + seam note dropped; four charts moved *below* the engine panels at half height.
+- `page-llm.js` rewritten: `CELLS` flags `hero/spark/alarm` drive markup — two `Dash.hero()` throughput heroes per engine (neutral ink, `Δ<window>` provenance in `.sub`, exact `source` in `title`); compact `.cells` rows with right-aligned tabular values, per-cell `sparkline()` (align:'right'+span, fixedMax 100 on ratio cells, `max` foot, KV coloured from payload `severity` via `severityFrom`); TTFT/TPOT/queue-time show mean + `p95 …` foot line; `preemptions`/`device_lost` render only as `.chip.fail` when non-zero (absent at zero); engine-scoped rows (`only:`) so gufo has no latency mystery; tokens lifetime demoted to footnote; **one** batched `/api/llm/history` request feeds charts + all sparks; both `severityFor(…,75,90)` call sites deleted.
+- `sources/llm.py`: additive `HISTORY_METRICS` `tpot_seconds` (vLLM) + `spec_accept_rate` (gufo) per NOTES-chart-set.
+- `dash.css`: `.cells/.cell` (label | tabular value | 160px spark column, p95 foot, KV verdict classes, alarm chip row, narrow-screen fallback).
+- Fixed two defects found only against live data: hero `cellValue` key mapping and `drawChart` legend-sibling null crash.
+
+**Tests/lint** — `bash .agents/validate.sh` exit 0: **483 passed, 7 skipped**, coverage 96.87 % (≥94), ruff + pyright clean, `node --check` clean. New pins: LLM payload-colour, hero row, spark/p95 contract, one-request + `HISTORY_METRICS` round-trip, alarm-chip rule, totals-card shape, chart height/aria; `test_severity_for_is_kept_as_the_documented_fallback` narrowed to `/ups`. Live: `uv run pytest -m live --live` → 7 passed. Browser (both engines live): heroes/sparks/p95 verified; mocked payload in devtools → red chips at non-zero, absent at zero; single-engine instance (`VLLM_BASE_URL=` unset, :8091, throwaway DB) → one clean panel, totals hidden, no dead panel. Screenshots: `data/logs/llm-layout-both-engines.png`, `llm-alarm-chips-mock.png`, `llm-single-engine.png`. One tiny vLLM chat request (16 tok) was sent to make latency rows non-empty — read-only class of traffic, same kind llm_monitor watches.
+
+**Docs** — AGENTS.md (`/llm` layout convention + `severityFor` now only `/ups` remains) and README (`web/` section `/llm` description) updated.
+
+**Deviations** — Totals card also hides below 2 engines (llm_monitor rule cited by NOTES); heroes neutral per the phase-01 colour language (documented in NOTES + AGENTS.md).
+
+**Next pending task:** `04_side_by_side_verification.md` (same phase).
@@ -0,0 +1 @@
+ok
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 14%]
+........................................................................ [ 29%]
+........................................................................ [ 44%]
+....................................sssssss............................. [ 58%]
+........................................................................ [ 73%]
+........................................................................ [ 88%]
+..........................................................               [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   228
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+483 passed, 7 skipped, 1 warning in 47.36s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1 @@
+Now let me fix the layout void I measured (the shorter engine card stretched to the taller one, leaving 172 px of empty card):
@@ -0,0 +1 @@
+ok
@@ -0,0 +1,46 @@
+== uv run pytest --cov (hermetic: no network, live tests skip) ==
+........................................................................ [ 14%]
+........................................................................ [ 29%]
+........................................................................ [ 44%]
+....................................sssssss............................. [ 58%]
+........................................................................ [ 73%]
+........................................................................ [ 88%]
+..........................................................               [100%]
+=============================== warnings summary ===============================
+.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.14.4-final-0 ________________
+
+Name                                 Stmts   Miss  Cover   Missing
+------------------------------------------------------------------
+src/dashboards/__init__.py               3      0   100%
+src/dashboards/__main__.py              14      1    93%   28
+src/dashboards/alerts.py               175      0   100%
+src/dashboards/api.py                  193      0   100%
+src/dashboards/app.py                  109      6    94%   50-51, 56, 58, 60, 131
+src/dashboards/collectors.py           142      5    96%   97-98, 102-103, 120
+src/dashboards/config.py               113      0   100%
+src/dashboards/debug.py                 14      0   100%
+src/dashboards/poller.py               139      1    99%   226
+src/dashboards/prom.py                 105      5    95%   107, 144, 158, 161, 164
+src/dashboards/rates.py                 37      1    97%   51
+src/dashboards/severity.py              49      0   100%
+src/dashboards/sources/__init__.py       0      0   100%
+src/dashboards/sources/host.py         199      1    99%   147
+src/dashboards/sources/llm.py          150      1    99%   228
+src/dashboards/sources/nut.py          367     22    94%   118, 132, 138, 150, 156, 168, 176, 181, 252-253, 255, 414, 458, 470, 505, 538, 542, 595, 599-600, 605, 617
+src/dashboards/sources/nut_sim.py      122     21    83%   141, 167-185, 226
+src/dashboards/store.py                178      2    99%   243-244
+------------------------------------------------------------------
+TOTAL                                 2109     66    97%
+Required test coverage of 94% reached. Total coverage: 96.87%
+483 passed, 7 skipped, 1 warning in 45.25s
+== uv run ruff check src tests agent ==
+All checks passed!
+== uv run pyright ==
+0 errors, 0 warnings, 0 informations
+== node --check src/dashboards/web/static/*.js ==
				`@@ -0,0 +1 @@`
				`Now I have everything verified. Writing the decision note:`
				`@@ -0,0 +1 @@`
				`error: assistant error: litellm.APIConnectionError: APIConnectionError: OpenAIException - Response payload is not completed: <TransferEncodingError: 400, message='Not…`
				`@@ -0,0 +1 @@`
				`Now let me fix the layout void I measured (the shorter engine card stretched to the taller one, leaving 172 px of empty card):`