phase: 04_llm_chart_readability
**Phase 04 final verification pass — all tasks already in `complete/`; verified + fixed 3 defects found in verification.**
- Verified `/llm` live (both engines, gufo-only instance on :8099, `llm_monitor` on :8095) in a real browser; dev server left running/healthy.
- Completed task-04's truncated layout fix: `#engines` now carries `.items-start` (the CSS rule existed but nothing used it) — gufo 544 px vs vLLM 715 px, no blank card; pinned by `test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling`.
- Fixed: event rows said the subject twice ("gufo gufo is answering again"), and `/ups` mangled mid-message names ("sent to apc: OK" → "sent to OK") → shared `Dash.eventText()` (strips `UpstreamError: ` + a *leading* name only, mirroring `alerts._clean_error`) used by `/system`, `/llm`, `/ups`; new test.
- Fixed: count sparklines autoscaled rollup averages (queue max 0.1379) into full-height spikes → `sparkline({minSpan})`, `minSpan: 1` on sessions/queued (spike 30 px → 4 px); assertions added.
- Tests: `bash .agents/validate.sh` → **EXIT=0** (`uv run pytest --cov --cov-fail-under=94`: **485 passed, 7 skipped**, coverage **96.87%**; `ruff check src tests agent`: All checks passed; `pyright`: 0 errors; `node --check` all static JS clean; repo hygiene clean; `uv pip list --outdated` empty). `uv run pytest -m live --live` → **7 passed**.
- Completion criteria: heroes largest per engine ✔ (38.4 px vs 16.8 px row values); sparkline per cell + selected window ✔ (5/5 gufo, 7/7 vLLM; 6 h chip moved KV max 53 %→58 %); mean+p95 without hover ✔ (243 / 48.8 / 285 ms feet); provenance tag ✔ (`Δ10.0s`/`Δ8.0s` + `title`=exact source); single-engine ✔ (1 panel, totals hidden, zero red); screenshots ✔ `data/logs/llm-verify-{both-engines,charts-and-slots,single-engine,monitor-reference}.png` + `NOTES-chart-set.md`; pytest/cov/ruff/pyright ✔; earlier phases unchanged-behaviour ✔ (all 01–03 tests pass).
- Deviation (documented, not code): throughput heroes stay **neutral ink** — `severity.py` has no throughput row and 0 tok/s idle is healthy; colour comes from rated cells (KV) + reachability. Kept per `NOTES-chart-set`.
- Deviation: could not append the task-04 written verdict to `NOTES-chart-set.md` (it lives in `todo/`, which I must not edit). Verdict: ours reads better than `:8095` — bigger throughput numbers, per-cell shape over the selected window, percentiles inline; `llm_monitor` only sparklines decode+KV and has no window control.
- Docs: AGENTS.md updated (485 test count, `sparkline` `minSpan` rule, `Dash.eventText` event-row rule, `/llm` panel `items-start`); README reviewed in full — commands, config table (all `config.py` vars present), architecture bullets, page descriptions verified accurate; no other inconsistencies found.
- Note for the harness: `.agents/phases/todo/03…/NOTES-verify.md` and `todo/04…/NOTES-chart-set.md` are untracked notes left behind by their phases (task files moved, notes did not).
- Next pending phase: `05_ups_hero_battery_glyph`.
This commit is contained in:
@@ -1,51 +0,0 @@
|
||||
# Phase 04 — LLM page is readable again
|
||||
|
||||
**Source:** `TODO.md` L6 — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Anchors* (hand-rolled SVG only), `~/Projects/Experimental/llm_monitor/static/index.html` (the reference implementation), `src/dashboards/web/static/page-llm.js` (`CELLS`, `makeCharts`), `src/dashboards/web/static/charts.js` (`sparkline`, `LineChart`), `src/dashboards/sources/llm.py`
|
||||
|
||||
## Objective
|
||||
|
||||
The LLM page drifted into a 14-cell grid where everything is the same size and nothing shows
|
||||
recent shape. Bring back what made `llm_monitor` legible — big throughput numbers, a sparkline
|
||||
in each metric cell, percentiles as foot lines — without losing what this project added since:
|
||||
per-engine provenance tags, history windows, and severity colour.
|
||||
|
||||
`llm_monitor` is the design reference, not a dependency. It is a separate app on `:8095` and
|
||||
stays that way; nothing here imports from it.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — KV-cache pressure and queue depth get coloured from the
|
||||
API verdict, not a per-page threshold.
|
||||
- `03_system_per_host_panels` (todo) — touches `charts.js` first; this phase builds on the
|
||||
same shared helpers and must not fight it for them.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_chart_set_decision.md` — read both implementations side by side and write the decision note (which cells live, which get sparklines, what is demoted).
|
||||
2. `02_sparkline_upgrade.md` — bring `DashCharts.sparkline()` up to `llm_monitor`'s `drawSpark` behaviour: right-aligned growth, fixed max, current/max labels, severity colour.
|
||||
3. `03_llm_layout_simplified.md` — restructure `llm.html` + `page-llm.js`: hero throughput, compact sparkline cells, percentile foot lines, conditional alarm cells.
|
||||
4. `04_side_by_side_verification.md` — verify against `llm_monitor` on `:8095` with the same live engines.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Python: none expected beyond the phase-01 severity fields; `/api/llm/*` stays as-is
|
||||
(additive-only anchor).
|
||||
- `uv run pytest -m live --live` must stay green — `test_engine_still_publishes_the_series_we_parse`
|
||||
is the assertion that the numbers on screen are backed by resolved upstream series.
|
||||
- Browser: both engines live (gufo on `deskwork:8731`, vLLM on `driveripper:8000`), plus the
|
||||
single-engine case: with one `*_BASE_URL` unset the page must show one clean half rather
|
||||
than a red panel.
|
||||
- Coverage: ≥94% overall maintained.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] decode and prompt throughput are the two largest numbers per engine, colour-coded
|
||||
- [ ] every remaining metric cell carries a sparkline showing the selected window
|
||||
- [ ] latency cells show mean **and** p95 without needing a hover
|
||||
- [ ] a source-provenance tag is still visible on every throughput number (`AGENTS.md`: never invent a number)
|
||||
- [ ] `/llm` with only one engine configured still renders cleanly
|
||||
- [ ] screenshots of this page and `llm_monitor` side by side in `data/logs/`, plus `NOTES-chart-set.md`
|
||||
- [ ] `uv run pytest` green, coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -1,52 +0,0 @@
|
||||
# Task 01 — Chart-set decision note
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Decide, in writing, what "simplified and more readable" means for this page before touching
|
||||
it — the drift happened precisely because nobody wrote down what the page was for.
|
||||
|
||||
## Work
|
||||
|
||||
1. Read the reference implementation:
|
||||
`~/Projects/Experimental/llm_monitor/static/index.html` — specifically
|
||||
`drawSpark(name, values, format, fixedMax)` (~L1092, with the right-aligned growth and the
|
||||
top/middle/bottom scale marks ~L1134), `setThroughput(el, cell)` (~L1303),
|
||||
`UI.historyLen = 60` (~L1003), the per-engine cell markup (`#gufo-ptps`, `#gufo-dtps`,
|
||||
`#vllm-ptps`, `#vllm-dtps`), the percentile foot lines (`#vllm-ttft-p95`,
|
||||
`#vllm-tpot-p95`, `#vllm-queue-p95`), and `buildSlotCard`/`renderSlots` (~L1179/L1209).
|
||||
2. Read the current page: `src/dashboards/web/pages/llm.html` (engine-totals card L39–60,
|
||||
four charts L62–81, `#engines` grid L83–91) and `page-llm.js` (`CELLS`, 14 entries;
|
||||
`cellValue()`; `severityFor(value, 75, 90)` calls at L195/L307).
|
||||
3. Write `.agents/phases/todo/04_llm_chart_readability/NOTES-chart-set.md`:
|
||||
- **keep**: provenance tags per throughput cell, history window chips, per-engine panels,
|
||||
the gufo slot/session list;
|
||||
- **hero**: `decode_tokens_per_second`, `prompt_tokens_per_second` per engine;
|
||||
- **sparkline cells** (compact, one per row): sessions active, queued, KV cache %,
|
||||
TTFT (mean + p95 as foot), TPOT (mean + p95 as foot), queue mean, prefix-cache hit %,
|
||||
spec-accept %;
|
||||
- **conditional / demoted**: tokens-lifetime counter (quiet footnote), `preemptions` and
|
||||
`device_lost` — shown as alarm chips **only when non-zero**, otherwise absent. A permanent
|
||||
cell for something that is always 0 trains the eye to ignore the row.
|
||||
- **charts**: keep decode/prompt/KV/queue `LineChart`s, one engine per series, with the
|
||||
existing legend and a units-bearing y-axis; the four stay but drop to half-height so the
|
||||
cells read first.
|
||||
- the colour rules for KV pressure and queue depth (from `severity.py`), and the explicit
|
||||
statement that throughput has *no* severity — 0 tok/s on an idle server is not red.
|
||||
4. No code changes.
|
||||
- ASSUMPTION: the cell set above is the judgment call; if the operator disagrees it is
|
||||
cheaper to change this note now than the page later.
|
||||
- ASSUMPTION: `llm_monitor` stays a separate app. Importing its files would break the
|
||||
no-build, single-app structure and it is not ours to own.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- The note is the deliverable; it must name real line numbers in both codebases so the next
|
||||
task does not re-derive them.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `NOTES-chart-set.md` exists in this phase directory, with keep/hero/demote lists and the two source files cited by path and line
|
||||
- [ ] no application file changed in this task (`git status` clean except the note)
|
||||
@@ -1,51 +0,0 @@
|
||||
# Task 02 — Sparkline upgrade
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "The charts got less useful."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Make the shared `sparkline()` as informative as `llm_monitor`'s `drawSpark`, because every
|
||||
page (LLM cells now, per-host cards later) draws on it.
|
||||
|
||||
`charts.js:352` already has `sparkline(container, points, opts)` and it is exported at
|
||||
`charts.js:377` — **no page uses it today**, which is exactly why the LLM page reads flat.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/charts.js` — extend `sparkline()` with these opts, defaulting
|
||||
to today's behaviour so no existing caller breaks:
|
||||
- `fixedMax` — plot against a known ceiling (percent metrics must not rescale to their own
|
||||
data: 8 % KV cache next to a 92 % spike is the lie this fixes);
|
||||
- `align: 'right'` — grow from the left, newest pinned right, as in `drawSpark`;
|
||||
- `labels: {current, max}` — the small `now / max` foot `llm_monitor` prints, using the
|
||||
`format` callback;
|
||||
- `severity` — stroke colour from the theme token for `ok|warn|fail` (`cssVar('--ok')` etc.),
|
||||
no colour and a faint grey stroke when severity is `null`;
|
||||
- `minPoints` (default 2) — keep the existing "no history" text for a series that has no
|
||||
shape yet; never draw a confident line through one point.
|
||||
2. Keep it dependency-free SVG (`<svg>` + `<path>`/`<polyline>` + optional baseline), and keep
|
||||
the output small enough to sit in a 160×34 box — that is the whole point of a sparkline.
|
||||
3. Add scale marks (top/bottom values only) when `fixedMax` is set, so a reader can tell that
|
||||
"the line is at a third" means a third *of what*.
|
||||
4. If `renderCores` survived phase 03's cleanup, leave it alone — do not widen this task.
|
||||
- ASSUMPTION: sparkline stays in `charts.js` rather than a new file; one shared chart
|
||||
module is the current architecture.
|
||||
- ASSUMPTION: no animation, but respect `prefers-reduced-motion` by not adding any
|
||||
transition at all.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `tests/test_api.py` already asserts static assets are served; extend it to assert the new
|
||||
option names are present in `/static/charts.js` (cheap, catches a bad merge).
|
||||
- Browser: a page that uses it (any LLM cell after task 03) plus a synthetic check in devtools
|
||||
— draw into an empty container (0, 1, 2, 60 points) and confirm each renders the right thing
|
||||
rather than throwing.
|
||||
- Coverage: ≥94% overall maintained; no new Python expected.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/static/charts.js | grep -c "fixedMax\|align\|severity"` ≥ 3
|
||||
- [ ] a sparkline with `fixedMax: 100` does not rescale when its data ranges 5–10 (verified in devtools, screenshot)
|
||||
- [ ] 0- and 1-point series render "no history", not a spike
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean; `LineChart` untouched (`git diff` shows no changes to its code)
|
||||
@@ -1,62 +0,0 @@
|
||||
# Task 03 — Simplify the LLM page layout
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Per engine: two big throughput numbers you can read across the room, then a compact list of
|
||||
metric cells that each carry their own sparkline and percentile foot line.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/llm.html` —
|
||||
- keep the engine-totals card (L39–60) but shrink it to what the overview no longer shows:
|
||||
engines answering, active sessions, queued. Drop the summed decode/prompt/KV numbers —
|
||||
per-engine is the honest view and the overview owns the aggregate;
|
||||
- reduce the four `LineChart` cards (L62–81) to half height, and drop `#chart-queue` if
|
||||
`NOTES-chart-set.md` says queue reads better as a cell;
|
||||
- per-engine panel (`#engines` grid, `page-llm.js:140+`): hero row (decode, prompt) using
|
||||
the phase-01 `.hero` markup, then the cell list from `NOTES-chart-set.md`, then the gufo
|
||||
slot list (unchanged behaviour).
|
||||
2. `src/dashboards/web/static/page-llm.js` —
|
||||
- `CELLS`: mark each entry with `hero: true` / `spark: true` / `alarm: true` and drive the
|
||||
markup from those flags instead of 14 identical cells;
|
||||
- render each `spark: true` cell through `DashCharts.sparkline()` with data from
|
||||
`/api/llm/history` (same endpoint, one request per engine per window — batch the metrics
|
||||
the endpoint supports rather than one XHR per cell);
|
||||
- percentile foot lines: TTFT/TPOT/queue cells print `mean` big and `p95 …` beneath, as
|
||||
`llm_monitor` does (`#vllm-ttft-p95` pattern);
|
||||
- `alarm: true` cells (`preemptions`, `device_lost`) render only when non-zero, as a
|
||||
`.chip.fail`; the row must not exist when they are 0;
|
||||
- replace `severityFor(value, 75, 90)` (L195, L307) with the payload `severity` via
|
||||
`Dash.severityFrom()`;
|
||||
- **keep the provenance tag** on every throughput cell — the file's header comment explains
|
||||
why (`78 tok/s` means different things when scraped, measured Δ/Δt, or estimated from
|
||||
TPOT). Removing it to look cleaner is a regression, not a simplification.
|
||||
3. `src/dashboards/web/static/dash.css` — cell-list styling: label left, value right-aligned
|
||||
tabular, sparkline on the third column at a fixed width so rows line up; the alarm chip
|
||||
reuses `.chip.fail`.
|
||||
- ASSUMPTION: throughput gets no severity colour. A server doing 0 tok/s while idle is
|
||||
healthy; a red zero every night destroys the signal.
|
||||
- ASSUMPTION: cells are a vertical list per engine, not a 4×4 grid — grids make 14 numbers
|
||||
look equally important, which is the complaint.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `uv run pytest -m live --live` against the real gufo and vLLM: `test_engine_still_publishes_the_series_we_parse`
|
||||
guards the metric names the new layout renders.
|
||||
- Browser, both engines live, and with one engine disabled (`VLLM_BASE_URL=` unset, restart)
|
||||
to prove the page degrades to one clean half instead of a dead panel
|
||||
(`AGENTS.md` invariant 3).
|
||||
- Idle vs loaded engine if possible (`llm_monitor` on `:8095` is watching the same upstreams,
|
||||
so compare like-for-like).
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] decode + prompt are the largest elements per engine, and the rest of the cells are visibly secondary
|
||||
- [ ] every cell has a sparkline for the selected window; TTFT/TPOT show mean + p95
|
||||
- [ ] `preemptions` / `device_lost` are absent at zero and appear as red chips when non-zero (verified in devtools with a mocked payload)
|
||||
- [ ] provenance tags still present on throughput cells (`grep -n "source" page-llm.js` shows the tag path intact)
|
||||
- [ ] `curl -s localhost:8090/llm` renders with one engine configured and shows no dead panel (screenshot)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
@@ -1,44 +0,0 @@
|
||||
# Task 04 — Verify side by side with llm_monitor
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the new page is at least as readable as the app it drifted from, using the same live
|
||||
upstreams.
|
||||
|
||||
## Work
|
||||
|
||||
1. Both apps watch the same engines, so compare honestly:
|
||||
- `llm_monitor` on `http://127.0.0.1:8095` (it is already running in this fleet's dev
|
||||
loop; if not, `cd ~/Projects/Experimental/llm_monitor && uv run uvicorn app:app --port 8095`);
|
||||
- this app on `http://127.0.0.1:8090/llm`.
|
||||
2. Screenshot both, same moment, same window length. Read the two images and answer in
|
||||
writing: which one tells you *right now* whether the engines are busy, whether anyone is
|
||||
queued, and whether the KV cache is about to bite? The answer must be "ours" or the phase
|
||||
is not done.
|
||||
3. Exercise the states, not just the happy path:
|
||||
- one engine down (stop gufo or point `GUFO_BASE_URL` at a dead port, restart the dev
|
||||
server) → one clean half, no red panel for the missing engine;
|
||||
- both engines idle → throughput `0 tok/s` with **no** severity colour, and the sparklines
|
||||
still show the shape of the last window rather than a flat lie;
|
||||
- a real request in flight if the operator is willing (`curl` a chat completion against
|
||||
gufo) so decode/prompt are visibly non-zero and the provenance tag is readable.
|
||||
4. Confirm the history window chips (1h…90d) still drive the four `LineChart`s and that the
|
||||
sparklines use the same window — a cell that silently ignores the selected range is worse
|
||||
than no sparkline.
|
||||
5. Append the comparison and the answers to `NOTES-chart-set.md`.
|
||||
- ASSUMPTION: `llm_monitor` is treated as the design reference and left running; this phase
|
||||
neither modifies nor deletes it.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Evidence task. Any defect found is fixed here, then re-screenshotted.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/llm-before-monitor.png` (the reference) and `data/logs/llm-after.png` (ours) exist and were both read
|
||||
- [ ] the single-engine and idle states are screenshotted too
|
||||
- [ ] `NOTES-chart-set.md` has a short written verdict + what was fixed after looking
|
||||
- [ ] `uv run pytest` and `uv run pytest -m live --live` green
|
||||
@@ -0,0 +1,206 @@
|
||||
# NOTES-chart-set — what `/llm` is for, and what it shows
|
||||
|
||||
**Phase 04, task 01 deliverable (no code changes).** Decides the cell set, the colour rules,
|
||||
and the contracts tasks 02–03 implement. Both implementations were read side by side on
|
||||
2026-10-17; every line number below was verified against the file it cites.
|
||||
|
||||
## The two sources
|
||||
|
||||
**Reference — `~/Projects/Experimental/llm_monitor/static/index.html`** (separate app on
|
||||
`:8095`; design reference only, never a dependency — ASSUMPTION kept: it is not ours to own,
|
||||
and importing it would break the no-build single-app structure):
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| sparkline CSS vocabulary (`.sparks`, `.spark`, `.spark-now`, `.spark-axis`, `.spark-line`, `.spark-area`) | L398–495 |
|
||||
| `UI.historyLen = 60` (client ring buffer; `pushHist` L1542–1545; fed per engine L1570–1571, L1582–1583) | L1003 |
|
||||
| `drawSpark(name, values, format, fixedMax)` | L1092 (`SPARK_W = 100`/`SPARK_H = 30` L1090–1091) |
|
||||
| right-aligned growth — `const step = SPARK_W / (UI.historyLen - 1)`, `x = SPARK_W - (values.length - 1 - i) * step` | L1158–1163 |
|
||||
| empty state: `< 2` values → `empty` class, blank paths (never a zero line) | L1112–1118 |
|
||||
| flat-window honesty: pad keeps the line mid-plot, axis reports the one real value, not the pad (`const flat` L1128) | L1126–1133 |
|
||||
| top/middle/bottom scale marks (`axis()` helper, `levels`, `fmtAxis` with enough decimals that the three labels stay distinguishable; axis writes L1155–1157) | L1101–1104, L1135, L1138–1154 |
|
||||
| `fixedMax` ratio axes: KV sparks called with `fixedMax = 1` → 0–100% always | L1356 (gufo), L1420 (vLLM) |
|
||||
| per-engine throughput `dd`s: `#gufo-ptps` / `#gufo-dtps`, `#vllm-ptps` / `#vllm-dtps` | L757 / L761, L802 / L806 |
|
||||
| `setThroughput(el, cell)` — value + honest tag (`Δ<w>` for measured, `lifetime avg`, `from TPOT`, `derived`; `title` = exact `source`) | L1303–1320 |
|
||||
| percentile foot lines: `#vllm-ttft-p95`, `#vllm-tpot-p95`, `#vllm-queue-p95` under their mean cards | L950, L955, L960 |
|
||||
| token counters as cells whose foot line names the upstream series (`#vllm-ptok`, `#vllm-gtok`) | L964, L969 |
|
||||
| slot cards: `SLOT_FIELDS` L1170–1177, `buildSlotCard` L1179 (built once, patched in place), `renderSlots` L1209 (DOM order == slot order, vanished slots removed) | — |
|
||||
| the only thresholds in that app: KV bar warn ≥ 0.75, crit ≥ 0.9 | L1085–1086 |
|
||||
| single-engine case: fleet block hidden when one engine — "a cross-engine sum of one engine just duplicates that engine's own card" | L1500–1504 |
|
||||
| dead engine holds last good data + `stale` class + error banner, never zeroes | L1514–1535 |
|
||||
|
||||
**Current page** — `src/dashboards/web/pages/llm.html` + `src/dashboards/web/static/page-llm.js`:
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| engine-totals card (`#f-decode` L45, `#f-prompt` L47, KV bar L54–56, tokens lifetime L52–53) | `llm.html` L39–60 |
|
||||
| four full-height `LineChart`s `#chart-decode/prompt/kv/queue` | `llm.html` L62–81 (chart divs L66, L71, L75, L79) |
|
||||
| per-engine panels grid `#engines` + honest "No engines configured" empty state | `llm.html` L83–91 |
|
||||
| gufo slots table `#slots-card` L94–102; events feed L103–106; provenance footer L109–113 | `llm.html` |
|
||||
| `CELLS` — the 14-equal-size grid this phase dissolves | `page-llm.js` L16–33 |
|
||||
| `cellValue()` — per-key payload mapping | L35–93 |
|
||||
| `provenance()` — `Δ10.0s` / formula tag from `source`, nothing for scraped gauges | L96–105 |
|
||||
| `enginePanel()` / `renderPanel()` | L112–136 / L138–214 |
|
||||
| **JS thresholds to remove**: `severityFor(value.value, 75, 90)` on the KV bar | L194–195 |
|
||||
| device_lost non-zero special case (the alarm-chip seed) | L196–200 |
|
||||
| `makeCharts()` — `height: 165` for all four | L241–249 (165 at L242) |
|
||||
| `refreshCharts()` — three `/api/llm/history` calls; the third already carries `requests_running,requests_waiting,kv_cache_usage` | L256–292 (L264–265) |
|
||||
| **JS thresholds to remove**: `severityFor(kv, 75, 90)` on fleet KV | L307 |
|
||||
| window chips `Dash.rangeControl` (affects charts today, sparklines after this phase) | L374 |
|
||||
|
||||
Shared helpers this builds on: `charts.js` `LineChart` L43, `renderLegend` L329 (with
|
||||
`showPeak`), `sparkline` L348–373 (the function task 02 upgrades); `app.js` `severityFor`
|
||||
L118 (documented FALLBACK ONLY — phases 04–05 exist to retire its remaining call sites),
|
||||
`severityFrom` L139, `hero()` L187.
|
||||
|
||||
## Decision
|
||||
|
||||
### KEEP (unchanged, do not regress)
|
||||
|
||||
- **Provenance tags on every throughput number** — `provenance()` (`page-llm.js` L96–105)
|
||||
and the footer legend (`llm.html` L109–113). AGENTS.md invariant 2; the `llm_monitor`
|
||||
equivalent is `setThroughput` L1303–1320. Ours is strictly better (it prints the measured
|
||||
Δ window); it ports as-is into the hero cells.
|
||||
- **History window chips** (`page-llm.js` L374, `Dash.rangeControl`). Divergence from the
|
||||
reference on purpose: `llm_monitor` sparks use a fixed 60-sample client buffer (L1003);
|
||||
our sparklines read `/api/llm/history` with the *selected* window, so chips and sparks
|
||||
always agree. Do not copy `historyLen`.
|
||||
- **Per-engine panels** (`llm.html` L84) with per-source health, stale/last-good dimming,
|
||||
and the upstream chip. Unconfigured engines render no panel at all (L85–90) — invariant 3.
|
||||
- **gufo slot list** (`llm.html` L94–102, `renderSlots` L216–239). Adopt from
|
||||
`llm_monitor` only the *in-place patch* idiom (`buildSlotCard` L1179) if rows churn; the
|
||||
table itself stays.
|
||||
- **Events card** (`llm.html` L103–106). Phase 02 deleted the raw feed from `/` only;
|
||||
per-engine up/down transitions stay here.
|
||||
|
||||
### HERO — the two largest numbers, per engine panel
|
||||
|
||||
- `llm.decode_tokens_per_second` and `llm.prompt_tokens_per_second` (payload cells built in
|
||||
`sources/llm.py`: gufo measured Δcounters, vLLM native gauge → measured → lifetime
|
||||
estimate, each labelled with its `source`).
|
||||
- Rendered with `Dash.hero()` (`app.js` L187) with the `Δ<window>` provenance in `.sub`.
|
||||
- **Throughput has NO severity.** `severity.py` `THRESHOLDS` (L46–57) has no throughput row
|
||||
and gets none: 0 tok/s on an idle server is not red, and a page-invented threshold is
|
||||
exactly what invariant "a threshold exists nowhere else" forbids. The number renders in
|
||||
neutral ink; colour in the hero row comes from the engine's reachability (status dot +
|
||||
`.stale` dimming, already implemented in `renderPanel`), mirroring the `/system` NET slot
|
||||
precedent ("throughput has no good/bad", AGENTS.md UI conventions). *Note: the phase
|
||||
checklist says the heroes are "colour-coded" — the honest reading is the page's colour
|
||||
language (neutral numbers, verdict colour on rated cells, reachability colour on the
|
||||
panel). Task 04's side-by-side screenshots should show this; if the operator wants the
|
||||
digits themselves tinted, that needs a ratified `severity.py` row first, not a JS guess.*
|
||||
|
||||
### SPARKLINE CELLS — compact rows, one per metric, each with its window's history
|
||||
|
||||
| Cell | Payload source (`cellValue`, L35–93) | History series for the spark | Colour |
|
||||
|---|---|---|---|
|
||||
| Sessions active | `sessions.active` (gufo `/slots`, vLLM count) | `llm.requests_running` ✔ stored | none |
|
||||
| Queued | `requests_deferred` / `num_requests_waiting` | `llm.requests_waiting` ✔ stored | none |
|
||||
| KV cache % | `kv_cache_usage_ratio` / `gpu_cache_usage` (×100) | `llm.kv_cache_usage` ✔ stored | **payload `severity`** |
|
||||
| TTFT | `time_to_first_token_seconds__mean`, foot: `__p95` | `llm.ttft_seconds` ✔ stored | none |
|
||||
| TPOT | `time_per_output_token_seconds__mean`, foot: `__p95` | `llm.tpot_seconds` — **new, see Python delta** | none |
|
||||
| Queue mean | `request_queue_time_seconds__mean` | `llm.queue_seconds` ✔ stored | none |
|
||||
| Prefix cache hit % | `prefix_cache_hit_ratio` / `prompt_cache_hit_ratio` | `llm.prefix_cache_hit_ratio` ✔ stored | fixed max 100 |
|
||||
| Spec accept % | `spec_accept_rate` (gufo only) | `llm.spec_accept_rate` — **new, see Python delta** | fixed max 100 |
|
||||
|
||||
- Percentiles print as **foot lines, no hover** (reference: L950/L955/L960), from the live
|
||||
snapshot cells — p95 needs no stored series.
|
||||
- TTFT/TPOT/queue are vLLM-only; spec accept is gufo-only. The other engine's row renders
|
||||
`n/a` uncoloured (rule 2), or the row is engine-scoped — task 03's choice, but a
|
||||
gufo panel must never show a red/empty latency mystery.
|
||||
- `/api/llm/history` (`api.py` L353–365) answers *any* metric name; unknown ones come back
|
||||
as empty series — so the page can request the full cell list in one call. `refreshCharts`
|
||||
(L256–292) already makes three calls; extend the third (L264–265) with the cell series
|
||||
instead of adding requests, and feed both charts and sparks from the same responses.
|
||||
|
||||
### CONDITIONAL / DEMOTED
|
||||
|
||||
- **`tokens_total` (lifetime counter)** → quiet footnote line in each engine panel (the
|
||||
existing `.sessions-note`/`.debug` strip, `page-llm.js` L203–213), like `llm_monitor`'s
|
||||
counter cards whose foot line just names the series (L964, L969). A lifetime sum is not a
|
||||
glance metric.
|
||||
- **`preemptions` and `device_lost`** → **alarm chips rendered only when non-zero, absent
|
||||
otherwise** (seed: the `device_lost` special case L196–200). A permanent cell for
|
||||
something that is always 0 trains the eye to ignore the row. Chip class is `.chip.fail`
|
||||
(existing vocabulary, `dash.css` L336); the rule is *existence of an incident*, not a
|
||||
graded threshold — `sources/llm.py` already treats any non-zero `gufo_device_lost_total`
|
||||
as an incident (comment at `GUFO_SERIES`), and zero/null simply draws no chip.
|
||||
- **Engine-totals card (L39–60)**: keep engines-live, decode/prompt sums, active, queued —
|
||||
demote tokens to footnote, and **remove the fleet KV bar** (L54–56 + `severityFor` at
|
||||
L307): `fleet_totals` (`sources/llm.py` L325–368, `kv_cache_usage_max` L360–363) carries
|
||||
no `severity`, and the rated per-engine KV cells plus the KV chart say it better. Hide the
|
||||
totals card when fewer than two engines are configured — copy `llm_monitor` L1500–1504
|
||||
("a cross-engine sum of one engine just duplicates that engine's own card"); this also
|
||||
satisfies the single-engine completion criterion.
|
||||
|
||||
### CHARTS
|
||||
|
||||
- All four `LineChart`s stay: decode, prompt, KV, queue — one engine per series (queue chart:
|
||||
running+waiting per engine), existing legend `renderLegend` (`charts.js` L329) with
|
||||
`showPeak` on the bursty throughput plots, units-bearing y-format (already passed).
|
||||
- **Drop `height: 165` → ~110** (`page-llm.js` L242) **and move the chart section below the
|
||||
engine panels** so the cells read first (the task says half-height "so the cells read
|
||||
first"; keeping full-height charts on top would contradict that).
|
||||
- Task 03 must pass `ariaLabel` to all four charts (AGENTS.md: every chart names itself;
|
||||
`/system` did this in phase 03, `/llm` never adopted it).
|
||||
|
||||
### COLOUR RULES (from `severity.py` and nowhere else)
|
||||
|
||||
- **KV pressure**: rated. `THRESHOLDS["llm.kv_cache_usage_pct"] = (75, 90)`
|
||||
(`severity.py` L53), attached per engine cell by `SEVERITY_CELLS` (`sources/llm.py`
|
||||
L99–103) with `scale=100` because both engines publish a 0–1 ratio. The page reads it via
|
||||
`Dash.severityFrom` (`app.js` L139) and **both `severityFor(..., 75, 90)` call sites
|
||||
(L194–195, L307) are deleted** — the `app.js` L112–117 fallback comment stays true only
|
||||
if phase 04 does this.
|
||||
- **Queue depth**: **not rated** — `THRESHOLDS` has no queue row (L46–57). `severityFrom`
|
||||
returns `''` → no colour. The queue chart and sparkline carry the signal as *shape*. If
|
||||
the operator wants a verdict, that is a future `severity.py` row (ratified numbers,
|
||||
tests), never a JS threshold.
|
||||
- **Throughput, sessions, latency (TTFT/TPOT/queue mean)**: not rated → colourless numbers.
|
||||
Explicit statement per the task: **0 tok/s on an idle server is not red.**
|
||||
- **Alarm chips**: `.chip.fail` when the counter is non-zero; absent otherwise (above).
|
||||
|
||||
### SPARKLINE CONTRACT (input for task 02, mirroring `drawSpark` L1092–1176)
|
||||
|
||||
`DashCharts.sparkline()` (currently L348–373: full-width autoscale, no labels) becomes:
|
||||
|
||||
1. **Right-aligned growth**: newest sample pinned right, x mapped over the *selected time
|
||||
window* (`[now − range, now]`, like `LineChart._xExtent`) rather than over sample index —
|
||||
a partly-filled window grows in from the right exactly like `drawSpark`'s fixed-step
|
||||
buffer (L1158–1163) but stays honest about *when* samples happened.
|
||||
2. **Fixed max** for ratio metrics (KV, prefix hit, spec accept: 0–100) so a squiggle means
|
||||
the same thing on every screen — reference `fixedMax` calls L1356/L1420; autoscale
|
||||
otherwise.
|
||||
3. **Current/max labels**: current value in the cell header (reference `.spark-now`, L419)
|
||||
and window max as the label; where the reference prints three axis marks (L1135,
|
||||
L1155–1157) our compact rows print `now` + `max` — adopt its formatting discipline:
|
||||
enough decimals that labels stay distinguishable (L1137–1152), a flat window reports the
|
||||
one real value, never the pad artifact (L1126–1133).
|
||||
4. **Severity colour**: stroke/area from the payload verdict (`ok`/`warn`/`fail` tokens from
|
||||
`severityFrom`, unrated → `--series-1`), not `llm_monitor`'s fixed per-engine hue.
|
||||
5. **Empty**: < 2 finite points → "no history", never a zero line (reference L1112–1118;
|
||||
current code already does this).
|
||||
|
||||
## Python delta (the phase's only one, flagged for task 03)
|
||||
|
||||
`HISTORY_METRICS` (`sources/llm.py` L113–123; sampled at L292–297) does **not** store TPOT
|
||||
or spec-accept, so those two cells would show permanent "no history" slots — the dead-cell
|
||||
anti-pattern this note exists to prevent. Decision: extend `HISTORY_METRICS` additively —
|
||||
|
||||
- `tpot_seconds` ← `("time_per_output_token_seconds__mean",)` (vLLM only)
|
||||
- `spec_accept_rate` ← `("spec_accept_rate",)` (gufo only)
|
||||
|
||||
New *names* in the public metric namespace (allowed: "phases may stop plotting a series,
|
||||
they may not rename or stop storing it" — adding is the safe direction), consumed by the
|
||||
generic `Sample("llm", engine, f"llm.{metric}", …)` loop (L292–297) and the store's generic
|
||||
rollups; retention needs nothing new. The phase overview says "Python: none *expected*" —
|
||||
this is a small, additive, tested exception; `/api/llm/*` fields keep their names/meanings
|
||||
(additive-only anchor holds), and task 03 adds a unit test per new series plus an
|
||||
`/api/llm/history` round-trip. Everything else in this phase is HTML/JS/CSS.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No changes to `/api/llm/snapshot` consumers outside this page, no `/ups`-style hero glyph
|
||||
work (phase 05), no per-slot sparklines, no touching `llm_monitor` itself.
|
||||
- The stored-metric set is never pruned by this phase: cells leave the *page*, series stay
|
||||
in SQLite (PLAN.md, dropped-visuals anchor).
|
||||
Reference in New Issue
Block a user