dashboards/.agents/phases/todo/04_llm_chart_readability/NOTES-chart-set.md

NOTES-chart-set — what /llm is for, and what it shows

Phase 04, task 01 deliverable (no code changes). Decides the cell set, the colour rules, and the contracts tasks 02–03 implement. Both implementations were read side by side on 2026-10-17; every line number below was verified against the file it cites.

The two sources

Reference — ~/Projects/Experimental/llm_monitor/static/index.html (separate app on :8095; design reference only, never a dependency — ASSUMPTION kept: it is not ours to own, and importing it would break the no-build single-app structure):

What	Where
sparkline CSS vocabulary (.sparks, .spark, .spark-now, .spark-axis, .spark-line, .spark-area)	L398–495
UI.historyLen = 60 (client ring buffer; pushHist L1542–1545; fed per engine L1570–1571, L1582–1583)	L1003
drawSpark(name, values, format, fixedMax)	L1092 (SPARK_W = 100/SPARK_H = 30 L1090–1091)
right-aligned growth — const step = SPARK_W / (UI.historyLen - 1), x = SPARK_W - (values.length - 1 - i) * step	L1158–1163
empty state: < 2 values → empty class, blank paths (never a zero line)	L1112–1118
flat-window honesty: pad keeps the line mid-plot, axis reports the one real value, not the pad (const flat L1128)	L1126–1133
top/middle/bottom scale marks (axis() helper, levels, fmtAxis with enough decimals that the three labels stay distinguishable; axis writes L1155–1157)	L1101–1104, L1135, L1138–1154
fixedMax ratio axes: KV sparks called with fixedMax = 1 → 0–100% always	L1356 (gufo), L1420 (vLLM)
per-engine throughput dds: #gufo-ptps / #gufo-dtps, #vllm-ptps / #vllm-dtps	L757 / L761, L802 / L806
setThroughput(el, cell) — value + honest tag (Δ<w> for measured, lifetime avg, from TPOT, derived; title = exact source)	L1303–1320
percentile foot lines: #vllm-ttft-p95, #vllm-tpot-p95, #vllm-queue-p95 under their mean cards	L950, L955, L960
token counters as cells whose foot line names the upstream series (#vllm-ptok, #vllm-gtok)	L964, L969
slot cards: SLOT_FIELDS L1170–1177, buildSlotCard L1179 (built once, patched in place), renderSlots L1209 (DOM order == slot order, vanished slots removed)	—
the only thresholds in that app: KV bar warn ≥ 0.75, crit ≥ 0.9	L1085–1086
single-engine case: fleet block hidden when one engine — "a cross-engine sum of one engine just duplicates that engine's own card"	L1500–1504
dead engine holds last good data + stale class + error banner, never zeroes	L1514–1535

Current page — src/dashboards/web/pages/llm.html + src/dashboards/web/static/page-llm.js:

What	Where
engine-totals card (#f-decode L45, #f-prompt L47, KV bar L54–56, tokens lifetime L52–53)	llm.html L39–60
four full-height LineCharts #chart-decode/prompt/kv/queue	llm.html L62–81 (chart divs L66, L71, L75, L79)
per-engine panels grid #engines + honest "No engines configured" empty state	llm.html L83–91
gufo slots table #slots-card L94–102; events feed L103–106; provenance footer L109–113	llm.html
CELLS — the 14-equal-size grid this phase dissolves	page-llm.js L16–33
cellValue() — per-key payload mapping	L35–93
provenance() — Δ10.0s / formula tag from source, nothing for scraped gauges	L96–105
enginePanel() / renderPanel()	L112–136 / L138–214
JS thresholds to remove: severityFor(value.value, 75, 90) on the KV bar	L194–195
device_lost non-zero special case (the alarm-chip seed)	L196–200
makeCharts() — height: 165 for all four	L241–249 (165 at L242)
refreshCharts() — three /api/llm/history calls; the third already carries requests_running,requests_waiting,kv_cache_usage	L256–292 (L264–265)
JS thresholds to remove: severityFor(kv, 75, 90) on fleet KV	L307
window chips Dash.rangeControl (affects charts today, sparklines after this phase)	L374

Shared helpers this builds on: charts.js LineChart L43, renderLegend L329 (with showPeak), sparkline L348–373 (the function task 02 upgrades); app.js severityFor L118 (documented FALLBACK ONLY — phases 04–05 exist to retire its remaining call sites), severityFrom L139, hero() L187.

Decision

KEEP (unchanged, do not regress)

• Provenance tags on every throughput number — provenance() (page-llm.js L96–105) and the footer legend (llm.html L109–113). AGENTS.md invariant 2; the llm_monitor equivalent is setThroughput L1303–1320. Ours is strictly better (it prints the measured Δ window); it ports as-is into the hero cells.
• History window chips (page-llm.js L374, Dash.rangeControl). Divergence from the reference on purpose: llm_monitor sparks use a fixed 60-sample client buffer (L1003); our sparklines read /api/llm/history with the selected window, so chips and sparks always agree. Do not copy historyLen.
• Per-engine panels (llm.html L84) with per-source health, stale/last-good dimming, and the upstream chip. Unconfigured engines render no panel at all (L85–90) — invariant 3.
• gufo slot list (llm.html L94–102, renderSlots L216–239). Adopt from llm_monitor only the in-place patch idiom (buildSlotCard L1179) if rows churn; the table itself stays.
• Events card (llm.html L103–106). Phase 02 deleted the raw feed from / only; per-engine up/down transitions stay here.

HERO — the two largest numbers, per engine panel

• llm.decode_tokens_per_second and llm.prompt_tokens_per_second (payload cells built in sources/llm.py: gufo measured Δcounters, vLLM native gauge → measured → lifetime estimate, each labelled with its source).
• Rendered with Dash.hero() (app.js L187) with the Δ<window> provenance in .sub.
• Throughput has NO severity. severity.py THRESHOLDS (L46–57) has no throughput row and gets none: 0 tok/s on an idle server is not red, and a page-invented threshold is exactly what invariant "a threshold exists nowhere else" forbids. The number renders in neutral ink; colour in the hero row comes from the engine's reachability (status dot + .stale dimming, already implemented in renderPanel), mirroring the /system NET slot precedent ("throughput has no good/bad", AGENTS.md UI conventions). Note: the phase checklist says the heroes are "colour-coded" — the honest reading is the page's colour language (neutral numbers, verdict colour on rated cells, reachability colour on the panel). Task 04's side-by-side screenshots should show this; if the operator wants the digits themselves tinted, that needs a ratified severity.py row first, not a JS guess.

SPARKLINE CELLS — compact rows, one per metric, each with its window's history

Cell	Payload source (cellValue, L35–93)	History series for the spark	Colour
Sessions active	sessions.active (gufo /slots, vLLM count)	llm.requests_running ✔ stored	none
Queued	requests_deferred / num_requests_waiting	llm.requests_waiting ✔ stored	none
KV cache %	kv_cache_usage_ratio / gpu_cache_usage (×100)	llm.kv_cache_usage ✔ stored	payload severity
TTFT	time_to_first_token_seconds__mean, foot: __p95	llm.ttft_seconds ✔ stored	none
TPOT	time_per_output_token_seconds__mean, foot: __p95	llm.tpot_seconds — new, see Python delta	none
Queue mean	request_queue_time_seconds__mean	llm.queue_seconds ✔ stored	none
Prefix cache hit %	prefix_cache_hit_ratio / prompt_cache_hit_ratio	llm.prefix_cache_hit_ratio ✔ stored	fixed max 100
Spec accept %	spec_accept_rate (gufo only)	llm.spec_accept_rate — new, see Python delta	fixed max 100

• Percentiles print as foot lines, no hover (reference: L950/L955/L960), from the live snapshot cells — p95 needs no stored series.
• TTFT/TPOT/queue are vLLM-only; spec accept is gufo-only. The other engine's row renders n/a uncoloured (rule 2), or the row is engine-scoped — task 03's choice, but a gufo panel must never show a red/empty latency mystery.
• /api/llm/history (api.py L353–365) answers any metric name; unknown ones come back as empty series — so the page can request the full cell list in one call. refreshCharts (L256–292) already makes three calls; extend the third (L264–265) with the cell series instead of adding requests, and feed both charts and sparks from the same responses.

CONDITIONAL / DEMOTED

• tokens_total (lifetime counter) → quiet footnote line in each engine panel (the existing .sessions-note/.debug strip, page-llm.js L203–213), like llm_monitor's counter cards whose foot line just names the series (L964, L969). A lifetime sum is not a glance metric.
• preemptions and device_lost → alarm chips rendered only when non-zero, absent otherwise (seed: the device_lost special case L196–200). A permanent cell for something that is always 0 trains the eye to ignore the row. Chip class is .chip.fail (existing vocabulary, dash.css L336); the rule is existence of an incident, not a graded threshold — sources/llm.py already treats any non-zero gufo_device_lost_total as an incident (comment at GUFO_SERIES), and zero/null simply draws no chip.
• Engine-totals card (L39–60): keep engines-live, decode/prompt sums, active, queued — demote tokens to footnote, and remove the fleet KV bar (L54–56 + severityFor at L307): fleet_totals (sources/llm.py L325–368, kv_cache_usage_max L360–363) carries no severity, and the rated per-engine KV cells plus the KV chart say it better. Hide the totals card when fewer than two engines are configured — copy llm_monitor L1500–1504 ("a cross-engine sum of one engine just duplicates that engine's own card"); this also satisfies the single-engine completion criterion.

CHARTS

• All four LineCharts stay: decode, prompt, KV, queue — one engine per series (queue chart: running+waiting per engine), existing legend renderLegend (charts.js L329) with showPeak on the bursty throughput plots, units-bearing y-format (already passed).
• Drop height: 165 → ~110 (page-llm.js L242) and move the chart section below the engine panels so the cells read first (the task says half-height "so the cells read first"; keeping full-height charts on top would contradict that).
• Task 03 must pass ariaLabel to all four charts (AGENTS.md: every chart names itself; /system did this in phase 03, /llm never adopted it).

COLOUR RULES (from severity.py and nowhere else)

• KV pressure: rated. THRESHOLDS["llm.kv_cache_usage_pct"] = (75, 90) (severity.py L53), attached per engine cell by SEVERITY_CELLS (sources/llm.py L99–103) with scale=100 because both engines publish a 0–1 ratio. The page reads it via Dash.severityFrom (app.js L139) and both severityFor(..., 75, 90) call sites (L194–195, L307) are deleted — the app.js L112–117 fallback comment stays true only if phase 04 does this.
• Queue depth: not rated — THRESHOLDS has no queue row (L46–57). severityFrom returns '' → no colour. The queue chart and sparkline carry the signal as shape. If the operator wants a verdict, that is a future severity.py row (ratified numbers, tests), never a JS threshold.
• Throughput, sessions, latency (TTFT/TPOT/queue mean): not rated → colourless numbers. Explicit statement per the task: 0 tok/s on an idle server is not red.
• Alarm chips: .chip.fail when the counter is non-zero; absent otherwise (above).

SPARKLINE CONTRACT (input for task 02, mirroring drawSpark L1092–1176)

DashCharts.sparkline() (currently L348–373: full-width autoscale, no labels) becomes:

1. Right-aligned growth: newest sample pinned right, x mapped over the selected time window ([now − range, now], like LineChart._xExtent) rather than over sample index — a partly-filled window grows in from the right exactly like drawSpark's fixed-step buffer (L1158–1163) but stays honest about when samples happened.
2. Fixed max for ratio metrics (KV, prefix hit, spec accept: 0–100) so a squiggle means the same thing on every screen — reference fixedMax calls L1356/L1420; autoscale otherwise.
3. Current/max labels: current value in the cell header (reference .spark-now, L419) and window max as the label; where the reference prints three axis marks (L1135, L1155–1157) our compact rows print now + max — adopt its formatting discipline: enough decimals that labels stay distinguishable (L1137–1152), a flat window reports the one real value, never the pad artifact (L1126–1133).
4. Severity colour: stroke/area from the payload verdict (ok/warn/fail tokens from severityFrom, unrated → --series-1), not llm_monitor's fixed per-engine hue.
5. Empty: < 2 finite points → "no history", never a zero line (reference L1112–1118; current code already does this).

Python delta (the phase's only one, flagged for task 03)

HISTORY_METRICS (sources/llm.py L113–123; sampled at L292–297) does not store TPOT or spec-accept, so those two cells would show permanent "no history" slots — the dead-cell anti-pattern this note exists to prevent. Decision: extend HISTORY_METRICS additively —

• tpot_seconds ← ("time_per_output_token_seconds__mean",) (vLLM only)
• spec_accept_rate ← ("spec_accept_rate",) (gufo only)

New names in the public metric namespace (allowed: "phases may stop plotting a series, they may not rename or stop storing it" — adding is the safe direction), consumed by the generic Sample("llm", engine, f"llm.{metric}", …) loop (L292–297) and the store's generic rollups; retention needs nothing new. The phase overview says "Python: none expected" — this is a small, additive, tested exception; /api/llm/* fields keep their names/meanings (additive-only anchor holds), and task 03 adds a unit test per new series plus an /api/llm/history round-trip. Everything else in this phase is HTML/JS/CSS.

Non-goals

• No changes to /api/llm/snapshot consumers outside this page, no /ups-style hero glyph work (phase 05), no per-slot sparklines, no touching llm_monitor itself.
• The stored-metric set is never pruned by this phase: cells leave the page, series stay in SQLite (PLAN.md, dropped-visuals anchor).