chore(agent): phase roadmap from TODO.md, 5 phases
The five UI items in TODO.md become an executable pipeline. Planning files only — no application code changed. - 01_severity_language: one Python threshold table + `severity` on the values the pages colour, plus the .hero style. Today the thresholds are magic numbers at six JS call sites (75/90 here, 80/92 there), which is exactly how "same colour, four meanings" happens. - 02_overview_fleet_and_alerts: /api/summary gains a derived `alerts` list; the page drops to fleet + warnings. Unconfigured sources must not alert — that is invariant 3, tested. - 03_system_per_host_panels: per-host hero cards, no per-core, shared multi-host charts and the detail picker deleted. The six card metrics already exist in the payload, so the first task pins them with tests instead of inventing fields. - 04_llm_chart_readability: charts.js already exports sparkline() and no page calls it — that is why the page reads flat. Port llm_monitor's drawSpark behaviour (fixed max, right growth, p95 foots) and keep the provenance tags. - 05_ups_hero_battery_glyph: charge/load/runtime hero + an SVG battery whose fill is the charge, verified against both real UPSes and the low-charge/on-battery states only the simulator can produce. PLAN.md records the ratified anchors and the deviation worth knowing: severity vocabulary is ok|warn|fail, matching dash.css and nut.py, not the "danger" wording in the proposal — one synonym for one colour. TODO.md cleared to a bare heading; the items are now phases. .gitignore stops ignoring all of .agents/ (the roadmap is project history) while keeping phase-sessions/ and pipeline.log out. deploy/nut-sudoers rewritten: its header still blamed a port mystery that is solved; the remaining need is reading upsd.users so adding a scoped dashboards user does not delete the accounts upsmon authenticates as. LLM-Generated: true
This commit is contained in:
+129
@@ -0,0 +1,129 @@
|
||||
# Homelab Dashboards — Master Plan
|
||||
|
||||
> Master design document for a phased-execution project. The phase roadmap below was
|
||||
> derived from `TODO.md` on 2026-10-16; each phase overview and task file cites its source
|
||||
> lines in that file. The **LOCKED DECISIONS** below are binding: no phase may introduce
|
||||
> technology outside them without explicit user permission.
|
||||
>
|
||||
> `AGENTS.md` (repo root) is the sibling document: the invariants, the fleet facts, and the
|
||||
> known gaps. It is not superseded by this file — read both.
|
||||
|
||||
## Assumptions & Design Principles
|
||||
|
||||
- **The dashboard's job is a fast, honest glance.** Every TODO item is a legibility
|
||||
complaint, not a feature request: the same truth with less ink and clearer colour.
|
||||
- **One colour language.** Green/amber/red means exactly one thing across all four pages,
|
||||
computed server-side in Python so it is unit-tested and cannot drift per page.
|
||||
- **Fewer numbers, larger.** Removing a metric from a page is a feature, but only when the
|
||||
history stays in the DB. Pages stop plotting what the operator stopped asking for; the
|
||||
stored series are kept so the decision is reversible.
|
||||
- **The existing pages are payload-driven and stay that way.** No page scrapes HTML, no page
|
||||
grows its own business logic. If a card needs a judgement ("is 5.3 years old bad?"), the
|
||||
API supplies it.
|
||||
- **Charts stay hand-rolled SVG.** `llm_monitor`'s per-cell sparklines are the reference for
|
||||
"useful and readable"; they are inline SVG with no library, which is exactly the house
|
||||
style, so they port rather than get replaced.
|
||||
- **Every UI change is verified in a browser against live data** (`AGENTS.md` definition of
|
||||
done) — a screenshot in `data/logs/`, not a claim.
|
||||
|
||||
## Architectural Anchors
|
||||
|
||||
| COMPONENT | DECISION | RATIONALE | STATUS |
|
||||
|-----------|----------|-----------|--------|
|
||||
| Backend | FastAPI, single process, one domain, four routes (`/`, `/system`, `/llm`, `/ups`) | Inherited from the existing code | LOCKED |
|
||||
| Frontend | Vanilla JS + one stylesheet. **No framework, no CDN, no build step, nothing fetched at runtime** | The engines live on a network with no outbound access; a CDN reference is a dead request | LOCKED |
|
||||
| Charts | Hand-rolled SVG in `web/static/charts.js` | Same reason as above; `llm_monitor` proved it works for a year | LOCKED |
|
||||
| History | SQLite (`sample(source,host,metric,res,ts)` + `event`), raw → 1m → 1h rollups | Inherited; bounded, no TSDB to operate | LOCKED |
|
||||
| Writes | Read-only except `POST /api/ups/command`, gated four times | Invariant 1 of `AGENTS.md` | LOCKED |
|
||||
| API shape | **Additive only**: existing `/api/*` fields keep their names and meaning | The four pages, the live tests, and future consumers depend on them | LOCKED (ratified 2026-10-16) |
|
||||
| Severity | New `src/dashboards/severity.py` is the only place thresholds live; API responses carry `severity` per value; CSS/JS only render the class | Testable once, identical on every page; JS business logic is untested by definition | LOCKED (ratified 2026-10-16) |
|
||||
| JS tests | **None added.** Coverage gate (>90%) applies to new/modified Python; UI is proven by payload-contract tests plus browser screenshots | Adding a JS toolchain would violate the no-build-step anchor for no extra confidence in the colour logic | LOCKED (ratified 2026-10-16) |
|
||||
| Dropped visuals | Per-core CPU charts and shared multi-host charts are removed from `/system`; the underlying series stay in SQLite | Explicit user request; reversible because nothing is deleted from history | LOCKED (ratified 2026-10-16) |
|
||||
| Toolchain | Python 3.14, pytest, ruff, pyright, `uv` | Inherited; the lock resolves for 3.14 only | LOCKED |
|
||||
| Planning tree | `.agents/` is tracked and committed; `.agents/phase-sessions/` and `.agents/pipeline.log` are ignored | The roadmap is project history, not scratch | LOCKED (ratified 2026-10-16) |
|
||||
|
||||
## High-Level Architecture
|
||||
|
||||
```
|
||||
agent/host_agent.py (each host) gufo / vLLM upsd (NUT, port 3051)
|
||||
│ GET /snapshot │ GET /metrics │ LIST UPS / LIST VAR / LIST CMD
|
||||
▼ ▼ ▼
|
||||
collectors.py ─── fetch, one UpstreamError type ────────────────────────────────┐
|
||||
│ parse │
|
||||
▼ ▼
|
||||
sources/host.py · sources/llm.py · sources/nut.py poller.py (task per source)
|
||||
│ Result(payload, samples, events) ◄── NEW: severity.py classifies values
|
||||
▼ ▼
|
||||
store.py ── SQLite (WAL): sample + event ──────────────────────────► api.py ──► web/
|
||||
/api/summary is the
|
||||
overview's one call
|
||||
```
|
||||
|
||||
Pages and their scripts (`src/dashboards/web/`):
|
||||
|
||||
| Page | HTML | Script | Charts it builds today |
|
||||
|------|------|--------|------------------------|
|
||||
| `/` | `pages/index.html` | `page-index.js` | none (three summary cards + fleet table + events) |
|
||||
| `/system` | `pages/system.html` | `page-system.js` | `#chart-cpu`, `#chart-mem`, `#chart-disk`, `#chart-net`, `#chart-gpu`, `#chart-temp`, `#chart-detail` — all shared, multi-host |
|
||||
| `/llm` | `pages/llm.html` | `page-llm.js` | `#chart-decode`, `#chart-prompt`, `#chart-kv`, `#chart-queue` |
|
||||
| `/ups` | `pages/ups.html` | `page-ups.js` | battery charge, runtime estimate, voltages, load |
|
||||
|
||||
Shared helpers: `app.js` (nav, poll loop, `fetchJSON`, formatting), `charts.js`
|
||||
(`DashCharts.LineChart`, escape/axis helpers), `dash.css` (tokens, cards, bars).
|
||||
|
||||
## Data Model
|
||||
|
||||
- `sample(source, host, metric, res, ts)` — `source` ∈ {host, llm, ups}; `res` ∈
|
||||
{raw, 1m, 1h}; `metric` is dotted with `@stream` for per-device series
|
||||
(`cpu.usage_pct@core7`, `disk.used_pct@/srv`, `gpu.busy_pct@0`). **The metric namespace is
|
||||
a public interface**: phases may stop plotting a series, they may not rename or stop
|
||||
storing it.
|
||||
- `event(ts, source, kind, severity, message)` — transitions only, which is what the
|
||||
overview's alert list will reuse.
|
||||
- Host payload (`/api/system/snapshot`, per host): `cpu.{usage_pct,temp_c,load1}`,
|
||||
`memory.used_pct`, `disks.{worst_used_pct,worst_mount,util_pct}`,
|
||||
`network.{rx_bytes_per_s,tx_bytes_per_s,interfaces[]}`,
|
||||
`gpus[].{busy_pct,vram_used_bytes,vram_total_bytes,vram_used_pct,mem_used_pct,temp_c}`,
|
||||
`temperatures.{cpu_c,max_c}`, `processes`, `agent.errors`. Everything the per-host cards
|
||||
ask for **already exists** — phase 03 tests the contract rather than inventing fields.
|
||||
- UPS payload (`/api/ups/snapshot`): per card `status`, `states[]`, `on_battery`, `numbers`
|
||||
(`charge_pct`, `load_pct`, `battery_runtime`, `battery_voltage`, `line_voltage`,
|
||||
`battery_temperature`), `health.{overall,severity,checks[],counts}`, `commands[]`
|
||||
(each with `enabled`, `reason`, `confirm_text`, `risk`), `simulated` at the top level.
|
||||
Real fleet today: `apc` (APC Back-UPS RS 1500MS) and `cyberpower` (CyberPower
|
||||
PR1500LCDRT2U) on driveripper's upsd at port **3051**, forwarded to `127.0.0.1:3494` in
|
||||
dev. Neither driver reports a battery temperature; `battery.mfr.date` is the only date
|
||||
that means anything.
|
||||
- `/api/summary` returns `{ts, system:{enabled,fleet,hosts[],states}, llm:{...}, ups:{...},
|
||||
recent_events[]}` — phase 02 adds `alerts` and the overview page shrinks to fleet+alerts.
|
||||
|
||||
## Validation / Verification Workflow
|
||||
|
||||
Per task, in order:
|
||||
|
||||
1. `uv run pytest` — hermetic, no network; must stay green (baseline at conversion:
|
||||
**258 passed, 6 skipped, 94% coverage**).
|
||||
2. `uv run pytest --cov` — coverage on new/modified Python **>90%**; overall must not drop
|
||||
below 94%.
|
||||
3. `uv run ruff check src tests agent && uv run pyright` — clean.
|
||||
4. `uv pip list --outdated` — empty whenever dependencies moved.
|
||||
5. `uv run pytest -m live --live` (with `.env.dev` sourced) — the opt-in suite that drives
|
||||
the production code paths against the real fleet, both engines, and real upsd. Baseline:
|
||||
**6 passed**. UI phases must not break it.
|
||||
6. Browser verification of every touched page with live data, screenshot saved to
|
||||
`data/logs/` (`./scripts/dev-server.sh restart` keeps http://127.0.0.1:8090 up).
|
||||
7. Commit per task with `--no-gpg-sign` and the `LLM-Generated: true` trailer.
|
||||
|
||||
## Phase Roadmap
|
||||
|
||||
| # | Phase | TODO source | Objective (one line) |
|
||||
|---|-------|-------------|----------------------|
|
||||
| 01 | `01_severity_language` | `TODO.md` L5, L7 (colour-coding clauses) | One server-side threshold table + severity on every value the pages colour, plus the CSS/JS vocabulary to render it |
|
||||
| 02 | `02_overview_fleet_and_alerts` | `TODO.md` L3 | Overview is fleet + warnings/errors and nothing else |
|
||||
| 03 | `03_system_per_host_panels` | `TODO.md` L4, L5 | `/system` is per-host hero cards (cpu/mem/temp/disk/gpu/net, no per-core) with per-host charts, no shared charts |
|
||||
| 04 | `04_llm_chart_readability` | `TODO.md` L6 | LLM page readable again: per-cell sparklines and percentiles in the `llm_monitor` idiom, big colour-coded rates |
|
||||
| 05 | `05_ups_hero_battery_glyph` | `TODO.md` L7 | Charge / load / runtime big and colour-coded, with an SVG battery that shows charge |
|
||||
|
||||
There is deliberately **no foundation phase**: pytest, ruff and pyright are configured, the
|
||||
suite is green, and the project launches, which are the only conditions that would have
|
||||
required one.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Phase 01 — Severity language
|
||||
|
||||
**Source:** `TODO.md` L5, L7 — "proper green (good), yellow (warn), red (danger) color coding" / "charge, load, and runtime left to be big, obvious, and color coded"
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Architectural Anchors* (severity computed in Python, JS renders classes), `src/dashboards/sources/host.py`, `src/dashboards/sources/nut.py`, `src/dashboards/api.py`, `src/dashboards/web/static/dash.css`, `src/dashboards/web/static/app.js`
|
||||
|
||||
## Objective
|
||||
|
||||
Introduce one place that decides whether a number is fine, worrying, or bad, and put that
|
||||
verdict on the values the API already returns. Phases 02–05 all want the same colour
|
||||
language; doing it here means they render a class instead of each inventing its own
|
||||
thresholds.
|
||||
|
||||
Deliverable: `severity` next to the value in the payloads the pages colour, and CSS/JS
|
||||
helpers that make a "hero number" one call.
|
||||
|
||||
## Dependencies
|
||||
|
||||
— (none)
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_severity_module.md` — `src/dashboards/severity.py`: the threshold table and the classifier, unit-tested.
|
||||
2. `02_severity_in_payloads.md` — attach `severity` to host and UPS values and to the `/api/summary` rows; API tests.
|
||||
3. `03_hero_number_style.md` — a `.hero` big-number style on the existing `.ok/.warn/.fail` colours, plus `Dash.hero()` and `Dash.severityFrom(payload)` so pages read the verdict instead of passing thresholds.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: every threshold boundary (below warn, exactly at warn, between, exactly at danger,
|
||||
above), `None` input, and the "unknown is not ok" rule — a value the upstream does not
|
||||
publish must get `severity: null`, never a green pass.
|
||||
- API: `/api/system/snapshot`, `/api/ups/snapshot`, `/api/summary` carry `severity` where
|
||||
the spec says, and **no existing field changes name or meaning** (additive-only anchor).
|
||||
- Coverage: **>90%** on new/modified code.
|
||||
- Live: `uv run pytest -m live --live` must stay at 6 passed.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run python -c "from dashboards import severity; print(severity.classify('cpu.usage_pct', 95))"` → `fail`; `60` → `ok`; `None` → `None`
|
||||
- [ ] `curl -s localhost:8090/api/ups/snapshot | python3 -m json.tool` shows `severity` beside `charge_pct`, `load_pct`, `battery_runtime`
|
||||
- [ ] no new colour vocabulary: classes stay `.ok` / `.warn` / `.fail` as in `dash.css`
|
||||
- [ ] `uv run pytest` green; coverage ≥94% overall, >90% on `severity.py`
|
||||
- [ ] `uv run ruff check src tests agent && uv run pyright` clean
|
||||
- [ ] Existing pages render unchanged (severity is additive; nothing consumes it yet) — one screenshot in `data/logs/`
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,61 @@
|
||||
# Task 01 — Severity module
|
||||
|
||||
**Phase:** `01_severity_language` · **Source:** `TODO.md:5` — "proper green (good), yellow (warn), red (danger) color coding" and `TODO.md:7` — "charge, load, and runtime left to be big, obvious, and color coded"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Move the good/worrying/bad decision out of the browser and into one Python module, so the
|
||||
four pages cannot drift into four different opinions about what red means.
|
||||
|
||||
Today the thresholds live in call sites: `page-system.js:129` passes `75, 90`,
|
||||
`page-system.js:139` and `:144` pass `80, 92`, `page-index.js:89` passes `80, 92`,
|
||||
`page-llm.js:195` and `:307` pass `75, 90`, and `Dash.severityFor` (`app.js:105`) defaults
|
||||
to `75/90`. That is the drift this task ends.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/severity.py` (new) —
|
||||
- `THRESHOLDS: dict[str, tuple[float, float]]` keyed by metric name, with the ratified
|
||||
defaults: `cpu.usage_pct` 75/90, `mem.used_pct` 80/92, `disk.used_pct` 80/90,
|
||||
`cpu.temp_c` 75/90, `gpu.busy_pct` 85/95, `gpu.mem_used_pct` 85/95,
|
||||
`llm.kv_cache_usage_pct` 75/90, `ups.charge_pct` 50/20, `ups.battery_runtime` 600/300,
|
||||
`ups.load_pct` 70/90.
|
||||
- `INVERTED: set[str]` — metrics where **lower is worse** (`ups.charge_pct`,
|
||||
`ups.battery_runtime`), so callers cannot get the direction wrong.
|
||||
- `classify(metric: str, value: float | None) -> str | None` returning `ok`, `warn`,
|
||||
`fail`, or `None`. `None` in → `None` out, always.
|
||||
- `attach(metric: str, value: float | None) -> dict` returning
|
||||
`{"value": value, "severity": classify(...)}` for the payload shapes that are plain
|
||||
numbers, and `with_severity(metric, cell)` for payloads that already are
|
||||
`{"value":…, "source":…, "derived":…}` (LLM metrics) — additive, never a rename.
|
||||
- Module docstring stating the two rules: unknown is **not** ok (it is `None`, the page
|
||||
renders `n/a` with no colour), and a threshold exists nowhere else in the app. Ties to
|
||||
the `AGENTS.md` "never invent a number" rule.
|
||||
2. `tests/test_severity.py` (new) —
|
||||
- boundaries inclusive on the warn edge (`75 → warn`, `74.9 → ok`), danger edge
|
||||
(`90 → fail`), every metric in the table, both directions for the inverted ones,
|
||||
unknown metric name → `None` (no exception), `None` and `float("nan")` → `None`.
|
||||
- a table sanity test asserting each entry's bounds are ordered correctly for its
|
||||
direction, so a future edit cannot silently invert a metric.
|
||||
- ASSUMPTION: the numbers above were confirmed in chat on 2026-10-16; they match what
|
||||
the call sites already pass, except the two UPS metrics which are new.
|
||||
- ASSUMPTION: the vocabulary is **`ok|warn|fail`**, not `ok|warn|danger` — it is what
|
||||
`dash.css` (`.bar.ok`, `.dot.warn`, `.chip.fail`, `ul.checks li.fail`) and
|
||||
`sources/nut.py`'s `health.checks[].status` already use. The TODO's word "danger" maps
|
||||
to `fail`; do not introduce a synonym for the same colour.
|
||||
- ASSUMPTION: `sources/nut.py`'s existing `health.checks[].status` values are left
|
||||
untouched (additive-only anchor). `severity` is a new field on *values*, computed by
|
||||
this module; `nut.assess()` keeps its own judgement about *conditions*.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit only, no I/O, no fixtures needed.
|
||||
- Coverage: **>90%** on `severity.py`.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest tests/test_severity.py -v` green
|
||||
- [ ] `uv run python -c "from dashboards.severity import classify as c; print(c('cpu.usage_pct',95), c('cpu.usage_pct',60), c('cpu.usage_pct',None), c('ups.charge_pct',15))"` → `fail ok None fail`
|
||||
- [ ] `uv run pytest` green overall; `ruff` + `pyright` clean
|
||||
- [ ] Nothing else changed yet — this task adds a leaf module and its tests
|
||||
@@ -0,0 +1,62 @@
|
||||
# Task 02 — Severity in the payloads
|
||||
|
||||
**Phase:** `01_severity_language` · **Source:** `TODO.md:5` — "proper green (good), yellow (warn), red (danger) color coding" and `TODO.md:7` — "big, obvious, and color coded"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Put the verdict where the pages can read it: every value that will be colour-coded travels
|
||||
with a `severity`, computed server-side.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/host.py` — in `parse_host_snapshot()`'s payload, add a
|
||||
`"severity"` sub-dict to the sections that have rated values, leaving every existing key
|
||||
exactly as it is:
|
||||
- `cpu.severity = {"usage_pct": …, "temp_c": …}`
|
||||
- `memory.severity = {"used_pct": …}`
|
||||
- `disks.severity = {"worst_used_pct": …}`
|
||||
- each row of `gpus[]` gets `"severity": {"busy_pct": …, "mem_used_pct": …}`
|
||||
Use `severity.classify(metric, value)`; import it as `from ..severity import classify`
|
||||
(or `from .. import severity` if that reads better against the existing imports).
|
||||
2. `src/dashboards/sources/nut.py` — in `payload()`, add
|
||||
`"severity": {"charge_pct": …, "load_pct": …, "battery_runtime": …}` to each UPS card.
|
||||
Do not touch `health`, `checks[].status`, or `numbers` — the checks answer "is this
|
||||
condition satisfied", severity answers "how bad is this number", and both stay.
|
||||
3. `src/dashboards/sources/llm.py` — for the metrics that have thresholds
|
||||
(`kv_cache_usage`-family and any other entry in `THRESHOLDS`), attach `severity` inside
|
||||
the existing `{"value":…, "source":…, "derived":…}` cell via `severity.with_severity`,
|
||||
which must leave `value`/`source`/`derived` untouched.
|
||||
4. `src/dashboards/api.py` — in `/api/summary`, add a `severity` key to each host row
|
||||
(`cpu_usage_pct`, `mem_used_pct`, `disk_worst_used_pct`, `temp_c`), each UPS row
|
||||
(`charge_pct`, `load_pct`, `runtime_seconds`), and reuse the section severities for the
|
||||
fleet table. `/api/health` and `/api/config` are unchanged.
|
||||
5. Tests — extend `tests/test_host_source.py`, `tests/test_nut.py`, `tests/test_llm_source.py`
|
||||
and `tests/test_api.py`:
|
||||
- a fixture host whose values straddle thresholds yields the expected severity per field
|
||||
(drive it from the real `tests/fixtures/agent_driveripper.json`, do not hand-copy a
|
||||
snapshot);
|
||||
- the APC fixture/UPS reading with `battery.charge=100` → `ok`, and the same reading with
|
||||
charge forced to 15 → `fail`;
|
||||
- a value the upstream omits (`battery_temperature` is missing on both real UPSes) gets
|
||||
`severity: null`, and the existing `unknown` health check still says `unknown`;
|
||||
- `/api/summary` still contains every key it had before — assert the shape did not shrink
|
||||
(additive-only anchor), then assert the new `severity` keys are present.
|
||||
- ASSUMPTION: severity is attached as a sibling `"severity"` dict rather than by
|
||||
restructuring `cpu.usage_pct` into `cpu.usage_pct.value`, because the existing
|
||||
pages, `tests/test_api.py`, and the live tests read the flat form.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit + API tests above; run the full suite (`uv run pytest`).
|
||||
- Coverage: **>90%** on the changed functions in `sources/host.py`, `sources/nut.py`,
|
||||
`sources/llm.py`, `api.py`.
|
||||
- `uv run pytest -m live --live` against the real fleet + real upsd must stay green — this
|
||||
is the first task whose output can be checked against actual numbers.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/api/system/snapshot | python3 -c "import json,sys; d=json.load(sys.stdin); print(d['hosts'][0]['cpu']['severity'], d['hosts'][0]['memory']['severity'])"` prints a verdict per rated field
|
||||
- [ ] `curl -s localhost:8090/api/ups/snapshot` shows `severity` for charge / load / runtime on both `apc` and `cyberpower`, and `battery_temperature` is `null`
|
||||
- [ ] no existing key was renamed or removed in any payload (diff the JSON keys before/after)
|
||||
- [ ] `uv run pytest` green, coverage not below 94%, `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,54 @@
|
||||
# Task 03 — Hero-number style and the JS side of severity
|
||||
|
||||
**Phase:** `01_severity_language` · **Source:** `TODO.md:5,7` — "Larger numbers" / "big, obvious, and color coded"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Give the pages one way to render a big, colour-coded number, reading the verdict from the
|
||||
payload instead of comparing thresholds in JavaScript.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/dash.css` — add a `.hero` block built entirely from the
|
||||
existing tokens (`--ok`, `--ok-soft`, `--warn`, `--warn-soft`, `--fail`, `--fail-soft`,
|
||||
`--text`, `--text-muted`, `--text-faint`):
|
||||
- `.hero` (label + value + unit), `.hero .value` at a display size (roughly 2.4rem, tabular
|
||||
numerals so digits do not jitter between polls), `.hero .unit` small and muted,
|
||||
`.hero .sub` for the `Δ10.0s` / "at current load" footnotes;
|
||||
- `.hero.ok .value { color: var(--ok); }` and the `.warn` / `.fail` equivalents — **reuse
|
||||
the `ok|warn|fail` classes the stylesheet already uses** (`.bar.ok`, `.dot.warn`,
|
||||
`.chip.fail`), do not add a parallel `.sev-*` family;
|
||||
- a `.hero-bar` variant that pairs the number with the existing `.bar` element, since the
|
||||
per-host cards (phase 03) and the UPS hero row (phase 05) both want number + bar.
|
||||
2. `src/dashboards/web/static/app.js` —
|
||||
- `hero(el, {label, value, unit, digits, severity, sub})`: renders/updates a hero, writes
|
||||
`naText` when `value` is null **and applies no severity class** (an unknown number must
|
||||
not look green);
|
||||
- `severityFrom(cellOrSeverity)`: accepts a payload cell (`{value, severity}`), a bare
|
||||
severity string, or `null`, and returns the class name to apply — this is what lets a
|
||||
page stop knowing thresholds;
|
||||
- keep `severityFor(value, warnAt, failAt)` working (pages still call it in
|
||||
`page-index.js`, `page-system.js`, `page-llm.js`, `page-ups.js`) but add a comment that
|
||||
it is the fallback for values the API does not rate, and that new call sites should read
|
||||
`severity` from the payload instead.
|
||||
3. Do **not** convert the existing pages in this task — phases 02–05 do that page by page so
|
||||
each page's change is reviewable and revertible on its own.
|
||||
- ASSUMPTION: leaving `severityFor` in place during this phase keeps every page rendering
|
||||
exactly as it does today, which is what "no behaviour change until your phase" requires.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- No JS test framework exists (locked anchor). The contract that matters here is that the
|
||||
helpers are the only path to a colour class, which the next phases' diffs make obvious.
|
||||
- If `tests/test_api.py` or a new test can assert the static assets are served and the CSS
|
||||
contains the `.hero` rules (a string check on `GET /static/dash.css`), add it — that is
|
||||
the cheap, honest end-to-end signal available without a browser test runner.
|
||||
- Coverage: >90% on any Python touched (ideally none beyond that static-asset assertion).
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/static/dash.css | grep -c "\.hero"` > 0
|
||||
- [ ] Every page still renders as before, unchanged, verified by a screenshot of `/`, `/system`, `/llm`, `/ups` in `data/logs/`
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
- [ ] no `.sev-*` classes were introduced; colour still comes only from `--ok/--warn/--fail`
|
||||
@@ -0,0 +1,47 @@
|
||||
# Phase 02 — Overview is fleet and alerts
|
||||
|
||||
**Source:** `TODO.md` L3 — "The overview page should only show Fleet and warnings/errors. Should be very simple."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Data Model* (`/api/summary`), `01_severity_language` (alerts reuse the verdicts), `src/dashboards/api.py:153` (`/api/summary`), `src/dashboards/web/pages/index.html`, `src/dashboards/web/static/page-index.js`
|
||||
|
||||
## Objective
|
||||
|
||||
Turn the landing page into one question answered instantly: *is anything wrong, and which
|
||||
page do I need?* Fleet table plus a warnings/errors panel — the three summary cards and the
|
||||
raw event feed go away.
|
||||
|
||||
The event list is what makes the current overview busy. Its useful content is not "vLLM is
|
||||
answering again" (that is history) but "something is wrong right now", which is a derived
|
||||
state and belongs server-side.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — alerts are built from the `severity` verdicts and the UPS
|
||||
`health.checks[]` results, not from new thresholds invented in the browser.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_alerts_builder.md` — `src/dashboards/alerts.py`: pure function from hub states + payloads to an alert list; `/api/summary` returns it.
|
||||
2. `02_rewrite_overview_page.md` — `index.html` + `page-index.js` reduced to fleet table + alerts panel.
|
||||
3. `03_retire_unused_overview_code.md` — delete the dead cards/events code and confirm the API contract other pages depend on is intact.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_alerts.py` — a source down, a source stale-but-serving, a UPS `fail`
|
||||
check, a threshold breach, an agent reporting `degraded`, an unconfigured source (must
|
||||
**not** alert — unconfigured is not broken, `AGENTS.md` invariant 3), a simulated UPS
|
||||
(warn, labelled), and the all-clear case producing `[]`.
|
||||
- API: `/api/summary` gains `alerts` and keeps every existing key.
|
||||
- Coverage: **>90%** on `alerts.py` and the changed part of `api.py`.
|
||||
- Browser: screenshot of `/` with a real problem present (stop the driveripper agent tunnel
|
||||
and re-shoot) — an overview that has never been seen in its alarm state has not been
|
||||
tested.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/` shows exactly two panels: Fleet and Warnings/Errors
|
||||
- [ ] `curl -s localhost:8090/api/summary | python3 -c "import json,sys; print(json.load(sys.stdin)['alerts'])"` is `[]` when everything is healthy
|
||||
- [ ] Killing one host tunnel produces exactly one alert naming that host, with a link to `/system`
|
||||
- [ ] `uv run pytest` green (plus `-m live --live` unchanged), coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] no behaviour change on `/system`, `/llm`, `/ups`
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,56 @@
|
||||
# Task 01 — Alerts builder
|
||||
|
||||
**Phase:** `02_overview_fleet_and_alerts` · **Source:** `TODO.md:3` — "The overview page should only show Fleet and warnings/errors."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Derive "what is wrong right now" on the server, so the overview is a list of verdicts and
|
||||
every other consumer (future mobile view, notification later) gets the same answer.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/alerts.py` (new) — `build(settings, hub_states, host_payloads, engine_payloads, ups_payload, recent_events) -> list[dict]`,
|
||||
each alert `{"id", "severity": "ok|warn|fail", "source", "message", "href", "at"}`:
|
||||
- **source down** — a configured job whose `Job.state.ok` is false → `fail`, message from
|
||||
the recorded error, `href` to its dashboard. Stale-but-serving (`age_seconds` beyond
|
||||
~3× its poll interval) is `warn`, not `fail`: the last good value is still on screen and
|
||||
saying "dead" would be a lie.
|
||||
- **host threshold breach** — from the `severity` sub-dicts added by phase 01: any
|
||||
`fail` → an alert naming host + metric + value; `warn` → `warn`.
|
||||
- **agent degraded** — `payload["degraded"]` non-empty (a `/proc` read failed on the box)
|
||||
→ `warn`, naming the sections.
|
||||
- **UPS** — any `health.checks[]` entry with `status: "fail"` → `fail`, `"warn"` → `warn`,
|
||||
carrying the check name and its detail (`"Battery age: 5.3 years — past the 4-year
|
||||
replacement window"` is exactly the kind of thing the landing page should say);
|
||||
`simulated: true` → one `warn` alert saying the numbers are modelled, not batteries.
|
||||
- **bad target config** — `settings.host_target_problems` → `warn` per problem.
|
||||
- **never** for an unset/unconfigured source. `AGENTS.md` invariant 3: a source nobody
|
||||
installed must not raise a red flag.
|
||||
- **dedupe and bound** — one alert per (source, metric); cap the list (say 12) with the
|
||||
count of suppressed ones so a 40-disk fleet cannot flood the page.
|
||||
- sort fail → warn, then by source name.
|
||||
2. `src/dashboards/api.py` — `/api/summary` calls it and adds `"alerts": [...]`. Keep
|
||||
`system`, `llm`, `ups`, `recent_events` exactly as they are (additive-only anchor),
|
||||
because `/api/summary` is also the cheapest whole-fleet probe anyone has.
|
||||
3. `tests/test_alerts.py` (new) — every bullet above, using the existing fixtures
|
||||
(`tests/fixtures/agent_*.json`) and the fakes in `tests/conftest.py`; the all-clear case
|
||||
asserts `[]`, and the unconfigured-source case asserts no alert is produced even though
|
||||
the state map has nothing for it.
|
||||
- ASSUMPTION: `recent_events` is passed in only so a *currently failing* source can show
|
||||
when it started failing; the event list itself is not re-rendered as alerts.
|
||||
- ASSUMPTION: staleness threshold is 3× the source's own poll interval rather than a
|
||||
single global number, because system polls at 10s and UPS at 5s.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit tests for all rules; no I/O in `alerts.py` (it is a pure projection, which is what
|
||||
makes it worth its own module).
|
||||
- Coverage: **>90%**.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest tests/test_alerts.py -v` green
|
||||
- [ ] `curl -s localhost:8090/api/summary | python3 -m json.tool | head -40` shows `alerts` alongside the untouched `system`/`llm`/`ups`
|
||||
- [ ] with a healthy fleet the list is `[]`; with `NUT_SIMULATE=1` it contains exactly the simulated-data warning
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,59 @@
|
||||
# Task 02 — Rewrite the overview as fleet + alerts
|
||||
|
||||
**Phase:** `02_overview_fleet_and_alerts` · **Source:** `TODO.md:3` — "The overview page should only show Fleet and warnings/errors. Should be very simple."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The landing page becomes two things: the fleet table, and a list of what needs attention.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/index.html` —
|
||||
- delete the three dashboard summary cards: the whole
|
||||
`<section class="grid auto" aria-label="Dashboards">` block, i.e. `#sys-hosts-live`,
|
||||
`#sys-cpu`, `#sys-mem`, `#sys-mem-unit`, `#sys-disk`, `#sys-gpus`, `#llm-live`,
|
||||
`#llm-decode`, `#llm-prompt`, `#llm-active`, `#llm-waiting`, `#llm-kv`, `#ups-list`
|
||||
and the `#dot-system` / `#dot-llm` / `#dot-ups` dots they own;
|
||||
- delete the events panel (`#events` and its card) — alerts replace it;
|
||||
- keep the masthead, `#nav-status` (the header source dots already live in `app.js`
|
||||
chrome), `#clock`, `#theme-toggle`, `#config-chip`, `#banners`, `#fleet-table`,
|
||||
`#fleet-empty`, `#foot-versions`;
|
||||
- add one panel **Warnings & errors** (`#alerts`, `aria-live="polite"`) with an empty
|
||||
state that says the fleet is clear — an empty list is a message, not a blank box.
|
||||
- keep `#fleet-table` columns as they are (host, CPU, load 1, mem, worst disk, GPU busy,
|
||||
temp, up). It is already the glanceable fleet view; simplifying it further loses the
|
||||
comparison that makes it worth having.
|
||||
2. `src/dashboards/web/static/page-index.js` —
|
||||
- delete `renderEvents()` and the `#sys-*` / `#llm-*` / `#ups-list` population code in
|
||||
`renderSummary()`;
|
||||
- add `renderAlerts(alerts)`: one row per alert with the severity class from phase 01
|
||||
(`.ok`/`.warn`/`.fail`), the source name as a link to its dashboard (`href` from the
|
||||
API), the message, and the age via `Dash.age()`; empty → the "all clear" state;
|
||||
- `refresh()` still calls `/api/summary` once and updates `#summary-line`
|
||||
("3 hosts · 2 engines · 2 UPS · 1 warning") — that one line is the only summary left;
|
||||
- keep `Dash.bindPause`, `Dash.rangeControl` (if the page used it) and the poll loop.
|
||||
3. Layout: alerts panel sits above the fleet table when something is wrong and below it when
|
||||
the list is empty, so a clean morning looks calm and a bad one puts the problem first.
|
||||
- ASSUMPTION: the UPS/LLM summary content the cards showed is *available* on their own
|
||||
pages, so removing it here loses nothing that is not one click away. The header status
|
||||
dots stay, because "which of the three is down" is the overview's job.
|
||||
- ASSUMPTION: `#banners` (config/setup hints, simulated-data notice) stays — it is a
|
||||
warning surface, which is exactly what the TODO keeps.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Update `tests/test_api.py` only if it asserts page structure; the page is not unit-tested
|
||||
(locked anchor: no JS tests).
|
||||
- Browser check with live data, both states:
|
||||
healthy (`alerts: []` → "all clear") and broken (stop one host agent:
|
||||
`ssh <host> "pkill -f '[h]ost_agent.py'"` or stop its tunnel) → one red row naming it.
|
||||
Screenshots to `data/logs/overview-clear.png` and `data/logs/overview-alert.png`.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/` renders exactly two panels; `document.querySelectorAll('section').length` is 2 (check via the browser console or an `evaluate`)
|
||||
- [ ] `#events`, `#sys-cpu`, `#llm-kv`, `#ups-list` no longer exist in the served HTML (`curl -s localhost:8090/ | grep -c 'id="sys-cpu"'` → 0)
|
||||
- [ ] alert rows link to `/system`, `/llm`, `/ups` correctly
|
||||
- [ ] both screenshots exist in `data/logs/` and show what they claim
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,43 @@
|
||||
# Task 03 — Retire the dead overview code, keep the API honest
|
||||
|
||||
**Phase:** `02_overview_fleet_and_alerts` · **Source:** `TODO.md:3` — "Should be very simple." (simplicity includes not carrying code nothing renders)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Leave no orphaned code behind, and prove the `/api/summary` fields the *other* consumers
|
||||
rely on were not quietly dropped along with the cards.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-index.js` — remove helpers that only the deleted cards
|
||||
used; run a grep for each symbol before deleting so nothing still referenced goes:
|
||||
`grep -n "sys-cpu\|llm-kv\|ups-list\|renderEvents" src/dashboards/web/static/*`.
|
||||
2. `src/dashboards/web/static/dash.css` — drop rules that only the deleted cards matched
|
||||
(check with `grep -n "metric\b" dash.css` against the remaining pages before removing;
|
||||
`.metric` is used by `/system` and `/llm` too, so most of it stays).
|
||||
3. `src/dashboards/api.py` — **do not remove** the `system`/`llm`/`ups` sections from
|
||||
`/api/summary` even though the page no longer renders them: it is the single cheapest
|
||||
whole-fleet probe, the live tests and future alerting want it, and the API anchor is
|
||||
additive-only. Add a short comment on `/api/summary` saying the page renders
|
||||
`alerts` + `system.fleet`/`hosts` and that the rest is for programmatic use.
|
||||
4. `tests/test_api.py` — add an explicit regression test: `/api/summary` contains the keys
|
||||
`system.enabled`, `system.fleet`, `system.hosts`, `llm.engines`, `ups.ups`,
|
||||
`recent_events`, `alerts`, so a later cleanup cannot silently shrink the contract.
|
||||
5. `README.md` — the overview row in the top table still says "one request, all of it";
|
||||
update the wording to "fleet + alerts in one request".
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Full suite green, including the new contract test.
|
||||
- Coverage: >90% on changed Python (`api.py` only).
|
||||
- `grep` evidence that no dangling selector survives (paste the empty grep result in the
|
||||
commit body or task notes).
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `grep -rn "sys-cpu\|llm-kv\|ups-list\|renderEvents" src/dashboards/web/` returns nothing
|
||||
- [ ] `curl -s localhost:8090/api/summary` still returns every key listed in step 4
|
||||
- [ ] `uv run pytest` green (contract test included), coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] `uv run pytest -m live --live` still 6 passed
|
||||
- [ ] no behavior change in completed phases (01's severity fields still present)
|
||||
@@ -0,0 +1,50 @@
|
||||
# Phase 03 — System page is per-host panels
|
||||
|
||||
**Source:** `TODO.md` L4 — "The system page doesn't need shared charts for anything. It should only have per-host charts." · L5 — "Per host charts need to be simpler. Larger numbers, proper green/yellow/red color coding. Don't need per-core. Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Anchors* (dropped visuals are intentional, series stay in SQLite), `01_severity_language`, `src/dashboards/web/pages/system.html`, `src/dashboards/web/static/page-system.js`, `src/dashboards/sources/host.py`
|
||||
|
||||
## Objective
|
||||
|
||||
`/system` stops drawing seven fleet-wide charts and a "pick a host, pick a metric" detail
|
||||
chart. It becomes one panel per host: six big colour-coded numbers — cpu %, mem %, cpu temp,
|
||||
disk usage %, GPU usage + VRAM, network up/down — and that host's own chart underneath.
|
||||
|
||||
Nothing per-core anywhere on the page, and nothing that requires a selection to make sense.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — the cards render `severity` from the payload; they do not
|
||||
get to keep their own thresholds.
|
||||
- `02_overview_fleet_and_alerts` (todo) — the fleet-wide "what stands out" role moved to the
|
||||
overview's alerts list, which is what makes removing the shared charts safe.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_host_card_payload_contract.md` — prove the six card metrics already exist in `/api/system/snapshot` and pin them with tests (add fields only if genuinely missing).
|
||||
2. `02_hero_cards_without_per_core.md` — rewrite `hostCard()`/`renderHostCard()` into big colour-coded numbers, no `renderCores`.
|
||||
3. `03_per_host_charts_replace_shared.md` — delete the fleet-chart and series-detail sections; give each host its own compact charts.
|
||||
4. `04_verify_every_host_shape.md` — browser verification across a discrete GPU, an APU, and a box with no GPU.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Python: contract tests for the payload fields the cards read; coverage **>90%** on any
|
||||
changed Python (ideally none — this phase is frontend plus tests).
|
||||
- `uv run pytest -m live --live` must stay green: `test_every_host_agent_answers_and_parses`
|
||||
is what proves the fields exist on all three real hosts, including the ones that report
|
||||
`n/a`.
|
||||
- Browser, live data, every host shape: driveripper (discrete R9700), deskwork (Strix Halo
|
||||
APU — VRAM *and* GTT, so "3% VRAM" and "78% memory" are both true), gateway (no GPU at all
|
||||
→ the honest "no GPU reported by this agent" state).
|
||||
- The `cpu.usage_pct@coreN` series must still be stored and still be queryable from
|
||||
`/api/system/history` — removing the plot is not removing the data.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/system` contains no chart whose legend lists more than one host
|
||||
- [ ] no per-core visualisation anywhere on `/system` (`grep -rn "renderCores\|core_usage" src/dashboards/web/` → nothing, or charts.js-only if kept unused on purpose)
|
||||
- [ ] each host panel shows exactly: cpu %, mem %, cpu temp, disk %, GPU busy + VRAM (or the no-GPU state), network ↓/↑
|
||||
- [ ] `curl -s "localhost:8090/api/system/history?host=driveripper&metric=cpu.usage_pct@core7&range=1h"` still returns points
|
||||
- [ ] three screenshots in `data/logs/`, one per host shape
|
||||
- [ ] `uv run pytest` green, coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,50 @@
|
||||
# Task 01 — Host card payload contract
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:5` — "Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Establish, with tests, that everything the new cards need is already served — so the card
|
||||
rewrite is a rendering change and nobody "fixes" a field that was never missing.
|
||||
|
||||
## Work
|
||||
|
||||
1. Read the current payload in `parse_host_snapshot()` (`src/dashboards/sources/host.py`)
|
||||
and confirm the six card metrics map to:
|
||||
- cpu % → `cpu.usage_pct`
|
||||
- mem % → `memory.used_pct`
|
||||
- cpu temp → `temperatures.cpu_c` **and** `cpu.temp_c` (both exist; pick one and say which
|
||||
in the card code comment — `cpu.temp_c` is the one `/api/summary` already uses)
|
||||
- disk usage % → `disks.worst_used_pct` + `disks.worst_mount` (the mount is what makes the
|
||||
number actionable)
|
||||
- GPU → `gpus[].busy_pct`, `gpus[].vram_used_bytes`, `gpus[].vram_total_bytes`,
|
||||
`gpus[].vram_used_pct`, `gpus[].mem_used_pct` (the honest APU view), `gpus[].model`
|
||||
- network ↓/↑ → `network.rx_bytes_per_s`, `network.tx_bytes_per_s`
|
||||
plus the phase-01 `severity` sub-dicts (`cpu.severity`, `memory.severity`,
|
||||
`disks.severity`, `gpus[].severity`).
|
||||
2. `tests/test_system_contract.py` (new) — one test that runs a real fixture snapshot
|
||||
(`tests/fixtures/agent_driveripper.json`) through `parse_host_snapshot()` and asserts each
|
||||
of the paths above exists (keys present, values may be `None`), and one that runs the
|
||||
`agent_gateway.json` fixture (no GPU) asserting `gpus == []` rather than a missing key.
|
||||
3. Add the **only** payload change this phase is allowed, if the card needs it:
|
||||
`disks.worst.severity` must be present (it comes from phase 01) and
|
||||
`network` gets no severity (throughput has no good/bad — say so in a comment so nobody
|
||||
adds a meaningless green later).
|
||||
4. Note in the test docstring what is *deliberately* not on the card: per-core usage
|
||||
(still stored as `cpu.usage_pct@coreN`), swap, processes, sensors, per-interface rows.
|
||||
- ASSUMPTION: the card reads `cpu.temp_c`; if the executor finds it absent where
|
||||
`temperatures.cpu_c` is present on a real host, use `temperatures.cpu_c` and record the
|
||||
choice in the commit message rather than changing the payload.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- New tests green; they must fail if a field is renamed, which is the point.
|
||||
- `uv run pytest -m live --live` for the same assertions against the real fleet.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest tests/test_system_contract.py -v` green
|
||||
- [ ] no payload field renamed or removed (`git diff src/dashboards/sources/host.py` is empty or adds only a comment)
|
||||
- [ ] `uv run pytest`, `ruff`, `pyright` clean
|
||||
- [ ] no visual change yet — this task changes no page code
|
||||
@@ -0,0 +1,60 @@
|
||||
# Task 02 — Hero cards, no per-core
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:5` — "Per host charts need to be simpler. Larger numbers, proper green (good), yellow (warn), red (danger) color coding. Don't need per-core. Just need cpu%, mem%, cpu temp, disk usage %, graphics card details (usage and vram), and network up and down."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Each host panel leads with six big colour-coded numbers and nothing else in the way.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-system.js` — `hostCard()`:
|
||||
- replace the two `.metrics` blocks with a single hero row built from the phase-01 `.hero`
|
||||
markup: CPU %, MEM %, TEMP °C, DISK %, GPU %, NET ↓/↑ (network is one hero with two
|
||||
lines — in and out — rather than two competing numbers);
|
||||
- **delete the per-core block** (`<div class="label faint">Per-core</div>` and
|
||||
`data-k="cores"`);
|
||||
- drop from the top row: Load 1/15, Clock GHz, Disk read/write/busy, Up — into the
|
||||
existing `<details>` (see below), not deleted from the page's reach;
|
||||
- keep `data-k` names for whatever survives so `renderHostCard()` keeps working, and give
|
||||
each hero a `data-sev` hook for the severity class.
|
||||
2. `renderHostCard()`:
|
||||
- drive colour from `host.cpu.severity.usage_pct`, `host.memory.severity.used_pct`,
|
||||
`host.disks.severity.worst_used_pct`, `gpu.severity.busy_pct` — via
|
||||
`Dash.severityFrom(...)`; delete the `severityFor(value, 75, 90)` calls on
|
||||
`page-system.js:129/139/144`;
|
||||
- value `null` → `n/a` with **no** severity class;
|
||||
- memory hero shows `%` big, `200 / 253 GiB` as the `.sub` line (the percentage is the
|
||||
verdict, the bytes are the detail);
|
||||
- disk hero shows `%` big, the mount (`disks.worst_mount`) as `.sub` — "80 % of what?"
|
||||
must be answerable without a click;
|
||||
- GPU block: one line per card — model, `busy %` with severity, `VRAM used / total` and
|
||||
the `.sub` pool view (`mem_used_pct`) for APUs. No GPU → the existing
|
||||
"No GPU reported by this agent" line;
|
||||
- network hero uses `Dash.rate()` on `network.rx_bytes_per_s` / `tx_bytes_per_s`, and
|
||||
renders `n/a` when `any_net` produced nulls rather than a confident `0 B/s`;
|
||||
- the existing `<details>` becomes "More detail": load averages, clock, disk read/write/
|
||||
busy, uptime, filesystems, top processes. Collapsed by default.
|
||||
3. `src/dashboards/web/static/dash.css` — add only what the hero row needs if phase 01 did
|
||||
not already (grid of 6 heroes, wraps to 2×3 and 3×2 at narrow widths; `.cores` rule can
|
||||
stay until task 03 confirms nothing else uses it).
|
||||
- ASSUMPTION: `temp` uses `cpu.temp_c` (see task 01). Hosts with no CPU sensor show `n/a`
|
||||
uncoloured — `gateway` in this fleet.
|
||||
- ASSUMPTION: the `.details`/`<details>` disclosure keeps the information that is not
|
||||
glance-worthy. The TODO asks for a simpler surface, not a less complete page.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- No JS tests (locked anchor). Contract is covered by `tests/test_system_contract.py`.
|
||||
- Browser: verify against live data on driveripper (GPU), and confirm the severity colours
|
||||
change when a value crosses a threshold — e.g. temporarily override with
|
||||
`THRESHOLDS` in a REPL or by pointing at a host under load. One screenshot per state.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] each host card shows exactly six heroes in this order: CPU, MEM, TEMP, DISK, GPU, NET
|
||||
- [ ] `grep -n "cores\|renderCores" src/dashboards/web/static/page-system.js` → no matches in the card code
|
||||
- [ ] no `severityFor(` call remains in `page-system.js`
|
||||
- [ ] a host with no GPU and no CPU temp renders `n/a` with no colour (screenshot)
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,66 @@
|
||||
# Task 03 — Per-host charts replace the shared ones
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:4` — "The system page doesn't need shared charts for anything. It should only have per-host charts." and `TODO.md:5` — "Per host charts need to be simpler"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Delete the six fleet charts and the "pick a host / pick a metric" detail chart. Every chart
|
||||
that survives belongs to exactly one host and is inside that host's panel.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/system.html` — delete:
|
||||
- the whole `<section class="grid cols-2" aria-label="Fleet charts">` (L65–91):
|
||||
`#chart-cpu`, `#legend-cpu`, `#chart-mem`, `#legend-mem`, `#chart-disk`, `#legend-disk`,
|
||||
`#chart-net`, `#legend-net`, `#chart-gpu`, `#legend-gpu`, `#chart-temp`, `#legend-temp`;
|
||||
- the whole "Series detail" `<section>` (L93–107): `#detail-host`, `#detail-metric`,
|
||||
`#chart-detail`;
|
||||
- the `<section class="card flat" aria-label="Fleet totals">` (L40–63) — its job moved to
|
||||
the overview's fleet table + alerts in phase 02;
|
||||
- inside each host panel, add a chart slot: `<div class="chart sm" data-chart="util"></div>`
|
||||
and `<div class="chart sm" data-chart="net"></div>` (plus `data-chart="gpu"` rendered
|
||||
only when the host reports GPUs).
|
||||
2. `src/dashboards/web/static/page-system.js` —
|
||||
- delete `makeCharts()`'s seven `state.charts.*` constructions, `refreshCharts()`,
|
||||
`refreshGpuChart()`, and the `#detail-*` select wiring;
|
||||
- add `renderHostCharts(host)`: per host, **one** utilisation chart (cpu %, mem %,
|
||||
disk-worst %, shared 0–100 y-axis, three lines, one legend) and **one** network chart
|
||||
(↓ and ↑ on the same axis), plus a GPU utilisation chart when `host.gpus.length`;
|
||||
- source them from `/api/system/history` — the endpoint already returns per-host series
|
||||
(`{"hosts": [...], "series": [{"host", "metric", "points"}]}`), so this is a query per
|
||||
host-metric set, not a new endpoint; keep one request per host per refresh window, not
|
||||
one per metric, if the endpoint supports a metric list (it accepts
|
||||
`host=`/`metric=`/`range=` — reuse `seriesFor()`);
|
||||
- respect the existing range control (`state.range`) and `Dash.bindPause`;
|
||||
- height ~120px: these are supporting evidence under a number, not the main event.
|
||||
3. `src/dashboards/web/static/charts.js` — if `renderCores` now has no callers
|
||||
(`grep -rn "renderCores" src/dashboards/web/`), delete it and its CSS. Do not otherwise
|
||||
touch `LineChart`; phase 04 depends on it.
|
||||
4. `src/dashboards/web/static/dash.css` — remove `.cores` rules and any now-unused chart
|
||||
grid classes, verified by grep before deletion.
|
||||
- ASSUMPTION: three charts per host (utilisation, network, GPU-if-present) rather than
|
||||
one chart per metric. Six metrics × three hosts = eighteen charts on screen is the
|
||||
unreadable version of "per-host charts".
|
||||
- ASSUMPTION: no cross-host chart stays anywhere. Comparing hosts is the overview fleet
|
||||
table's job; `/system` answers "what is this machine doing".
|
||||
- ASSUMPTION: history retention and the stored series are untouched —
|
||||
`cpu.usage_pct@coreN` keeps being polled and stored, and remains queryable.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `tests/test_api.py`: if it asserts the removed chart endpoints, update it; the endpoints
|
||||
themselves stay (`/api/system/history` is used by the per-host charts and the live tests).
|
||||
- Browser, live data: chart per host, no legend with two host names in it, and the range
|
||||
chips (1h/6h/24h/7d/30d/90d) still re-fetch. Watch for the old bug class: a chart that
|
||||
keeps drawing after its host disappears from the payload.
|
||||
- Performance sanity: with three hosts the page must still poll smoothly — count the XHRs per
|
||||
refresh in devtools and keep it under ~6.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/system | grep -c 'id="chart-cpu"\|id="chart-detail"'` → 0
|
||||
- [ ] every `LineChart` on `/system` is inside a `[data-host]` panel
|
||||
- [ ] `curl -s "localhost:8090/api/system/history?host=driveripper&metric=cpu.usage_pct@core7&range=1h"` still returns points
|
||||
- [ ] `uv run pytest` green (including `-m live --live`), coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] screenshot of `/system` with all three hosts in `data/logs/`
|
||||
@@ -0,0 +1,49 @@
|
||||
# Task 04 — Verify every host shape in a browser
|
||||
|
||||
**Phase:** `03_system_per_host_panels` · **Source:** `TODO.md:4–5` — the page must be right for *every* host, not just the one that was convenient to develop against
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the new `/system` page on all three real hosts — including the two shapes that usually
|
||||
get missed: an APU with two memory pools, and a box with no GPU and no CPU temperature.
|
||||
|
||||
## Work
|
||||
|
||||
1. Confirm the dev loop is live: `./scripts/dev-tunnels.sh status` (three agents + the upsd
|
||||
forward OK), `./scripts/dev-server.sh restart`, `/api/health` all `ok`.
|
||||
2. Verify and screenshot each host panel on `http://127.0.0.1:8090/system`:
|
||||
- **driveripper** — discrete Radeon AI PRO R9700: busy %, VRAM used/total (~32 GiB),
|
||||
power/temp where reported;
|
||||
- **deskwork** — Strix Halo APU: `vram_used_pct` low while `mem_used_pct` is high is
|
||||
*correct*, and the `.sub` pool line must make that legible rather than contradictory;
|
||||
- **gateway** — no GPU: the "No GPU reported by this agent" state, GPU hero `n/a` with
|
||||
**no** colour; CPU temp `n/a` likewise.
|
||||
3. Check the failure path, not just the happy path: stop one agent
|
||||
(`./scripts/dev-tunnels.sh stop` for a single host, or `ssh <host> "pkill -f '[h]ost_agent.py'"`)
|
||||
and confirm the panel keeps its last good numbers, marks them stale
|
||||
(`stalenessText`), turns its dot amber/red, and does **not** blank or zero the heroes.
|
||||
Screenshot it. Restore afterwards (`./scripts/dev-tunnels.sh start`).
|
||||
4. Accessibility pass on the new markup: hero values are real text (selectable, not SVG
|
||||
paths), each chart container has a label or `aria-label`, colour is never the *only*
|
||||
signal — the number itself must carry the meaning.
|
||||
5. Light and dark theme both checked (the header toggle), since severity colours come from
|
||||
theme tokens.
|
||||
6. Write a short verification note under this phase's directory as
|
||||
`NOTES-verify.md`: date, what was checked, what looked wrong (if anything), screenshots
|
||||
referenced by filename.
|
||||
- ASSUMPTION: the note lives next to the phase rather than in git history alone, so the
|
||||
next reader can tell what was actually seen on screen.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- This task produces evidence, not code. If verification finds a defect, fix it in this task
|
||||
and re-shoot; do not leave it as a note.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/system-driveripper.png`, `data/logs/system-deskwork.png`, `data/logs/system-gateway.png` exist and were opened/read before declaring success
|
||||
- [ ] a stale-host screenshot shows last-good values marked stale, not zeros
|
||||
- [ ] both themes checked; no colour-only signalling
|
||||
- [ ] `NOTES-verify.md` written in this phase directory
|
||||
- [ ] `uv run pytest` still green after any fix made during verification
|
||||
@@ -0,0 +1,51 @@
|
||||
# Phase 04 — LLM page is readable again
|
||||
|
||||
**Source:** `TODO.md` L6 — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Anchors* (hand-rolled SVG only), `~/Projects/Experimental/llm_monitor/static/index.html` (the reference implementation), `src/dashboards/web/static/page-llm.js` (`CELLS`, `makeCharts`), `src/dashboards/web/static/charts.js` (`sparkline`, `LineChart`), `src/dashboards/sources/llm.py`
|
||||
|
||||
## Objective
|
||||
|
||||
The LLM page drifted into a 14-cell grid where everything is the same size and nothing shows
|
||||
recent shape. Bring back what made `llm_monitor` legible — big throughput numbers, a sparkline
|
||||
in each metric cell, percentiles as foot lines — without losing what this project added since:
|
||||
per-engine provenance tags, history windows, and severity colour.
|
||||
|
||||
`llm_monitor` is the design reference, not a dependency. It is a separate app on `:8095` and
|
||||
stays that way; nothing here imports from it.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — KV-cache pressure and queue depth get coloured from the
|
||||
API verdict, not a per-page threshold.
|
||||
- `03_system_per_host_panels` (todo) — touches `charts.js` first; this phase builds on the
|
||||
same shared helpers and must not fight it for them.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_chart_set_decision.md` — read both implementations side by side and write the decision note (which cells live, which get sparklines, what is demoted).
|
||||
2. `02_sparkline_upgrade.md` — bring `DashCharts.sparkline()` up to `llm_monitor`'s `drawSpark` behaviour: right-aligned growth, fixed max, current/max labels, severity colour.
|
||||
3. `03_llm_layout_simplified.md` — restructure `llm.html` + `page-llm.js`: hero throughput, compact sparkline cells, percentile foot lines, conditional alarm cells.
|
||||
4. `04_side_by_side_verification.md` — verify against `llm_monitor` on `:8095` with the same live engines.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Python: none expected beyond the phase-01 severity fields; `/api/llm/*` stays as-is
|
||||
(additive-only anchor).
|
||||
- `uv run pytest -m live --live` must stay green — `test_engine_still_publishes_the_series_we_parse`
|
||||
is the assertion that the numbers on screen are backed by resolved upstream series.
|
||||
- Browser: both engines live (gufo on `deskwork:8731`, vLLM on `driveripper:8000`), plus the
|
||||
single-engine case: with one `*_BASE_URL` unset the page must show one clean half rather
|
||||
than a red panel.
|
||||
- Coverage: ≥94% overall maintained.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] decode and prompt throughput are the two largest numbers per engine, colour-coded
|
||||
- [ ] every remaining metric cell carries a sparkline showing the selected window
|
||||
- [ ] latency cells show mean **and** p95 without needing a hover
|
||||
- [ ] a source-provenance tag is still visible on every throughput number (`AGENTS.md`: never invent a number)
|
||||
- [ ] `/llm` with only one engine configured still renders cleanly
|
||||
- [ ] screenshots of this page and `llm_monitor` side by side in `data/logs/`, plus `NOTES-chart-set.md`
|
||||
- [ ] `uv run pytest` green, coverage ≥94%, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,52 @@
|
||||
# Task 01 — Chart-set decision note
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Decide, in writing, what "simplified and more readable" means for this page before touching
|
||||
it — the drift happened precisely because nobody wrote down what the page was for.
|
||||
|
||||
## Work
|
||||
|
||||
1. Read the reference implementation:
|
||||
`~/Projects/Experimental/llm_monitor/static/index.html` — specifically
|
||||
`drawSpark(name, values, format, fixedMax)` (~L1092, with the right-aligned growth and the
|
||||
top/middle/bottom scale marks ~L1134), `setThroughput(el, cell)` (~L1303),
|
||||
`UI.historyLen = 60` (~L1003), the per-engine cell markup (`#gufo-ptps`, `#gufo-dtps`,
|
||||
`#vllm-ptps`, `#vllm-dtps`), the percentile foot lines (`#vllm-ttft-p95`,
|
||||
`#vllm-tpot-p95`, `#vllm-queue-p95`), and `buildSlotCard`/`renderSlots` (~L1179/L1209).
|
||||
2. Read the current page: `src/dashboards/web/pages/llm.html` (engine-totals card L39–60,
|
||||
four charts L62–81, `#engines` grid L83–91) and `page-llm.js` (`CELLS`, 14 entries;
|
||||
`cellValue()`; `severityFor(value, 75, 90)` calls at L195/L307).
|
||||
3. Write `.agents/phases/todo/04_llm_chart_readability/NOTES-chart-set.md`:
|
||||
- **keep**: provenance tags per throughput cell, history window chips, per-engine panels,
|
||||
the gufo slot/session list;
|
||||
- **hero**: `decode_tokens_per_second`, `prompt_tokens_per_second` per engine;
|
||||
- **sparkline cells** (compact, one per row): sessions active, queued, KV cache %,
|
||||
TTFT (mean + p95 as foot), TPOT (mean + p95 as foot), queue mean, prefix-cache hit %,
|
||||
spec-accept %;
|
||||
- **conditional / demoted**: tokens-lifetime counter (quiet footnote), `preemptions` and
|
||||
`device_lost` — shown as alarm chips **only when non-zero**, otherwise absent. A permanent
|
||||
cell for something that is always 0 trains the eye to ignore the row.
|
||||
- **charts**: keep decode/prompt/KV/queue `LineChart`s, one engine per series, with the
|
||||
existing legend and a units-bearing y-axis; the four stay but drop to half-height so the
|
||||
cells read first.
|
||||
- the colour rules for KV pressure and queue depth (from `severity.py`), and the explicit
|
||||
statement that throughput has *no* severity — 0 tok/s on an idle server is not red.
|
||||
4. No code changes.
|
||||
- ASSUMPTION: the cell set above is the judgment call; if the operator disagrees it is
|
||||
cheaper to change this note now than the page later.
|
||||
- ASSUMPTION: `llm_monitor` stays a separate app. Importing its files would break the
|
||||
no-build, single-app structure and it is not ours to own.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- The note is the deliverable; it must name real line numbers in both codebases so the next
|
||||
task does not re-derive them.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `NOTES-chart-set.md` exists in this phase directory, with keep/hero/demote lists and the two source files cited by path and line
|
||||
- [ ] no application file changed in this task (`git status` clean except the note)
|
||||
@@ -0,0 +1,51 @@
|
||||
# Task 02 — Sparkline upgrade
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "The charts got less useful."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Make the shared `sparkline()` as informative as `llm_monitor`'s `drawSpark`, because every
|
||||
page (LLM cells now, per-host cards later) draws on it.
|
||||
|
||||
`charts.js:352` already has `sparkline(container, points, opts)` and it is exported at
|
||||
`charts.js:377` — **no page uses it today**, which is exactly why the LLM page reads flat.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/charts.js` — extend `sparkline()` with these opts, defaulting
|
||||
to today's behaviour so no existing caller breaks:
|
||||
- `fixedMax` — plot against a known ceiling (percent metrics must not rescale to their own
|
||||
data: 8 % KV cache next to a 92 % spike is the lie this fixes);
|
||||
- `align: 'right'` — grow from the left, newest pinned right, as in `drawSpark`;
|
||||
- `labels: {current, max}` — the small `now / max` foot `llm_monitor` prints, using the
|
||||
`format` callback;
|
||||
- `severity` — stroke colour from the theme token for `ok|warn|fail` (`cssVar('--ok')` etc.),
|
||||
no colour and a faint grey stroke when severity is `null`;
|
||||
- `minPoints` (default 2) — keep the existing "no history" text for a series that has no
|
||||
shape yet; never draw a confident line through one point.
|
||||
2. Keep it dependency-free SVG (`<svg>` + `<path>`/`<polyline>` + optional baseline), and keep
|
||||
the output small enough to sit in a 160×34 box — that is the whole point of a sparkline.
|
||||
3. Add scale marks (top/bottom values only) when `fixedMax` is set, so a reader can tell that
|
||||
"the line is at a third" means a third *of what*.
|
||||
4. If `renderCores` survived phase 03's cleanup, leave it alone — do not widen this task.
|
||||
- ASSUMPTION: sparkline stays in `charts.js` rather than a new file; one shared chart
|
||||
module is the current architecture.
|
||||
- ASSUMPTION: no animation, but respect `prefers-reduced-motion` by not adding any
|
||||
transition at all.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `tests/test_api.py` already asserts static assets are served; extend it to assert the new
|
||||
option names are present in `/static/charts.js` (cheap, catches a bad merge).
|
||||
- Browser: a page that uses it (any LLM cell after task 03) plus a synthetic check in devtools
|
||||
— draw into an empty container (0, 1, 2, 60 points) and confirm each renders the right thing
|
||||
rather than throwing.
|
||||
- Coverage: ≥94% overall maintained; no new Python expected.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/static/charts.js | grep -c "fixedMax\|align\|severity"` ≥ 3
|
||||
- [ ] a sparkline with `fixedMax: 100` does not rescale when its data ranges 5–10 (verified in devtools, screenshot)
|
||||
- [ ] 0- and 1-point series render "no history", not a spike
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean; `LineChart` untouched (`git diff` shows no changes to its code)
|
||||
@@ -0,0 +1,62 @@
|
||||
# Task 03 — Simplify the LLM page layout
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor. The charts got less useful. Need simplified and more readable."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Per engine: two big throughput numbers you can read across the room, then a compact list of
|
||||
metric cells that each carry their own sparkline and percentile foot line.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/llm.html` —
|
||||
- keep the engine-totals card (L39–60) but shrink it to what the overview no longer shows:
|
||||
engines answering, active sessions, queued. Drop the summed decode/prompt/KV numbers —
|
||||
per-engine is the honest view and the overview owns the aggregate;
|
||||
- reduce the four `LineChart` cards (L62–81) to half height, and drop `#chart-queue` if
|
||||
`NOTES-chart-set.md` says queue reads better as a cell;
|
||||
- per-engine panel (`#engines` grid, `page-llm.js:140+`): hero row (decode, prompt) using
|
||||
the phase-01 `.hero` markup, then the cell list from `NOTES-chart-set.md`, then the gufo
|
||||
slot list (unchanged behaviour).
|
||||
2. `src/dashboards/web/static/page-llm.js` —
|
||||
- `CELLS`: mark each entry with `hero: true` / `spark: true` / `alarm: true` and drive the
|
||||
markup from those flags instead of 14 identical cells;
|
||||
- render each `spark: true` cell through `DashCharts.sparkline()` with data from
|
||||
`/api/llm/history` (same endpoint, one request per engine per window — batch the metrics
|
||||
the endpoint supports rather than one XHR per cell);
|
||||
- percentile foot lines: TTFT/TPOT/queue cells print `mean` big and `p95 …` beneath, as
|
||||
`llm_monitor` does (`#vllm-ttft-p95` pattern);
|
||||
- `alarm: true` cells (`preemptions`, `device_lost`) render only when non-zero, as a
|
||||
`.chip.fail`; the row must not exist when they are 0;
|
||||
- replace `severityFor(value, 75, 90)` (L195, L307) with the payload `severity` via
|
||||
`Dash.severityFrom()`;
|
||||
- **keep the provenance tag** on every throughput cell — the file's header comment explains
|
||||
why (`78 tok/s` means different things when scraped, measured Δ/Δt, or estimated from
|
||||
TPOT). Removing it to look cleaner is a regression, not a simplification.
|
||||
3. `src/dashboards/web/static/dash.css` — cell-list styling: label left, value right-aligned
|
||||
tabular, sparkline on the third column at a fixed width so rows line up; the alarm chip
|
||||
reuses `.chip.fail`.
|
||||
- ASSUMPTION: throughput gets no severity colour. A server doing 0 tok/s while idle is
|
||||
healthy; a red zero every night destroys the signal.
|
||||
- ASSUMPTION: cells are a vertical list per engine, not a 4×4 grid — grids make 14 numbers
|
||||
look equally important, which is the complaint.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `uv run pytest -m live --live` against the real gufo and vLLM: `test_engine_still_publishes_the_series_we_parse`
|
||||
guards the metric names the new layout renders.
|
||||
- Browser, both engines live, and with one engine disabled (`VLLM_BASE_URL=` unset, restart)
|
||||
to prove the page degrades to one clean half instead of a dead panel
|
||||
(`AGENTS.md` invariant 3).
|
||||
- Idle vs loaded engine if possible (`llm_monitor` on `:8095` is watching the same upstreams,
|
||||
so compare like-for-like).
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] decode + prompt are the largest elements per engine, and the rest of the cells are visibly secondary
|
||||
- [ ] every cell has a sparkline for the selected window; TTFT/TPOT show mean + p95
|
||||
- [ ] `preemptions` / `device_lost` are absent at zero and appear as red chips when non-zero (verified in devtools with a mocked payload)
|
||||
- [ ] provenance tags still present on throughput cells (`grep -n "source" page-llm.js` shows the tag path intact)
|
||||
- [ ] `curl -s localhost:8090/llm` renders with one engine configured and shows no dead panel (screenshot)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,44 @@
|
||||
# Task 04 — Verify side by side with llm_monitor
|
||||
|
||||
**Phase:** `04_llm_chart_readability` · **Source:** `TODO.md:6` — "LLM usage drifed from initial request to copy Experimental/llm_monitor."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the new page is at least as readable as the app it drifted from, using the same live
|
||||
upstreams.
|
||||
|
||||
## Work
|
||||
|
||||
1. Both apps watch the same engines, so compare honestly:
|
||||
- `llm_monitor` on `http://127.0.0.1:8095` (it is already running in this fleet's dev
|
||||
loop; if not, `cd ~/Projects/Experimental/llm_monitor && uv run uvicorn app:app --port 8095`);
|
||||
- this app on `http://127.0.0.1:8090/llm`.
|
||||
2. Screenshot both, same moment, same window length. Read the two images and answer in
|
||||
writing: which one tells you *right now* whether the engines are busy, whether anyone is
|
||||
queued, and whether the KV cache is about to bite? The answer must be "ours" or the phase
|
||||
is not done.
|
||||
3. Exercise the states, not just the happy path:
|
||||
- one engine down (stop gufo or point `GUFO_BASE_URL` at a dead port, restart the dev
|
||||
server) → one clean half, no red panel for the missing engine;
|
||||
- both engines idle → throughput `0 tok/s` with **no** severity colour, and the sparklines
|
||||
still show the shape of the last window rather than a flat lie;
|
||||
- a real request in flight if the operator is willing (`curl` a chat completion against
|
||||
gufo) so decode/prompt are visibly non-zero and the provenance tag is readable.
|
||||
4. Confirm the history window chips (1h…90d) still drive the four `LineChart`s and that the
|
||||
sparklines use the same window — a cell that silently ignores the selected range is worse
|
||||
than no sparkline.
|
||||
5. Append the comparison and the answers to `NOTES-chart-set.md`.
|
||||
- ASSUMPTION: `llm_monitor` is treated as the design reference and left running; this phase
|
||||
neither modifies nor deletes it.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Evidence task. Any defect found is fixed here, then re-screenshotted.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/llm-before-monitor.png` (the reference) and `data/logs/llm-after.png` (ours) exist and were both read
|
||||
- [ ] the single-engine and idle states are screenshotted too
|
||||
- [ ] `NOTES-chart-set.md` has a short written verdict + what was fixed after looking
|
||||
- [ ] `uv run pytest` and `uv run pytest -m live --live` green
|
||||
@@ -0,0 +1,48 @@
|
||||
# Phase 05 — UPS hero numbers and a battery glyph
|
||||
|
||||
**Source:** `TODO.md` L7 — "UPS looks good but need charge, load, and runtime left to be big, obvious, and color coded. Also an SVG battery symbol that shows current charge would be a nice touch."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` → *Architectural Anchors* (severity from the API, hand-rolled SVG), `01_severity_language`, `src/dashboards/web/pages/ups.html`, `src/dashboards/web/static/page-ups.js`, `src/dashboards/web/static/charts.js`, `src/dashboards/sources/nut.py`
|
||||
|
||||
## Objective
|
||||
|
||||
Each UPS card opens with a battery glyph that visibly holds its charge, next to three big,
|
||||
colour-coded numbers: **charge %**, **load %**, **runtime left**. The health checks, diagnostics
|
||||
buttons and history stay below them, working exactly as they do now.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_severity_language` (todo) — charge/load/runtime verdicts come from the API, and the
|
||||
`.hero` style comes from `dash.css`.
|
||||
- `03_system_per_host_panels` (todo) — proves the hero pattern on `/system` first, so the UPS
|
||||
page is not the place that pattern gets invented.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_ups_hero_data.md` — confirm the API gives what the hero row and glyph need (`numbers.charge_pct`, `numbers.load_pct`, `numbers.battery_runtime`, `numbers.battery_voltage`, `charger_status`, `on_battery`) with severity attached; contract tests against real upsd.
|
||||
2. `02_battery_glyph.md` — `DashCharts.battery(container, {charge, severity, state, label})`: inline SVG with a fill proportional to charge, severity colour, charging/on-battery treatment, accessible label.
|
||||
3. `03_ups_hero_row.md` — restructure the card in `page-ups.js`/`ups.html`: hero row + glyph on top, checks and diagnostics below, `n/a` never coloured.
|
||||
4. `04_verify_real_batteries.md` — verify against both real UPSes plus the low-charge and on-battery states reachable only through the simulator.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Python: the only expected change is in `sources/nut.py`'s `payload()` (severity + any
|
||||
missing glyph field); coverage **>90%** there.
|
||||
- Contract: `tests/test_ups_contract.py` asserts the hero fields and their severities exist for
|
||||
the shapes the real drivers produce — including `battery_temperature` being `None`, which is
|
||||
true of both the APC and the CyberPower here.
|
||||
- Glyph states are exercised in devtools with synthetic values (`100 ok`, `60 warn`,
|
||||
`12 fail`, `null`) **and** in the live simulator (`NUT_SIMULATE=1`), which can be driven to
|
||||
low charge and on-battery — real hardware will not do that on demand, and `nut_sim.py`'s
|
||||
quick test already flips `OB` for ~12s.
|
||||
- Live: `uv run pytest -m live --live` covers the real UPSes' numbers.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] each UPS card shows charge / load / runtime as its three largest elements, coloured from API `severity`
|
||||
- [ ] the glyph's fill visibly tracks charge: `data/logs/ups-glyph-*.png` at 100 %, mid, low, and the unknown state
|
||||
- [ ] on battery the glyph/card state changes visibly (simulator is acceptable proof — say which in `NOTES-verify.md`)
|
||||
- [ ] `SIMULATED` still badges the card when `NUT_SIMULATE=1`, and never when reading real upsd
|
||||
- [ ] battery-test buttons unchanged: `Quick battery test` → confirm dialog → **Cancel** works, and `DANGER` is still required for the power-cutting ones
|
||||
- [ ] `uv run pytest` green, coverage ≥94%, `ruff` + `pyright` clean, `-m live --live` green
|
||||
- [ ] no behavior change in completed phases
|
||||
@@ -0,0 +1,51 @@
|
||||
# Task 01 — UPS hero data contract
|
||||
|
||||
**Phase:** `05_ups_hero_battery_glyph` · **Source:** `TODO.md:7` — "charge, load, and runtime left to be big, obvious, and color coded"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Know exactly what the hero row reads, and prove it — with real batteries, since the
|
||||
CyberPower and the APC disagree in ways the fixtures do not cover.
|
||||
|
||||
## Work
|
||||
|
||||
1. Inspect what `/api/ups/snapshot` returns today for the two real UPSes:
|
||||
`curl -s localhost:8090/api/ups/snapshot | python3 -m json.tool`
|
||||
(dev forwards `127.0.0.1:3494` → driveripper's upsd on `3051`).
|
||||
2. Confirm `payload()` in `src/dashboards/sources/nut.py` supplies, per card:
|
||||
- `numbers.charge_pct`, `numbers.load_pct`, `numbers.battery_runtime`,
|
||||
`numbers.battery_voltage`, `numbers.battery_temperature`
|
||||
- `charger_status` (`full` / `ol` / `dchg`) and `on_battery` — the glyph's state comes from
|
||||
these, not from guessing
|
||||
- `severity` for charge / load / runtime (from phase 01; `sources/nut.py`'s
|
||||
`payload()` is where to attach it if it is not there yet)
|
||||
- `health.overall` and `health.counts` (unchanged — the card header already uses them).
|
||||
3. `tests/test_ups_contract.py` (new) —
|
||||
- a `UpsReading` built from captured-style vars asserts each of the paths above exists, and
|
||||
that a missing `battery.temperature` yields `None` rather than 0 (both real drivers omit
|
||||
it — this is the exact real-world shape);
|
||||
- charge 100 → `ok`; charge 45 → `warn`; charge 15 → `fail` (inverted metric);
|
||||
- runtime 3600 s → `ok`; 400 s → `warn`; 200 s → `fail`; `None` → `None` severity, and the
|
||||
page-side formatting of `None` must be `n/a` (assert whatever helper is used returns the
|
||||
app's `naText`, not `0s`);
|
||||
- `charger_status == "full"` with `on_battery == False` renders "charging" nowhere, and
|
||||
`on_battery == True` makes the glyph's on-battery state available.
|
||||
- ASSUMPTION: the glyph derives its state from `charger_status` + `on_battery` rather than
|
||||
NUT status flags being re-parsed in JS — the flags are already parsed server-side into
|
||||
`states[]` (`UPS_STATE_LABELS`) and re-parsing them client-side duplicates that map.
|
||||
- ASSUMPTION: runtime prose stays `Dash.duration()`-formatted (`10m 21s`, `1h 32m`) — no new
|
||||
formatter, because the UPS page already prints runtime that way.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Coverage **>90%** on `sources/nut.py` changes.
|
||||
- Live: `uv run pytest -m live --live` — `test_upsd_reports_both_upses_with_sane_units` is the
|
||||
real-hardware version of the same assertions.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `uv run pytest tests/test_ups_contract.py -v` green
|
||||
- [ ] `curl -s localhost:8090/api/ups/snapshot` shows `severity` for charge/load/runtime on **both** `apc` and `cyberpower`
|
||||
- [ ] `battery_temperature` is `null` on both, and nothing in the API turned it into 0
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,58 @@
|
||||
# Task 02 — Battery glyph
|
||||
|
||||
**Phase:** `05_ups_hero_battery_glyph` · **Source:** `TODO.md:7` — "an SVG battery symbol that shows current charge would be a nice touch"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
An inline SVG battery whose fill *is* the charge, whose colour *is* the severity, and whose
|
||||
outline says "unknown" when there is no number — in the same hand-rolled-SVG house style as
|
||||
the charts.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/charts.js` — `battery(container, opts)` with
|
||||
`{charge: number|null, severity: 'ok'|'warn'|'fail'|null, state: 'online'|'onbattery'|'charging'|'unknown', label?: string}`:
|
||||
- horizontal cell: rounded body + terminal nub on the right, drawn as one `<svg>` with a
|
||||
`<rect>` fill sized `charge/100` of the interior, clipped so a 0 % charge leaves an
|
||||
empty shell rather than a stray pixel;
|
||||
- colour from the theme tokens via the existing `cssVar()`: `--ok` / `--warn` / `--fail`;
|
||||
`severity: null` → outline only, no fill, muted stroke;
|
||||
- `charge` clamped to 0–100 but never *rounded up* to visible: 2 % must look nearly empty,
|
||||
because a reassuring full-looking battery at 2 % is a lie;
|
||||
- state affordances drawn in SVG, no icon font: a bolt overlay when
|
||||
`state === 'charging'`, a subtle hatch or dashed outline when on battery, and a dash
|
||||
pattern through the body when charge is unknown;
|
||||
- accessible by construction: `role="img"`, `<title>` built from the inputs
|
||||
("Charge 100 percent, on line"), and `aria-hidden` only when the adjacent hero number
|
||||
already carries the same text — pick one, and comment which;
|
||||
- scales by container width (`viewBox` + `preserveAspectRatio`) so the hero row can make it
|
||||
big on desktop and small on a phone.
|
||||
2. No animation. If a fill transition is added at all, it must sit behind
|
||||
`@media (prefers-reduced-motion: no-preference)`.
|
||||
3. Export it from the `window.DashCharts = { … }` object at `charts.js:377`.
|
||||
4. `src/dashboards/web/static/dash.css` — only what the glyph needs (sizing, `.battery`
|
||||
wrapper, the on-battery hatch if it is CSS rather than SVG).
|
||||
- ASSUMPTION: horizontal cell, not vertical — runtime/charge read left-to-right next to the
|
||||
hero numbers and it matches how NUT prints battery bars in `upsc`.
|
||||
- ASSUMPTION: no emoji/Unicode battery character anywhere; it would render differently per
|
||||
platform and cannot be coloured by severity.
|
||||
- ASSUMPTION: `state` is derived in the page from `status`/`charger_status`/`on_battery`
|
||||
(`page-ups.js`), because that mapping is presentation, not data.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Static-asset assertion in `tests/test_api.py` that `battery` is exported from
|
||||
`/static/charts.js`.
|
||||
- Browser/devtools matrix — call it directly with synthetic inputs and screenshot each:
|
||||
`{charge:100, severity:'ok', state:'online'}`, `{charge:60, severity:'ok', state:'charging'}`,
|
||||
`{charge:35, severity:'warn', state:'onbattery'}`, `{charge:12, severity:'fail', state:'onbattery'}`,
|
||||
`{charge:2, severity:'fail', state:'onbattery'}`, `{charge:null, severity:null, state:'unknown'}`,
|
||||
and `charge:0`. Also zoom to 200 % and check both themes.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `curl -s localhost:8090/static/charts.js | grep -c "function battery"` → 1
|
||||
- [ ] the six-state devtools matrix is screenshotted into `data/logs/` and each state is distinguishable without colour (shape/hatch/label)
|
||||
- [ ] `<title>` present and correct for at least the online and on-battery cases
|
||||
- [ ] `uv run pytest` green; `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,64 @@
|
||||
# Task 03 — The hero row
|
||||
|
||||
**Phase:** `05_ups_hero_battery_glyph` · **Source:** `TODO.md:7` — "charge, load, and runtime left to be big, obvious, and color coded. Also an SVG battery symbol that shows current charge"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Each UPS card opens with the battery glyph and three big numbers. Everything else on the card
|
||||
moves below them and changes nothing about how it works.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-ups.js` — `upsCard()` (L34):
|
||||
- replace the top of the `.metrics` block (L50–68) with a hero row: the `data-k="glyph"`
|
||||
container plus three `.hero` blocks — **Charge %**, **Load %**, **Runtime left** — with
|
||||
`.sub` lines for the context that matters (charge: `12 V nominal · 27.3 V`; load:
|
||||
`128 W of 900 W` when `realpower`/`realpower_nominal` allow it, otherwise just the %;
|
||||
runtime: `at the current load`);
|
||||
- demote to a secondary, visibly smaller row or the existing disclosure: line in, output,
|
||||
battery volts, battery amps, battery °C. Do not delete them — they are the numbers you
|
||||
want the moment the UPS is on battery;
|
||||
- keep unchanged: the state chips (`OL`/`OB`/`LB`), the `SIMULATED` badge, the
|
||||
`degraded`/`healthy` chip, health checks (`data-k="checks"`), diagnostics
|
||||
(`data-k="commands"`), the NUT variable dump (`data-k="raw"`), and the whole
|
||||
`#confirm-dialog` flow including typing `DANGER`.
|
||||
2. `renderCard()` (L84):
|
||||
- hero values via `Dash.hero()`, colours via `Dash.severityFrom(item.severity.charge_pct)`
|
||||
etc. — **delete the `severityFor(...)` call in this file**;
|
||||
- `runtimeText()` (L24) stays the single place runtime prose is built (`1h 32m`,
|
||||
`10m 21s`); `null` → `n/a` with no colour — never `0m`;
|
||||
- derive the glyph state: `on_battery` → `'onbattery'`; else `charger_status` in
|
||||
('dchg',) → `'charging'`; `charger_status === 'full'` or `status` containing `OL` →
|
||||
`'online'`; nothing known → `'unknown'`. Comment the mapping — NUT's flags are subtle
|
||||
(`OL` and `CHRG` can both be present, and a fully-charged-on-line unit reports
|
||||
`charger.status: full`, which is not "charging");
|
||||
- draw the glyph through `DashCharts.battery()`; re-draw only when charge/state changes so
|
||||
the poll loop does not rebuild DOM every 5s.
|
||||
3. `src/dashboards/web/pages/ups.html` — the cards are built in JS into `#ups-cards`; only
|
||||
the four history charts (`#chart-charge`, `#chart-runtime`, `#chart-voltage`,
|
||||
`#chart-load`), the events card and the dialog live here. Leave them; if a hero row needs a
|
||||
wrapper class, put it in `dash.css`.
|
||||
4. `src/dashboards/web/static/dash.css` — hero row layout (glyph + three heroes, wraps to a
|
||||
stack on narrow screens), secondary row visibly secondary (muted, smaller), and the card
|
||||
must not grow so tall that two cards stop fitting side by side on a laptop.
|
||||
- ASSUMPTION: the four history charts and the event log stay where they are. The TODO says
|
||||
the page "looks good"; only the top of each card changes.
|
||||
- ASSUMPTION: charge wins the space next to the glyph, runtime is the number people
|
||||
actually act on. Both are heroes; neither is buried.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Contract already covered by `tests/test_ups_contract.py` (task 01).
|
||||
- Browser: real fleet (`apc`, `cyberpower`), both themes, plus the synthetic low/on-battery
|
||||
states from task 04's simulator pass.
|
||||
- Watch the poll: numbers update every `NUT_INTERVAL_SECONDS` (5s) without layout shift — the
|
||||
tabular numerals from phase 01 make that true; verify rather than assume.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] each card's three largest elements are Charge / Load / Runtime, colour-coded from API severity
|
||||
- [ ] the glyph sits with them and visibly matches charge (both real UPSes are at 100 %, so verify the match with the synthetic states too)
|
||||
- [ ] battery °C renders `n/a` on both real UPSes without a failure colour
|
||||
- [ ] battery-test buttons + `DANGER` confirmation still work (click one through to the dialog, then **Cancel** — do not send a command)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
@@ -0,0 +1,57 @@
|
||||
# Task 04 — Verify on real batteries, and the states they will not give you
|
||||
|
||||
**Phase:** `05_ups_hero_battery_glyph` · **Source:** `TODO.md:7` — the whole item; the colour coding only means something if the alarming states have been seen
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
See the hero row in every state that matters — including the two that only the simulator can
|
||||
produce, because unplugging a live UPS to test a dashboard is not a test.
|
||||
|
||||
## Work
|
||||
|
||||
1. Real fleet (dev env points at `127.0.0.1:3494` → upsd on driveripper:3051):
|
||||
- `apc` — Back-UPS RS 1500MS: charge 100 %, load ~47 %, runtime ~10m, battery age
|
||||
**5.3 years → warn**, no battery temp;
|
||||
- `cyberpower` — PR1500LCDRT2U: charge 100 %, load ~14 %, runtime ~56m, battery age
|
||||
**unknown** (`battery.mfr.date: CPS` is not a date and must not become an age), no
|
||||
battery temp.
|
||||
Screenshot both cards. The two "unknown/warn" differences are the point: they prove the
|
||||
page is not rendering a template.
|
||||
2. States only `nut_sim.py` can produce — start the app with `NUT_SIMULATE=1` and
|
||||
`NUT_ALLOW_ACTIONS=0` and reach them the honest way (drive `SimUps.tick()`/`step()` in a
|
||||
unit test or a REPL and read `NutSimulator.read()` output; the quick battery test puts a
|
||||
unit on `OB` for ~12s):
|
||||
- 100 % online → green, no bolt;
|
||||
- ~60 % charging → bolt visible;
|
||||
- ~35 % on battery → amber, on-battery treatment;
|
||||
- 12 % on battery (`OB LB`) → red;
|
||||
- charge absent → outline only, `n/a`, no colour, no fake runtime.
|
||||
Screenshot each. `SIMULATED` must be visible in every one of these shots — if it is not,
|
||||
that is a bug in this phase, not a cosmetic miss.
|
||||
3. Failure path: kill the upsd forward (`./scripts/dev-tunnels.sh stop`/`start`, or stop
|
||||
`nut-server` remotely **only** if the operator agrees) and confirm the cards keep the last
|
||||
good numbers and mark them stale rather than showing `n/a` everywhere
|
||||
(`AGENTS.md` invariant 4).
|
||||
4. Commands: click **Quick battery test** → dialog appears → type `DANGER` → then **Cancel**.
|
||||
Verifying the gate without sending the command is the correct amount of clever here; if
|
||||
the operator explicitly asks for a live quick test, run it and watch `ups.test.result`
|
||||
change to `PASSED` (and the health check go green), which is the payoff for the whole
|
||||
phase.
|
||||
5. Record what was seen in `NOTES-verify.md` in this phase directory, listing each screenshot
|
||||
and which state it proves.
|
||||
- ASSUMPTION: no instant command is sent as part of automated verification. A deep test
|
||||
discharges a battery that something real is plugged into.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Evidence task; fix on discovery, then re-shoot.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/ups-real-apc.png` and `data/logs/ups-real-cyberpower.png` exist and were read
|
||||
- [ ] five synthetic states screenshotted, each with the `SIMULATED` badge visible
|
||||
- [ ] stale-source screenshot shows last-good values, not zeros
|
||||
- [ ] the `DANGER` dialog path exercised and cancelled; no command was sent without the operator asking
|
||||
- [ ] `NOTES-verify.md` written in this phase directory
|
||||
- [ ] `uv run pytest` and `uv run pytest -m live --live` green
|
||||
+4
-3
@@ -31,9 +31,10 @@ agent/*.tar.gz
|
||||
.idea/
|
||||
.DS_Store
|
||||
|
||||
# Agent/pipeline scratch. This project's plan lives in AGENTS.md (tracked) — nothing under
|
||||
# .agents/ is anything another checkout needs, so all of it stays out.
|
||||
.agents/
|
||||
# Agent/pipeline scratch ONLY. The planning tree (.agents/PLAN.md, .agents/phases/) is
|
||||
# tracked on purpose — it is the project's roadmap and history, not throwaway state.
|
||||
.agents/phase-sessions/
|
||||
.agents/pipeline.log
|
||||
|
||||
# The original brief for this project. Kept in the checkout as local context, not history:
|
||||
# README.md and AGENTS.md carry the decisions that came out of it.
|
||||
|
||||
@@ -4,6 +4,19 @@ The plan, the rules, and the environment facts an agent needs to work on this re
|
||||
re-deriving them. `README.md` is the operator-facing how-to; this file is the why, the
|
||||
invariants, and the state of the fleet.
|
||||
|
||||
## Working protocol
|
||||
|
||||
This project runs the `.agents/phases/` phased-execution pipeline.
|
||||
|
||||
1. Always read `.agents/PLAN.md` first to understand the project context and goals.
|
||||
2. Follow the phased execution protocol in `.agents/phases/`.
|
||||
3. Never modify `.agents/PLAN.md` or any files in `.agents/phases/complete/`.
|
||||
4. If you need to update any file in `.agents/phases/todo/`, you must ask the user for permission first.
|
||||
5. Strictly adhere to the **LOCKED DECISIONS** listed in `.agents/PLAN.md`.
|
||||
|
||||
Everything below this section is the project's own content: what the app is, the rules that
|
||||
keep its numbers honest, and the state of the fleet it watches.
|
||||
|
||||
## What this is
|
||||
|
||||
Three dashboards for one homelab, behind one domain, in one process, backed by one SQLite
|
||||
|
||||
+42
-43
@@ -1,55 +1,54 @@
|
||||
# driveripper-ops sudo rules for the UPS dashboard work
|
||||
# driveripper-ops sudo rules for the homelab dashboards UPS work.
|
||||
#
|
||||
# Install as /etc/sudoers.d/dashboards-nut (visudo -cf it first):
|
||||
# Install (from the repo checkout, as a user with sudo):
|
||||
#
|
||||
# sudo visudo -cf - <<'EOF' ... EOF # syntax check
|
||||
# install -m 0440 deploy/nut-sudoers /tmp/dashboards-nut
|
||||
# sudo mv /tmp/dashboards-nut /etc/sudoers.d/dashboards-nut
|
||||
# sudo visudo -cf deploy/nut-sudoers # syntax check FIRST
|
||||
# sudo install -m 0440 -o root -g root deploy/nut-sudoers /etc/sudoers.d/dashboards-nut
|
||||
# sudo visudo -cf /etc/sudoers.d/dashboards-nut # and check the installed copy
|
||||
#
|
||||
# Keep it scoped: paths are absolute (sudoers uses `secure_path`), arguments are wildcards
|
||||
# only where the command is read-only or idempotent, and nothing here allows a shell,
|
||||
# `EDITOR=`, `dnf install` of arbitrary code, or `upsd -c` with an attacker-chosen file.
|
||||
# Scope: absolute paths only (`secure_path` is in effect), wildcards only where the command
|
||||
# is read-only or idempotent, and nothing here grants a shell, `sh -c`, an editor, or a
|
||||
# client that can send instant commands.
|
||||
#
|
||||
# WHY THESE ARE NEEDED
|
||||
# Reads already work with zero privileges — upsd on driveripper serves `LIST UPS` /
|
||||
# `LIST VAR` unauthenticated (on port 3051, not 3493). What is missing is a upsd.users
|
||||
# entry with `instcmds`, which is the only thing standing between the dashboard and its
|
||||
# battery-test buttons. `tee /etc/ups/*` is already granted, but writing upsd.users blind
|
||||
# would drop the accounts upsmon authenticates as — upsmon must run as a user with
|
||||
# `shutdown`, so an overwritten file means no orderly power-down. So: read first.
|
||||
|
||||
# ---- already present on driveripper (this is what the dashboard setup used) ---------- #
|
||||
# (root) NOPASSWD: /usr/bin/tee /etc/ups/* # write upsd.conf / upsd.users
|
||||
# (root) NOPASSWD: /usr/bin/cp /etc/ups/* /home/ops/* # back up a config before editing
|
||||
# ---- already present on driveripper (used during setup) ------------------------------ #
|
||||
# (root) NOPASSWD: /usr/bin/tee /etc/ups/* # write a config file
|
||||
# (root) NOPASSWD: /usr/bin/cp /etc/ups/* /home/ops/* # copy one out (root-owned)
|
||||
# (root) NOPASSWD: /usr/bin/systemctl start|restart|reload|enable *
|
||||
# (root) NOPASSWD: /usr/bin/udevadm * # USB device discovery/replug
|
||||
# (root) NOPASSWD: /usr/bin/udevadm * # USB device discovery
|
||||
# (root) NOPASSWD: /usr/bin/firewall-cmd * # only if upsd must serve the LAN
|
||||
# (root) NOPASSWD: /usr/bin/dnf install * # nut, nut-client, nut-dev
|
||||
# (root) NOPASSWD: /usr/bin/dnf install * # nut, nut-client
|
||||
|
||||
# ---- still needed: diagnose why upsd runs but binds nothing -------------------------- #
|
||||
# Reading is the blocker. `upsd` is active, both drivers are up, and 3493 has no listener,
|
||||
# but /etc/ups/* is 0640 root:nut and the journal needs group membership:
|
||||
# $ sudo cat /etc/ups/upsd.conf -> Permission denied
|
||||
# $ journalctl -u nut-server -n 50 -> No entries (not in systemd-journal/wheel)
|
||||
# $ upsc -l -> Error: Connection failure: Connection refused
|
||||
|
||||
Cmnd_Alias NUT_DIAGNOSE = \
|
||||
# ---- read: what is actually configured, before anything is rewritten ----------------- #
|
||||
Cmnd_Alias NUT_READ = \
|
||||
/usr/bin/cat /etc/ups/*, \
|
||||
/usr/bin/systemctl status *, \
|
||||
/usr/bin/journalctl -u nut-server *, /usr/bin/journalctl -u nut-driver@*, \
|
||||
/usr/bin/journalctl -u nut*, \
|
||||
/usr/sbin/ss -ltnp, \
|
||||
/usr/sbin/getenforce, \
|
||||
/usr/bin/ausearch *, /usr/sbin/sealert -l *, \
|
||||
/usr/sbin/semanage port -l, \
|
||||
/usr/bin/upsdrvctl status, /usr/bin/upsdrvctl status *, \
|
||||
/usr/bin/systemctl status nut*, \
|
||||
/usr/bin/upsdrvctl status, \
|
||||
/usr/bin/upsc *, \
|
||||
/usr/bin/lsusb
|
||||
/usr/sbin/ss -ltnp, \
|
||||
/usr/bin/lsusb, \
|
||||
/usr/sbin/getenforce, \
|
||||
/usr/bin/ausearch -m avc *, /usr/sbin/sealert -l *
|
||||
|
||||
# ---- still needed: apply the fix and verify ------------------------------------------ #
|
||||
# `tee /etc/ups/*` already writes configs; these cover the reload path, SELinux booleans,
|
||||
# and a bounded cleanup of the .bak files `tee`-based edits leave behind.
|
||||
Cmnd_Alias NUT_APPLY = \
|
||||
/usr/bin/tee /etc/ups/upsd.conf, /usr/bin/tee /etc/ups/upsd.users, \
|
||||
/usr/bin/tee /etc/ups/ups.conf, /usr/bin/tee /etc/ups/nut.conf, \
|
||||
/usr/bin/rm -f /etc/ups/*.bak-*, \
|
||||
/usr/bin/upsdrvctl restart, /usr/bin/upsdrvctl restart *, \
|
||||
/usr/sbin/setsebool -P allow_upsd_tcp_server 1, \
|
||||
/usr/sbin/setsebool -P nis_enabled 0
|
||||
# ---- write: add/replace a scoped user, then reload ----------------------------------- #
|
||||
Cmnd_Alias NUT_WRITE = \
|
||||
/usr/bin/tee /etc/ups/upsd.users, \
|
||||
/usr/bin/tee /etc/ups/upsd.conf, \
|
||||
/usr/bin/systemctl restart nut-server, \
|
||||
/usr/bin/systemctl reload nut-server
|
||||
|
||||
# NOTE: deliberately NOT allowed, so an agent cannot power-cycle anything by accident:
|
||||
# /usr/bin/upscmd (any form) — the dashboard's own gated POST is the only command path
|
||||
# systemctl stop * — stopping nut-server would blind the UPS view
|
||||
# editors, shells, `sh -c` — no escape hatches
|
||||
# ---- deliberately NOT granted -------------------------------------------------------- #
|
||||
# /usr/bin/upscmd, /usr/bin/upsrw — the dashboard's gated POST /api/ups/command is the
|
||||
# ONLY command path in this project, by design
|
||||
# systemctl stop * — stopping nut-server blinds the UPS view
|
||||
# tee /etc/ups/ups.conf, upsmon.conf — driver instances and shutdown policy are the
|
||||
# operator's, not the dashboard's
|
||||
# sh, bash, vim, tee to anything else — no escape hatches
|
||||
|
||||
Reference in New Issue
Block a user