Files
dashboards/tests/card_contract.py
T
ducoterra 9757e3c947 phase: 03_system_per_host_panels
**Phase 03 final verification pass — all 4 tasks already in `complete/`; verified criteria, fixed 3 pre-existing defects.**

- **Verified live** (headless Chromium + curl against `:8090`, all 6 sources `ok`): 3 panels × six heroes in order `cpu,mem,temp,disk,gpus,net`; charts per panel `util,net[,gpu]` (gateway: no GPU chart/hole); legends name metrics only — no legend names a foreign host; every `<svg>` `role="img"` + host-scoped `aria-label`; 2 sections, 0 `<select>`; `grep -rn "renderCores|core_usage" src/dashboards/web/` → nothing; console error-free; new shot `data/logs/final-system-panels.png`.
- **Defect fixed (credential hygiene, invariant 6)**: `collectors.py` put the *dialed* URL into every `UpstreamError`, and that string lands in the health record (`/api/health`) and the `event` table (`/api/events` → page); `/api/config` also echoed `GUFO_BASE_URL`/`VLLM_BASE_URL` verbatim. Added `config.strip_userinfo()` (now what `Target.display_url` uses) and redacted all message/config URLs. Tests: `test_a_failing_host_never_echoes_its_token_in_the_message` (×3 paths), `test_a_failing_engine_never_echoes_credentials_in_its_url`, `test_a_dead_host_never_writes_its_token_into_the_feed`, `test_engine_urls_are_redacted_the_same_way_hosts_are`, `test_strip_userinfo_is_the_one_redaction_every_url_leaves_by`. (`parse_targets` already moves `HOST_TARGETS` userinfo into a header, so this is a backstop for hosts and a real fix for engine URLs.)
- **Defect fixed (invariant 2)**: `/system` footnote printed `Δ{HOST_INTERVAL}` as the measurement; it now prints each host's measured `window_seconds` (range when they disagree) — `rates over Δ10.0s · poll 10s`. Pinned by `test_system_footnote_prints_the_window_it_actually_measured`. Also fixed the page's stale "window in each header" note and the empty-state's `--token` (argv) advice → `--token-file`.
- `bash .agents/validate.sh` → **EXIT=0**: **472 passed, 7 skipped**, coverage **96.87%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` clean · `uv run pytest -m live --live` → **7 passed** · `uv pip list --outdated` empty.
- Criteria: no multi-host legend ✅ · no per-core visual ✅ · six heroes/panel incl. no-GPU `n/a` uncoloured ✅ · per-core history still queryable ✅ (354 pts) · ≥3 shape screenshots ✅ (`system-driveripper/deskwork/gateway.png` + dark/light/stale) · pytest+coverage+lint ✅ · phases 01/02 unchanged (their pin tests + `/api/summary` shape tests pass; live UPS gap in AGENTS.md re-confirmed read-only against real upsd) ✅.
- **Docs**: AGENTS.md — invariant 6 rewritten to cover message/event redaction, `collectors.py` line, new "Δ is a measurement" rule, "hosts answering total" → "nine-figure fleet-totals panel" (the subtitle still counts hosts), test count 452→472, coverage 96.8→96.9, rewrapped an over-long line. README.md — same fleet-totals wording, `RATE_WINDOW_SECONDS` now covers every measured rate (not just tokens), agent route list completed (`/`, `/schema`, `?top=N`), `LIST CMD` in the diagram, new paragraph on URL redaction.
- **Deviation**: the criterion's literal `curl …/api/system/history?host=…&metric=cpu.usage_pct@core7` uses singular params that endpoint has never accepted (it takes `hosts=`/`metrics=` and ignores unknown ones). Verified 354 points via `…system/history?hosts=driveripper&metrics=cpu.usage_pct@core7&range=1h` and `/api/history?source=host&host=…&metric=…`; I did **not** add singular param aliases (unauthorized API surface). No live infra changes made — reads only.
- **Next pending phase**: `04_llm_chart_readability`.
2026-10-17 04:42:09 -04:00

105 lines
5.2 KiB
Python

"""The `/system` hero-card payload contract, shared by the hermetic and the live tests.
Phase 03 makes `/system` one panel per host with six big colour-coded numbers. The paths
below are the payload fields those cards are allowed to read; they were read out of
`parse_host_snapshot()` and are pinned hermetically in `tests/test_system_contract.py`
(driven from `tests/fixtures/agent_*.json`) and against the real fleet in
`tests/test_live_fleet.py::test_every_host_serves_the_hero_card_contract`.
Keeping the list here rather than in one test file means the live assertions cannot drift
away from the fixture assertions: there is one table, and it is the thing under test.
=========================== ==========================================================
Card Path in `/api/system/snapshot` `hosts[]`
=========================== ==========================================================
cpu % ``cpu.usage_pct`` (+ ``cpu.severity.usage_pct``)
mem % ``memory.used_pct`` (+ ``memory.severity.used_pct``)
cpu temp ``cpu.temp_c`` (+ ``cpu.severity.temp_c``)
disk usage % ``disks.worst_used_pct`` + ``disks.worst_mount`` (+ verdict)
graphics card ``gpus[].model/busy_pct/vram_*/mem_used_pct`` (+ verdict)
network down/up ``network.rx_bytes_per_s`` / ``tx_bytes_per_s`` (no verdict)
=========================== ==========================================================
CPU temp comes from ``cpu.temp_c`` rather than ``temperatures.cpu_c``: both are the agent's
own ``cpu_temperature_celsius``, and ``cpu.temp_c`` is the field ``/api/summary`` flattens
into its fleet row, so the card and the overview cannot drift apart.
Deliberately *not* card metrics (legibility, not missing data — the series stay in SQLite):
per-core usage (``cpu.usage_pct@coreN``), swap, the process table, the raw sensor list, and
per-interface rows. Network carries no ``severity`` on purpose; throughput has no good/bad,
so ``severity.THRESHOLDS`` has no ``net.*`` row.
"""
from __future__ import annotations
from typing import Any
#: Card metric -> dotted payload paths. Present is the contract; a value may be ``None``
#: on a host that genuinely does not have the number (gateway reports no CPU temperature).
CARD_METRICS: dict[str, tuple[str, ...]] = {
"cpu %": ("cpu.usage_pct", "cpu.severity.usage_pct"),
"mem %": ("memory.used_pct", "memory.severity.used_pct"),
"cpu temp": ("cpu.temp_c", "cpu.severity.temp_c", "temperatures.cpu_c"),
# the mount is what makes the percentage actionable: 88 % used — of which disk?
"disk %": ("disks.worst_used_pct", "disks.worst_mount", "disks.severity.worst_used_pct"),
"network down/up": ("network.rx_bytes_per_s", "network.tx_bytes_per_s"),
}
#: Per-GPU paths, checked on every row of `gpus[]` — and on none of them when the list is
#: empty, which is how a host without a GPU stays honest (no card, not a card at 0 % busy).
GPU_METRICS: tuple[str, ...] = (
"model",
"busy_pct",
"vram_used_bytes",
"vram_total_bytes",
"vram_used_pct",
# the honest APU view: vram + GTT, because a Strix Halo keeps the model in GTT
"mem_used_pct",
"severity.busy_pct",
"severity.mem_used_pct",
)
#: `network` must never grow a `severity` dict (see the module docstring).
SEVERITY_FREE_SECTIONS: tuple[str, ...] = ("network",)
#: The host shapes the captures in `tests/fixtures/` cover, and therefore the shapes a card
#: must handle: a discrete GPU, an APU (VRAM *and* GTT), and no GPU at all. A new capture
#: has to be added here — `test_the_contract_covers_every_captured_host` fails otherwise.
HOST_SHAPES: tuple[str, ...] = ("driveripper", "deskwork", "gateway")
def dig(node: Any, path: str) -> Any:
"""Follow a dotted payload path, failing with the path in the message.
A missing key is a contract break; a ``None`` value is not (that renders `n/a`), which is
why this returns the value instead of asserting on it.
"""
walked = ""
for part in path.split("."):
assert isinstance(node, dict), f"{path}: hit a non-dict at {walked or 'payload'}"
assert part in node, (
f"{path}: no key {part!r} under {walked or 'payload'} — a hero card would print "
f"n/a for a number the API used to serve"
)
node = node[part]
walked = f"{walked}.{part}" if walked else part
return node
def assert_card_contract(payload: Any) -> None:
"""Every card path exists on this host payload, GPU rows included."""
for paths in CARD_METRICS.values():
for path in paths:
dig(payload, path)
gpus = payload.get("gpus")
assert isinstance(gpus, list), "gpus must be a list — [] when the agent reports no GPU"
for gpu in gpus:
for path in GPU_METRICS:
dig(gpu, path)
for section in SEVERITY_FREE_SECTIONS:
block = payload.get(section)
assert isinstance(block, dict), f"{section} must be a dict"
assert "severity" not in block, (
f"{section}.severity would be a colour nobody measured — throughput has no "
f"good/bad and severity.THRESHOLDS has no {section}.* row"
)