phase: 01_severity_language

**Gate defect found & fixed (the "does not lint" bug).** `.agents/validate.sh` — the phase gate — ran `python3 -m pytest` on `/usr/bin/python3` + `~/.local` site-packages (not the uv venv), ran `ruff check .` *only if* a ruff happened to be on PATH, and **never ran pyright or the coverage floor**. Rewrote it to the project's real gate: cd to repo root → `uv run pytest --cov --cov-fail-under=94` → `uv run ruff check src tests agent` → `uv run pyright` → `node --check web/static/*.js` → hygiene gates; a missing toolchain now fails loudly. Provoked it to confirm it bites (floor + a seeded F401 both fail).

**Verification (all green)**
- `bash .agents/validate.sh` → **exit 0**: 356 passed, 6 skipped · coverage **94.75%** (floor 94) · ruff `All checks passed!` · pyright `0 errors` · node syntax clean
- `uv run pytest --cov` → `severity.py` **100%**; `uv run pytest -m live --live` → **6 passed**; `uv pip list --outdated` → empty
- Criteria: `classify('cpu.usage_pct',…)` → `fail`/`ok`/`None` ✓ · `/api/ups/snapshot` carries `severity` beside `charge_pct`/`load_pct`/`battery_runtime` (apc now live-`warn` at 498 s — threshold working) ✓ · no `.sev-*`/synonym vocabulary (only a comment saying so) ✓ · pytest/coverage/lint ✓ · dev server restarted, `/api/health` ok 6/6 sources, all pages+assets 200 ✓
- Chromium against live fleet: `severityFrom` cell→`fail`, bare→`warn`, null→`''`, unrated→`''`; `hero` fail → `rgb(207,34,46)`, null → `n/a` with **no** verdict class, warn+bar 42%; all four pages render unchanged → `data/logs/phase01-final-{index,system,llm,ups}.png`

**Docs (full review, incl. pre-existing drift).** README: `/ups` row claimed "upsd on 3493" (fleet is 3051, stated elsewhere in the same file); config table omitted `DASH_HOST/PORT`, llm/ups `*_INTERVAL_SECONDS`, `*_TIMEOUT_SECONDS`, `GUFO/VLLM_API_KEY`, `POLL_MS_*`, `DEBUGPY` (+ dedup); severity bullet omitted `disks`/`gpus[]`/rated LLM cells and the ratio rescale; agent section listed only `/snapshot` though `/info`+`/health` exist. AGENTS.md: same 3493 claim; `install-agent.sh --token-file` (its flags are `--bind/--port/--token/--rotate-token/--status/--uninstall` — it hands *the agent* `--token-file`); gap 3 "Two commits" (now 6, still no `origin`); gap 1 given the sudoers install command + verified live gating reason; coverage line now 80% repo floor vs 94% gate floor; cyberpower `identity.model: GX1500U` vs `ups.description` quirk. `.env.dev.example`: `DEV_FORWARDS` remote port 3493→3051 and engine URLs that pointed at `127.0.0.1` with nothing forwarding them.

**Deviations.** No `ruff format` (30 files would reformat; the project gate is `ruff check` + pyright — reformatting completed phases is out of scope). **No live-infra changes**: enabling the battery-test buttons needs `deploy/nut-sudoers` installed on driveripper + a nut-server restart and a scoped `[dashboards]` `instcmds` user — prod write with a documented shutdown risk (AGENTS gap 1), so repo-side only. 36 pre-existing `ResourceWarning: unclosed database` from test fixtures are noise, not failures; left rather than churn ~15 fixtures. `severity.attach()` is unit-tested but has no production call site until phases 02–05.

**Next pending phase:** `02_overview_fleet_and_alerts` (`01_alerts_builder.md`).
This commit is contained in:
2026-10-16 22:22:26 -04:00
parent 4b4962d7fb
commit ad993c7b03
40 changed files with 1384 additions and 25 deletions
+75
View File
@@ -13,6 +13,7 @@ from typing import Any
import pytest
from dashboards.rates import RateEstimator
from dashboards.severity import classify
from dashboards.sources.host import HostCollector, fleet_summary, parse_host_snapshot
TICKS_PER_SECOND = 100.0
@@ -234,6 +235,80 @@ def test_fleet_summary_worst_disk_and_memory_totals(agent_snapshot) -> None:
base["memory"]["used_bytes"] / base["memory"]["total_bytes"] * 100)
# ---- severity: the verdict travels with the value ------------------------- #
def test_payloads_carry_a_severity_beside_every_rated_value(agent_snapshot) -> None:
"""The page paints a class; it must never hold its own opinion about where amber starts."""
_first, payload = parse_twice(agent_snapshot)
assert payload["cpu"]["severity"] == {
"usage_pct": classify("cpu.usage_pct", payload["cpu"]["usage_pct"]),
"temp_c": classify("cpu.temp_c", payload["cpu"]["temp_c"]),
}
assert payload["cpu"]["severity"]["usage_pct"] == "ok" # the window is 10 % busy
assert payload["cpu"]["severity"]["temp_c"] == "ok" # 68 °C in the capture
assert payload["memory"]["severity"]["used_pct"] == "ok" # 68 % of 128 GB
assert payload["disks"]["severity"]["worst_used_pct"] == "ok" # 74.6 % worst mount
gpu = payload["gpus"][0]
assert gpu["severity"] == {
"busy_pct": classify("gpu.busy_pct", gpu["busy_pct"]),
"mem_used_pct": classify("gpu.mem_used_pct", gpu["mem_used_pct"]),
}
assert gpu["severity"]["busy_pct"] == "ok"
def test_severity_trips_on_the_ratified_boundaries(agent_snapshot) -> None:
"""Values pushed onto the table's edges in the real fixture, not on a hand-drawn snapshot."""
snap = copy.deepcopy(agent_snapshot)
snap["memory"]["used_bytes"] = snap["memory"]["total_bytes"] * 0.92 # mem.used_pct danger
snap["cpu_temperature_celsius"] = 90.0 # cpu.temp_c danger
for fs in snap["filesystems"]:
fs["used_pct"] = 80.0 # disk.used_pct warn
snap["gpus"][0]["busy_pct"] = 95.0 # gpu.busy_pct danger
snap["gpus"][0]["gtt_used_bytes"] = 65e9 # whole-pool usage well past danger
payload = parse_host_snapshot("driveripper", snap, make_collector(), now=1_000.0).payload
assert payload["memory"]["severity"]["used_pct"] == "fail"
assert payload["cpu"]["severity"]["temp_c"] == "fail"
assert payload["disks"]["severity"]["worst_used_pct"] == "warn"
assert payload["gpus"][0]["severity"]["busy_pct"] == "fail"
assert payload["gpus"][0]["severity"]["mem_used_pct"] == "fail"
def test_a_value_the_agent_did_not_report_gets_no_colour(agent_snapshot) -> None:
"""Unknown is not ok: `severity: null` renders `n/a`, a green pass would be a lie."""
snap = copy.deepcopy(agent_snapshot)
snap["cpu_temperature_celsius"] = None
snap["gpus"][0]["busy_pct"] = None
snap["filesystems"] = []
payload = parse_host_snapshot("driveripper", snap, make_collector(), now=1_000.0).payload
assert payload["cpu"]["severity"]["temp_c"] is None
assert payload["cpu"]["severity"]["usage_pct"] is None, "one scrape is not a rate"
assert payload["gpus"][0]["severity"]["busy_pct"] is None
assert payload["disks"]["severity"]["worst_used_pct"] is None
# the numbers themselves are unchanged by the verdict sitting next to them
assert payload["memory"]["used_pct"] == pytest.approx(
snap["memory"]["used_bytes"] / snap["memory"]["total_bytes"] * 100)
def test_attaching_severity_renamed_or_removed_nothing(agent_snapshot) -> None:
"""Additive-only anchor, at the source: the flat fields the pages read are all still there."""
payload = parse_host_snapshot("driveripper", agent_snapshot, make_collector(),
now=1_000.0).payload
expected = {
"cpu": {"usage_pct", "modes_pct", "load1", "load5", "load15", "cores",
"core_usage_pct", "freq_mhz", "freq_max_mhz", "context_switches_per_s",
"forks_per_s", "procs_running", "procs_blocked", "temp_c"},
"memory": {"total_bytes", "used_bytes", "available_bytes", "used_pct", "cached_bytes",
"buffers_bytes", "swap_total_bytes", "swap_used_bytes", "swap_used_pct",
"committed_bytes"},
"disks": {"filesystems", "worst_used_pct", "worst_mount", "read_bytes_per_s",
"write_bytes_per_s", "reads_per_s", "writes_per_s", "util_pct", "devices"},
}
for section, keys in expected.items():
assert keys <= set(payload[section]), f"{section} lost {keys - set(payload[section])}"
for gpu in payload["gpus"]:
assert {"busy_pct", "mem_used_pct"} <= set(gpu)
def test_fleet_summary_of_nothing_is_empty_not_zero() -> None:
fleet = fleet_summary([])
assert fleet["hosts_total"] == 0