phase: 01_severity_language

**Gate defect found & fixed (the "does not lint" bug).** `.agents/validate.sh` — the phase gate — ran `python3 -m pytest` on `/usr/bin/python3` + `~/.local` site-packages (not the uv venv), ran `ruff check .` *only if* a ruff happened to be on PATH, and **never ran pyright or the coverage floor**. Rewrote it to the project's real gate: cd to repo root → `uv run pytest --cov --cov-fail-under=94` → `uv run ruff check src tests agent` → `uv run pyright` → `node --check web/static/*.js` → hygiene gates; a missing toolchain now fails loudly. Provoked it to confirm it bites (floor + a seeded F401 both fail). **Verification (all green)** - `bash .agents/validate.sh` → **exit 0**: 356 passed, 6 skipped · coverage **94.75%** (floor 94) · ruff `All checks passed!` · pyright `0 errors` · node syntax clean - `uv run pytest --cov` → `severity.py` **100%**; `uv run pytest -m live --live` → **6 passed**; `uv pip list --outdated` → empty - Criteria: `classify('cpu.usage_pct',…)` → `fail`/`ok`/`None` ✓ · `/api/ups/snapshot` carries `severity` beside `charge_pct`/`load_pct`/`battery_runtime` (apc now live-`warn` at 498 s — threshold working) ✓ · no `.sev-*`/synonym vocabulary (only a comment saying so) ✓ · pytest/coverage/lint ✓ · dev server restarted, `/api/health` ok 6/6 sources, all pages+assets 200 ✓ - Chromium against live fleet: `severityFrom` cell→`fail`, bare→`warn`, null→`''`, unrated→`''`; `hero` fail → `rgb(207,34,46)`, null → `n/a` with **no** verdict class, warn+bar 42%; all four pages render unchanged → `data/logs/phase01-final-{index,system,llm,ups}.png` **Docs (full review, incl. pre-existing drift).** README: `/ups` row claimed "upsd on 3493" (fleet is 3051, stated elsewhere in the same file); config table omitted `DASH_HOST/PORT`, llm/ups `*_INTERVAL_SECONDS`, `*_TIMEOUT_SECONDS`, `GUFO/VLLM_API_KEY`, `POLL_MS_*`, `DEBUGPY` (+ dedup); severity bullet omitted `disks`/`gpus[]`/rated LLM cells and the ratio rescale; agent section listed only `/snapshot` though `/info`+`/health` exist. AGENTS.md: same 3493 claim; `install-agent.sh --token-file` (its flags are `--bind/--port/--token/--rotate-token/--status/--uninstall` — it hands *the agent* `--token-file`); gap 3 "Two commits" (now 6, still no `origin`); gap 1 given the sudoers install command + verified live gating reason; coverage line now 80% repo floor vs 94% gate floor; cyberpower `identity.model: GX1500U` vs `ups.description` quirk. `.env.dev.example`: `DEV_FORWARDS` remote port 3493→3051 and engine URLs that pointed at `127.0.0.1` with nothing forwarding them. **Deviations.** No `ruff format` (30 files would reformat; the project gate is `ruff check` + pyright — reformatting completed phases is out of scope). **No live-infra changes**: enabling the battery-test buttons needs `deploy/nut-sudoers` installed on driveripper + a nut-server restart and a scoped `[dashboards]` `instcmds` user — prod write with a documented shutdown risk (AGENTS gap 1), so repo-side only. 36 pre-existing `ResourceWarning: unclosed database` from test fixtures are noise, not failures; left rather than churn ~15 fixtures. `severity.attach()` is unit-tested but has no production call site until phases 02–05. **Next pending phase:** `02_overview_fleet_and_alerts` (`01_alerts_builder.md`).
2026-10-16 22:22:26 -04:00
parent 4b4962d7fb
commit ad993c7b03
40 changed files with 1384 additions and 25 deletions
@@ -0,0 +1,176 @@
+"""The single threshold table and its classifier.
+
+Colour is a judgement, and a judgement made in four places drifts into four meanings.
+These tests pin the boundaries (reaching a threshold trips it, in both directions), the
+"unknown is not ok" rule, and the shape of the two payload helpers that put `severity`
+next to a value.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+
+import pytest
+
+from dashboards import severity
+from dashboards.severity import FAIL, INVERTED, OK, THRESHOLDS, WARN, attach, classify, with_severity
+
+# The ratified table (chat, 2026-10-16). Edit a number here and in THRESHOLDS together —
+# that coupling is the point: thresholds are a contract with the pages, not a tuning knob.
+RATIFIED = {
+    "cpu.usage_pct": (75.0, 90.0),
+    "mem.used_pct": (80.0, 92.0),
+    "disk.used_pct": (80.0, 90.0),
+    "cpu.temp_c": (75.0, 90.0),
+    "gpu.busy_pct": (85.0, 95.0),
+    "gpu.mem_used_pct": (85.0, 95.0),
+    "llm.kv_cache_usage_pct": (75.0, 90.0),
+    "ups.charge_pct": (50.0, 20.0),
+    "ups.battery_runtime": (600.0, 300.0),
+    "ups.load_pct": (70.0, 90.0),
+}
+
+
+def _samples(metric: str) -> list[tuple[float, str]]:
+    """(value, expected) walking across the boundaries of one metric, in its own direction."""
+    warn, danger = THRESHOLDS[metric]
+    middle = (warn + danger) / 2
+    if metric in INVERTED:  # lower is worse: reaching warn trips it, going below danger fails
+        step = 0.1
+        return [
+            (warn + step, OK),
+            (warn, WARN),
+            (middle, WARN),
+            (danger, FAIL),
+            (danger - 1.0, FAIL),
+        ]
+    return [
+        (warn - 0.1, OK),
+        (warn, WARN),
+        (middle, WARN),
+        (danger, FAIL),
+        (danger + 1.0, FAIL),
+    ]
+
+
+@pytest.mark.parametrize("metric", sorted(THRESHOLDS))
+def test_boundaries_hold_for_every_metric_in_the_table(metric: str) -> None:
+    for value, expected in _samples(metric):
+        assert classify(metric, value) == expected, f"{metric}={value} should be {expected}"
+
+
+def test_completion_criteria_examples() -> None:
+    assert classify("cpu.usage_pct", 95) == "fail"
+    assert classify("cpu.usage_pct", 60) == "ok"
+    assert classify("cpu.usage_pct", None) is None
+    assert classify("ups.charge_pct", 15) == "fail"
+
+
+def test_warn_edge_is_inclusive_and_just_below_is_still_good() -> None:
+    assert classify("cpu.usage_pct", 74.9) == "ok"
+    assert classify("cpu.usage_pct", 75) == "warn"
+    assert classify("cpu.usage_pct", 75.0001) == "warn"
+    assert classify("cpu.usage_pct", 89.9) == "warn"
+    assert classify("cpu.usage_pct", 90) == "fail"
+
+
+def test_inverted_metrics_read_the_other_way() -> None:
+    """A flat battery and a flat runtime are the bad end, not the good one."""
+    assert classify("ups.charge_pct", 100) == "ok"
+    assert classify("ups.charge_pct", 50) == "warn"
+    assert classify("ups.charge_pct", 19.9) == "fail"
+    assert classify("ups.battery_runtime", 3600) == "ok"
+    assert classify("ups.battery_runtime", 600) == "warn"
+    assert classify("ups.battery_runtime", 120) == "fail"
+
+
+def test_a_metric_is_only_in_one_direction() -> None:
+    """Sanity: bounds are ordered for their direction, so an edit cannot invert a metric."""
+    for metric, (warn, danger) in THRESHOLDS.items():
+        assert math.isfinite(warn) and math.isfinite(danger), metric
+        if metric in INVERTED:
+            assert warn > danger, f"{metric} is inverted: warn must sit above danger"
+        else:
+            assert warn < danger, f"{metric} is not inverted: warn must sit below danger"
+
+
+def test_inverted_is_a_subset_of_the_table() -> None:
+    assert set(THRESHOLDS) >= INVERTED, "an inverted metric with no thresholds has no colour"
+
+
+def test_the_ratified_numbers_are_the_ones_in_the_table() -> None:
+    assert THRESHOLDS == RATIFIED
+
+
+def test_unknown_metric_is_none_rather_than_an_exception() -> None:
+    assert classify("gpu.vram_used_pct", 99) is None
+    assert classify("nope.nope", 0) is None
+    assert classify("", 100) is None
+
+
+@pytest.mark.parametrize("metric", sorted(THRESHOLDS))
+def test_none_is_never_ok(metric: str) -> None:
+    assert classify(metric, None) is None
+
+
+@pytest.mark.parametrize("metric", sorted(THRESHOLDS))
+@pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf")])
+def test_non_finite_values_are_treated_as_missing(metric: str, value: float) -> None:
+    assert classify(metric, value) is None
+
+
+def test_per_device_streams_inherit_their_base_metric() -> None:
+    assert classify("disk.used_pct@/srv", 95) == "fail"
+    assert classify("disk.used_pct@/srv", 10) == "ok"
+    assert classify("gpu.busy_pct@0", 84) == "ok"
+    assert classify("gpu.busy_pct@0", 90) == "warn"
+    assert classify("gpu.busy_pct@1", 95) == "fail"
+    assert classify("cpu.usage_pct@core7", 80) == "warn"
+
+
+def test_vocabulary_is_exactly_ok_warn_fail_or_none() -> None:
+    seen = {classify(metric, value) for metric in THRESHOLDS for value, _ in _samples(metric)}
+    assert seen == {OK, WARN, FAIL}
+    inverted = severity.INVERTED
+    assert inverted == {"ups.charge_pct", "ups.battery_runtime"}
+
+
+def test_attach_pairs_a_plain_number_with_its_verdict() -> None:
+    assert attach("cpu.usage_pct", 95) == {"value": 95.0, "severity": "fail"}
+    assert attach("mem.used_pct", 10) == {"value": 10.0, "severity": "ok"}
+    assert attach("ups.load_pct", 70) == {"value": 70.0, "severity": "warn"}
+
+
+def test_attach_keeps_the_value_when_it_has_no_opinion() -> None:
+    assert attach("cpu.usage_pct", None) == {"value": None, "severity": None}
+    assert attach("telemetry.mystery", 42) == {"value": 42.0, "severity": None}
+
+
+def test_attach_never_puts_a_non_finite_number_in_a_payload() -> None:
+    for value in (float("nan"), float("inf"), float("-inf")):
+        assert attach("cpu.usage_pct", value) == {"value": None, "severity": None}
+    assert json.dumps(attach("cpu.usage_pct", float("nan")))  # would raise otherwise
+
+
+def test_with_severity_is_additive_on_a_llm_shaped_cell() -> None:
+    cell = {"value": 82.0, "source": "llamacpp:kv_cache_usage_ratio", "derived": False}
+    out = with_severity("llm.kv_cache_usage_pct", cell)
+    assert out["source"] == "llamacpp:kv_cache_usage_ratio"
+    assert out["derived"] is False
+    assert out["value"] == 82.0
+    assert out["severity"] == "warn"
+    assert cell == {"value": 82.0, "source": "llamacpp:kv_cache_usage_ratio", "derived": False}, \
+        "the caller's mapping must not be mutated"
+
+
+def test_with_severity_handles_missing_and_non_numeric_cells() -> None:
+    # purely additive: a missing cell gains `severity` and nothing else
+    assert with_severity("llm.kv_cache_usage_pct", None) == {"severity": None}
+    assert with_severity("llm.kv_cache_usage_pct", {"value": None}) == {"value": None, "severity": None}
+    assert with_severity("llm.decode_tps", {"value": "n/a", "source": None}) == {
+        "value": "n/a", "source": None, "severity": None,
+    }
+    # a value the table has no thresholds for still keeps its other keys
+    out = with_severity("llm.ttft_seconds", {"value": 1.2, "source": "computed"})
+    assert out == {"value": 1.2, "source": "computed", "severity": None}