phase: 01_severity_language
**Gate defect found & fixed (the "does not lint" bug).** `.agents/validate.sh` — the phase gate — ran `python3 -m pytest` on `/usr/bin/python3` + `~/.local` site-packages (not the uv venv), ran `ruff check .` *only if* a ruff happened to be on PATH, and **never ran pyright or the coverage floor**. Rewrote it to the project's real gate: cd to repo root → `uv run pytest --cov --cov-fail-under=94` → `uv run ruff check src tests agent` → `uv run pyright` → `node --check web/static/*.js` → hygiene gates; a missing toolchain now fails loudly. Provoked it to confirm it bites (floor + a seeded F401 both fail).
**Verification (all green)**
- `bash .agents/validate.sh` → **exit 0**: 356 passed, 6 skipped · coverage **94.75%** (floor 94) · ruff `All checks passed!` · pyright `0 errors` · node syntax clean
- `uv run pytest --cov` → `severity.py` **100%**; `uv run pytest -m live --live` → **6 passed**; `uv pip list --outdated` → empty
- Criteria: `classify('cpu.usage_pct',…)` → `fail`/`ok`/`None` ✓ · `/api/ups/snapshot` carries `severity` beside `charge_pct`/`load_pct`/`battery_runtime` (apc now live-`warn` at 498 s — threshold working) ✓ · no `.sev-*`/synonym vocabulary (only a comment saying so) ✓ · pytest/coverage/lint ✓ · dev server restarted, `/api/health` ok 6/6 sources, all pages+assets 200 ✓
- Chromium against live fleet: `severityFrom` cell→`fail`, bare→`warn`, null→`''`, unrated→`''`; `hero` fail → `rgb(207,34,46)`, null → `n/a` with **no** verdict class, warn+bar 42%; all four pages render unchanged → `data/logs/phase01-final-{index,system,llm,ups}.png`
**Docs (full review, incl. pre-existing drift).** README: `/ups` row claimed "upsd on 3493" (fleet is 3051, stated elsewhere in the same file); config table omitted `DASH_HOST/PORT`, llm/ups `*_INTERVAL_SECONDS`, `*_TIMEOUT_SECONDS`, `GUFO/VLLM_API_KEY`, `POLL_MS_*`, `DEBUGPY` (+ dedup); severity bullet omitted `disks`/`gpus[]`/rated LLM cells and the ratio rescale; agent section listed only `/snapshot` though `/info`+`/health` exist. AGENTS.md: same 3493 claim; `install-agent.sh --token-file` (its flags are `--bind/--port/--token/--rotate-token/--status/--uninstall` — it hands *the agent* `--token-file`); gap 3 "Two commits" (now 6, still no `origin`); gap 1 given the sudoers install command + verified live gating reason; coverage line now 80% repo floor vs 94% gate floor; cyberpower `identity.model: GX1500U` vs `ups.description` quirk. `.env.dev.example`: `DEV_FORWARDS` remote port 3493→3051 and engine URLs that pointed at `127.0.0.1` with nothing forwarding them.
**Deviations.** No `ruff format` (30 files would reformat; the project gate is `ruff check` + pyright — reformatting completed phases is out of scope). **No live-infra changes**: enabling the battery-test buttons needs `deploy/nut-sudoers` installed on driveripper + a nut-server restart and a scoped `[dashboards]` `instcmds` user — prod write with a documented shutdown risk (AGENTS gap 1), so repo-side only. 36 pre-existing `ResourceWarning: unclosed database` from test fixtures are noise, not failures; left rather than churn ~15 fixtures. `severity.attach()` is unit-tested but has no production call site until phases 02–05.
**Next pending phase:** `02_overview_fleet_and_alerts` (`01_alerts_builder.md`).
This commit is contained in:
@@ -7,6 +7,8 @@ asset the network can fail to deliver.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
@@ -17,6 +19,9 @@ from fastapi.testclient import TestClient
|
||||
|
||||
from dashboards.app import create_app
|
||||
from dashboards.config import Settings, Target
|
||||
from dashboards.rates import RateEstimator
|
||||
from dashboards.severity import classify
|
||||
from dashboards.sources.host import HostCollector, parse_host_snapshot
|
||||
from dashboards.store import Event, Sample, SampleStore
|
||||
|
||||
# refused immediately, and never dials out of the test environment
|
||||
@@ -63,6 +68,107 @@ def wait_for_payload(client: TestClient, path: str, *, timeout: float = 5.0) ->
|
||||
pytest.fail(f"{path} never became available")
|
||||
|
||||
|
||||
def host_payload(agent_snapshot: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Two scrapes of the real driveripper capture, nudged onto known sides of the table.
|
||||
|
||||
Driven from the fixture rather than hand-written so a schema drift breaks here first.
|
||||
"""
|
||||
snap = copy.deepcopy(agent_snapshot)
|
||||
snap["memory"]["used_bytes"] = snap["memory"]["total_bytes"] * 0.95 # mem.used_pct -> fail
|
||||
snap["cpu_temperature_celsius"] = 91.0 # cpu.temp_c -> fail
|
||||
for fs in snap["filesystems"]:
|
||||
fs["used_pct"] = 93.0 # disk -> fail
|
||||
collector = HostCollector(RateEstimator(span=60.0))
|
||||
parse_host_snapshot("driveripper", snap, collector, now=1_000.0)
|
||||
second = copy.deepcopy(snap)
|
||||
second["ts"] = 1_010.0
|
||||
modes = second["cpu"]["modes"]
|
||||
modes["total"] += 1_000.0
|
||||
modes["idle"] += 900.0 # 10 % busy over the window -> ok
|
||||
return parse_host_snapshot("driveripper", second, collector, now=1_010.0).payload
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def data_client(tmp_path: Path, monkeypatch: pytest.MonkeyPatch,
|
||||
agent_snapshot: dict[str, Any]) -> Iterator[TestClient]:
|
||||
"""Overview-shaped app: one host payload in the hub plus the simulated UPSes.
|
||||
|
||||
The host job points at a dead port, so a poll can never overwrite the injected payload
|
||||
(a failed poll keeps the last good one) — the numbers below are ours, not the fleet's.
|
||||
"""
|
||||
monkeypatch.setenv("NUT_SIMULATE", "1")
|
||||
db = str(tmp_path / "summary.sqlite3")
|
||||
store = SampleStore(db)
|
||||
app = create_app(Settings(db_path=db, nut_host="127.0.0.1",
|
||||
host_targets=(Target("driveripper", DEAD_URL),)), store)
|
||||
test_client = TestClient(app)
|
||||
test_client.__enter__()
|
||||
try:
|
||||
app.state.ctx.hub.jobs["host:driveripper"].state.payload = host_payload(agent_snapshot)
|
||||
wait_for_payload(test_client, "/api/ups/snapshot")
|
||||
yield test_client
|
||||
finally:
|
||||
test_client.__exit__(None, None, None)
|
||||
store.close()
|
||||
|
||||
|
||||
# ---- severity travels with the value --------------------------------------- #
|
||||
|
||||
def test_summary_carries_the_verdict_for_every_column_the_page_colours(data_client) -> None:
|
||||
body = data_client.get("/api/summary").json()
|
||||
row = body["system"]["hosts"][0]
|
||||
assert row["severity"] == {"cpu_usage_pct": "ok", "mem_used_pct": "fail",
|
||||
"disk_worst_used_pct": "fail", "temp_c": "fail"}
|
||||
# the fleet card is judged by the same table, so amber means the same thing at both sizes
|
||||
assert body["system"]["fleet"]["severity"] == {
|
||||
"cpu_usage_pct_avg": "ok", "memory_used_pct": "fail",
|
||||
"disk_worst_used_pct": "fail", "gpu_busy_pct_avg": "ok",
|
||||
}
|
||||
for ups in body["ups"]["ups"]:
|
||||
assert set(ups["severity"]) == {"charge_pct", "load_pct", "runtime_seconds"}
|
||||
assert ups["severity"]["charge_pct"] == classify("ups.charge_pct", ups["charge_pct"])
|
||||
assert ups["severity"]["charge_pct"] == "ok", "the simulator starts at float charge"
|
||||
|
||||
|
||||
def test_summary_shape_did_not_shrink(data_client) -> None:
|
||||
"""Additive-only anchor: /api/summary gained `severity` and lost nothing.
|
||||
|
||||
The overview, the live tests, and any external consumer read the flat keys; this test is
|
||||
the tripwire for a well-meaning refactor that "tidies" them away.
|
||||
"""
|
||||
body = data_client.get("/api/summary").json()
|
||||
fleet_keys = {"hosts_total", "hosts_live", "cpu_cores", "cpu_usage_pct_avg",
|
||||
"memory_total_bytes", "memory_used_bytes", "memory_used_pct",
|
||||
"disk_worst_used_pct", "load1_total", "net_rx_bytes_per_s",
|
||||
"net_tx_bytes_per_s", "gpu_count", "gpu_busy_pct_avg", "power_estimate_w"}
|
||||
host_keys = {"name", "os", "cpu_usage_pct", "load1", "cores", "mem_used_pct",
|
||||
"disk_worst_used_pct", "gpu_busy_pct", "temp_c", "uptime_seconds"}
|
||||
ups_keys = {"name", "overall", "status", "charge_pct", "load_pct", "runtime_seconds"}
|
||||
assert fleet_keys <= set(body["system"]["fleet"]), fleet_keys - set(body["system"]["fleet"])
|
||||
assert host_keys <= set(body["system"]["hosts"][0]), host_keys - set(body["system"]["hosts"][0])
|
||||
assert ups_keys <= set(body["ups"]["ups"][0]), ups_keys - set(body["ups"]["ups"][0])
|
||||
assert set(body) == {"ts", "system", "llm", "ups", "recent_events"}
|
||||
# ...and the new keys are there beside the old ones, not instead of them
|
||||
assert "severity" in body["system"]["fleet"]
|
||||
assert "severity" in body["system"]["hosts"][0]
|
||||
assert "severity" in body["ups"]["ups"][0]
|
||||
|
||||
|
||||
def test_snapshot_payloads_carry_their_own_severity_sections(data_client) -> None:
|
||||
"""/api/system/snapshot and /api/ups/snapshot ship the same verdicts the summary reuses."""
|
||||
system = data_client.get("/api/system/snapshot").json()
|
||||
host = system["hosts"][0]
|
||||
assert set(host["cpu"]["severity"]) == {"usage_pct", "temp_c"}
|
||||
assert set(host["memory"]["severity"]) == {"used_pct"}
|
||||
assert set(host["disks"]["severity"]) == {"worst_used_pct"}
|
||||
assert set(host["gpus"][0]["severity"]) == {"busy_pct", "mem_used_pct"}
|
||||
ups = data_client.get("/api/ups/snapshot").json()
|
||||
for card in ups["ups"]:
|
||||
assert set(card["severity"]) == {"charge_pct", "load_pct", "battery_runtime"}
|
||||
# numbers keep their own shape; severity is a sibling, never a replacement
|
||||
assert isinstance(card["numbers"], dict) and "charge_pct" in card["numbers"]
|
||||
|
||||
|
||||
# ---- unconfigured: say so, and do not dial -------------------------------- #
|
||||
|
||||
def test_config_reports_every_source_as_off(client) -> None:
|
||||
@@ -175,6 +281,104 @@ def test_static_assets_are_served(client) -> None:
|
||||
assert client.get(asset).status_code == 200, asset
|
||||
|
||||
|
||||
# ---- the severity language in the static assets --------------------------- #
|
||||
#
|
||||
# There is no JS test runner (a locked anchor: adding one would mean a build step).
|
||||
# What *can* be asserted end-to-end, from the served bytes, is the contract the pages
|
||||
# depend on: the hero style exists, its colour comes from the shared ok/warn/fail tokens,
|
||||
# no parallel `.sev-*` vocabulary appeared, and `Dash.hero()` reads the verdict instead of
|
||||
# comparing the number to a threshold in JavaScript.
|
||||
|
||||
ASSETS = ("/static/dash.css", "/static/app.js", "/static/charts.js",
|
||||
"/static/page-index.js", "/static/page-system.js", "/static/page-llm.js",
|
||||
"/static/page-ups.js")
|
||||
|
||||
|
||||
def _css(client: TestClient) -> str:
|
||||
return client.get("/static/dash.css").text
|
||||
|
||||
|
||||
def _hero_css(css: str) -> str:
|
||||
"""Just the `hero numbers` section of the stylesheet."""
|
||||
return css.split("/* ---- hero numbers", 1)[1].split("/* ---- bars", 1)[0]
|
||||
|
||||
|
||||
def test_css_ships_the_hero_number_style(client) -> None:
|
||||
css = _css(client)
|
||||
hero_block = _hero_css(css)
|
||||
for rule in (".hero {", ".hero .label", ".hero .value", ".hero .unit", ".hero .sub",
|
||||
".hero.ok .value", ".hero.warn .value", ".hero.fail .value",
|
||||
".hero-bar", ".hero-bar.ok .bar"):
|
||||
assert rule in hero_block, f"dash.css lost the {rule} rule"
|
||||
# a display size with fixed-width digits, so the number does not jitter between polls
|
||||
assert "2.4rem" in hero_block
|
||||
assert "tabular-nums" in hero_block
|
||||
|
||||
|
||||
def test_hero_colour_comes_from_the_shared_verdict_tokens(client) -> None:
|
||||
"""`.hero` reuses `--ok/--warn/--fail`; it does not invent a palette of its own."""
|
||||
hero_block = _hero_css(_css(client))
|
||||
for token in ("var(--ok)", "var(--warn)", "var(--fail)", "var(--text)",
|
||||
"var(--text-muted)", "var(--text-faint)"):
|
||||
assert token in hero_block, f"the hero style no longer uses {token}"
|
||||
|
||||
|
||||
def test_no_stylesheet_rule_uses_an_undeclared_token(client) -> None:
|
||||
"""A typo'd `var(--x)` renders unstyled and reads as a broken panel, not an error."""
|
||||
css = _css(client)
|
||||
root = re.search(r":root\s*\{(.*?)\n\}", css, re.S)
|
||||
assert root is not None
|
||||
declared = set(re.findall(r"(--[a-z0-9-]+)\s*:", root.group(1)))
|
||||
used = set(re.findall(r"var\(\s*(--[a-z0-9-]+)", css))
|
||||
assert not used - declared
|
||||
|
||||
|
||||
def _code(body: str) -> str:
|
||||
"""An asset with its comments stripped — prose may *name* a class the code must not define."""
|
||||
body = re.sub(r"/\*.*?\*/", " ", body, flags=re.S)
|
||||
body = re.sub(r"<!--.*?-->", " ", body, flags=re.S)
|
||||
return re.sub(r"^\s*//.*$", " ", body, flags=re.M)
|
||||
|
||||
|
||||
def test_no_parallel_severity_vocabulary_exists_anywhere(client) -> None:
|
||||
"""One word per colour: `ok|warn|fail`, as in severity.py and the existing components."""
|
||||
for asset in (*ASSETS, "/", "/system", "/llm", "/ups"):
|
||||
body = _code(client.get(asset).text)
|
||||
assert ".sev-" not in body, f"{asset} invented a .sev-* class"
|
||||
assert not re.search(r"classList\.(?:add|toggle)\(\s*[\"'](?:danger|good|warning)[\"']",
|
||||
body), f"{asset} invented a synonym for fail"
|
||||
|
||||
|
||||
def test_app_js_exposes_the_hero_helpers(client) -> None:
|
||||
js = client.get("/static/app.js").text
|
||||
assert "function hero(" in js
|
||||
assert "function severityFrom(" in js
|
||||
exports = js.rsplit("return {", 1)[1].split("}", 1)[0]
|
||||
for name in ("hero", "severityFrom", "severityFor"):
|
||||
assert re.search(rf"\b{name}\b", exports), f"Dash.{name} is not exported"
|
||||
|
||||
|
||||
def test_hero_reads_the_verdict_instead_of_a_threshold(client) -> None:
|
||||
"""`hero()` may only colour what the API rated — no thresholds re-derived in JS."""
|
||||
js = client.get("/static/app.js").text
|
||||
start = js.index("function hero(")
|
||||
body = js[start:js.index("\n function setDot", start)]
|
||||
assert "severityFrom(" in body
|
||||
assert "severityFor(" not in body
|
||||
# an unknown value gets n/a and no colour class at all
|
||||
assert "naText" in body
|
||||
assert re.search(r"raw != null", body)
|
||||
|
||||
|
||||
def test_severity_for_is_kept_as_the_documented_fallback(client) -> None:
|
||||
"""The old call sites still work until phases 02-05 convert each page."""
|
||||
js = client.get("/static/app.js").text
|
||||
assert "FALLBACK ONLY" in js
|
||||
for page in ("/static/page-index.js", "/static/page-system.js",
|
||||
"/static/page-llm.js", "/static/page-ups.js"):
|
||||
assert "severityFor" in client.get(page).text, f"{page} lost its fallback call site"
|
||||
|
||||
|
||||
def test_api_responses_are_not_cacheable(client) -> None:
|
||||
for path in ("/api/config", "/api/health", "/api/summary"):
|
||||
assert "no-store" in client.get(path).headers.get("cache-control", "")
|
||||
|
||||
Reference in New Issue
Block a user