phase: 12_nut_self_test_verdict
Final state verified. Report: **Verified (phase 12 — all 3 tasks already complete; this was the verification pass)** - Re-ran the wire capture live: `ups.test.result` = `No test initiated` (apc) / `Done and passed` (cyberpower) — matches `LIVE_SELF_TEST_WORDS` + both fixtures; `/api/summary` has **no** self-test row, `/api/ups/snapshot` reads `cyberpower ok | Done and passed`, `apc unknown | …run a battery test`; both screenshots read back correct. - Fixed a defect found in prior work while checking the "unrelated live noise" note: `NutMonitor.collect()` reused one upsd socket after a *partially-read* reply, so a 4 s stall produced bogus `expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'` errors (same hazard on `INSTCMD`, where a late `OK` could be read as a different command's success). `NutClient` now marks a socket whose reply stopped early as `UNUSABLE`, naming the command that stalled; a complete `ERR …` line still leaves it usable (the per-UPS tolerance `collect()` depends on). +3 hermetic pins against the fake upsd; live re-check (09:33–09:44) now logs one line per stall, no cascade. **Tests / lint / coverage (all green)** - `bash .agents/validate.sh` → **exit 0**; `uv run pytest --cov --cov-fail-under=94` → **1172 passed, 11 skipped**, coverage **98.63 %** (floor 94); `uv run ruff check src tests agent` clean; `uv run pyright` 0 errors; `node --check` clean. Modified modules: `sources/nut.py` 95 %, `collectors.py` 97 % (no new line uncovered). - `uv run pytest -m live --live` → **8 passed / 3 failed**, all three the pre-existing `driveripper` agent-transport failures AGENTS.md records (verified still true: unit `inactive`, no installed file, nothing on `:9120`). - `./scripts/dev-server.sh status`: `nut ok`, `llm:* ok`, `host/disks:deskwork|gateway ok`; the two `driveripper` jobs DOWN. **Completion criteria** - Live capture recorded + asserts `ok` — met (re-confirmed from the wire today). - `/ups` renders `ok`, overview silent, screenshots present — met. - pytest / cov / ruff / pyright / validate — met (above). - No other `health.checks[]` behaviour changed — met (diff confined to the `test_result` branch; all prior nut/ups-contract tests unmodified). **Documentation**: README verified consistent (self-test wording, config table cross-checked against `config.py`, all documented commands/flags exist) — no changes needed beyond phase tasks' edit. AGENTS.md fixes: stale test count (1140 → 1172), `nvme0n1` power-on hours 19 158 → 19 160, gap 4 store-error claim corrected (last stored `store-error` 2026-10-18 06:33, none in the previous 24 h), new `collectors.py`/`sources/nut.py` socket-hygiene note, new gap-1 bullet on the upsd stall, plus live evidence appended to `data/logs/NOTES-self-test.md`. **Deviations**: made no live *changes* (read-only probes only) — `driveripper`'s agent install needs root (`ops` has no installer/visudo grant) and a battery test stays the operator's call; the upsd 4 s stall itself is upstream-side. `uv pip list --outdated` still advisory (certifi/fastapi/starlette) — dependencies untouched this phase. **Next pending phase**: `13_llm_page_and_decode_rate`.
This commit is contained in:
Vendored
+4
-4
@@ -9,7 +9,7 @@ VAR apc battery.charge.low "10"
|
||||
VAR apc battery.charge.warning "50"
|
||||
VAR apc battery.date "2001/09/25"
|
||||
VAR apc battery.mfr.date "2021/06/24"
|
||||
VAR apc battery.runtime "633"
|
||||
VAR apc battery.runtime "621"
|
||||
VAR apc battery.runtime.low "120"
|
||||
VAR apc battery.type "PbAc"
|
||||
VAR apc battery.voltage "27.3"
|
||||
@@ -36,13 +36,13 @@ VAR apc driver.version.usb "libusb-1.0.30 (API: 0x0100010C)"
|
||||
VAR apc input.sensitivity "medium"
|
||||
VAR apc input.transfer.high "144"
|
||||
VAR apc input.transfer.low "88"
|
||||
VAR apc input.voltage "123.0"
|
||||
VAR apc input.voltage "124.0"
|
||||
VAR apc input.voltage.nominal "120"
|
||||
VAR apc ups.beeper.status "disabled"
|
||||
VAR apc ups.beeper.status "enabled"
|
||||
VAR apc ups.delay.shutdown "20"
|
||||
VAR apc ups.firmware "952.e5 .D"
|
||||
VAR apc ups.firmware.aux "e5 "
|
||||
VAR apc ups.load "50"
|
||||
VAR apc ups.load "46"
|
||||
VAR apc ups.mfr "American Power Conversion"
|
||||
VAR apc ups.mfr.date "2021/01/08"
|
||||
VAR apc ups.model "Back-UPS RS 1500MS"
|
||||
|
||||
Vendored
+2
-2
@@ -49,11 +49,11 @@ VAR cyberpower ups.model "GX1500U"
|
||||
VAR cyberpower ups.power "132"
|
||||
VAR cyberpower ups.power.nominal "1500"
|
||||
VAR cyberpower ups.productid "0601"
|
||||
VAR cyberpower ups.realpower "129"
|
||||
VAR cyberpower ups.realpower "126"
|
||||
VAR cyberpower ups.realpower.nominal "900"
|
||||
VAR cyberpower ups.serial "SCRUBBEDSERIAL1"
|
||||
VAR cyberpower ups.status "OL"
|
||||
VAR cyberpower ups.test.result "No test initiated"
|
||||
VAR cyberpower ups.test.result "Done and passed"
|
||||
VAR cyberpower ups.timer.shutdown "-60"
|
||||
VAR cyberpower ups.timer.start "-60"
|
||||
VAR cyberpower ups.vendorid "0764"
|
||||
|
||||
@@ -382,6 +382,171 @@ def test_snapshot_payloads_carry_their_own_severity_sections(data_client) -> Non
|
||||
assert isinstance(card["numbers"], dict) and "charge_pct" in card["numbers"]
|
||||
|
||||
|
||||
# ---- the self-test verdict on the two surfaces that were wrong (phase 12) --- #
|
||||
|
||||
#: The `ups.test.result` sentences as the hardware answers them. `Done and passed` is the
|
||||
#: CyberPower's verbatim word from the 2026-10-19 capture (`tests/test_nut.py
|
||||
#: ::LIVE_SELF_TEST_WORDS`); `FAILED` is what a driver says when a test ran and did not pass,
|
||||
#: and is the reason the check exists at all. `tests/test_nut.py` pins the *word map*; what is
|
||||
#: pinned here is the surface `TODO.md:3` names — a warning in the overview's Warnings panel —
|
||||
#: because a verdict that grades correctly inside `assess()` and still reaches the alert feed
|
||||
#: as a warning is the same bug wearing a different hat.
|
||||
SELF_TEST_PASS_WORD = "Done and passed"
|
||||
SELF_TEST_FAIL_WORD = "FAILED"
|
||||
|
||||
#: How `/api/summary` says the check out loud (`alerts._ups_alerts`): `"<check name>: <detail>"`.
|
||||
SELF_TEST_ALERT_TEXT = "Last self-test"
|
||||
|
||||
|
||||
def set_self_test_verdict(payload: dict[str, Any], status: str, detail: str) -> None:
|
||||
"""Rewrite every card's `Last self-test` check in the hub's own UPS payload.
|
||||
|
||||
Mutating the payload the job actually serves (instead of hand-building a fake one) is what
|
||||
makes this an end-to-end pin: `/api/ups/snapshot` and `alerts.build()` then read the *same*
|
||||
object, so the test cannot pass by having the two surfaces disagree about the verdict.
|
||||
"""
|
||||
graded = 0
|
||||
for card in payload["ups"]:
|
||||
for check in card["health"]["checks"]:
|
||||
if check["name"] == SELF_TEST_ALERT_TEXT:
|
||||
check["status"] = status
|
||||
check["detail"] = detail
|
||||
graded += 1
|
||||
assert graded, "no `Last self-test` check in the UPS payload — nothing was tested"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def self_test_app(tmp_path: Path, monkeypatch: pytest.MonkeyPatch,
|
||||
) -> Iterator[tuple[TestClient, dict[str, Any]]]:
|
||||
"""A UPS app whose served payload cannot be replaced by a poll mid-assertion.
|
||||
|
||||
The simulated UPSes answer `PASSED`, so the pass direction would survive a race with the
|
||||
poll loop and the *fail* direction would not — a test that flakes on a poll is a test of
|
||||
timing. Hence `healthy_client`'s trick for the `nut` job: the collector hands back the very
|
||||
dict this harness holds, so the health record says `ok` truthfully (invariant 4) and the
|
||||
payload stays ours. The interval is shortened for the same reason — waiting a real
|
||||
`NUT_INTERVAL_SECONDS` per test would only make the suite slower at proving the same thing.
|
||||
"""
|
||||
with build_client(tmp_path, monkeypatch, simulate=True, nut_host="127.0.0.1",
|
||||
nut_interval=0.5) as test_client:
|
||||
from dashboards.poller import Result # noqa: PLC0415 - only this harness needs it
|
||||
|
||||
hub = test_client.app.state.ctx.hub # type: ignore[attr-defined]
|
||||
job = hub.jobs["nut"]
|
||||
wait_for_payload(test_client, "/api/ups/snapshot")
|
||||
payload = job.state.payload
|
||||
|
||||
async def collect() -> Result:
|
||||
return Result(payload=payload, samples=[])
|
||||
|
||||
job.fn = collect
|
||||
# let one poll of *our* collector land before the caller mutates the payload, so the
|
||||
# health record and the served bytes are both demonstrably ours
|
||||
baseline = job.state.attempts
|
||||
deadline = time.time() + 5.0
|
||||
while time.time() < deadline and job.state.attempts <= baseline:
|
||||
time.sleep(0.02)
|
||||
assert job.state.payload is payload and job.state.ok, (
|
||||
"the nut job is not serving the payload under test — the pin below would be fiction"
|
||||
)
|
||||
yield test_client, payload
|
||||
|
||||
|
||||
def self_test_alerts(alerts_list: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
return [item for item in alerts_list
|
||||
if SELF_TEST_ALERT_TEXT.lower() in item["message"].lower()
|
||||
or SELF_TEST_ALERT_TEXT.lower() in item["id"].lower()]
|
||||
|
||||
|
||||
def test_a_passed_self_test_is_quiet_on_the_overview_and_ok_on_the_ups_page(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The TODO's sentence, end to end: `Done and passed` is not a warning anywhere.
|
||||
|
||||
Both halves of the fix are asserted on one payload: `/api/ups/snapshot` carries the `ok`
|
||||
entry the `/ups` check list renders verbatim, and `/api/summary` — the only call the
|
||||
overview makes — carries **no** row naming it. The simulated card still contributes its
|
||||
`NUT_SIMULATE` line, so the list is not empty and this cannot be a vacuous pass.
|
||||
"""
|
||||
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
|
||||
set_self_test_verdict(payload, "ok", SELF_TEST_PASS_WORD)
|
||||
cards = wait_for_payload(test_client, "/api/ups/snapshot")["ups"]
|
||||
for card in cards:
|
||||
check = next(c for c in card["health"]["checks"]
|
||||
if c["name"] == SELF_TEST_ALERT_TEXT)
|
||||
assert check["status"] == "ok", f"{card['name']} reports a pass as {check['status']}"
|
||||
assert check["detail"] == SELF_TEST_PASS_WORD, "the driver's word is the useful half"
|
||||
rows = self_test_alerts(test_client.get("/api/summary").json()["alerts"])
|
||||
assert rows == [], f"a passed battery test is on the overview as {rows}"
|
||||
assert any(item["source"] == "ups" for item in
|
||||
test_client.get("/api/summary").json()["alerts"]), (
|
||||
"the simulated-UPS line disappeared, so this assertion has nothing left to be quiet about"
|
||||
)
|
||||
|
||||
|
||||
def test_a_never_tested_battery_is_not_an_alert_either(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The APC's factory `No test initiated` reaches the overview as silence, not amber.
|
||||
|
||||
This is the state the live hardware is actually in on one of the two UPSes, so the
|
||||
screenshot proves the absence of a warning for a *never-run* test while the payload pin
|
||||
above proves it for a passed one. `unknown` is not a verdict an operator can act on, and
|
||||
`alerts._RANK` admits only `fail` and `warn`.
|
||||
"""
|
||||
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
|
||||
set_self_test_verdict(payload, "unknown", nut.NEVER_TESTED_DETAIL)
|
||||
cards = wait_for_payload(test_client, "/api/ups/snapshot")["ups"]
|
||||
assert all(next(c for c in card["health"]["checks"]
|
||||
if c["name"] == SELF_TEST_ALERT_TEXT)["status"] == "unknown"
|
||||
for card in cards)
|
||||
assert self_test_alerts(test_client.get("/api/summary").json()["alerts"]) == []
|
||||
|
||||
|
||||
def test_a_failed_self_test_still_reaches_the_overview(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The check keeps its reason for existing: a test that failed is a red line, once per UPS.
|
||||
|
||||
Without this direction the absence test above is satisfiable by deleting the check, which
|
||||
is the failure mode the phase's completion criterion calls out.
|
||||
"""
|
||||
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
|
||||
set_self_test_verdict(payload, "fail", SELF_TEST_FAIL_WORD)
|
||||
names = {card["name"] for card in
|
||||
wait_for_payload(test_client, "/api/ups/snapshot")["ups"]}
|
||||
assert names, "no UPS card to fail"
|
||||
rows = self_test_alerts(test_client.get("/api/summary").json()["alerts"])
|
||||
assert {row["source"] for row in rows} == names
|
||||
for row in rows:
|
||||
assert row["severity"] == "fail", f"a failed battery test reads {row['severity']}"
|
||||
assert row["href"] == "/ups", "the alert must link to the card that names the UPS"
|
||||
assert SELF_TEST_ALERT_TEXT in row["message"] and SELF_TEST_FAIL_WORD in row["message"]
|
||||
assert row["message"].count(SELF_TEST_ALERT_TEXT) == 1, (
|
||||
"the row names the check twice — see alerts._clean_error")
|
||||
assert all(row["id"] == f"{row['source']}:health:{SELF_TEST_ALERT_TEXT}" for row in rows)
|
||||
|
||||
|
||||
def test_the_ups_check_list_needs_no_new_markup_for_a_passed_verdict(client) -> None:
|
||||
"""Task 03 expected no page change, and this pins *why* there is none.
|
||||
|
||||
`page-ups.js` puts `check.status` straight on the `<li>`, and `dash.css` colours only the
|
||||
`.mark` glyph from the shared `--ok`/`--warn`/`--fail` tokens while the check's name and
|
||||
detail stay `--text`/`--text-muted` — so an `ok` row is quiet ink with a tick, and a verdict
|
||||
fixed in `assess()` is a verdict rendered. Nothing on either asset names the self-test
|
||||
check, which is the point: a special case added here would be a second opinion about the
|
||||
driver's word, and the API already made it.
|
||||
"""
|
||||
js = _code(client.get("/static/page-ups.js").text)
|
||||
assert '<li class="${check.status}">' in js, "the row no longer wears the payload verdict as its class"
|
||||
for verdict in ("ok", "warn", "fail", "unknown"):
|
||||
assert f"{verdict}: '" in js[js.index("const MARKS"):], f"MARKS lost the {verdict} glyph"
|
||||
rows = js[js.index("const health = item.health"):js.index("const commands = item.commands")]
|
||||
assert "self-test" not in rows.lower(), "the check list grew an opinion about one check by name"
|
||||
css = _code(_css(client))
|
||||
ok_rule = re.search(r"ul\.checks li\.ok \.mark \{([^}]*)\}", css)
|
||||
assert ok_rule and "var(--ok)" in ok_rule.group(1), "the ok tick is not the shared token"
|
||||
assert "ul.checks li.ok {" not in css, "the whole ok row is coloured — a pass is not a banner"
|
||||
assert "self-test" not in css.lower(), "the stylesheet special-cases one check by name"
|
||||
|
||||
|
||||
# ---- unconfigured: say so, and do not dial -------------------------------- #
|
||||
|
||||
def test_config_reports_every_source_as_off(client) -> None:
|
||||
|
||||
@@ -557,6 +557,11 @@ async def test_a_self_test_result_is_reported_when_it_changes() -> None:
|
||||
assert events[0].severity == "warn"
|
||||
monitor.simulator = FakeSimulator([reading("OL", **{"ups.test.result": "OK"})])
|
||||
assert (await monitor.collect()).events[0].severity == "info"
|
||||
# the word this fleet's CyberPower actually answers (phase 12 capture): the feed and
|
||||
# the health check grade one outcome, so a passed test is never an amber line
|
||||
monitor.simulator = FakeSimulator(
|
||||
[reading("OL", **{"ups.test.result": "Done and passed"})])
|
||||
assert (await monitor.collect()).events[0].severity == "info"
|
||||
|
||||
|
||||
async def test_gating_is_applied_where_the_commands_are_built() -> None:
|
||||
|
||||
+264
-2
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
@@ -16,6 +17,7 @@ import pytest_asyncio
|
||||
from dashboards.sources import nut
|
||||
from dashboards.sources.nut import NutClient, NutError, UpsReading, normalise_celsius
|
||||
from dashboards.sources.nut_sim import NutSimulator, SimUps, from_env
|
||||
from ups_contract import parse_capture
|
||||
|
||||
VAR_ROWS = {
|
||||
"battery.charge": "100",
|
||||
@@ -47,10 +49,18 @@ class FakeUpsd:
|
||||
"""Just enough of upsd's protocol to exercise the client, over a real socket."""
|
||||
|
||||
def __init__(self, *, require_auth: bool = False, close_early: bool = False,
|
||||
hang: bool = False) -> None:
|
||||
hang: bool = False, partial_list: str | None = None,
|
||||
deny: tuple[str, ...] = ()) -> None:
|
||||
self.require_auth = require_auth
|
||||
self.close_early = close_early
|
||||
self.hang = hang
|
||||
# Answer this `LIST` kind with `BEGIN` + one row and then go silent: the row is already
|
||||
# in the client's buffer when its read times out, which is the exact state a slow upsd
|
||||
# leaves a shared connection in.
|
||||
self.partial_list = partial_list
|
||||
# Answer these `LIST` kinds with a complete `ERR` line — a refusal is the whole reply,
|
||||
# so a client may (and must) keep using the connection afterwards.
|
||||
self.deny = deny
|
||||
self.received: list[str] = []
|
||||
self.authenticated = False
|
||||
self._server: asyncio.AbstractServer | None = None
|
||||
@@ -84,7 +94,16 @@ class FakeUpsd:
|
||||
|
||||
if verb == "LIST":
|
||||
kind, _, target = rest.partition(" ")
|
||||
if kind == "UPS":
|
||||
if kind in self.deny:
|
||||
send("ERR ACCESS-DENIED")
|
||||
elif kind == self.partial_list:
|
||||
# One row of the real shape, then silence: no END, so the client times out
|
||||
# with this row sitting in its socket buffer.
|
||||
send(f"BEGIN LIST {kind}")
|
||||
send(f'VAR "{_strip(target)}" "ups.status" "OL"' if kind == "VAR"
|
||||
else f'{kind} "{_strip(target)}" "beeper.mute"')
|
||||
return
|
||||
elif kind == "UPS":
|
||||
send("BEGIN LIST UPS")
|
||||
send('UPS "cyberpower" "Cyber Power Systems PR1500LCDRT2U"')
|
||||
send("END LIST UPS")
|
||||
@@ -301,6 +320,77 @@ async def test_a_silent_upsd_times_out_instead_of_hanging() -> None:
|
||||
await server.stop()
|
||||
|
||||
|
||||
async def test_a_reply_that_stops_early_makes_the_socket_refuse_the_next_command() -> None:
|
||||
"""A stalled `LIST` is reported as the stall it was, not as a protocol mystery.
|
||||
|
||||
This fleet's upsd answers every poll within `NUT_TIMEOUT_SECONDS` except every so often,
|
||||
and the lines it had already sent stay in the socket buffer. `collectors.NutMonitor`
|
||||
tolerates a failed `LIST CMD`/`LIST RW` and reads the next UPS on the same connection, so
|
||||
those leftovers used to be taken for that command's `BEGIN` and the poll died saying
|
||||
`expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'` — a sentence about the
|
||||
wire protocol, when the truth was a 4-second stall two commands earlier (the pair of
|
||||
messages is in the dev `event` table, which is how this got noticed).
|
||||
"""
|
||||
server = FakeUpsd(partial_list="CMD")
|
||||
port = await server.start()
|
||||
try:
|
||||
async with NutClient("127.0.0.1", port, timeout=0.3) as client:
|
||||
with pytest.raises(NutError) as stalled:
|
||||
await client.list_cmds("apc")
|
||||
assert stalled.value.code == "TIMEOUT"
|
||||
with pytest.raises(NutError) as next_command:
|
||||
await client.list_vars("apc")
|
||||
finally:
|
||||
await server.stop()
|
||||
assert next_command.value.code == "UNUSABLE"
|
||||
assert "did not answer" in str(next_command.value), "the cause is named, not the symptom"
|
||||
assert "LIST CMD" in str(next_command.value), "and it names the command that stalled"
|
||||
|
||||
|
||||
async def test_an_instant_command_that_never_answers_spends_the_connection() -> None:
|
||||
"""The guard is on the write path too, where it matters most.
|
||||
|
||||
`instcmd` is the only thing in this app that dials a write. If upsd answers it after we
|
||||
gave up, that late `OK` sits on the socket, and a client allowed to ask another question
|
||||
would read a *different* command's success as the answer to it — the one mistake the four
|
||||
gates cannot survive, since it would print `OK` beside a `load.off` nobody confirmed.
|
||||
"""
|
||||
server = FakeUpsd(hang=True)
|
||||
port = await server.start()
|
||||
try:
|
||||
async with NutClient("127.0.0.1", port, timeout=0.3,
|
||||
username="monitor", password="pw") as client:
|
||||
with pytest.raises(NutError) as hung:
|
||||
await client.instcmd("apc", "beeper.mute")
|
||||
assert hung.value.code == "TIMEOUT"
|
||||
with pytest.raises(NutError) as next_command:
|
||||
await client.instcmd("apc", "load.off")
|
||||
finally:
|
||||
await server.stop()
|
||||
assert next_command.value.code == "UNUSABLE"
|
||||
assert "INSTCMD" in str(next_command.value)
|
||||
|
||||
|
||||
async def test_an_upsd_error_line_leaves_the_connection_usable() -> None:
|
||||
"""The other half: `ERR …` is a *complete* reply, so the tolerance in `collect()` stands.
|
||||
|
||||
One UPS answering `LIST CMD` with `ACCESS-DENIED` must cost that UPS its command buttons
|
||||
and nothing else — the same poll keeps reading the other battery. A guard that poisoned the
|
||||
connection on a refusal would turn a per-UPS refusal into a dead UPS page.
|
||||
"""
|
||||
server = FakeUpsd(deny=("CMD",))
|
||||
port = await server.start()
|
||||
try:
|
||||
async with NutClient("127.0.0.1", port, timeout=2.0) as client:
|
||||
with pytest.raises(NutError) as denied:
|
||||
await client.list_cmds("apc")
|
||||
assert denied.value.code == "ACCESS-DENIED"
|
||||
variables = await client.list_vars("apc")
|
||||
finally:
|
||||
await server.stop()
|
||||
assert variables["ups.status"] == "OL CHRG"
|
||||
|
||||
|
||||
# ---- reading semantics ---------------------------------------------------- #
|
||||
|
||||
def reading(**overrides: str) -> UpsReading:
|
||||
@@ -407,6 +497,178 @@ def test_missing_test_history_is_unknown_not_ok() -> None:
|
||||
assert check["status"] == "unknown"
|
||||
|
||||
|
||||
# ---- the self-test word the hardware actually answers (phase 12 pin) ------- #
|
||||
|
||||
#: `ups.test.result` exactly as the two real UPSes answered it, captured from the running
|
||||
#: upsd on **2026-10-19** from the dev box (which reaches driveripper's 3051 — the transport
|
||||
#: phase 10 re-verified) with::
|
||||
#:
|
||||
#: printf 'LIST VAR apc ups.test.result\nLIST VAR cyberpower ups.test.result\n' |
|
||||
#: nc <upsd-host> 3051
|
||||
#:
|
||||
#: These are the sentences the "Last self-test" check has to grade, and they are not the
|
||||
#: words its pass tuple was written for (AGENTS.md rule 7 — verify against the deployed
|
||||
#: upstream): the CyberPower has had a test run and answers `Done and passed`, the APC has
|
||||
#: not and still answers the factory `No test initiated`. The same two words are what
|
||||
#: `/api/summary` was putting in the Warnings panel on the capture date, one per UPS.
|
||||
#: Re-captured with `./scripts/capture-fixtures.sh` the same day, so `nut_*.vars` carries
|
||||
#: them verbatim; if a future capture changes one, the first test below fails and the word
|
||||
#: map and this table get updated together.
|
||||
LIVE_SELF_TEST_WORDS = {
|
||||
"apc": "No test initiated",
|
||||
"cyberpower": "Done and passed",
|
||||
}
|
||||
|
||||
|
||||
def self_test_check(health: dict) -> dict:
|
||||
return next(check for check in health["checks"] if check["name"] == "Last self-test")
|
||||
|
||||
|
||||
def test_the_captured_self_test_words_are_what_the_fixtures_carry() -> None:
|
||||
"""The capture and the verbatim fixture must not drift apart.
|
||||
|
||||
`assess()` is asserted against `LIVE_SELF_TEST_WORDS` below, so a fixture re-captured
|
||||
after somebody runs a real battery test has to land in the same table — the day the two
|
||||
disagree, the verdict is being tested against a memory of the hardware instead of the
|
||||
hardware (`AGENTS.md` rule 7).
|
||||
"""
|
||||
for ups, word in LIVE_SELF_TEST_WORDS.items():
|
||||
capture = (Path(__file__).parent / "fixtures" / f"nut_{ups}.vars").read_text()
|
||||
assert f'VAR {ups} ups.test.result "{word}"' in capture, (
|
||||
f"nut_{ups}.vars no longer answers the captured word {word!r} — re-run "
|
||||
f"./scripts/capture-fixtures.sh and update LIVE_SELF_TEST_WORDS with the diff"
|
||||
)
|
||||
assert parse_capture(ups).get("test_result") == word
|
||||
|
||||
|
||||
def test_a_passed_self_test_reads_ok_and_not_a_warning() -> None:
|
||||
"""The task-01 pin, now green: the CyberPower's pass word must read `ok`.
|
||||
|
||||
Before phase 12 `assess()` compared the driver's word against
|
||||
`("ok", "passed", "pass", "completed")`, so `Done and passed` fell through to `warn` and
|
||||
the overview printed a warning for a battery that passed.
|
||||
"""
|
||||
health = nut.assess(parse_capture("cyberpower"), now=1_767_225_600.0)
|
||||
check = self_test_check(health)
|
||||
assert check["status"] == "ok", (
|
||||
f"the driver reported {check['detail']!r} and it graded {check['status']!r}"
|
||||
)
|
||||
assert check["detail"].startswith(LIVE_SELF_TEST_WORDS["cyberpower"]), (
|
||||
"the driver's own sentence is the useful part of this line")
|
||||
|
||||
|
||||
def test_a_self_test_that_was_never_initiated_is_not_a_warning() -> None:
|
||||
"""The second half of the same pin: the APC's factory state is not a fault.
|
||||
|
||||
A pack nobody has ever self-tested is a suggestion to run one, so it reads `unknown`
|
||||
with the unset branch's sentence instead of `warn`.
|
||||
"""
|
||||
health = nut.assess(parse_capture("apc"), now=1_767_225_600.0)
|
||||
check = self_test_check(health)
|
||||
assert check["status"] == "unknown", (
|
||||
f"never tested ({check['detail']!r}) must read like the unset case, not "
|
||||
f"{check['status']!r}"
|
||||
)
|
||||
assert check["detail"] == nut.NEVER_TESTED_DETAIL
|
||||
assert "run a battery test" in check["detail"]
|
||||
|
||||
|
||||
#: The whole `Last self-test` outcome map: the driver's word -> the verdict `assess()` gives.
|
||||
#: The pass tier holds every pass word this fleet's drivers, the simulator, and the older
|
||||
#: NUT drivers answer — including `Done and passed`, task 01's live capture and the one the
|
||||
#: old tuple missed. The fail tier is unchanged: a test that ran and failed is the reason the
|
||||
#: check exists. `unknown` holds both "no test has ever run" states. `warn` is the deliberate
|
||||
#: fall-through for a word this map has never seen — a driver inventing a verdict is upstream
|
||||
#: drift worth a line, and the check prints that word verbatim so it can be added.
|
||||
SELF_TEST_WORD_MAP = {
|
||||
"ok": "ok",
|
||||
"passed": "ok",
|
||||
"pass": "ok",
|
||||
"completed": "ok",
|
||||
LIVE_SELF_TEST_WORDS["cyberpower"]: "ok",
|
||||
"FAILED": "fail",
|
||||
"fail": "fail",
|
||||
"Error": "fail",
|
||||
"aborted": "fail",
|
||||
"test incomplete": "fail",
|
||||
LIVE_SELF_TEST_WORDS["apc"]: "unknown",
|
||||
"": "unknown",
|
||||
"mystery": "warn",
|
||||
"DONE AND FAILED": "warn",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("word,expected", sorted(SELF_TEST_WORD_MAP.items()))
|
||||
def test_the_self_test_check_grades_the_outcome_and_not_the_phrasing(word, expected) -> None:
|
||||
check = self_test_check(nut.assess(reading(**{"battery.test.result": word}),
|
||||
now=1_767_225_600.0))
|
||||
assert check["status"] == expected, f"{word!r} graded {check['status']!r}"
|
||||
if expected == "unknown":
|
||||
assert check["detail"] == nut.NEVER_TESTED_DETAIL
|
||||
else:
|
||||
assert check["detail"].startswith(word), "the raw word is what the operator reads"
|
||||
assert "2026-09-01" in check["detail"], "the test_date suffix stays on the line"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("variant", [
|
||||
"Done and passed", "DONE AND PASSED", "done and passed", " Done and passed ",
|
||||
])
|
||||
def test_the_live_pass_word_is_matched_case_and_whitespace_insensitively(variant) -> None:
|
||||
assert nut.self_test_outcome(variant) == "ok"
|
||||
check = self_test_check(nut.assess(reading(**{"battery.test.result": variant}),
|
||||
now=1_767_225_600.0))
|
||||
assert check["status"] == "ok"
|
||||
|
||||
|
||||
def test_never_tested_says_the_same_sentence_as_no_variable_at_all() -> None:
|
||||
"""`No test initiated` and a driver exporting nothing are one sentence, not two.
|
||||
|
||||
Both are "run a battery test": a check that invented a second phrasing for the same
|
||||
nothing would be two warnings to learn instead of one thing to do.
|
||||
"""
|
||||
values = {key: value for key, value in VAR_ROWS.items()
|
||||
if not key.startswith("battery.test")}
|
||||
unset = self_test_check(nut.assess(nut.UpsReading(name="apc", vars=values, commands=[]),
|
||||
now=1_767_225_600.0))
|
||||
never = self_test_check(nut.assess(reading(**{
|
||||
"battery.test.result": LIVE_SELF_TEST_WORDS["apc"]}), now=1_767_225_600.0))
|
||||
assert unset["status"] == never["status"] == "unknown"
|
||||
assert never["detail"] == unset["detail"] == nut.NEVER_TESTED_DETAIL
|
||||
|
||||
|
||||
def test_no_word_the_real_drivers_answer_lands_in_warn() -> None:
|
||||
"""Whatever the two boxes answer must be a word the map knows.
|
||||
|
||||
`warn` is the fall-through, so a captured word reaching it means the overview is warning
|
||||
about a battery this code has simply never read before — the exact bug this phase fixes.
|
||||
"""
|
||||
for ups, word in LIVE_SELF_TEST_WORDS.items():
|
||||
check = self_test_check(nut.assess(parse_capture(ups), now=1_767_225_600.0))
|
||||
assert check["status"] != "warn", f"{ups} answers {word!r} and it graded warn"
|
||||
assert check["status"] == nut.self_test_outcome(word)
|
||||
|
||||
|
||||
def test_the_outcome_map_holds_only_verdicts_the_pages_render() -> None:
|
||||
"""A typo'd tier in the table would reach the browser as an unknown CSS class."""
|
||||
assert set(nut.SELF_TEST_OUTCOMES.values()) <= {"ok", "fail", "unknown"}
|
||||
assert nut.self_test_outcome(None) == "unknown", "no variable is not a verdict"
|
||||
|
||||
|
||||
def test_the_simulator_only_answers_words_the_map_grades() -> None:
|
||||
"""The model's `PASSED` / `FAILED` were already in the right tiers (task 02 item 4).
|
||||
|
||||
If the simulator ever learns a third word it goes into `SELF_TEST_OUTCOMES`; it gets no
|
||||
special case here, because a model the health check grades differently from the hardware
|
||||
is a dev box that lies about `/ups`.
|
||||
"""
|
||||
sim = SimUps("apc", "Smart-UPS 1500", "APC", "AS-1")
|
||||
now = time.time()
|
||||
assert nut.self_test_outcome(sim.test_result) == "ok"
|
||||
assert nut.self_test_outcome("FAILED") == "fail"
|
||||
card = nut.payload(UpsReading(name=sim.name, vars=sim.vars(now), commands=[]), now=now)
|
||||
assert self_test_check(card["health"])["status"] == "ok"
|
||||
|
||||
|
||||
def test_alarms_drive_the_overall_verdict() -> None:
|
||||
health = nut.assess(reading(**{"ups.alarm": "BATT.FAIL", "battery.temperature": "27.4"}),
|
||||
now=1_767_225_600.0)
|
||||
|
||||
Reference in New Issue
Block a user