services/dashboards
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 12_nut_self_test_verdict

Browse Source 
Final state verified. Report:

**Verified (phase 12 — all 3 tasks already complete; this was the verification pass)**
- Re-ran the wire capture live: `ups.test.result` = `No test initiated` (apc) / `Done and passed` (cyberpower) — matches `LIVE_SELF_TEST_WORDS` + both fixtures; `/api/summary` has **no** self-test row, `/api/ups/snapshot` reads `cyberpower ok | Done and passed`, `apc unknown | …run a battery test`; both screenshots read back correct.
- Fixed a defect found in prior work while checking the "unrelated live noise" note: `NutMonitor.collect()` reused one upsd socket after a *partially-read* reply, so a 4 s stall produced bogus `expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'` errors (same hazard on `INSTCMD`, where a late `OK` could be read as a different command's success). `NutClient` now marks a socket whose reply stopped early as `UNUSABLE`, naming the command that stalled; a complete `ERR …` line still leaves it usable (the per-UPS tolerance `collect()` depends on). +3 hermetic pins against the fake upsd; live re-check (09:33–09:44) now logs one line per stall, no cascade.

**Tests / lint / coverage (all green)**
- `bash .agents/validate.sh` → **exit 0**; `uv run pytest --cov --cov-fail-under=94` → **1172 passed, 11 skipped**, coverage **98.63 %** (floor 94); `uv run ruff check src tests agent` clean; `uv run pyright` 0 errors; `node --check` clean. Modified modules: `sources/nut.py` 95 %, `collectors.py` 97 % (no new line uncovered).
- `uv run pytest -m live --live` → **8 passed / 3 failed**, all three the pre-existing `driveripper` agent-transport failures AGENTS.md records (verified still true: unit `inactive`, no installed file, nothing on `:9120`).
- `./scripts/dev-server.sh status`: `nut ok`, `llm:* ok`, `host/disks:deskwork|gateway ok`; the two `driveripper` jobs DOWN.

**Completion criteria**
- Live capture recorded + asserts `ok` — met (re-confirmed from the wire today).
- `/ups` renders `ok`, overview silent, screenshots present — met.
- pytest / cov / ruff / pyright / validate — met (above).
- No other `health.checks[]` behaviour changed — met (diff confined to the `test_result` branch; all prior nut/ups-contract tests unmodified).

**Documentation**: README verified consistent (self-test wording, config table cross-checked against `config.py`, all documented commands/flags exist) — no changes needed beyond phase tasks' edit. AGENTS.md fixes: stale test count (1140 → 1172), `nvme0n1` power-on hours 19 158 → 19 160, gap 4 store-error claim corrected (last stored `store-error` 2026-10-18 06:33, none in the previous 24 h), new `collectors.py`/`sources/nut.py` socket-hygiene note, new gap-1 bullet on the upsd stall, plus live evidence appended to `data/logs/NOTES-self-test.md`.

**Deviations**: made no live *changes* (read-only probes only) — `driveripper`'s agent install needs root (`ops` has no installer/visudo grant) and a battery test stays the operator's call; the upsd 4 s stall itself is upstream-side. `uv pip list --outdated` still advisory (certifi/fastapi/starlette) — dependencies untouched this phase.

**Next pending phase**: `13_llm_page_and_decode_rate`.
This commit is contained in:
ducoterra
2026-10-19 09:54:03 -04:00
parent 1734d81e48
commit 28139df6d6
29 changed files with 943 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/12_nut_self_test_verdict/00_phase.md → .agents/phases/complete/12_nut_self_test_verdict/00_phase.md
View File
.agents/phases/todo/12_nut_self_test_verdict/01_pin_self_test_wording.md → .agents/phases/complete/12_nut_self_test_verdict/01_pin_self_test_wording.md
View File
.agents/phases/todo/12_nut_self_test_verdict/02_verdict_map_for_test_results.md → .agents/phases/complete/12_nut_self_test_verdict/02_verdict_map_for_test_results.md
View File
.agents/phases/todo/12_nut_self_test_verdict/03_no_false_warning_overview_and_ups.md → .agents/phases/complete/12_nut_self_test_verdict/03_no_false_warning_overview_and_ups.md
View File
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__00_phase.a1.err
View File
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__00_phase.a1.md
+22
View File
@@ -0,0 +1,22 @@
Final state verified. Report:
**Verified (phase 12 — all 3 tasks already complete; this was the verification pass)**
- Re-ran the wire capture live: `ups.test.result` = `No test initiated` (apc) / `Done and passed` (cyberpower) — matches `LIVE_SELF_TEST_WORDS` + both fixtures; `/api/summary` has **no** self-test row, `/api/ups/snapshot` reads `cyberpower ok | Done and passed`, `apc unknown | …run a battery test`; both screenshots read back correct.
- Fixed a defect found in prior work while checking the "unrelated live noise" note: `NutMonitor.collect()` reused one upsd socket after a *partially-read* reply, so a 4 s stall produced bogus `expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'` errors (same hazard on `INSTCMD`, where a late `OK` could be read as a different command's success). `NutClient` now marks a socket whose reply stopped early as `UNUSABLE`, naming the command that stalled; a complete `ERR …` line still leaves it usable (the per-UPS tolerance `collect()` depends on). +3 hermetic pins against the fake upsd; live re-check (09:33–09:44) now logs one line per stall, no cascade.
**Tests / lint / coverage (all green)**
- `bash .agents/validate.sh` → **exit 0**; `uv run pytest --cov --cov-fail-under=94` → **1172 passed, 11 skipped**, coverage **98.63 %** (floor 94); `uv run ruff check src tests agent` clean; `uv run pyright` 0 errors; `node --check` clean. Modified modules: `sources/nut.py` 95 %, `collectors.py` 97 % (no new line uncovered).
- `uv run pytest -m live --live` → **8 passed / 3 failed**, all three the pre-existing `driveripper` agent-transport failures AGENTS.md records (verified still true: unit `inactive`, no installed file, nothing on `:9120`).
- `./scripts/dev-server.sh status`: `nut ok`, `llm:* ok`, `host/disks:deskwork|gateway ok`; the two `driveripper` jobs DOWN.
**Completion criteria**
- Live capture recorded + asserts `ok` — met (re-confirmed from the wire today).
- `/ups` renders `ok`, overview silent, screenshots present — met.
- pytest / cov / ruff / pyright / validate — met (above).
- No other `health.checks[]` behaviour changed — met (diff confined to the `test_result` branch; all prior nut/ups-contract tests unmodified).
**Documentation**: README verified consistent (self-test wording, config table cross-checked against `config.py`, all documented commands/flags exist) — no changes needed beyond phase tasks' edit. AGENTS.md fixes: stale test count (1140 → 1172), `nvme0n1` power-on hours 19 158 → 19 160, gap 4 store-error claim corrected (last stored `store-error` 2026-10-18 06:33, none in the previous 24 h), new `collectors.py`/`sources/nut.py` socket-hygiene note, new gap-1 bullet on the upsd stall, plus live evidence appended to `data/logs/NOTES-self-test.md`.
**Deviations**: made no live *changes* (read-only probes only) — `driveripper`'s agent install needs root (`ops` has no installer/visudo grant) and a battery test stays the operator's call; the upsd 4 s stall itself is upstream-side. `uv pip list --outdated` still advisory (certifi/fastapi/starlette) — dependencies untouched this phase.
**Next pending phase**: `13_llm_page_and_decode_rate`.
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__00_phase.a1.validate
+62
View File
@@ -0,0 +1,62 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 6%]
........................................................................ [ 12%]
........................................................................ [ 18%]
........................................................................ [ 24%]
........................................................................ [ 30%]
........................................................................ [ 36%]
........................................................................ [ 42%]
........................................................................ [ 48%]
........................................................................ [ 54%]
........................................................................ [ 60%]
........................................................................ [ 66%]
...................sssssssssss.......................................... [ 73%]
........................................................................ [ 79%]
........................................................................ [ 85%]
........................................................................ [ 91%]
........................................................................ [ 97%]
............................... [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 348 0 100%
src/dashboards/app.py 178 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
src/dashboards/rates.py 37 1 97% 51
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 130 0 100%
src/dashboards/sources/host.py 199 1 99% 147
src/dashboards/sources/llm.py 150 1 99% 228
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 188 2 99% 255-256
------------------------------------------------------------------
TOTAL 2843 39 99%
Required test coverage of 94% reached. Total coverage: 98.63%
1172 passed, 11 skipped, 1 warning in 94.23s (0:01:34)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.7.22 2026.10.18 wheel
fastapi 0.143.5 0.148.0 wheel
starlette 1.8.0 1.9.0 wheel
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__01_pin_self_test_wording.a1.err
View File
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__01_pin_self_test_wording.a1.md
+20
View File
@@ -0,0 +1,20 @@
**Implemented (task 12/01 — pin the self-test wording)**
- Captured `ups.test.result` from the live upsd (`10.1.0.126:3051`, 2026-10-19, direct from the dev box): `apc` = `No test initiated`, **`cyberpower` = `Done and passed`** (a test has run since the phase-05 capture).
- `tests/test_nut.py`: new `LIVE_SELF_TEST_WORDS` table + 3 tests — fixture/capture drift pin (green), and two bug pins asserting the correct verdicts (`ok` for a pass, `unknown` for never-initiated).
- Re-captured `tests/fixtures/nut_apc.vars` / `nut_cyberpower.vars` with the script's own NUT+scrub blocks, since the live CyberPower word differed from the fixture.
- Confirmed the symptom live against the running dev server: `/api/summary` shows `warn cyberpower | Last self-test: Done and passed` and `warn apc | Last self-test: No test initiated`; `/api/ups/snapshot` `health.checks[]` carries both `warn` entries. Evidence in `data/logs/NOTES-self-test.md`.
**Tests / lint / coverage**
- `uv run pytest --cov --cov-fail-under=94` → **1141 passed, 11 skipped, 2 xfailed**, coverage 98.62% (floor 94%).
- `uv run ruff check src tests agent` clean · `uv run pyright` 0 errors · `bash .agents/validate.sh` **exit 0**.
- `uv run pytest tests/test_nut.py -k self_test` → `2 passed, 2 xfailed`; with `--runxfail` both fail as intended: `assert 'warn' == 'ok'` and `assert 'warn' == 'unknown'`.
- `uv run pytest -m live --live` → only the 3 pre-existing `driveripper` agent-transport failures recorded in AGENTS.md; UPS/engine live tests pass.
**Docs**: AGENTS.md fleet fact corrected — `ups.test.result` is no longer "the factory state on both"; it names each box's captured word and the `LIVE_SELF_TEST_WORDS` pin. No README impact (no behaviour change yet).
**Decisions / deviations**
- The two pinning tests use `xfail(strict=True)` instead of being left red: the pipeline gate requires a green suite per task, and strict xfail becomes a hard failure the moment `assess()` is fixed, so task 02 must delete the marker as part of the fix (`--runxfail` is the recorded "it fails today" evidence).
- Re-captured only the NUT fixtures; the scrub pass also wanted to normalise `Local Time is:` in the 16 `smartctl_driveripper_*.txt` captures (they predate that rule) — reverted as unrelated churn.
- Not touched (out of scope, deps unmoved): `uv pip list --outdated` currently reports certifi/fastapi/starlette updates — worth a dependency pass, not a NUT-verdict task.
**Next pending task**: `.agents/phases/todo/12_nut_self_test_verdict/02_verdict_map_for_test_results.md`.
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__01_pin_self_test_wording.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__01_pin_self_test_wording.a1.validate
+62
View File
@@ -0,0 +1,62 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 6%]
........................................................................ [ 12%]
........................................................................ [ 18%]
........................................................................ [ 24%]
........................................................................ [ 31%]
........................................................................ [ 37%]
........................................................................ [ 43%]
........................................................................ [ 49%]
........................................................................ [ 56%]
........................................................................ [ 62%]
........................................................................ [ 68%]
...............sssssssssss.............................................. [ 74%]
.......................................................xx............... [ 81%]
........................................................................ [ 87%]
........................................................................ [ 93%]
........................................................................ [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 348 0 100%
src/dashboards/app.py 178 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
src/dashboards/rates.py 37 1 97% 51
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 130 0 100%
src/dashboards/sources/host.py 199 1 99% 147
src/dashboards/sources/llm.py 150 1 99% 228
src/dashboards/sources/nut.py 390 20 95% 137, 151, 157, 169, 175, 187, 195, 208, 279-280, 282, 498, 510, 545, 578, 582, 639-640, 645, 657
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 188 2 99% 255-256
------------------------------------------------------------------
TOTAL 2820 39 99%
Required test coverage of 94% reached. Total coverage: 98.62%
1141 passed, 11 skipped, 2 xfailed, 1 warning in 61.73s (0:01:01)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.7.22 2026.10.18 wheel
fastapi 0.143.5 0.147.0 wheel
starlette 1.8.0 1.9.0 wheel
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__02_verdict_map_for_test_results.a1.err
View File
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__02_verdict_map_for_test_results.a1.md
+19
View File
@@ -0,0 +1,19 @@
Done — all green.
**Implemented**
- `src/dashboards/sources/nut.py`: new explicit outcome map `SELF_TEST_OUTCOMES` + `self_test_outcome()` + `NEVER_TESTED_DETAIL`; the `assess()` `Last self-test` branch now grades the *outcome* — pass words → `ok` (incl. the captured live `Done and passed`), fail words → `fail`, `No test initiated`/empty/no variable → `unknown` with the existing "run a battery test" sentence, unrecognised word → `warn` printing the driver's raw word (+ `test_date` suffix kept).
- `src/dashboards/collectors.py`: the UPS self-test **event** severity now calls `nut.self_test_outcome()` instead of its own duplicate pass tuple (same bug: a passed test was logged `warn` while the card beside it read green).
- `tests/test_nut.py`: dropped the two task-01 `xfail(strict)` markers (now assert `ok` / `unknown` + exact detail), added `SELF_TEST_WORD_MAP` parametrized table (pass/fail/unknown/warn tiers incl. both live words), case/whitespace variants, unset-vs-never-initiated same-sentence, "no live driver word lands in warn", map-verdict sanity, simulator-word test.
- `tests/test_collectors.py`: added `Done and passed` → `info` event case.
- Docs: AGENTS.md (`sources/nut.py`, `collectors.py` bullets), README.md (`/ups` health-check paragraph + fixture-quirks paragraph).
**Verification**
- `uv run pytest` → **1165 passed, 11 skipped**; `bash .agents/validate.sh` → **EXIT=0**, coverage **98.62%** (gate 94%), `nut.py` 95 % with the new `self_test_outcome()`/map lines fully covered, `ruff` "All checks passed!", `pyright` "0 errors, 0 warnings".
- Live **read-only** check (dev server restarted, signed-in curl against real upsd): `cyberpower → ok | Done and passed`, `apc → unknown | no self-test result reported — run a battery test`; `/api/summary` emits **no** `Last self-test` alert anymore (only apc's legitimate `Battery age: 5.3 years` warn). No live writes.
**Notes / deviations**
- Fixed the duplicate word list in `collectors.py` rather than special-casing the simulator; `nut_sim.py` needed no change (its `PASSED`/`FAILED` land in the right tiers, now asserted).
- A word outside the map (e.g. an in-progress phrasing) stays `warn` by design, per the task spec.
- validate.sh prints an advisory that certifi/fastapi/starlette are outdated; dependencies were not touched by this task.
**Next pending task:** `.agents/phases/todo/12_nut_self_test_verdict/03_no_false_warning_overview_and_ups.md` (API-level alert-absence test + the two screenshots).
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__02_verdict_map_for_test_results.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__02_verdict_map_for_test_results.a1.validate
+62
View File
@@ -0,0 +1,62 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 6%]
........................................................................ [ 12%]
........................................................................ [ 18%]
........................................................................ [ 24%]
........................................................................ [ 30%]
........................................................................ [ 36%]
........................................................................ [ 42%]
........................................................................ [ 48%]
........................................................................ [ 55%]
........................................................................ [ 61%]
........................................................................ [ 67%]
...............sssssssssss.............................................. [ 73%]
........................................................................ [ 79%]
........................................................................ [ 85%]
........................................................................ [ 91%]
........................................................................ [ 97%]
........................ [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 348 0 100%
src/dashboards/app.py 178 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
src/dashboards/rates.py 37 1 97% 51
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 130 0 100%
src/dashboards/sources/host.py 199 1 99% 147
src/dashboards/sources/llm.py 150 1 99% 228
src/dashboards/sources/nut.py 395 20 95% 137, 151, 157, 169, 175, 187, 195, 208, 279-280, 282, 539, 551, 586, 617, 621, 678-679, 684, 696
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 188 2 99% 255-256
------------------------------------------------------------------
TOTAL 2825 39 99%
Required test coverage of 94% reached. Total coverage: 98.62%
1165 passed, 11 skipped, 1 warning in 60.54s (0:01:00)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.7.22 2026.10.18 wheel
fastapi 0.143.5 0.148.0 wheel
starlette 1.8.0 1.9.0 wheel
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__03_no_false_warning_overview_and_ups.a1.err
View File
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__03_no_false_warning_overview_and_ups.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Implemented (task 12/03 — no false warning on the overview and `/ups`)**
- `tests/test_api.py`: new `self_test_app` harness (swaps the `nut` job's collector so it keeps serving the *same* payload the test mutates — no poll race) plus 4 pins: passed word → **no** `/api/summary` alert row *and* an `ok`/verbatim-word `health.checks[]` entry on `/api/ups/snapshot`; never-tested (`unknown`) → also silent; `FAILED` → exactly one `fail` row per UPS, `href=/ups`, id `<ups>:health:Last self-test`, check named once.
- 4th pin proves **no page/CSS change was needed**: `<li class="${check.status}">` + `ul.checks li.ok .mark { color: var(--ok) }` is the whole render path, and no page/CSS rule names one check.
- Live verification (dev server pid 960540, real upsd, `nut ok` before *and* after): `/api/summary` alerts = driveripper unreachable, apc battery age, deskwork memory — **no self-test row**; `/api/ups/snapshot` reads `cyberpower ok | Done and passed`, `apc unknown | …run a battery test`.
- Screenshots read and confirmed: `data/logs/self-test-passed-overview.png`, `data/logs/self-test-passed-ups.png`; evidence appended to `data/logs/NOTES-self-test.md`.
**Tests / lint / coverage**
- `uv run pytest` → **1169 passed, 11 skipped** (exit 0). `bash .agents/validate.sh` → **exit 0**: coverage **98.62%** (floor 94%), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` clean.
- `uv run pytest -m live --live` → 8 passed / 3 failed, all three the pre-existing `driveripper` agent-transport failures AGENTS.md already records; every UPS/engine live test passes.
**Docs**: AGENTS.md testing-policy bullet (verdicts pinned on the surface that shows them, both directions; the no-new-markup rule). No README change — task 02's `/ups` paragraph already stated the behaviour and this task changed none.
**Decisions / deviations**: no `page-ups.js`/`ups.html`/`dash.css` edit (task said don't restyle — pinned instead); no battery test run on live hardware (phase assumption), so the APC screenshot shows the `unknown` dim state; live upsd emitted unrelated transient `did not answer in 4s` events before the pass — poller handled them per invariant 4, noted in NOTES, not a phase-12 regression; left `uv pip list --outdated` advisory (certifi/fastapi/starlette) untouched since dependencies were not moved here.
**Next pending task**: `.agents/phases/todo/13_llm_page_and_decode_rate/01_fleet_strip_spacing.md`.
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__03_no_false_warning_overview_and_ups.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/12_nut_self_test_verdict/12_nut_self_test_verdict__03_no_false_warning_overview_and_ups.a1.validate
+62
View File
@@ -0,0 +1,62 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 6%]
........................................................................ [ 12%]
........................................................................ [ 18%]
........................................................................ [ 24%]
........................................................................ [ 30%]
........................................................................ [ 36%]
........................................................................ [ 42%]
........................................................................ [ 48%]
........................................................................ [ 54%]
........................................................................ [ 61%]
........................................................................ [ 67%]
...................sssssssssss.......................................... [ 73%]
........................................................................ [ 79%]
........................................................................ [ 85%]
........................................................................ [ 91%]
........................................................................ [ 97%]
............................ [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 348 0 100%
src/dashboards/app.py 178 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
src/dashboards/rates.py 37 1 97% 51
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 130 0 100%
src/dashboards/sources/host.py 199 1 99% 147
src/dashboards/sources/llm.py 150 1 99% 228
src/dashboards/sources/nut.py 395 20 95% 137, 151, 157, 169, 175, 187, 195, 208, 279-280, 282, 539, 551, 586, 617, 621, 678-679, 684, 696
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 188 2 99% 255-256
------------------------------------------------------------------
TOTAL 2825 39 99%
Required test coverage of 94% reached. Total coverage: 98.62%
1169 passed, 11 skipped, 1 warning in 62.95s (0:01:02)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
certifi 2026.7.22 2026.10.18 wheel
fastapi 0.143.5 0.148.0 wheel
starlette 1.8.0 1.9.0 wheel
AGENTS.md
+56 -16
View File
@@ -213,8 +213,17 @@ Responsibilities, one line each:
Events are stored under the **dashboard** name (`system`/`disks`/`llm`/`ups`), not the sample Events are stored under the **dashboard** name (`system`/`disks`/`llm`/`ups`), not the sample
source (`host`) — a page filtering `/api/events` by `host` gets an empty feed forever source (`host`) — a page filtering `/api/events` by `host` gets an empty feed forever
(found on `/system` during phase 03 browser verification). (found on `/system` during phase 03 browser verification).
- `collectors.py` — the fetch half. `NutMonitor` owns per-UPS transition bookkeeping. Failure - `collectors.py` — the fetch half. `NutMonitor` owns per-UPS transition bookkeeping (its
messages name the **redacted** URL (`strip_userinfo` / `Target.display_url`), never the one self-test event takes its severity from `nut.self_test_outcome()` and not its own word list,
so a passed test is never an amber line in the feed while the card beside it reads green).
Its per-UPS tolerance for a failed `LIST CMD`/`LIST RW` is for a *complete* error reply only:
one battery whose command list is denied loses its buttons and the poll goes on, while a
**stalled** reply fails the poll, because `NutClient` marks a socket it could not read to its
`END` as unusable (a reply still arriving would otherwise be taken for the *next* command's
answer — the real cause of the `expected 'BEGIN LIST VAR', got 'RW apc …'` lines this fleet's
event table used to fill with, and the same hazard on `INSTCMD`, where a late `OK` read as a
different command's success would be a gate that lies).
Failure messages name the **redacted** URL (`strip_userinfo` / `Target.display_url`), never the one
that was dialled, because they become health records and event rows. `fetch_snapshot()` is that was dialled, because they become health records and event rows. `fetch_snapshot()` is
the one agent `/snapshot` read that `system` and `disks` share, so a dead host says the the one agent `/snapshot` read that `system` and `disks` share, so a dead host says the
same sentence on both pages and only the `timeout` differs (the disks poll has to outlast same sentence on both pages and only the `timeout` differs (the disks poll has to outlast
@@ -247,7 +256,13 @@ Responsibilities, one line each:
`VLLM_HISTOGRAMS`), measured Δ rates, prefill speed as `VLLM_HISTOGRAMS`), measured Δ rates, prefill speed as
`mean(prompt tokens)/mean(prefill time)`, session list from gufo `/slots`. `mean(prompt tokens)/mean(prefill time)`, session list from gufo `/slots`.
- `sources/nut.py` — upsd client, `UpsReading`, `VAR_MAP`, `assess()` (explicit checks, not - `sources/nut.py` — upsd client, `UpsReading`, `VAR_MAP`, `assess()` (explicit checks, not
a mystery score), `COMMANDS` catalog with risk tiers, `COMMAND_ALIASES` + `resolve_command()` a mystery score), `SELF_TEST_OUTCOMES` + `self_test_outcome()` (the `Last self-test` verdict
follows the *outcome* of the driver's `*.test.result` word, not its phrasing: pass words →
`ok` (including `Done and passed`, what the CyberPower here answers), fail words → `fail`,
`No test initiated` / empty / no variable → `unknown` printing `NEVER_TESTED_DETAIL` so both
"never tested" states say one sentence, and an unrecognised word → `warn`, because a driver
inventing a verdict is drift worth noticing and the check prints its own word to be added),
`COMMANDS` catalog with risk tiers, `COMMAND_ALIASES` + `resolve_command()`
(drivers answer two names for the same action — `beeper.enable`/`beeper.on`, (drivers answer two names for the same action — `beeper.enable`/`beeper.on`,
`beeper.disable`/`beeper.off` — so one entry is offered per action and any of its names hits `beeper.disable`/`beeper.off` — so one entry is offered per action and any of its names hits
that entry's verdict, never a gate the rendered name missed), `available_commands()` filtering, that entry's verdict, never a gate the rendered name missed), `available_commands()` filtering,
@@ -566,7 +581,11 @@ That scheme is the history API's key — treat it as a public interface.
CyberPower reports `battery.mfr.date: CPS` (not a date → `unknown`, never a fake age), CyberPower reports `battery.mfr.date: CPS` (not a date → `unknown`, never a fake age),
**neither driver reports a battery temperature or the NUT 2.8 `battery.charger.status`** **neither driver reports a battery temperature or the NUT 2.8 `battery.charger.status`**
(so the battery glyph's state has to come from `on_battery`, and `charger_status` must stay (so the battery glyph's state has to come from `on_battery`, and `charger_status` must stay
optional), and `ups.test.result: No test initiated` is the factory state on both. optional), and **`ups.test.result` is where the two boxes disagree**: the APC still answers
the factory `No test initiated` while the CyberPower answers `Done and passed` (a test has
run on it since the phase-05 capture; both words re-read from the wire on 2026-10-19 and
pinned as `LIVE_SELF_TEST_WORDS` in `tests/test_nut.py`, which fails if the fixture and the
capture ever drift).
Both UPSes are recorded verbatim (`LIST UPS`/`LIST VAR`/`LIST CMD`, serials scrubbed) as Both UPSes are recorded verbatim (`LIST UPS`/`LIST VAR`/`LIST CMD`, serials scrubbed) as
`tests/fixtures/nut_<ups>.vars` by `./scripts/capture-fixtures.sh`, and the wire quotes a `tests/fixtures/nut_<ups>.vars` by `./scripts/capture-fixtures.sh`, and the wire quotes a
field only when it must — `VAR apc ups.status "OL"`, not three quoted fields. field only when it must — `VAR apc ups.status "OL"`, not three quoted fields.
@@ -588,7 +607,8 @@ That scheme is the history API's key — treat it as a public interface.
What changed by 2026-10-19 is that `deskwork`'s **installed** service runs as What changed by 2026-10-19 is that `deskwork`'s **installed** service runs as
`dashboards-agent`, which is in the `disk` group (`getent group disk` → `disk:dashboards-agent`) `dashboards-agent`, which is in the `disk` group (`getent group disk` → `disk:dashboards-agent`)
on an agent installed after the SMART pass: `/disks` shows real numbers there — on an agent installed after the SMART pass: `/disks` shows real numbers there —
`smartctl_available: true`, `nvme0n1` healthy at 52 °C, 19 158 power-on hours, 11 % of rated `smartctl_available: true`, `nvme0n1` healthy at 52 °C, 19 160 power-on hours (an odometer,
so that figure is a capture date, not a fact about the drive), 11 % of rated
wear — so the wear bar and the alarm chips are drawn from live hardware, not only fixtures. wear — so the wear bar and the alarm chips are drawn from live hardware, not only fixtures.
`gateway` answers `smartctl_available: false` (no smartmontools in its VM: the *install it* `gateway` answers `smartctl_available: false` (no smartmontools in its VM: the *install it*
half of the three-way flag), and `driveripper` answers nothing at all until its agent exists, half of the three-way flag), and `driveripper` answers nothing at all until its agent exists,
@@ -636,7 +656,7 @@ set -a; . ./.env.dev; set +a # dev dials the fleet the prod way, no
./scripts/dev-server.sh status # per-source ok/DOWN — unauthenticated: /api/health ./scripts/dev-server.sh status # per-source ok/DOWN — unauthenticated: /api/health
./scripts/dev-server.sh logs ./scripts/dev-server.sh logs
uv run pytest # hermetic: 1140 passed + 11 live-skipped, no network uv run pytest # hermetic: 1172 passed + 11 live-skipped, no network
uv run pytest --cov # coverage gate (98.6%; floor 94%) uv run pytest --cov # coverage gate (98.6%; floor 94%)
uv run pytest -m live --live # opt-in: real fleet / engines / upsd uv run pytest -m live --live # opt-in: real fleet / engines / upsd
uv run ruff check src tests agent && uv run pyright uv run ruff check src tests agent && uv run pyright
@@ -679,6 +699,17 @@ branch `main`); there is no CI on it, so `bash .agents/validate.sh` is the only
reading the end of an attribute row instead of its annotated RAW_VALUE column. reading the end of an attribute row instead of its annotated RAW_VALUE column.
- Anything that needs a failure mode — upsd refusing a command, an engine 500ing, a host - Anything that needs a failure mode — upsd refusing a command, an engine 500ing, a host
rebooting mid-poll, a counter reset — is a fake in `tests/conftest.py`. rebooting mid-poll, a counter reset — is a fake in `tests/conftest.py`.
- **A verdict the operator sees is pinned on the surface that shows it, in both directions.**
`tests/test_nut.py` grades the word; `tests/test_api.py`'s `self_test_app` harness holds the
`nut` job's own payload (its collector is swapped, so a poll cannot overwrite what the test
mutated) and rewrites the `Last self-test` check in the object `/api/ups/snapshot` serves:
a pass (`Done and passed`) and a never-tested pack produce **no** `/api/summary` alert row,
a fail produces exactly one `fail` row per UPS linking to `/ups`. The two directions ship
together on purpose — an absence-only assertion is satisfied by deleting the check, which is
not the fix the report asked for (phase 12). The same file pins that the `/ups` check list
needs **no** markup for a new verdict: `<li class="${check.status}">` plus
`ul.checks li.<verdict> .mark` in `dash.css` is the whole render path, and no page asset or
stylesheet rule names one check (a second opinion about the driver's word is the bug).
- **Page payloads that a rewrite must not break are pinned as a table.** - **Page payloads that a rewrite must not break are pinned as a table.**
`tests/card_contract.py` holds the dotted `/api/system/snapshot` paths the `/system` hero `tests/card_contract.py` holds the dotted `/api/system/snapshot` paths the `/system` hero
cards read (`CARD_METRICS`, `GPU_METRICS`, `HOST_SHAPES`); `tests/test_system_contract.py` cards read (`CARD_METRICS`, `GPU_METRICS`, `HOST_SHAPES`); `tests/test_system_contract.py`
@@ -1106,6 +1137,15 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
- **Pre-existing upsd log noise:** every upsd start logs `Unknown upsmon type =` (20 - **Pre-existing upsd log noise:** every upsd start logs `Unknown upsmon type =` (20
occurrences over 12 starts, oldest before this work). Harmless — `upsmon` logs into both occurrences over 12 starts, oldest before this work). Harmless — `upsmon` logs into both
UPSes right after it — but unexplained. UPSes right after it — but unexplained.
- **One connection every few minutes answers nothing for >4 s.** Live upsd here stalls a
single poll past `NUT_TIMEOUT_SECONDS` roughly every 2–4 minutes and then answers normally
again; `nut` reads `ok` throughout, because the poller's job is one amber line plus the
recovery, not a frozen page (invariant 4). What phase 12's verification pass fixed on our
side was the **cascade** that used to follow it — the event table filling with
`expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'`, which blamed the wire
protocol for a stall two commands earlier (see the `NutClient` bullet above). The stall
itself is an upsd/driver-side question with no repo bug in it, and a battery test is not
how to chase it.
Editing `upsd.users` at all: never grant `actions = -`, and never write it blind — the Editing `upsd.users` at all: never grant `actions = -`, and never write it blind — the
running `upsmon` authenticates as its own account from that same file for both UPSes, so running `upsmon` authenticates as its own account from that same file for both UPSes, so
dropping it means no orderly power-down. Copy the untouched sections from the live bytes dropping it means no orderly power-down. Copy the untouched sections from the live bytes
@@ -1118,16 +1158,16 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
3. **UPS history depth.** Events and the numeric series are stored; long-window battery 3. **UPS history depth.** Events and the numeric series are stored; long-window battery
health (capacity fade over months) has the rollups but no dedicated view. health (capacity fade over months) has the rollups but no dedicated view.
4. **The dev DB is big enough to refuse a write now and then.** The dev store (~640 MB on 4. **The dev DB is big enough to refuse a write now and then.** The dev store (~640 MB on
2026-10-19, its WAL grown past 140 MB at its worst) logs `store-error: OperationalError: 2026-10-19, its WAL grown past 140 MB at its worst) logged `store-error: OperationalError:
database is locked` a few times an database is locked` a few times an hour across `system`/`llm`/`ups` through 2026-10-18; the
hour across `system`/`llm`/`ups`; the poller handles it exactly as invariant 4 requires — poller handles it exactly as invariant 4 requires — the source goes red with the real reason,
the source goes red with the real reason, ages, and writes `recording` when the next poll ages, and writes `recording` when the next poll lands — so nothing freezes, but those polls'
lands — so nothing freezes, but those polls' samples are lost. The one cause this repo can samples are lost. The one cause this repo can prove is two servers on one file, which is why
prove is two servers on one file, which is why `scripts/dev-server.sh stop` waits for the `scripts/dev-server.sh stop` waits for the old pid to die (its comment records the incident).
old pid to die (its comment records the incident). A controlled 25 s external read A controlled 25 s external read transaction during the phase-09 pass produced none, and
transaction during this pass produced none, and neither did the dev-server log window open on phase 12's re-check on 2026-10-19 found the 190 stored `store-error` rows all older than
2026-10-19 — the absence of a sample, not the absence of the pressure, since the file only 2026-10-18 06:33 with none in the previous 24 h of the running dev store — the absence of a
keeps growing. sample, not the absence of the pressure, since the file only keeps growing.
Retention/`wal_checkpoint` tuning for a long-running store is its own phase, not a Retention/`wal_checkpoint` tuning for a long-running store is its own phase, not a
phase-09 follow-up. phase-09 follow-up.
5. **The login throttle keys on the socket peer address** (phase 11), and the one 5. **The login throttle keys on the socket peer address** (phase 11), and the one
README.md
+11 -3
View File
@@ -534,7 +534,13 @@ auth.py ── verifier + session key, loaded once at startup (no row and no see
`severity` dict, with the electrical readings (line in, output, battery volts and amps, `severity` dict, with the electrical readings (line in, output, battery volts and amps,
battery °C) kept below them in smaller muted type because they are what you read *while* the battery °C) kept below them in smaller muted type because they are what you read *while* the
UPS is on battery. Health checks, the battery-test buttons (with their `DANGER` UPS is on battery. Health checks, the battery-test buttons (with their `DANGER`
confirmation) and the NUT variable dump are unchanged, just second. confirmation) and the NUT variable dump are unchanged, just second. The `Last self-test`
check grades the **outcome** of the driver's word and not its phrasing: a pass
(`Done and passed`, the sentence this fleet's CyberPower answers) reads `ok` and raises no
overview warning, a fail reads `fail`, and a pack that has never been tested —
`No test initiated` or no variable at all — says the same "run a battery test" sentence as
`unknown`. Only a word the map has never seen stays a warning, printed verbatim so it can
be added.
The four event feeds (`/system`, `/disks`, `/llm`, `/ups`) are a **paged window** over the `event` The four event feeds (`/system`, `/disks`, `/llm`, `/ups`) are a **paged window** over the `event`
table rather than a printout of it: the list is a bounded box that scrolls, 50 events to a table rather than a printout of it: the list is a bounded box that scrolls, 50 events to a
page, with `1–50 of 200` between `‹ prev` and `next ›` (disabled at the ends) underneath. page, with `1–50 of 200` between `‹ prev` and `next ›` (disabled at the ends) underneath.
@@ -614,8 +620,10 @@ battery glyph (`charge %`, `load %`, `runtime left`, `battery_voltage`, the glyp
verbatim as `tests/fixtures/nut_<ups>.vars` by `./scripts/capture-fixtures.sh`, with the verbatim as `tests/fixtures/nut_<ups>.vars` by `./scripts/capture-fixtures.sh`, with the
device serials scrubbed — and again against live upsd. The captures earn their place: the device serials scrubbed — and again against live upsd. The captures earn their place: the
CyberPower answers `battery.mfr.date: CPS`, the APC's `battery.date` is a factory CyberPower answers `battery.mfr.date: CPS`, the APC's `battery.date` is a factory
placeholder, and **neither** publishes `battery.temperature` or `battery.charger.status`, so placeholder, **neither** publishes `battery.temperature` or `battery.charger.status`, and the
a hand-written variable table would have tested nobody's memory. `tests/slots_contract.py` two disagree on `ups.test.result` (the CyberPower has run a test and answers `Done and passed`,
the APC still answers the factory `No test initiated`), so a hand-written variable table would
have tested nobody's memory. `tests/slots_contract.py`
pins gufo's `/slots` rows the same way — hermetically from pins gufo's `/slots` rows the same way — hermetically from
`tests/fixtures/slots_gufo.json` (the verbatim `/slots` reply, captured whenever `tests/fixtures/slots_gufo.json` (the verbatim `/slots` reply, captured whenever
`GUFO_BASE_URL` is set; the idle-slot rows are the shape the grid must handle) and again `GUFO_BASE_URL` is set; the idle-slot rows are the shape the grid must handle) and again
src/dashboards/collectors.py
+8 -2
View File
@@ -191,6 +191,10 @@ class NutMonitor:
reading = nut.UpsReading(name=ups, description=entry.get("description", ""), reading = nut.UpsReading(name=ups, description=entry.get("description", ""),
ts=now) ts=now)
reading.vars = await client.list_vars(ups) reading.vars = await client.list_vars(ups)
# A refusal is tolerable per UPS (the card loses its buttons, the page says
# why); a stall is not, and is not swallowed — `NutClient` marks a socket
# whose reply stopped early as unusable, so the next read fails the poll
# naming the stall instead of misreading the leftover lines.
try: try:
reading.commands = await client.list_cmds(ups) reading.commands = await client.list_cmds(ups)
except nut.NutError: except nut.NutError:
@@ -256,9 +260,11 @@ class NutMonitor:
first_seen = reading.name not in self._test_results first_seen = reading.name not in self._test_results
self._test_results[reading.name] = result self._test_results[reading.name] = result
if not first_seen: # the first sighting is not news, a change is if not first_seen: # the first sighting is not news, a change is
passed = result.strip().lower() in ("ok", "passed", "pass", "completed") # Same outcome map the health check uses, so a driver's `Done and passed`
# is not an amber line in the feed while the card beside it reads green.
outcome = nut.self_test_outcome(result)
out.append(Event(now, "ups", reading.name, "test", out.append(Event(now, "ups", reading.name, "test",
"info" if passed else "warn", "info" if outcome == "ok" else "warn",
f"{reading.name}: self-test reported {result}")) f"{reading.name}: self-test reported {result}"))
return out return out
src/dashboards/sources/nut.py
+100 -21
View File
@@ -52,6 +52,18 @@ class NutError(Exception):
self.code = code self.code = code
#: The `_read_line` failures that mean "the rest of this reply may still be arriving". An
#: upsd `ERR …` line is the *whole* reply, so a denied `LIST` leaves the connection clean —
#: which is what lets `collectors.NutMonitor` tolerate one UPS's refused `LIST CMD` and go on
#: reading the next. Anything else (a stall, a dropped socket, a line out of place) means a
#: partial reply, and a partial reply is a real hazard here: this upsd answers every poll
#: within 4 s except every few minutes, when one connection gets no answer at all, and the
#: `CMD`/`RW` rows it had already sent used to be read as the *next* command's `BEGIN` — the
#: poll died saying `expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'`, a
#: sentence about the wire protocol, when the truth was a stall two commands earlier.
_INCOMPLETE_CODES = frozenset({"TIMEOUT", "CLOSED", "UNEXPECTED"})
#: upsd error tokens -> what the operator should actually do about them. #: upsd error tokens -> what the operator should actually do about them.
ERROR_HINTS = { ERROR_HINTS = {
# Reached only on an *authenticated* session — an unauthenticated client gets # Reached only on an *authenticated* session — an unauthenticated client gets
@@ -104,6 +116,9 @@ class NutClient:
self.password = password self.password = password
self._reader: asyncio.StreamReader | None = None self._reader: asyncio.StreamReader | None = None
self._writer = None self._writer = None
# `(command, why it stopped)` for the last reply this socket did not read to its end.
# Not `None` means the socket is spent: see `_INCOMPLETE_CODES` and `_mark_unread()`.
self._unread: tuple[str, str] | None = None
async def __aenter__(self) -> NutClient: async def __aenter__(self) -> NutClient:
await self.connect() await self.connect()
@@ -131,6 +146,7 @@ class NutClient:
f"cannot reach upsd at {self.host}:{self.port} ({reason})", f"cannot reach upsd at {self.host}:{self.port} ({reason})",
code="UNREACHABLE", code="UNREACHABLE",
) from exc ) from exc
self._unread = None # a fresh socket has nothing of anybody's left on it
async def close(self) -> None: async def close(self) -> None:
if self._writer is None: if self._writer is None:
@@ -146,9 +162,22 @@ class NutClient:
self._reader = None self._reader = None
# -- plumbing ------------------------------------------------------------ # # -- plumbing ------------------------------------------------------------ #
def _mark_unread(self, command: str, exc: NutError) -> None:
"""Remember a reply that stopped early, so this socket is never asked another question."""
if exc.code in _INCOMPLETE_CODES and self._unread is None:
self._unread = (command, str(exc))
async def _send(self, command: str) -> None: async def _send(self, command: str) -> None:
if self._writer is None: if self._writer is None:
raise NutError("not connected", code="NOCONNECTION") raise NutError("not connected", code="NOCONNECTION")
if self._unread is not None:
stale, reason = self._unread
raise NutError(
f"this upsd connection is unusable: the reply to {stale!r} stopped early "
f"({reason}), and its remaining lines would be read as the answer to "
f"{command.split(None, 1)[0]!r}",
code="UNUSABLE",
)
self._writer.write((command + "\n").encode()) self._writer.write((command + "\n").encode())
await self._writer.drain() await self._writer.drain()
@@ -180,19 +209,23 @@ class NutClient:
begin = f"BEGIN LIST {kind}" begin = f"BEGIN LIST {kind}"
end = f"END LIST {kind}" end = f"END LIST {kind}"
await self._send(command) await self._send(command)
# LIST replies begin with `BEGIN LIST <kind>`, not `OK`, so this path must not try:
# demand an OK line — upsd would otherwise be "broken" on every read. # LIST replies begin with `BEGIN LIST <kind>`, not `OK`, so this path must not
first = await self._read_line(expect_ok=False) # demand an OK line — upsd would otherwise be "broken" on every read.
if not first.startswith(begin): first = await self._read_line(expect_ok=False)
raise NutError(f"expected {begin!r}, got {first!r}", code="UNEXPECTED") if not first.startswith(begin):
rows: list[list[str]] = [] raise NutError(f"expected {begin!r}, got {first!r}", code="UNEXPECTED")
while True: rows: list[list[str]] = []
line = await self._read_line(expect_ok=False) while True:
if line.startswith(end): line = await self._read_line(expect_ok=False)
return rows if line.startswith(end):
rows.append(_split_fields(line)) return rows
if len(rows) > 4096: # a wedged upsd must not grow the heap rows.append(_split_fields(line))
raise NutError(f"{kind} list too long", code="UNEXPECTED") if len(rows) > 4096: # a wedged upsd must not grow the heap
raise NutError(f"{kind} list too long", code="UNEXPECTED")
except NutError as exc:
self._mark_unread(command, exc)
raise
async def login(self) -> bool: async def login(self) -> bool:
"""Authenticate for actions. Returns False when no credentials are configured. """Authenticate for actions. Returns False when no credentials are configured.
@@ -254,8 +287,15 @@ class NutClient:
if not (self.username and self.password): if not (self.username and self.password):
raise NutError("instant commands need NUT_USER/NUT_PASSWORD with `actions` " raise NutError("instant commands need NUT_USER/NUT_PASSWORD with `actions` "
"in upsd.users", code="ACCESS-DENIED") "in upsd.users", code="ACCESS-DENIED")
await self._send(f"INSTCMD {_quote(ups)} {_quote(command)}") sent = f'INSTCMD {_quote(ups)} {_quote(command)}'
return await self._read_line() await self._send(sent)
try:
return await self._read_line()
except NutError as exc:
# A stall here means upsd may still answer the command later; the socket cannot be
# asked anything else, and an `ERR …` line (the whole reply) leaves it usable.
self._mark_unread(sent, exc)
raise
def _quote(value: str) -> str: def _quote(value: str) -> str:
@@ -483,6 +523,47 @@ def _check(name: str, status: str, detail: str, *, metric: str | None = None,
return {"name": name, "status": status, "detail": detail, "metric": metric, "value": value} return {"name": name, "status": status, "detail": detail, "metric": metric, "value": value}
#: What the driver's `*.test.result` word means, matched lowercased and trimmed exactly as
#: `self_test_outcome()` normalises it. The two live words came off this fleet's upsd on
#: 2026-10-19 (`AGENTS.md` rule 7, pinned as `LIVE_SELF_TEST_WORDS` in `tests/test_nut.py`):
#: the CyberPower answers `Done and passed` after a successful test, the APC has never run
#: one and answers the factory `No test initiated`.
SELF_TEST_OUTCOMES: dict[str, str] = {
"ok": "ok",
"passed": "ok",
"pass": "ok",
"completed": "ok",
"done and passed": "ok",
"failed": "fail",
"fail": "fail",
"error": "fail",
"aborted": "fail",
"test incomplete": "fail",
# A pack nobody has ever self-tested is a suggestion to run one, not a fault, so it
# lands in the same tier as a driver that exports no variable at all — both print
# `NEVER_TESTED_DETAIL` rather than a warning nobody can act on.
"no test initiated": "unknown",
"": "unknown",
}
#: The one sentence both "never tested" states say.
NEVER_TESTED_DETAIL = "no self-test result reported — run a battery test"
def self_test_outcome(result: str | None) -> str:
"""Grade a driver's self-test word: `ok`, `fail`, `unknown`, or `warn` if unrecognised.
The verdict follows the *outcome*, not the phrasing: `Done and passed` is a pass even
though it is not the literal `passed` this map was written with, and grading it as a
warning put a healthy battery in the overview's Warnings panel. A word outside the map
is the one thing that still earns `warn` — a driver inventing a verdict is upstream
drift worth noticing, and it keeps printing its own word so the operator can add it.
"""
if result is None:
return "unknown"
return SELF_TEST_OUTCOMES.get(result.strip().lower(), "warn")
def assess(reading: UpsReading, *, now: float | None = None, def assess(reading: UpsReading, *, now: float | None = None,
battery_replacement_years: float = 4.0, min_runtime_seconds: float = 180.0) -> dict[str, Any]: battery_replacement_years: float = 4.0, min_runtime_seconds: float = 180.0) -> dict[str, Any]:
"""A list of explicit checks, not a mystery score. """A list of explicit checks, not a mystery score.
@@ -556,13 +637,11 @@ def assess(reading: UpsReading, *, now: float | None = None,
checks.append(_check("Battery age", "ok", f"{age:.1f} years", value=age)) checks.append(_check("Battery age", "ok", f"{age:.1f} years", value=age))
result = reading.get("test_result") result = reading.get("test_result")
if result is None: outcome = self_test_outcome(result)
checks.append(_check("Last self-test", "unknown", if outcome == "unknown":
"no self-test result reported — run a battery test")) checks.append(_check("Last self-test", "unknown", NEVER_TESTED_DETAIL))
else: else:
passed = result.strip().lower() in ("ok", "passed", "pass", "completed") checks.append(_check("Last self-test", outcome,
failed = result.strip().lower() in ("failed", "fail", "error", "aborted", "test incomplete")
checks.append(_check("Last self-test", "ok" if passed else "fail" if failed else "warn",
f"{result}" + (f" ({reading.get('test_date')})" if reading.get("test_date") else ""))) f"{result}" + (f" ({reading.get('test_date')})" if reading.get("test_date") else "")))
raw_temp = reading.number("battery_temperature") raw_temp = reading.number("battery_temperature")
tests/fixtures/nut_apc.vars
Vendored
+4 -4
View File
@@ -9,7 +9,7 @@ VAR apc battery.charge.low "10"
VAR apc battery.charge.warning "50" VAR apc battery.charge.warning "50"
VAR apc battery.date "2001/09/25" VAR apc battery.date "2001/09/25"
VAR apc battery.mfr.date "2021/06/24" VAR apc battery.mfr.date "2021/06/24"
VAR apc battery.runtime "633" VAR apc battery.runtime "621"
VAR apc battery.runtime.low "120" VAR apc battery.runtime.low "120"
VAR apc battery.type "PbAc" VAR apc battery.type "PbAc"
VAR apc battery.voltage "27.3" VAR apc battery.voltage "27.3"
@@ -36,13 +36,13 @@ VAR apc driver.version.usb "libusb-1.0.30 (API: 0x0100010C)"
VAR apc input.sensitivity "medium" VAR apc input.sensitivity "medium"
VAR apc input.transfer.high "144" VAR apc input.transfer.high "144"
VAR apc input.transfer.low "88" VAR apc input.transfer.low "88"
VAR apc input.voltage "123.0" VAR apc input.voltage "124.0"
VAR apc input.voltage.nominal "120" VAR apc input.voltage.nominal "120"
VAR apc ups.beeper.status "disabled" VAR apc ups.beeper.status "enabled"
VAR apc ups.delay.shutdown "20" VAR apc ups.delay.shutdown "20"
VAR apc ups.firmware "952.e5 .D" VAR apc ups.firmware "952.e5 .D"
VAR apc ups.firmware.aux "e5 " VAR apc ups.firmware.aux "e5 "
VAR apc ups.load "50" VAR apc ups.load "46"
VAR apc ups.mfr "American Power Conversion" VAR apc ups.mfr "American Power Conversion"
VAR apc ups.mfr.date "2021/01/08" VAR apc ups.mfr.date "2021/01/08"
VAR apc ups.model "Back-UPS RS 1500MS" VAR apc ups.model "Back-UPS RS 1500MS"
tests/fixtures/nut_cyberpower.vars
Vendored
+2 -2
View File
@@ -49,11 +49,11 @@ VAR cyberpower ups.model "GX1500U"
VAR cyberpower ups.power "132" VAR cyberpower ups.power "132"
VAR cyberpower ups.power.nominal "1500" VAR cyberpower ups.power.nominal "1500"
VAR cyberpower ups.productid "0601" VAR cyberpower ups.productid "0601"
VAR cyberpower ups.realpower "129" VAR cyberpower ups.realpower "126"
VAR cyberpower ups.realpower.nominal "900" VAR cyberpower ups.realpower.nominal "900"
VAR cyberpower ups.serial "SCRUBBEDSERIAL1" VAR cyberpower ups.serial "SCRUBBEDSERIAL1"
VAR cyberpower ups.status "OL" VAR cyberpower ups.status "OL"
VAR cyberpower ups.test.result "No test initiated" VAR cyberpower ups.test.result "Done and passed"
VAR cyberpower ups.timer.shutdown "-60" VAR cyberpower ups.timer.shutdown "-60"
VAR cyberpower ups.timer.start "-60" VAR cyberpower ups.timer.start "-60"
VAR cyberpower ups.vendorid "0764" VAR cyberpower ups.vendorid "0764"
tests/test_api.py
+165
View File
@@ -382,6 +382,171 @@ def test_snapshot_payloads_carry_their_own_severity_sections(data_client) -> Non
assert isinstance(card["numbers"], dict) and "charge_pct" in card["numbers"] assert isinstance(card["numbers"], dict) and "charge_pct" in card["numbers"]
# ---- the self-test verdict on the two surfaces that were wrong (phase 12) --- #
#: The `ups.test.result` sentences as the hardware answers them. `Done and passed` is the
#: CyberPower's verbatim word from the 2026-10-19 capture (`tests/test_nut.py
#: ::LIVE_SELF_TEST_WORDS`); `FAILED` is what a driver says when a test ran and did not pass,
#: and is the reason the check exists at all. `tests/test_nut.py` pins the *word map*; what is
#: pinned here is the surface `TODO.md:3` names — a warning in the overview's Warnings panel —
#: because a verdict that grades correctly inside `assess()` and still reaches the alert feed
#: as a warning is the same bug wearing a different hat.
SELF_TEST_PASS_WORD = "Done and passed"
SELF_TEST_FAIL_WORD = "FAILED"
#: How `/api/summary` says the check out loud (`alerts._ups_alerts`): `"<check name>: <detail>"`.
SELF_TEST_ALERT_TEXT = "Last self-test"
def set_self_test_verdict(payload: dict[str, Any], status: str, detail: str) -> None:
"""Rewrite every card's `Last self-test` check in the hub's own UPS payload.
Mutating the payload the job actually serves (instead of hand-building a fake one) is what
makes this an end-to-end pin: `/api/ups/snapshot` and `alerts.build()` then read the *same*
object, so the test cannot pass by having the two surfaces disagree about the verdict.
"""
graded = 0
for card in payload["ups"]:
for check in card["health"]["checks"]:
if check["name"] == SELF_TEST_ALERT_TEXT:
check["status"] = status
check["detail"] = detail
graded += 1
assert graded, "no `Last self-test` check in the UPS payload — nothing was tested"
@contextmanager
def self_test_app(tmp_path: Path, monkeypatch: pytest.MonkeyPatch,
) -> Iterator[tuple[TestClient, dict[str, Any]]]:
"""A UPS app whose served payload cannot be replaced by a poll mid-assertion.
The simulated UPSes answer `PASSED`, so the pass direction would survive a race with the
poll loop and the *fail* direction would not — a test that flakes on a poll is a test of
timing. Hence `healthy_client`'s trick for the `nut` job: the collector hands back the very
dict this harness holds, so the health record says `ok` truthfully (invariant 4) and the
payload stays ours. The interval is shortened for the same reason — waiting a real
`NUT_INTERVAL_SECONDS` per test would only make the suite slower at proving the same thing.
"""
with build_client(tmp_path, monkeypatch, simulate=True, nut_host="127.0.0.1",
nut_interval=0.5) as test_client:
from dashboards.poller import Result # noqa: PLC0415 - only this harness needs it
hub = test_client.app.state.ctx.hub # type: ignore[attr-defined]
job = hub.jobs["nut"]
wait_for_payload(test_client, "/api/ups/snapshot")
payload = job.state.payload
async def collect() -> Result:
return Result(payload=payload, samples=[])
job.fn = collect
# let one poll of *our* collector land before the caller mutates the payload, so the
# health record and the served bytes are both demonstrably ours
baseline = job.state.attempts
deadline = time.time() + 5.0
while time.time() < deadline and job.state.attempts <= baseline:
time.sleep(0.02)
assert job.state.payload is payload and job.state.ok, (
"the nut job is not serving the payload under test — the pin below would be fiction"
)
yield test_client, payload
def self_test_alerts(alerts_list: list[dict[str, Any]]) -> list[dict[str, Any]]:
return [item for item in alerts_list
if SELF_TEST_ALERT_TEXT.lower() in item["message"].lower()
or SELF_TEST_ALERT_TEXT.lower() in item["id"].lower()]
def test_a_passed_self_test_is_quiet_on_the_overview_and_ok_on_the_ups_page(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""The TODO's sentence, end to end: `Done and passed` is not a warning anywhere.
Both halves of the fix are asserted on one payload: `/api/ups/snapshot` carries the `ok`
entry the `/ups` check list renders verbatim, and `/api/summary` — the only call the
overview makes — carries **no** row naming it. The simulated card still contributes its
`NUT_SIMULATE` line, so the list is not empty and this cannot be a vacuous pass.
"""
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
set_self_test_verdict(payload, "ok", SELF_TEST_PASS_WORD)
cards = wait_for_payload(test_client, "/api/ups/snapshot")["ups"]
for card in cards:
check = next(c for c in card["health"]["checks"]
if c["name"] == SELF_TEST_ALERT_TEXT)
assert check["status"] == "ok", f"{card['name']} reports a pass as {check['status']}"
assert check["detail"] == SELF_TEST_PASS_WORD, "the driver's word is the useful half"
rows = self_test_alerts(test_client.get("/api/summary").json()["alerts"])
assert rows == [], f"a passed battery test is on the overview as {rows}"
assert any(item["source"] == "ups" for item in
test_client.get("/api/summary").json()["alerts"]), (
"the simulated-UPS line disappeared, so this assertion has nothing left to be quiet about"
)
def test_a_never_tested_battery_is_not_an_alert_either(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""The APC's factory `No test initiated` reaches the overview as silence, not amber.
This is the state the live hardware is actually in on one of the two UPSes, so the
screenshot proves the absence of a warning for a *never-run* test while the payload pin
above proves it for a passed one. `unknown` is not a verdict an operator can act on, and
`alerts._RANK` admits only `fail` and `warn`.
"""
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
set_self_test_verdict(payload, "unknown", nut.NEVER_TESTED_DETAIL)
cards = wait_for_payload(test_client, "/api/ups/snapshot")["ups"]
assert all(next(c for c in card["health"]["checks"]
if c["name"] == SELF_TEST_ALERT_TEXT)["status"] == "unknown"
for card in cards)
assert self_test_alerts(test_client.get("/api/summary").json()["alerts"]) == []
def test_a_failed_self_test_still_reaches_the_overview(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""The check keeps its reason for existing: a test that failed is a red line, once per UPS.
Without this direction the absence test above is satisfiable by deleting the check, which
is the failure mode the phase's completion criterion calls out.
"""
with self_test_app(tmp_path, monkeypatch) as (test_client, payload):
set_self_test_verdict(payload, "fail", SELF_TEST_FAIL_WORD)
names = {card["name"] for card in
wait_for_payload(test_client, "/api/ups/snapshot")["ups"]}
assert names, "no UPS card to fail"
rows = self_test_alerts(test_client.get("/api/summary").json()["alerts"])
assert {row["source"] for row in rows} == names
for row in rows:
assert row["severity"] == "fail", f"a failed battery test reads {row['severity']}"
assert row["href"] == "/ups", "the alert must link to the card that names the UPS"
assert SELF_TEST_ALERT_TEXT in row["message"] and SELF_TEST_FAIL_WORD in row["message"]
assert row["message"].count(SELF_TEST_ALERT_TEXT) == 1, (
"the row names the check twice — see alerts._clean_error")
assert all(row["id"] == f"{row['source']}:health:{SELF_TEST_ALERT_TEXT}" for row in rows)
def test_the_ups_check_list_needs_no_new_markup_for_a_passed_verdict(client) -> None:
"""Task 03 expected no page change, and this pins *why* there is none.
`page-ups.js` puts `check.status` straight on the `<li>`, and `dash.css` colours only the
`.mark` glyph from the shared `--ok`/`--warn`/`--fail` tokens while the check's name and
detail stay `--text`/`--text-muted` — so an `ok` row is quiet ink with a tick, and a verdict
fixed in `assess()` is a verdict rendered. Nothing on either asset names the self-test
check, which is the point: a special case added here would be a second opinion about the
driver's word, and the API already made it.
"""
js = _code(client.get("/static/page-ups.js").text)
assert '<li class="${check.status}">' in js, "the row no longer wears the payload verdict as its class"
for verdict in ("ok", "warn", "fail", "unknown"):
assert f"{verdict}: '" in js[js.index("const MARKS"):], f"MARKS lost the {verdict} glyph"
rows = js[js.index("const health = item.health"):js.index("const commands = item.commands")]
assert "self-test" not in rows.lower(), "the check list grew an opinion about one check by name"
css = _code(_css(client))
ok_rule = re.search(r"ul\.checks li\.ok \.mark \{([^}]*)\}", css)
assert ok_rule and "var(--ok)" in ok_rule.group(1), "the ok tick is not the shared token"
assert "ul.checks li.ok {" not in css, "the whole ok row is coloured — a pass is not a banner"
assert "self-test" not in css.lower(), "the stylesheet special-cases one check by name"
# ---- unconfigured: say so, and do not dial -------------------------------- # # ---- unconfigured: say so, and do not dial -------------------------------- #
def test_config_reports_every_source_as_off(client) -> None: def test_config_reports_every_source_as_off(client) -> None:
tests/test_collectors.py
+5
View File
@@ -557,6 +557,11 @@ async def test_a_self_test_result_is_reported_when_it_changes() -> None:
assert events[0].severity == "warn" assert events[0].severity == "warn"
monitor.simulator = FakeSimulator([reading("OL", **{"ups.test.result": "OK"})]) monitor.simulator = FakeSimulator([reading("OL", **{"ups.test.result": "OK"})])
assert (await monitor.collect()).events[0].severity == "info" assert (await monitor.collect()).events[0].severity == "info"
# the word this fleet's CyberPower actually answers (phase 12 capture): the feed and
# the health check grade one outcome, so a passed test is never an amber line
monitor.simulator = FakeSimulator(
[reading("OL", **{"ups.test.result": "Done and passed"})])
assert (await monitor.collect()).events[0].severity == "info"
async def test_gating_is_applied_where_the_commands_are_built() -> None: async def test_gating_is_applied_where_the_commands_are_built() -> None:
tests/test_nut.py
+264 -2
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import asyncio import asyncio
import time import time
from pathlib import Path
import pytest import pytest
import pytest_asyncio import pytest_asyncio
@@ -16,6 +17,7 @@ import pytest_asyncio
from dashboards.sources import nut from dashboards.sources import nut
from dashboards.sources.nut import NutClient, NutError, UpsReading, normalise_celsius from dashboards.sources.nut import NutClient, NutError, UpsReading, normalise_celsius
from dashboards.sources.nut_sim import NutSimulator, SimUps, from_env from dashboards.sources.nut_sim import NutSimulator, SimUps, from_env
from ups_contract import parse_capture
VAR_ROWS = { VAR_ROWS = {
"battery.charge": "100", "battery.charge": "100",
@@ -47,10 +49,18 @@ class FakeUpsd:
"""Just enough of upsd's protocol to exercise the client, over a real socket.""" """Just enough of upsd's protocol to exercise the client, over a real socket."""
def __init__(self, *, require_auth: bool = False, close_early: bool = False, def __init__(self, *, require_auth: bool = False, close_early: bool = False,
hang: bool = False) -> None: hang: bool = False, partial_list: str | None = None,
deny: tuple[str, ...] = ()) -> None:
self.require_auth = require_auth self.require_auth = require_auth
self.close_early = close_early self.close_early = close_early
self.hang = hang self.hang = hang
# Answer this `LIST` kind with `BEGIN` + one row and then go silent: the row is already
# in the client's buffer when its read times out, which is the exact state a slow upsd
# leaves a shared connection in.
self.partial_list = partial_list
# Answer these `LIST` kinds with a complete `ERR` line — a refusal is the whole reply,
# so a client may (and must) keep using the connection afterwards.
self.deny = deny
self.received: list[str] = [] self.received: list[str] = []
self.authenticated = False self.authenticated = False
self._server: asyncio.AbstractServer | None = None self._server: asyncio.AbstractServer | None = None
@@ -84,7 +94,16 @@ class FakeUpsd:
if verb == "LIST": if verb == "LIST":
kind, _, target = rest.partition(" ") kind, _, target = rest.partition(" ")
if kind == "UPS": if kind in self.deny:
send("ERR ACCESS-DENIED")
elif kind == self.partial_list:
# One row of the real shape, then silence: no END, so the client times out
# with this row sitting in its socket buffer.
send(f"BEGIN LIST {kind}")
send(f'VAR "{_strip(target)}" "ups.status" "OL"' if kind == "VAR"
else f'{kind} "{_strip(target)}" "beeper.mute"')
return
elif kind == "UPS":
send("BEGIN LIST UPS") send("BEGIN LIST UPS")
send('UPS "cyberpower" "Cyber Power Systems PR1500LCDRT2U"') send('UPS "cyberpower" "Cyber Power Systems PR1500LCDRT2U"')
send("END LIST UPS") send("END LIST UPS")
@@ -301,6 +320,77 @@ async def test_a_silent_upsd_times_out_instead_of_hanging() -> None:
await server.stop() await server.stop()
async def test_a_reply_that_stops_early_makes_the_socket_refuse_the_next_command() -> None:
"""A stalled `LIST` is reported as the stall it was, not as a protocol mystery.
This fleet's upsd answers every poll within `NUT_TIMEOUT_SECONDS` except every so often,
and the lines it had already sent stay in the socket buffer. `collectors.NutMonitor`
tolerates a failed `LIST CMD`/`LIST RW` and reads the next UPS on the same connection, so
those leftovers used to be taken for that command's `BEGIN` and the poll died saying
`expected 'BEGIN LIST VAR', got 'RW apc battery.charge.low "10"'` — a sentence about the
wire protocol, when the truth was a 4-second stall two commands earlier (the pair of
messages is in the dev `event` table, which is how this got noticed).
"""
server = FakeUpsd(partial_list="CMD")
port = await server.start()
try:
async with NutClient("127.0.0.1", port, timeout=0.3) as client:
with pytest.raises(NutError) as stalled:
await client.list_cmds("apc")
assert stalled.value.code == "TIMEOUT"
with pytest.raises(NutError) as next_command:
await client.list_vars("apc")
finally:
await server.stop()
assert next_command.value.code == "UNUSABLE"
assert "did not answer" in str(next_command.value), "the cause is named, not the symptom"
assert "LIST CMD" in str(next_command.value), "and it names the command that stalled"
async def test_an_instant_command_that_never_answers_spends_the_connection() -> None:
"""The guard is on the write path too, where it matters most.
`instcmd` is the only thing in this app that dials a write. If upsd answers it after we
gave up, that late `OK` sits on the socket, and a client allowed to ask another question
would read a *different* command's success as the answer to it — the one mistake the four
gates cannot survive, since it would print `OK` beside a `load.off` nobody confirmed.
"""
server = FakeUpsd(hang=True)
port = await server.start()
try:
async with NutClient("127.0.0.1", port, timeout=0.3,
username="monitor", password="pw") as client:
with pytest.raises(NutError) as hung:
await client.instcmd("apc", "beeper.mute")
assert hung.value.code == "TIMEOUT"
with pytest.raises(NutError) as next_command:
await client.instcmd("apc", "load.off")
finally:
await server.stop()
assert next_command.value.code == "UNUSABLE"
assert "INSTCMD" in str(next_command.value)
async def test_an_upsd_error_line_leaves_the_connection_usable() -> None:
"""The other half: `ERR …` is a *complete* reply, so the tolerance in `collect()` stands.
One UPS answering `LIST CMD` with `ACCESS-DENIED` must cost that UPS its command buttons
and nothing else — the same poll keeps reading the other battery. A guard that poisoned the
connection on a refusal would turn a per-UPS refusal into a dead UPS page.
"""
server = FakeUpsd(deny=("CMD",))
port = await server.start()
try:
async with NutClient("127.0.0.1", port, timeout=2.0) as client:
with pytest.raises(NutError) as denied:
await client.list_cmds("apc")
assert denied.value.code == "ACCESS-DENIED"
variables = await client.list_vars("apc")
finally:
await server.stop()
assert variables["ups.status"] == "OL CHRG"
# ---- reading semantics ---------------------------------------------------- # # ---- reading semantics ---------------------------------------------------- #
def reading(**overrides: str) -> UpsReading: def reading(**overrides: str) -> UpsReading:
@@ -407,6 +497,178 @@ def test_missing_test_history_is_unknown_not_ok() -> None:
assert check["status"] == "unknown" assert check["status"] == "unknown"
# ---- the self-test word the hardware actually answers (phase 12 pin) ------- #
#: `ups.test.result` exactly as the two real UPSes answered it, captured from the running
#: upsd on **2026-10-19** from the dev box (which reaches driveripper's 3051 — the transport
#: phase 10 re-verified) with::
#:
#: printf 'LIST VAR apc ups.test.result\nLIST VAR cyberpower ups.test.result\n' |
#: nc <upsd-host> 3051
#:
#: These are the sentences the "Last self-test" check has to grade, and they are not the
#: words its pass tuple was written for (AGENTS.md rule 7 — verify against the deployed
#: upstream): the CyberPower has had a test run and answers `Done and passed`, the APC has
#: not and still answers the factory `No test initiated`. The same two words are what
#: `/api/summary` was putting in the Warnings panel on the capture date, one per UPS.
#: Re-captured with `./scripts/capture-fixtures.sh` the same day, so `nut_*.vars` carries
#: them verbatim; if a future capture changes one, the first test below fails and the word
#: map and this table get updated together.
LIVE_SELF_TEST_WORDS = {
"apc": "No test initiated",
"cyberpower": "Done and passed",
}
def self_test_check(health: dict) -> dict:
return next(check for check in health["checks"] if check["name"] == "Last self-test")
def test_the_captured_self_test_words_are_what_the_fixtures_carry() -> None:
"""The capture and the verbatim fixture must not drift apart.
`assess()` is asserted against `LIVE_SELF_TEST_WORDS` below, so a fixture re-captured
after somebody runs a real battery test has to land in the same table — the day the two
disagree, the verdict is being tested against a memory of the hardware instead of the
hardware (`AGENTS.md` rule 7).
"""
for ups, word in LIVE_SELF_TEST_WORDS.items():
capture = (Path(__file__).parent / "fixtures" / f"nut_{ups}.vars").read_text()
assert f'VAR {ups} ups.test.result "{word}"' in capture, (
f"nut_{ups}.vars no longer answers the captured word {word!r} — re-run "
f"./scripts/capture-fixtures.sh and update LIVE_SELF_TEST_WORDS with the diff"
)
assert parse_capture(ups).get("test_result") == word
def test_a_passed_self_test_reads_ok_and_not_a_warning() -> None:
"""The task-01 pin, now green: the CyberPower's pass word must read `ok`.
Before phase 12 `assess()` compared the driver's word against
`("ok", "passed", "pass", "completed")`, so `Done and passed` fell through to `warn` and
the overview printed a warning for a battery that passed.
"""
health = nut.assess(parse_capture("cyberpower"), now=1_767_225_600.0)
check = self_test_check(health)
assert check["status"] == "ok", (
f"the driver reported {check['detail']!r} and it graded {check['status']!r}"
)
assert check["detail"].startswith(LIVE_SELF_TEST_WORDS["cyberpower"]), (
"the driver's own sentence is the useful part of this line")
def test_a_self_test_that_was_never_initiated_is_not_a_warning() -> None:
"""The second half of the same pin: the APC's factory state is not a fault.
A pack nobody has ever self-tested is a suggestion to run one, so it reads `unknown`
with the unset branch's sentence instead of `warn`.
"""
health = nut.assess(parse_capture("apc"), now=1_767_225_600.0)
check = self_test_check(health)
assert check["status"] == "unknown", (
f"never tested ({check['detail']!r}) must read like the unset case, not "
f"{check['status']!r}"
)
assert check["detail"] == nut.NEVER_TESTED_DETAIL
assert "run a battery test" in check["detail"]
#: The whole `Last self-test` outcome map: the driver's word -> the verdict `assess()` gives.
#: The pass tier holds every pass word this fleet's drivers, the simulator, and the older
#: NUT drivers answer — including `Done and passed`, task 01's live capture and the one the
#: old tuple missed. The fail tier is unchanged: a test that ran and failed is the reason the
#: check exists. `unknown` holds both "no test has ever run" states. `warn` is the deliberate
#: fall-through for a word this map has never seen — a driver inventing a verdict is upstream
#: drift worth a line, and the check prints that word verbatim so it can be added.
SELF_TEST_WORD_MAP = {
"ok": "ok",
"passed": "ok",
"pass": "ok",
"completed": "ok",
LIVE_SELF_TEST_WORDS["cyberpower"]: "ok",
"FAILED": "fail",
"fail": "fail",
"Error": "fail",
"aborted": "fail",
"test incomplete": "fail",
LIVE_SELF_TEST_WORDS["apc"]: "unknown",
"": "unknown",
"mystery": "warn",
"DONE AND FAILED": "warn",
}
@pytest.mark.parametrize("word,expected", sorted(SELF_TEST_WORD_MAP.items()))
def test_the_self_test_check_grades_the_outcome_and_not_the_phrasing(word, expected) -> None:
check = self_test_check(nut.assess(reading(**{"battery.test.result": word}),
now=1_767_225_600.0))
assert check["status"] == expected, f"{word!r} graded {check['status']!r}"
if expected == "unknown":
assert check["detail"] == nut.NEVER_TESTED_DETAIL
else:
assert check["detail"].startswith(word), "the raw word is what the operator reads"
assert "2026-09-01" in check["detail"], "the test_date suffix stays on the line"
@pytest.mark.parametrize("variant", [
"Done and passed", "DONE AND PASSED", "done and passed", " Done and passed ",
])
def test_the_live_pass_word_is_matched_case_and_whitespace_insensitively(variant) -> None:
assert nut.self_test_outcome(variant) == "ok"
check = self_test_check(nut.assess(reading(**{"battery.test.result": variant}),
now=1_767_225_600.0))
assert check["status"] == "ok"
def test_never_tested_says_the_same_sentence_as_no_variable_at_all() -> None:
"""`No test initiated` and a driver exporting nothing are one sentence, not two.
Both are "run a battery test": a check that invented a second phrasing for the same
nothing would be two warnings to learn instead of one thing to do.
"""
values = {key: value for key, value in VAR_ROWS.items()
if not key.startswith("battery.test")}
unset = self_test_check(nut.assess(nut.UpsReading(name="apc", vars=values, commands=[]),
now=1_767_225_600.0))
never = self_test_check(nut.assess(reading(**{
"battery.test.result": LIVE_SELF_TEST_WORDS["apc"]}), now=1_767_225_600.0))
assert unset["status"] == never["status"] == "unknown"
assert never["detail"] == unset["detail"] == nut.NEVER_TESTED_DETAIL
def test_no_word_the_real_drivers_answer_lands_in_warn() -> None:
"""Whatever the two boxes answer must be a word the map knows.
`warn` is the fall-through, so a captured word reaching it means the overview is warning
about a battery this code has simply never read before — the exact bug this phase fixes.
"""
for ups, word in LIVE_SELF_TEST_WORDS.items():
check = self_test_check(nut.assess(parse_capture(ups), now=1_767_225_600.0))
assert check["status"] != "warn", f"{ups} answers {word!r} and it graded warn"
assert check["status"] == nut.self_test_outcome(word)
def test_the_outcome_map_holds_only_verdicts_the_pages_render() -> None:
"""A typo'd tier in the table would reach the browser as an unknown CSS class."""
assert set(nut.SELF_TEST_OUTCOMES.values()) <= {"ok", "fail", "unknown"}
assert nut.self_test_outcome(None) == "unknown", "no variable is not a verdict"
def test_the_simulator_only_answers_words_the_map_grades() -> None:
"""The model's `PASSED` / `FAILED` were already in the right tiers (task 02 item 4).
If the simulator ever learns a third word it goes into `SELF_TEST_OUTCOMES`; it gets no
special case here, because a model the health check grades differently from the hardware
is a dev box that lies about `/ups`.
"""
sim = SimUps("apc", "Smart-UPS 1500", "APC", "AS-1")
now = time.time()
assert nut.self_test_outcome(sim.test_result) == "ok"
assert nut.self_test_outcome("FAILED") == "fail"
card = nut.payload(UpsReading(name=sim.name, vars=sim.vars(now), commands=[]), now=now)
assert self_test_check(card["health"])["status"] == "ok"
def test_alarms_drive_the_overall_verdict() -> None: def test_alarms_drive_the_overall_verdict() -> None:
health = nut.assess(reading(**{"ups.alarm": "BATT.FAIL", "battery.temperature": "27.4"}), health = nut.assess(reading(**{"ups.alarm": "BATT.FAIL", "battery.temperature": "27.4"}),
now=1_767_225_600.0) now=1_767_225_600.0)