**Phase 09 — final verification pass (all 4 task files already in `complete/`; nothing re-implemented)**
- Verified `GET /api/disks/snapshot` live: 3 hosts / 18 whole-`/sys/block` rows, `smartctl_available` three-way, per-row `severity`, API-summed `fleet`; 404 `{"configured": false}` proven *without dialling* (spies on `httpx.AsyncClient.get` + `collectors.fetch_snapshot`).
- Verified `/api/disks/history` live (`disk.temp_c@dev`, unknown names → empty series) + `test_store.py`'s 300 s rollup/prune buckets; `/api/metrics?source=disks` answers the `disks` family.
- Browser against live data: 18 drive cards (never a partition/mount), collapsed per-host `Charts` with `aria-label`, 3 `.stale` grids + one `.banner.info` why-line and **0 `.fail` classes / green `disks` dot** → criteria 3 and 4 hold; new shots `data/logs/09-verify-disks-live-dimmed.png`, `09-verify-overview-live.png` (13 total for the phase).
- Real-hardware criterion: read 4 driveripper drives through `ops`' `sudo -n smartctl -i -A -H` and ran the live output through `host_agent.parse_smart_output` → `sources/disks.parse_disks_snapshot`: `sda` 35 °C / 30 423 h, `nvme1n1` 58 % wear, `nvme5n1` 13 uncorrectable → `alarm_disks: 1`, temps graded `warn`; consistent with the 16 committed fixtures.
- `bash .agents/validate.sh` → **EXIT=0**; `uv run pytest --cov` → **769 passed, 11 skipped, coverage 98.33%** (floor 94); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean; hygiene + dependency advisories clean. `uv run pytest -m live --live` → 10 passed, 1 skipped (agents still predate the SMART pass). Dev server healthy: `/api/health` `ok`, 9/9 sources.
- Docs fixed: AGENTS.md test count 698→769 and coverage 98.2→98.33 %; `disks` added to the event dashboard-name list and to the "every feed renders `Dash.eventText`" sentence; README `POLL_MS_*` row now states `/disks`' deliberate fixed 30 s; new Known gap 4 records the dev-DB `database is locked` writes (reported/recovered per invariant 4, ~590 MB DB / ~70 MB WAL, not reproduced by a controlled 25 s external read txn).
- Deviations: **no live-infra change** — granting the agent user the `disk` group and redeploying `host_agent.py` on driveripper/deskwork stay the documented paired operator action (redeploying without the grant yields 16 permission-denied lines; a sudo-spawning agent would break the confirmed "direct read, no sudo" assumption). Store-lock issue left to its own phase rather than patched speculatively.
- Next pending phase: `10_dev_direct_dial`.
README advertised `uv run pytest -m live --live`, but there was no --live
option and no `live` marker registered — the command errored out, so the
one workflow that catches an upstream renaming a series did not exist.
- conftest adds --live and skips `live`-marked tests without it, so the
default run stays hermetic (258 passed, 6 skipped, no network).
- tests/test_live_fleet.py drives the real code paths (collectors.*,
NutMonitor.collect) against the fleet, both engines, and upsd. Each
skips when its source is unset, and none of them may write — a battery
test on live hardware is not a unit test. The host test polls twice
because a rate has no second point on the first scrape.
- The agent's real-/proc test stays in the default run and is explicitly
not `live`: it touches only the box running pytest.
- dev-tunnels.sh grows DEV_FORWARDS for services already listening
remotely, which is how upsd gets reachable without opening 3493;
pidfiles are now host+port so one host can carry two tunnels.
- requires-python, ruff and pyright were still on 3.13 while the venv and
Containerfile are 3.14, so the lock resolved for a stack nobody deploys.
LLM-Generated: true
The README referenced `deploy/install-agent.sh` and a `Containerfile` that
did not exist, so a fresh clone had no way to actually run this anywhere.
* agent: `--token-file` (and AGENT_TOKEN_FILE). The token guards hostname,
mounts, disk and GPU models and process names, and argv is world-readable
through `ps`, so a file has to win over the flag. Missing or empty token
files are fatal — falling back to "no token" would expose the host.
* deploy/install-agent.sh: creates a system user, generates the token once
(re-runs reuse it, --rotate-token for a new one), stores it 0600
root-owned, and runs the agent under ProtectSystem=strict with no
capabilities; --status and --uninstall included.
* Containerfile: two-stage, uv.lock resolved in the builder and slim Python
in the runtime. Verified with podman — the image serves /api/health as an
unprivileged user and writes its history under /var/lib/dashboards.
UV_PROJECT_ENVIRONMENT and --no-editable are what make the venv survive
being copied into the runtime stage.
* .containerignore keeps .git, .venv, data/ and .env out of the build
context.
258 tests, 94% coverage; ruff and pyright clean.
LLM-Generated: true
Three dashboards served by one FastAPI app with a SQLite history store
(raw / 1m / 1h) and hand-written SVG charts: no Prometheus, no Grafana,
no CDN, no build step. Nothing is fetched from the internet at runtime.
* system: a stdlib-only agent (agent/host_agent.py) reports /proc and
/sys counters; the dashboard turns them into rates over a labelled
window, so every number says how it was measured.
* llm: gufo/llama.cpp and vLLM /metrics are parsed by an in-house
exposition parser. Measured delta/elapsed throughput beats a native
per-request gauge, and each value carries its provenance.
* ups: a NUT wire-protocol client plus health checks that name what
they looked at. Commands are gated by NUT_ALLOW_ACTIONS,
DASH_READ_ONLY and a DANGER confirmation, re-checked server-side.
Missing values render as n/a rather than 0, and an unconfigured source
answers 404 {"configured": false} without dialling anything, so "not
deployed here" never looks like an outage.
253 tests at 94% coverage against captured real fixtures; ruff and
pyright clean.
LLM-Generated: true
ducoterra