phase: 16_agent_smart_pass_budget

**Report — phase 16 final verification pass** (`16_agent_smart_pass_budget`)

- Verified both tasks' work is present and non-tautological: `_smart_pass` scales `max(20 s, 5 s × disks)` with the override winning; `tests/test_agent.py` pins the 16-disk scale (no truncation), the 20 s floor, and the exact `pass budget of 80s exhausted` line, plus the untouched `0.0`-override test. No code change this pass (only the docstring reflow already in the tree).
- Completion criteria:
  - `uv run pytest tests/test_agent.py` → **105 passed** (4 pass-budget tests green, `-k pass_budget` → 4 passed) ✅
  - `grep -rn "20 s per pass" AGENTS.md README.md agent/ deploy/` → **no matches** (exit 1) ✅
  - `bash .agents/validate.sh` → **EXIT=0**: `uv run pytest --cov --cov-fail-under=94` **1427 passed, 15 skipped, coverage 98.94 %**; `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` + repo hygiene clean ✅
  - `uv run pytest -m live --live` (with `.env.dev`) → **11 passed**; `./scripts/dev-server.sh status` → nine sources `ok`; live `/api/disks/snapshot` → driveripper **16/16 drives with temps, no budget line** ✅
  - driveripper md5 byte-identity → **not met, deliberately no live change**: deployed copy is 1.0.0 and differs from the repo only by phase 15's TLS block (verified by diff); `ops` there *is* sudo-granted a staged `install-agent.sh` (new finding), but re-running it rewrites that host's hand-diverged `User=root`+`CapabilityBoundingSet` unit and re-breaks the 16-drive SMART read (gap 10) — needs root + phase 17's template fix. Recorded in AGENTS.md gap 11.
- Docs (full README + AGENTS review, incl. pre-existing drift): fixed 4 inaccuracies — AGENTS fleet record claimed *neither* host sudo-grants the installer (driveripper does, and that grant is the trap); `/health` publishes `version`, not `agent_version`; deskwork's "55 °C amber" marked as a dated capture (48 °C `ok` on re-read); gap 10's title still read as an open defect though resolved the same day. README verified accurate end-to-end (SMART budget wording, TLS rollout, config table, routes, test philosophy) — no further change.
- Notable: left `NOTES.md` (untracked operator scratch) alone; its "admin buttons disappear on `/certs`/`/settings`" item is deliberate, test-pinned behaviour (`test_every_page_links_to_both_admin_pages`) and belongs to a future phase, not this one.
- Next pending phase: `17_installer_device_filter`.
This commit is contained in:
2026-10-20 11:14:35 -04:00
parent 4b0f53c499
commit 7e367a9014
23 changed files with 337 additions and 17 deletions
@@ -0,0 +1 @@
(no final assistant message — see the .err log)
@@ -0,0 +1 @@
error: no final assistant text
@@ -0,0 +1,63 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
...........................s............................................ [ 29%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 64%]
........................................sssssssssss..................... [ 69%]
........................................................................ [ 74%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 89%]
........................................................................ [ 94%]
................sss..................................................... [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 436 0 100%
src/dashboards/app.py 185 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3490 37 99%
Required test coverage of 94% reached. Total coverage: 98.94%
1427 passed, 15 skipped, 1 warning in 128.28s (0:02:08)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,12 @@
**Report — phase 16 final verification pass** (`16_agent_smart_pass_budget`)
- Verified both tasks' work is present and non-tautological: `_smart_pass` scales `max(20 s, 5 s × disks)` with the override winning; `tests/test_agent.py` pins the 16-disk scale (no truncation), the 20 s floor, and the exact `pass budget of 80s exhausted` line, plus the untouched `0.0`-override test. No code change this pass (only the docstring reflow already in the tree).
- Completion criteria:
- `uv run pytest tests/test_agent.py` → **105 passed** (4 pass-budget tests green, `-k pass_budget` → 4 passed) ✅
- `grep -rn "20 s per pass" AGENTS.md README.md agent/ deploy/` → **no matches** (exit 1) ✅
- `bash .agents/validate.sh` → **EXIT=0**: `uv run pytest --cov --cov-fail-under=94` **1427 passed, 15 skipped, coverage 98.94 %**; `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` + repo hygiene clean ✅
- `uv run pytest -m live --live` (with `.env.dev`) → **11 passed**; `./scripts/dev-server.sh status` → nine sources `ok`; live `/api/disks/snapshot` → driveripper **16/16 drives with temps, no budget line** ✅
- driveripper md5 byte-identity → **not met, deliberately no live change**: deployed copy is 1.0.0 and differs from the repo only by phase 15's TLS block (verified by diff); `ops` there *is* sudo-granted a staged `install-agent.sh` (new finding), but re-running it rewrites that host's hand-diverged `User=root`+`CapabilityBoundingSet` unit and re-breaks the 16-drive SMART read (gap 10) — needs root + phase 17's template fix. Recorded in AGENTS.md gap 11.
- Docs (full README + AGENTS review, incl. pre-existing drift): fixed 4 inaccuracies — AGENTS fleet record claimed *neither* host sudo-grants the installer (driveripper does, and that grant is the trap); `/health` publishes `version`, not `agent_version`; deskwork's "55 °C amber" marked as a dated capture (48 °C `ok` on re-read); gap 10's title still read as an open defect though resolved the same day. README verified accurate end-to-end (SMART budget wording, TLS rollout, config table, routes, test philosophy) — no further change.
- Notable: left `NOTES.md` (untracked operator scratch) alone; its "admin buttons disappear on `/certs`/`/settings`" item is deliberate, test-pinned behaviour (`test_every_page_links_to_both_admin_pages`) and belongs to a future phase, not this one.
- Next pending phase: `17_installer_device_filter`.
@@ -0,0 +1,63 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
...........................s............................................ [ 29%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 64%]
........................................sssssssssss..................... [ 69%]
........................................................................ [ 74%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 89%]
........................................................................ [ 94%]
................sss..................................................... [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 436 0 100%
src/dashboards/app.py 185 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3490 37 99%
Required test coverage of 94% reached. Total coverage: 98.94%
1427 passed, 15 skipped, 1 warning in 128.57s (0:02:08)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,14 @@
**Report — task 01_pin_the_scaled_budget**
- **Finding:** the task's deliverables were already present and committed at HEAD (code change + all three pinning tests landed alongside phase 15). Verified them rather than duplicating:
- `tests/test_agent.py::test_the_pass_budget_scales_with_the_disk_count` (16 disks, 24 s pass > 20 s floor, no truncation, `timeout==5.0` for late devices)
- `test_the_pass_budget_floor_is_twenty_seconds` (2 disks → floor wins, `min(20−18, 5)` timeout)
- `test_the_pass_budget_error_line_names_the_scaled_figure` (exact line `smart: pass budget of 80s exhausted, 2 disk(s) skipped`, asserts `20s` absent)
- `test_the_pass_budget_skips_the_rest_with_one_error` (override `0.0`) untouched
- **Non-tautology proven:** mutation-checked `_smart_pass` back to a flat 20 s budget — the scales and scaled-figure tests fail, then restored; `agent/host_agent.py` byte-identical to HEAD (`git diff --quiet` yes).
- **Tests:** `uv run pytest` → **1427 passed, 15 skipped**; `uv run pytest tests/test_agent.py` → 105 passed; live suite `uv run pytest -m live --live` → **11 passed** (read-only).
- **Lint/gate:** `uv run ruff check src tests agent` + `uv run pyright` clean; `bash .agents/validate.sh` **EXIT=0**, coverage **98.94 %** (floor 94).
- **Coverage of modified code:** `agent/host_agent.py` 94 % overall under `tests/test_agent.py`; every line of `_smart_pass` incl. the scaled-budget and exhaustion branches covered (>90 % criterion met; the only smart-adjacent misses are `_refresh_smart`'s exception path).
- **Docs:** no doc impact from this task — README:98 and AGENTS.md already say "20 s floor scaled to 5 s × disk count"; `grep -rn "20 s per pass" AGENTS.md README.md agent/ deploy/` returns nothing.
- **No live changes made** (repo-side pinning task; driveripper md5 check and docs final pass belong to task 02).
- **Next pending task:** `16_agent_smart_pass_budget/02_docs_and_gate.md`.
@@ -0,0 +1,63 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
...........................s............................................ [ 29%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 64%]
........................................sssssssssss..................... [ 69%]
........................................................................ [ 74%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 89%]
........................................................................ [ 94%]
................sss..................................................... [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 436 0 100%
src/dashboards/app.py 185 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3490 37 99%
Required test coverage of 94% reached. Total coverage: 98.94%
1427 passed, 15 skipped, 1 warning in 133.78s (0:02:13)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
@@ -0,0 +1,10 @@
**Report — task 16/02_docs_and_gate**
- Docs verified + fixed: `AGENTS.md` (SMART bullet) and `README.md` (agent SMART paragraph) already read the floor-scaled truth, so `grep -rn "20 s per pass" AGENTS.md README.md agent/ deploy/` returns nothing; reflowed the `agent/host_agent.py` module docstring whose 2026-10-19 edit had broken "…cached 240 s so a fast | dashboard poll…" mid-clause (docstring only, no code change).
- Accuracy pass (measured, not assumed): `AGENTS.md` gap 10 now names phase 16's three pinning tests plus the live 16/16 re-read; gap 11 records that **all three fleet agents answer `/health` with `agent_version` 1.0.0** (phase 15's TLS code is installed nowhere) and that `driveripper`'s deployed copy differs from the repo by exactly that TLS block; `README.md` rollout step 2 now states the installer ships the agent file, so a pre-1.1.0 agent must be upgraded to gain `--cert-bundle`.
- **Live criterion not met, no live change made:** driveripper's deployed `host_agent.py` is **not** md5-identical to the repo (1.0.0 vs 1.1.0). The only `ops`-granted route to fix it is re-running `deploy/install-agent.sh`, which rewrites that host's hand-diverged unit (`User=root` + `CapabilityBoundingSet`) and would re-break the 16-drive SMART reads this phase pins (gap 10). Needs root and the phase-17 installer fix; recorded in gap 11 instead.
- Read-only live evidence gathered instead: installed agent answers **16/16 drives with temperatures and no `pass budget … exhausted` line**; `./scripts/dev-server.sh status` = nine sources `ok`.
- Tests/lint (exact): `bash .agents/validate.sh` → **EXIT=0** — `uv run pytest --cov --cov-fail-under=94`: **1427 passed, 15 skipped, coverage 98.94 %**; `uv run ruff check src tests agent` clean; `uv run pyright` **0 errors**; `node --check` + repo hygiene clean. `uv run pytest tests/test_agent.py` **105 passed**; `uv run pytest -m live --live` **11 passed**; `uv run uv pip list --outdated` empty.
- Doc impact: `AGENTS.md`, `README.md`, `agent/host_agent.py` docstring only; `tests/test_auth_docs.py` (19) and the page/doc pins still green.
- Left untouched deliberately: untracked `NOTES.md` (operator scratch — mTLS-only rollout, and the by-design single admin-masthead link pinned by `test_every_page_links_to_both_admin_pages`) and the modified `TODO.md`.
- Next pending task: `17_installer_device_filter/01_drop_the_broken_device_filter.md`.
@@ -0,0 +1,63 @@
== uv run pytest --cov (hermetic: no network, live tests skip) ==
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 24%]
...........................s............................................ [ 29%]
........................................................................ [ 34%]
........................................................................ [ 39%]
........................................................................ [ 44%]
........................................................................ [ 49%]
........................................................................ [ 54%]
........................................................................ [ 59%]
........................................................................ [ 64%]
........................................sssssssssss..................... [ 69%]
........................................................................ [ 74%]
........................................................................ [ 79%]
........................................................................ [ 84%]
........................................................................ [ 89%]
........................................................................ [ 94%]
................sss..................................................... [ 99%]
.. [100%]
=============================== warnings summary ===============================
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
Name Stmts Miss Cover Missing
------------------------------------------------------------------
src/dashboards/__init__.py 3 0 100%
src/dashboards/__main__.py 14 1 93% 32
src/dashboards/alerts.py 176 0 100%
src/dashboards/api.py 436 0 100%
src/dashboards/app.py 185 0 100%
src/dashboards/auth.py 265 0 100%
src/dashboards/certs.py 292 0 100%
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
src/dashboards/config.py 120 0 100%
src/dashboards/debug.py 14 0 100%
src/dashboards/poller.py 165 3 98% 192-194, 279
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
src/dashboards/rates.py 73 0 100%
src/dashboards/severity.py 49 0 100%
src/dashboards/sources/__init__.py 0 0 100%
src/dashboards/sources/disks.py 134 0 100%
src/dashboards/sources/host.py 209 0 100%
src/dashboards/sources/llm.py 178 1 99% 285
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
src/dashboards/sources/nut_sim.py 134 0 100%
src/dashboards/store.py 313 2 99% 553-554
------------------------------------------------------------------
TOTAL 3490 37 99%
Required test coverage of 94% reached. Total coverage: 98.94%
1427 passed, 15 skipped, 1 warning in 128.26s (0:02:08)
== uv run ruff check src tests agent ==
All checks passed!
== uv run pyright ==
0 errors, 0 warnings, 0 informations
== node --check src/dashboards/web/static/*.js ==
+36 -13
View File
@@ -854,10 +854,16 @@ That scheme is the history API's key — treat it as a public interface.
`upsc`, `udevadm`, the `/etc/ups` reads, and the `tee /etc/ups/upsd.users|upsd.conf` + `upsc`, `udevadm`, the `/etc/ups` reads, and the `tee /etc/ups/upsd.users|upsd.conf` +
`systemctl restart|reload nut-server` pair documented in `deploy/nut-sudoers`); `deskwork` `systemctl restart|reload nut-server` pair documented in `deploy/nut-sudoers`); `deskwork`
grants an unrelated set (`btrfs scrub`, `snapper`, `setfacl /srv/*`, `osbuild-auto.sh`). grants an unrelated set (`btrfs scrub`, `snapper`, `setfacl /srv/*`, `osbuild-auto.sh`).
Neither host grants `ops` the installer or `visudo` (re-checked 2026-10-19), so every install `deskwork` grants `ops` neither the installer nor `visudo`, so its install went through root by
that has happened on those two went through root by the operator — `driveripper`'s included, the operator. **`driveripper` does grant the installer** — `(root) NOPASSWD:
which is why its unit is not the one this repo writes (see the SMART block: `DevicePolicy=auto`, /home/ops/agent-install/deploy/install-agent.sh`, an Oct-18 snapshot of the script staged under
no `DeviceAllow`). Worth knowing before the next install: the allow-list `install-agent.sh` the operator's home (`sudo -n -l`, re-checked 2026-10-20) — and that grant is the trap, not the
shortcut: the installer rewrites the unit unconditionally, so running it there today replaces the
`User=root` + `CapabilityBoundingSet` unit that gap 10 resolved with this repo's
`User=dashboards-agent` + `DevicePolicy=strict` template and re-breaks all 16 SMART reads. The
live unit is not the one this repo writes (no `DevicePolicy`/`DeviceAllow` lines at all —
systemd's default `auto` — `PrivateDevices=false`, and the two named capabilities).
Worth knowing before the next install: the allow-list `install-agent.sh`
does write (`/dev/nvme*`, `/dev/sd*`, `/dev/vd*`, `/dev/sg*`) has **no `/dev/ng*n*` and no does write (`/dev/nvme*`, `/dev/sd*`, `/dev/vd*`, `/dev/sg*`) has **no `/dev/ng*n*` and no
`/dev/bsg/*`**, the nodes smartctl 7.5 reaches for NVMe admin and SATA pass-through, so a `/dev/bsg/*`**, the nodes smartctl 7.5 reaches for NVMe admin and SATA pass-through, so a
`strict` unit there would deny exactly what a SMART read needs — whatever phase takes `strict` unit there would deny exactly what a SMART read needs — whatever phase takes
@@ -870,9 +876,9 @@ That scheme is the history API's key — treat it as a public interface.
What changed by 2026-10-19 is that `deskwork`'s **installed** service runs as What changed by 2026-10-19 is that `deskwork`'s **installed** service runs as
`dashboards-agent`, which is in the `disk` group (`getent group disk` → `disk:dashboards-agent`) `dashboards-agent`, which is in the `disk` group (`getent group disk` → `disk:dashboards-agent`)
on an agent installed after the SMART pass: `/disks` shows real numbers there — on an agent installed after the SMART pass: `/disks` shows real numbers there —
`smartctl_available: true`, `nvme0n1` healthy at 55 °C (amber, past `disk.temp_c`'s 50 °C `smartctl_available: true`, `nvme0n1` healthy at 55 °C that day — amber, past `disk.temp_c`'s
threshold), 19 166 power-on hours (an odometer, 50 °C threshold, and a plain 48 °C `ok` on the 2026-10-20 re-read — 19 166 power-on hours (an
so that figure is a capture date, not a fact about the drive), 11 % of rated odometer, so that figure is a capture date, not a fact about the drive), 11 % of rated
wear — so the wear bar and the alarm chips are drawn from live hardware, not only fixtures. wear — so the wear bar and the alarm chips are drawn from live hardware, not only fixtures.
`gateway` answers `smartctl_available: false` (no smartmontools in its VM: the *install it* `gateway` answers `smartctl_available: false` (no smartmontools in its VM: the *install it*
half of the three-way flag), and **`driveripper` is the fourth case the flag was built for: half of the three-way flag), and **`driveripper` is the fourth case the flag was built for:
@@ -1705,7 +1711,8 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
(`test_a_fallback_inside_the_documented_ceiling_still_publishes`), so a span-relative floor (`test_a_fallback_inside_the_documented_ceiling_still_publishes`), so a span-relative floor
would be a re-decision of that phase, not a bug fix. Do not "fix" the residual by hiding the would be a re-decision of that phase, not a bug fix. Do not "fix" the residual by hiding the
short window or its label. short window or its label.
10. **`driveripper`'s 16 drives read `n/a`: kernel 7.2's unprivileged-passthrough gate (2026-10-19).** 10. **`driveripper`'s 16 drives read `n/a`: kernel 7.2's unprivileged-passthrough gate
(2026-10-19, resolved the same day — the installer template is what is left).**
The agent is installed, the `disk` group grant is in place, the block nodes open, and the The agent is installed, the `disk` group grant is in place, the block nodes open, and the
page renders all 16 real drive rows — but kernel 7.2.11 blocks the *agent user's* `smartctl` page renders all 16 real drive rows — but kernel 7.2.11 blocks the *agent user's* `smartctl`
at the command level (not the open; the probe matrix is in at the command level (not the open; the probe matrix is in
@@ -1716,7 +1723,7 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
read needs the drive's ACEL to advertise the page (the 980 PRO's `5B2QGXA7`, 2020-era, read needs the drive's ACEL to advertise the page (the 980 PRO's `5B2QGXA7`, 2020-era,
does not: `NVME_IOCTL_ADMIN_CMD: Permission denied`) or `CAP_SYS_ADMIN`; `deskwork` reads does not: `NVME_IOCTL_ADMIN_CMD: Permission denied`) or `CAP_SYS_ADMIN`; `deskwork` reads
live because its firmware does. Options, in order of narrowness: (a) accept the designed live because its firmware does. Options, in order of narrowness: (a) accept the designed
degradation (what `/disks` renders now — the identity is real, the values are `n/a` with a degradation (what `/disks` rendered then — the identity is real, the values are `n/a` with a
why-line); (b) `AmbientCapabilities=CAP_SYS_RAWIO` (+ `CapabilityBoundingSet`) on the unit why-line); (b) `AmbientCapabilities=CAP_SYS_RAWIO` (+ `CapabilityBoundingSet`) on the unit
— unblocks exactly the 8 HDDs, is the capability the kernel names for the gate, but a — unblocks exactly the 8 HDDs, is the capability the kernel names for the gate, but a
compromised agent could then write raw block I/O — test it in a `systemd-run --scope` compromised agent could then write raw block I/O — test it in a `systemd-run --scope`
@@ -1731,10 +1738,16 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
`CapEff` came back `0000000000220000` (SYS_ADMIN + systemd's auto CHROOT; RAWIO in the `CapEff` came back `0000000000220000` (SYS_ADMIN + systemd's auto CHROOT; RAWIO in the
bounding set but not effective) — the 8 SATA drives' reads were unblocked by SYS_ADMIN bounding set but not effective) — the 8 SATA drives' reads were unblocked by SYS_ADMIN
alone, so the effective grant is {SYS_ADMIN}; the RAWIO in the line stays as the named alone, so the effective grant is {SYS_ADMIN}; the RAWIO in the line stays as the named
intent for the `SG_IO` gate. The agent also gained the pass-budget scale-up (20 s floor intent for the `SG_IO` gate. The agent also gained the pass-budget scale-up — 20 s floor, 5 s ×
→ 5 s × disk count; an explicit override still wins, the budget test is untouched), disk count, an explicit override still winning — because a 16-disk pass takes more than 20 s
because a 16-disk pass takes more than 20 s when the drives answer. The `deploy/install-agent.sh` when the drives answer, and truncating it would null the tail drives. Phase 16 pinned that
template still ships the `DevicePolicy`/`DeviceAllow` lines that misbehaved on this host default in `tests/test_agent.py` (`test_the_pass_budget_scales_with_the_disk_count`,
`test_the_pass_budget_floor_is_twenty_seconds`,
`test_the_pass_budget_error_line_names_the_scaled_figure` — which demands the line read `80s`
for 16 disks, not the floor; the override test stays). Re-read live on 2026-10-20: the installed
agent answers 16/16 drives with temperatures and no `pass budget … exhausted` line. The
`deploy/install-agent.sh` template still ships the `DevicePolicy`/`DeviceAllow` lines that
misbehaved on this host
(a deny-all filter; the `nvme*` glob matches nothing on this kernel because (a deny-all filter; the `nvme*` glob matches nothing on this kernel because
`/proc/devices` lists NVMe as `blkext`) — the template change is its own phase. `/proc/devices` lists NVMe as `blkext`) — the template change is its own phase.
`tests/fixtures/smartctl_driveripper_*.txt` (captured via `sudo`) remain the ground truth `tests/fixtures/smartctl_driveripper_*.txt` (captured via `sudo`) remain the ground truth
@@ -1803,6 +1816,16 @@ memory plot; the `cpu.usage_pct@coreN` series is still stored and still queryabl
stays `http://` here until somebody installs a bundle on a host — which needs root on that host stays `http://` here until somebody installs a bundle on a host — which needs root on that host
(`ops` has no installer grant on `deskwork`, and `driveripper`'s unit must not be rewritten, gap (`ops` has no installer grant on `deskwork`, and `driveripper`'s unit must not be rewritten, gap
10), and flipping the URL before the bundle is installed turns that host red by construction. 10), and flipping the URL before the bundle is installed turns that host red by construction.
Re-checked from the dev box the same day, and worth knowing before that rollout: all three
agents answer `/health` with `version` **1.0.0** (`agent_version` is the `/snapshot` spelling),
so none of them carries task 02's TLS code either — the rollout ships the repo's
`agent/host_agent.py` (1.1.0) *together with* the
bundle, never the bundle alone. `driveripper`'s installed copy differs from the repo's by
exactly that TLS block; its `_smart_pass` and the phase-16 budget scale-up are already there.
Copying 1.1.0 into place is not an `install-agent.sh` re-run on that host (gap 10) — even
though `ops` there is sudo-granted a staged copy of the installer (see the fleet record above),
which is the one route that must not be taken until phase 17 fixes the unit template — and no
live change was made while pinning phase 16.
## Locked decisions (and why) ## Locked decisions (and why)
+2
View File
@@ -0,0 +1,2 @@
- [ ] The client should never allow non-mtls communication. deskwork and gateway are still non-mtls according to the dashboard. During dev, the dev server should generate certs in a non-committed directory and then distribute them to clients during registration so we never need non-mtls servers.
- [ ] Clicking the certs tab causes the certs button to disappear, then clicking the account tab causes the accounts button to disappear. These should remain visible.
+4 -2
View File
@@ -194,8 +194,10 @@ Three steps, and they are order-dependent because the dashboard picks its dial f
curl -s -H "Authorization: Bearer $SESSION" -o agent-deskwork.bundle \ curl -s -H "Authorization: Bearer $SESSION" -o agent-deskwork.bundle \
http://127.0.0.1:8090/api/certs/deskwork/bundle http://127.0.0.1:8090/api/certs/deskwork/bundle
# 2. install it beside the token — as root on the host, 0640 root:dashboards-agent. The unit # 2. install it beside the token — as root on the host, 0640 root:dashboards-agent. The same run
# gains `--cert-bundle` and its RuntimeDirectory, and restarts into TLS mode. # copies the repo's `agent/host_agent.py`, which is how a pre-1.1.0 agent gains TLS mode at all
# (an older file does not know the `--cert-bundle` flag). The unit gains `--cert-bundle` and its
# RuntimeDirectory, and restarts into TLS mode.
scp agent-deskwork.bundle <host>-ops:/tmp/ scp agent-deskwork.bundle <host>-ops:/tmp/
ssh <host>-ops sudo /tmp/install-agent.sh --cert-bundle /tmp/agent-deskwork.bundle ssh <host>-ops sudo /tmp/install-agent.sh --cert-bundle /tmp/agent-deskwork.bundle
ssh <host>-ops sudo /tmp/install-agent.sh --status # https, with the CA and no client cert ssh <host>-ops sudo /tmp/install-agent.sh --status # https, with the CA and no client cert
+2 -2
View File
@@ -23,8 +23,8 @@ Design rules
item; the snapshot still succeeds. The poll is degraded, never failed. item; the snapshot still succeeds. The poll is degraded, never failed.
* **SMART is a best-effort add-on.** The per-disk `smart` block runs `smartctl -i -A -H` * **SMART is a best-effort add-on.** The per-disk `smart` block runs `smartctl -i -A -H`
in a subprocess: bounded 5 s per device and a pass budget of 20 s scaled to the disk in a subprocess: bounded 5 s per device and a pass budget of 20 s scaled to the disk
count, cached 240 s so a fast count, cached 240 s so a fast dashboard poll cannot re-issue ATA commands every few
dashboard poll cannot re-issue ATA commands every few seconds. Refreshing an expired seconds. Refreshing an expired
cache happens on a background thread while the request is served the previous result — cache happens on a background thread while the request is served the previous result —
20 s of wedged disk must never cost a poll more than its own timeout. No binary, a denied 20 s of wedged disk must never cost a poll more than its own timeout. No binary, a denied
device or a wedged disk degrades *that disk's* `smart` to `null` plus one `errors` device or a wedged disk degrades *that disk's* `smart` to `null` plus one `errors`