phase: 09_disks_smartctl
**Phase 09 — final verification pass (all 4 task files already in `complete/`; nothing re-implemented)**
- Verified `GET /api/disks/snapshot` live: 3 hosts / 18 whole-`/sys/block` rows, `smartctl_available` three-way, per-row `severity`, API-summed `fleet`; 404 `{"configured": false}` proven *without dialling* (spies on `httpx.AsyncClient.get` + `collectors.fetch_snapshot`).
- Verified `/api/disks/history` live (`disk.temp_c@dev`, unknown names → empty series) + `test_store.py`'s 300 s rollup/prune buckets; `/api/metrics?source=disks` answers the `disks` family.
- Browser against live data: 18 drive cards (never a partition/mount), collapsed per-host `Charts` with `aria-label`, 3 `.stale` grids + one `.banner.info` why-line and **0 `.fail` classes / green `disks` dot** → criteria 3 and 4 hold; new shots `data/logs/09-verify-disks-live-dimmed.png`, `09-verify-overview-live.png` (13 total for the phase).
- Real-hardware criterion: read 4 driveripper drives through `ops`' `sudo -n smartctl -i -A -H` and ran the live output through `host_agent.parse_smart_output` → `sources/disks.parse_disks_snapshot`: `sda` 35 °C / 30 423 h, `nvme1n1` 58 % wear, `nvme5n1` 13 uncorrectable → `alarm_disks: 1`, temps graded `warn`; consistent with the 16 committed fixtures.
- `bash .agents/validate.sh` → **EXIT=0**; `uv run pytest --cov` → **769 passed, 11 skipped, coverage 98.33%** (floor 94); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean; hygiene + dependency advisories clean. `uv run pytest -m live --live` → 10 passed, 1 skipped (agents still predate the SMART pass). Dev server healthy: `/api/health` `ok`, 9/9 sources.
- Docs fixed: AGENTS.md test count 698→769 and coverage 98.2→98.33 %; `disks` added to the event dashboard-name list and to the "every feed renders `Dash.eventText`" sentence; README `POLL_MS_*` row now states `/disks`' deliberate fixed 30 s; new Known gap 4 records the dev-DB `database is locked` writes (reported/recovered per invariant 4, ~590 MB DB / ~70 MB WAL, not reproduced by a controlled 25 s external read txn).
- Deviations: **no live-infra change** — granting the agent user the `disk` group and redeploying `host_agent.py` on driveripper/deskwork stay the documented paired operator action (redeploying without the grant yields 16 permission-denied lines; a sudo-spawning agent would break the confirmed "direct read, no sudo" assumption). Store-lock issue left to its own phase rather than patched speculatively.
- Next pending phase: `10_dev_direct_dial`.
This commit is contained in:
+1
@@ -50,3 +50,4 @@ reads `n/a` for what `smartctl` cannot see — never red for a missing binary.
|
||||
- [ ] the `/disks` page renders one card per physical disk (never per-partition) with health, temp, power-on hours and non-zero alarm chips
|
||||
- [ ] a host without `smartctl` (or a denied read) shows dimmed `n/a` cards, never a red page
|
||||
- [ ] `bash .agents/validate.sh` green; screenshots in `data/logs/`
|
||||
- [ ] you have verified with real smartctl data from driveripper. The `ops` user has `sudo smartctl*` access via sudoers.
|
||||
@@ -0,0 +1,10 @@
|
||||
**Phase 09 — final verification pass (all 4 task files already in `complete/`; nothing re-implemented)**
|
||||
|
||||
- Verified `GET /api/disks/snapshot` live: 3 hosts / 18 whole-`/sys/block` rows, `smartctl_available` three-way, per-row `severity`, API-summed `fleet`; 404 `{"configured": false}` proven *without dialling* (spies on `httpx.AsyncClient.get` + `collectors.fetch_snapshot`).
|
||||
- Verified `/api/disks/history` live (`disk.temp_c@dev`, unknown names → empty series) + `test_store.py`'s 300 s rollup/prune buckets; `/api/metrics?source=disks` answers the `disks` family.
|
||||
- Browser against live data: 18 drive cards (never a partition/mount), collapsed per-host `Charts` with `aria-label`, 3 `.stale` grids + one `.banner.info` why-line and **0 `.fail` classes / green `disks` dot** → criteria 3 and 4 hold; new shots `data/logs/09-verify-disks-live-dimmed.png`, `09-verify-overview-live.png` (13 total for the phase).
|
||||
- Real-hardware criterion: read 4 driveripper drives through `ops`' `sudo -n smartctl -i -A -H` and ran the live output through `host_agent.parse_smart_output` → `sources/disks.parse_disks_snapshot`: `sda` 35 °C / 30 423 h, `nvme1n1` 58 % wear, `nvme5n1` 13 uncorrectable → `alarm_disks: 1`, temps graded `warn`; consistent with the 16 committed fixtures.
|
||||
- `bash .agents/validate.sh` → **EXIT=0**; `uv run pytest --cov` → **769 passed, 11 skipped, coverage 98.33%** (floor 94); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean; hygiene + dependency advisories clean. `uv run pytest -m live --live` → 10 passed, 1 skipped (agents still predate the SMART pass). Dev server healthy: `/api/health` `ok`, 9/9 sources.
|
||||
- Docs fixed: AGENTS.md test count 698→769 and coverage 98.2→98.33 %; `disks` added to the event dashboard-name list and to the "every feed renders `Dash.eventText`" sentence; README `POLL_MS_*` row now states `/disks`' deliberate fixed 30 s; new Known gap 4 records the dev-DB `database is locked` writes (reported/recovered per invariant 4, ~590 MB DB / ~70 MB WAL, not reproduced by a controlled 25 s external read txn).
|
||||
- Deviations: **no live-infra change** — granting the agent user the `disk` group and redeploying `host_agent.py` on driveripper/deskwork stay the documented paired operator action (redeploying without the grant yields 16 permission-denied lines; a sudo-spawning agent would break the confirmed "direct read, no sudo" assumption). Store-lock issue left to its own phase rather than patched speculatively.
|
||||
- Next pending phase: `10_dev_direct_dial`.
|
||||
@@ -0,0 +1 @@
|
||||
ok
|
||||
@@ -0,0 +1,51 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 55%]
|
||||
...........sssssssssss.................................................. [ 64%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 92%]
|
||||
............................................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 28
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 221 0 100%
|
||||
src/dashboards/app.py 101 0 100%
|
||||
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
|
||||
src/dashboards/config.py 115 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
|
||||
src/dashboards/rates.py 37 1 97% 51
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 130 0 100%
|
||||
src/dashboards/sources/host.py 199 1 99% 147
|
||||
src/dashboards/sources/llm.py 150 1 99% 228
|
||||
src/dashboards/sources/nut.py 389 20 95% 128, 142, 148, 160, 166, 178, 186, 199, 270-271, 273, 489, 501, 536, 569, 573, 630-631, 636, 648
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 178 2 99% 250-251
|
||||
------------------------------------------------------------------
|
||||
TOTAL 2335 39 98%
|
||||
Required test coverage of 94% reached. Total coverage: 98.33%
|
||||
769 passed, 11 skipped, 1 warning in 58.25s
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
Everything is green. Final state:
|
||||
|
||||
**Implemented (task 09/01 — agent smartctl collection)**
|
||||
- `agent/host_agent.py`: new `smart()` pass — `smartctl -i -A -H` per whole `/sys/block` disk (virtual/`dm-*` skipped), stdlib `subprocess`, 5 s/device, 20 s/pass, 240 s cache; `parse_smart_output()` covers ATA+NVMe (health verdict word, temp, power-on hours, reallocated/pending/offline-uncorrectable sectors, wear %; NVMe media-errors mapped; hex-packed raws refused); `smart: {…}` on each `disk_io.devices[]` row + SCHEMA/docstring; `--smartctl`/`--no-smart` flags; failure semantics: missing binary → **one** `errors` line, denied/wedged device → only that disk `null` + its own line, `/health` untouched; fixed a real-box-caused defect (error note quoted the version banner instead of the failure line).
|
||||
- Fixtures `tests/fixtures/smartctl_reference_{nvme,ata_hdd}.txt` — labels verbatim from the deployed smartmontools 7.5 binary (fleet runs the same build), values synthetic, honestly labelled **reference shapes** (see deviation below).
|
||||
- `scripts/capture-fixtures.sh`: `SMART_HOSTS=` pass records `smartctl_<host>_<dev>.txt` per disk (never overwrites good fixtures with permission-denied output); scrubber now also redacts smartctl serials/WWNs and normalizes capture timestamps. `.env.dev.example` documents it.
|
||||
- `collectors.py`: agent `errors` → payload `degraded` **excludes** `smart:` add-on lines (invariant 3: no smartmontools must not hold `/system` amber); they stay in `agent.errors` for the `/disks` page. 2 new collector tests pin both directions.
|
||||
- `deploy/install-agent.sh`: grants `disk` group to the agent user (idempotent), notes missing smartmontools, unit replaces `PrivateDevices` with `DevicePolicy=strict` + device-allowed disk nodes.
|
||||
- Docs: README (agent section + capture commands), AGENTS.md (agent/collectors bullets, fleet facts, capture command).
|
||||
|
||||
**Tests/lint**: `bash .agents/validate.sh` → **EXIT=0** — `pytest --cov --cov-fail-under=94`: **646 passed, 10 skipped, coverage 97.92%**; `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean. Agent file coverage 92% (new SMART code paths fully covered). `uv run pytest -m live --live`: 9 passed, my new smart live test skips with reason “no agent redeployed with the smart pass yet”. `uv pip list --outdated` empty. Dev server restarted; `/api/health` all sources ok.
|
||||
|
||||
**Deviations / live-infra note**
|
||||
- Fixtures are labelled reference shapes, not field captures: **no reachable environment can open a block device** — fleet `ops` gets `Permission denied` (no smartctl sudo rule, not in `disk`), and the dev box is an unprivileged toolbox container. I made **no live changes** (no group grants, no agent redeploy — a redeploy without the grant would emit 12 permission-denied lines on driveripper). Operator follow-up (documented in AGENTS.md): `sudo usermod -aG disk <agent-user>` on deskwork+driveripper **together with** an agent redeploy, then `SMART_HOSTS="deskwork-ops driveripper-ops" ./scripts/capture-fixtures.sh` to replace the reference fixtures with hardware truth (the live shape test then activates automatically). Installed `smartmontools` in the local dev toolbox to verify format strings; harmless, no service runs there.
|
||||
|
||||
**Next pending task**: `09_disks_smartctl/02_disks_source_poller_store.md`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 32%]
|
||||
........................................................................ [ 43%]
|
||||
.................................................ssssssssss............. [ 54%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 98%]
|
||||
........ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 28
|
||||
src/dashboards/alerts.py 166 0 100%
|
||||
src/dashboards/api.py 193 0 100%
|
||||
src/dashboards/app.py 107 6 94% 50-51, 56, 58, 60, 127
|
||||
src/dashboards/collectors.py 145 5 97% 105-106, 110-111, 128
|
||||
src/dashboards/config.py 113 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
|
||||
src/dashboards/rates.py 37 1 97% 51
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/host.py 199 1 99% 147
|
||||
src/dashboards/sources/llm.py 150 1 99% 228
|
||||
src/dashboards/sources/nut.py 389 20 95% 128, 142, 148, 160, 166, 178, 186, 199, 270-271, 273, 489, 501, 536, 569, 573, 630-631, 636, 648
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 178 2 99% 243-244
|
||||
------------------------------------------------------------------
|
||||
TOTAL 2161 45 98%
|
||||
Required test coverage of 94% reached. Total coverage: 97.92%
|
||||
646 passed, 10 skipped, 1 warning in 59.09s
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
**Implemented (task 09/02 — `disks` source, poller job, store)**
|
||||
- `src/dashboards/sources/disks.py` (new): one row per whole-`/sys/block` disk (never partition/mount), agent `smart` verbatim, `health` = drive's word normalised `ok/failed/None`, three-way `smartctl_available` (yes / "install smartmontools" / unknown) + `smart_errors`; samples `disk.temp_c|power_on_hours|reallocated|pending|uncorrectable|percentage_used@<dev>` in a new `disks` sample family; counters stored as reported (no `RateEstimator`, no window); health-transition events under dashboard name `disks`.
|
||||
- `config.py`: `DISKS_INTERVAL_SECONDS` (300) + `DISKS_TIMEOUT_SECONDS` (30).
|
||||
- `collectors.py`: shared `fetch_snapshot()` (system+disks, redacted messages, per-request timeout → httpx sentinel, since literal `None` disables timeouts), `fetch_disk_snapshot()`, `disks_jobs()` (`disks:<host>`; no targets ⇒ no jobs ⇒ never dials).
|
||||
- `app.py`: one `build_hub()` for both wiring paths (production + injected-store tests) now registers the disks jobs.
|
||||
- `alerts.py` + `store.py`: interval row for `disks`, disks-echo suppression, dashboard-filtered `_state_for`; rollup docstring for a 300 s source (no code change needed).
|
||||
- **Defect fixed in task 01's work**: an expired SMART cache now refreshes on a background thread (single-flight, keeps the old cache) — an inline 20 s wedged-disk pass blew `HOST_TIMEOUT_SECONDS` and turned `/system` red every 4 minutes.
|
||||
- Docs: README (env rows, diagram, poller bullet, agent SMART), AGENTS.md (source/poller/alerts/metric family/agent/gaps/counts), `.env.example`, `.env.dev.example`.
|
||||
|
||||
**Tests/lint** — `bash .agents/validate.sh` → **EXIT=0**: `pytest --cov --cov-fail-under=94` **698 passed, 11 skipped, coverage 98.25%** (floor 94; `sources/disks.py` 100%); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean; `uv pip list --outdated` empty. New `tests/test_disks_source.py` (33) + additions in test_store (300 s rollup buckets/pruning), test_poller (failed disks poll ages, not zeroes), test_collectors (fetch + timeout), test_alerts (echo + interval), test_config, test_api (health record per host), test_agent (background refresh, `not found` wording pin). `pytest -m live --live` (with `.env.dev`): 10 passed, 1 skip (agent SMART — fleet agents not redeployed). Dev server restarted: 9 sources ok incl. `disks:driveripper/deskwork/gateway`.
|
||||
|
||||
**Decisions / deviations**
|
||||
- Task text assumed `alerts.py` needed *no* change; it did — without a `disks` interval the job ambered `/` every 5 min, and one dead agent printed two identical red rows. Both fixed with tests.
|
||||
- One job **per host** (`disks:<host>`), not one fleet job: per-host staleness and the stub-per-job shape `/api/system/snapshot` already uses.
|
||||
- The `404 {"configured": false}` route is task 03; the job half (no targets ⇒ no task, no dial) is pinned here.
|
||||
- **No live changes** (no agent redeploy, no `disk`-group grant) — that stays the operator action documented in AGENTS.md; live work was read-only.
|
||||
|
||||
**Next pending task**: `09_disks_smartctl/03_disks_api_and_config.md` — and it should add `alerts.HREF["disks"] = "/disks"` together with the `/disks` page.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 10%]
|
||||
........................................................................ [ 20%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 50%]
|
||||
........................sssssssssss..................................... [ 60%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 91%]
|
||||
............................................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 28
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 193 0 100%
|
||||
src/dashboards/app.py 100 1 99% 139
|
||||
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
|
||||
src/dashboards/config.py 115 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
|
||||
src/dashboards/rates.py 37 1 97% 51
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 104 0 100%
|
||||
src/dashboards/sources/host.py 199 1 99% 147
|
||||
src/dashboards/sources/llm.py 150 1 99% 228
|
||||
src/dashboards/sources/nut.py 389 20 95% 128, 142, 148, 160, 166, 178, 186, 199, 270-271, 273, 489, 501, 536, 569, 573, 630-631, 636, 648
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 178 2 99% 250-251
|
||||
------------------------------------------------------------------
|
||||
TOTAL 2280 40 98%
|
||||
Required test coverage of 94% reached. Total coverage: 98.25%
|
||||
698 passed, 11 skipped, 1 warning in 55.89s
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
@@ -0,0 +1,12 @@
|
||||
**Implemented (task 09/03 — disks API, config wiring, severity rows)**
|
||||
- `api.py`: new `disks_router()` — `GET /api/disks/snapshot` (per host: disk rows + verbatim `smart` + `health` + per-row `severity` + three-way `smartctl_available` + `smart_errors` + `summary`, plus a `fleet` block summed by the API, and a configured-but-never-answered host as an `unreachable` stub whose counts are `None`, not `0`) and `GET /api/disks/history` (`disk.*@dev` through the existing history mechanism; unknown names → empty series; `hosts` defaults to every configured target). `require_hosts()` gate runs first: `404 {"configured": false}` with no dial. `/api/config` gained `disks {enabled, interval_seconds, timeout_seconds}`. No POST anywhere on the surface.
|
||||
- `severity.py`: `disk.temp_c` (50/70) and `disk.percentage_used` (80/100); the sector counters and `disk.power_on_hours` deliberately have **no row** (presence chips, never green).
|
||||
- `sources/disks.py`: `row_severity()` + `SEVERITY_FIELDS` / `UNGRADED_FIELDS` / `HEALTH_SEVERITY` (drive's own word → `ok|fail`, not a threshold) and `fleet_disk_summary()` (dead host counted, never summed).
|
||||
- `app.py`: `disks_router()` included; `alerts.py` **needed no change** — its existing `disks` interval + echo-suppression + unconfigured rules already cover stale/dead/unconfigured (added a test for the unconfigured case to prove it).
|
||||
- Docs: README (new `api.py` architecture bullet with the two 404/empty-series rules; severity bullet with the disk rows and the ungraded counters), AGENTS.md (`api.py`, `sources/disks.py`, `severity.py` bullets).
|
||||
|
||||
**Tests/lint** — `bash .agents/validate.sh` → **EXIT=0**: `pytest --cov --cov-fail-under=94` **734 passed, 11 skipped, coverage 98.33%** (`api.py` 100%, `sources/disks.py` 100%, `severity.py` 100%, `app.py` 100% after covering the `UpstreamError` route handler); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean; `uv pip list --outdated` empty. New pins: 10 disks API tests (payload paths, stub-of-`n/a`s, verdict-equals-`classify` per row, fleet sum, GET-only + route-verb enumeration, 404-without-dialling with spies on `httpx.AsyncClient.get` and `collectors.fetch_snapshot` plus `hub.jobs == {}`, history known/unknown/default-hosts, `/api/metrics?source=disks`, `/api/config` disks section), 9 disks-source tests (severity + fleet summary), 3 severity tests, 1 alerts test; the token-leak test now also asserts the `disks:driveripper` error string is redacted. `pytest -m live --live` (`.env.dev` sourced): 10 passed, 1 skipped, no writes.
|
||||
|
||||
**Notable decisions** — `/api/disks/history` follows its siblings (empty answers, not 404, since it reads only SQLite); `HREF["disks"] = "/disks"` left for task 04 so an alert never links to a route that does not exist. Live-verified against the real fleet (read-only): `/api/disks/snapshot` → 3 hosts / 18 physical disks, `smart: null` + `smartctl_available: null` (deployed agents predate the SMART pass — the designed `n/a`, no red), `/api/health` all 9 sources ok. No live infrastructure change made; no new env vars.
|
||||
|
||||
**Next pending task**: `09_disks_smartctl/04_disks_page_and_overview.md` (the `/disks` page, nav tab, overview entry, header dot, and `alerts.HREF["disks"]`).
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 48%]
|
||||
................................................sssssssssss............. [ 57%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 96%]
|
||||
......................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 28
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 220 0 100%
|
||||
src/dashboards/app.py 101 0 100%
|
||||
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
|
||||
src/dashboards/config.py 115 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
|
||||
src/dashboards/rates.py 37 1 97% 51
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 130 0 100%
|
||||
src/dashboards/sources/host.py 199 1 99% 147
|
||||
src/dashboards/sources/llm.py 150 1 99% 228
|
||||
src/dashboards/sources/nut.py 389 20 95% 128, 142, 148, 160, 166, 178, 186, 199, 270-271, 273, 489, 501, 536, 569, 573, 630-631, 636, 648
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 178 2 99% 250-251
|
||||
------------------------------------------------------------------
|
||||
TOTAL 2334 39 98%
|
||||
Required test coverage of 94% reached. Total coverage: 98.33%
|
||||
734 passed, 11 skipped, 1 warning in 58.36s
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
@@ -0,0 +1,27 @@
|
||||
Task 04 is complete and the repo is green.
|
||||
|
||||
## `/disks` — the fourth dashboard
|
||||
|
||||
**New:** `src/dashboards/web/pages/disks.html`, `src/dashboards/web/static/page-disks.js`. **Touched:** `app.py` (route), `app.js` (masthead `disks` dot), `dash.css` (drive-panel section), `page-index.js` (footer cadence), nav tab added to all five pages, `api.py` (`/api/summary` gained an additive `disks` block: fleet counts + per-host `health_failed`/`alarm_disks`/`smartctl_available`/`failed_devices`), `alerts.py` (disk-only SMART failure links to `/disks`).
|
||||
|
||||
- One card per **whole `/sys/block` drive** — device path, the drive's own verdict word (`OK`/`FAILED`/`unknown`), model·size·SSD/HDD, then Temp / Power-on / Wear. Colour only from the payload `severity`; power-on hours uncoloured; sector counters are `.chip.fail` **only while non-zero**.
|
||||
- Missing/denied `smartctl`: host dot stays green, the grid dims, one `.banner.info` why-line (`smartctl not available on <host> — install smartmontools…` / "No SMART data from … and redeploy the agent" / the agent's own sentence). Never red. No partition or mount rows.
|
||||
- Per-host `Charts` `<details>`: built and painted only while open, **one** `/api/disks/history` request per host per refresh naming every `disk.temp_c@dev` / `disk.percentage_used@dev`, cached response, `pruneHosts` drops card+charts+maps together.
|
||||
|
||||
## Real-hardware verification (the phase's acceptance line)
|
||||
|
||||
`ops` on driveripper has `NOPASSWD /usr/sbin/smartctl*` (not the `disk` group; deskwork grants neither), so `scripts/capture-fixtures.sh` now tries plain `smartctl` first, falls back to `sudo -n`, scrubs Serial / LU WWN / namespace EUI-64, and writes `tests/fixtures/smartctl_driveripper_<dev>.txt` — **all 16 real drives captured**.
|
||||
|
||||
That capture found a real parser bug, now fixed in `agent/host_agent.py`: attribute rows were read from the **end** of the line, but this smartmontools annotates RAW_VALUE — `35 (Min/Max 34/44)` on every Seagate, `43894h+43m+32.519s` on one Western Digital — so **all 8 HDDs reported no temperature and one had no power-on count**. Rows now read the RAW_VALUE column (`_smart_raw_number`, hex raws still refused). New tests: `test_smart_attribute_rows_are_read_from_the_raw_column_not_the_line_end`, `test_the_real_drives_answer_every_number_the_disks_page_prints` (every real capture), scrub test extended to EUI-64. Real findings the page now shows: `nvme5n1` has **13 media-and-data-integrity errors**, `nvme1n1` is at 58 % wear, NVMe temps 48–62 °C render amber.
|
||||
|
||||
## Verification
|
||||
|
||||
- `bash .agents/validate.sh` → **EXIT=0**: 769 passed / 11 skipped (+18 tests), coverage **98.33 %** (floor 94), ruff clean, pyright 0, `node --check` clean, repo hygiene clean.
|
||||
- `uv run pytest -m live --live` (with `.env.dev`): 10 passed, 1 skipped (`no agent redeployed with the smart pass yet`).
|
||||
- Dev server restarted; `/api/health` ok with `disks:driveripper/deskwork/gateway` all green. Nothing staged.
|
||||
- Screenshots in `data/logs/`: `09-disks-real-data.png` (16 real drives, amber temps, `13 uncorrectable` chip, `ALARM DRIVES 1`), `09-disks-real-alarm-drive.png`, `09-disks-real-charts.png`, `09-disks-smart-cards.png` / `-charts-expanded.png` / `-no-smartctl-host.png` (reference shapes + missing-binary state), `09-disks-live-fleet.png` / `-live-charts-open.png` / `-live-overview.png` (live fleet: 3 hosts answering, 18 drives, dimmed `n/a` + why-line; `Disks` tab, `disks` dot, `disks 300s` footer). No page errors on any page.
|
||||
- `README.md` + `AGENTS.md` updated: four dashboards, `/disks` row + UI conventions, the RAW_VALUE rule, corrected fleet SMART facts, real captures, and the two operator actions.
|
||||
|
||||
## Left for you (deliberately not done)
|
||||
|
||||
Live SMART needs **both**: `sudo usermod -aG disk <agent-user>` **and** a redeploy of `agent/host_agent.py` — the agent shells `smartctl` as itself and never calls `sudo`, so driveripper's sudoers rule alone only feeds the capture script. I did not add a sudo-spawning path to the agent unprompted; say the word if you'd rather that route exist (opt-in flag) than grant the group. deskwork has neither grant; gateway stays `n/a` permanently (VM). I did not modify any phase file — the `00_phase.md` diff in `git status` is your acceptance line.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 18%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 55%]
|
||||
...........sssssssssss.................................................. [ 64%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 92%]
|
||||
............................................................ [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 28
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 221 0 100%
|
||||
src/dashboards/app.py 101 0 100%
|
||||
src/dashboards/collectors.py 155 5 97% 141-142, 146-147, 164
|
||||
src/dashboards/config.py 115 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 105 5 95% 107, 144, 158, 161, 164
|
||||
src/dashboards/rates.py 37 1 97% 51
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 130 0 100%
|
||||
src/dashboards/sources/host.py 199 1 99% 147
|
||||
src/dashboards/sources/llm.py 150 1 99% 228
|
||||
src/dashboards/sources/nut.py 389 20 95% 128, 142, 148, 160, 166, 178, 186, 199, 270-271, 273, 489, 501, 536, 569, 573, 630-631, 636, 648
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 178 2 99% 250-251
|
||||
------------------------------------------------------------------
|
||||
TOTAL 2335 39 98%
|
||||
Required test coverage of 94% reached. Total coverage: 98.33%
|
||||
769 passed, 11 skipped, 1 warning in 60.88s (0:01:00)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
Reference in New Issue
Block a user