phase: 02_overview_fleet_and_alerts
**Phase 02 final verification pass — all three tasks already complete; verified criteria, fixed 3 defects found live.**
- Verified: `/` = exactly 2 `<section>`s (Warnings & errors + Fleet), alerts render from `/api/summary` verdicts with each alert's own `href`; `/system`, `/llm`, `/ups` unchanged (charts + data live).
- Fixed (`src/dashboards/alerts.py`): a dead source's message printed the name twice — `gateway: UpstreamError: gateway: http://…`; the poller's `UpstreamError:` wrapper is now stripped (other exception names kept). Tests: 2 new in `tests/test_alerts.py`.
- Fixed (`src/dashboards/api.py`): `/api/summary` dropped a configured host/engine that had never answered post-start, so the fleet said `2/2 hosts` while a third was down. Stub rows (`unreachable: true`, all `n/a`) now match `/api/*/snapshot`; `hosts_total=3 / hosts_live=2`. Test: `test_a_host_that_has_never_answered_is_still_a_row_in_the_fleet`.
- Fixed (`web/static/page-index.js`): fleet row of a non-answering host is now dimmed with the shared `.stale` class (invariant 4 on `/`). Test: `test_overview_dims_the_fleet_row_of_a_host_that_stopped_answering`.
- `bash .agents/validate.sh` exit 0: `uv run pytest --cov --cov-fail-under=94` → **439 passed, 6 skipped, 96.82%** (`alerts.py` 100%, `api.py` 100%); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check web/static` clean; hygiene clean; `uv pip list --outdated` empty; `uv run pytest -m live --live` → **6 passed**.
- Live proof (tunnel kill, then restored — all 6 sources `ok` again): killing the gateway forward produced exactly **one** alert, `fail gateway /system … unreachable (ConnectError)`; screenshots `data/logs/phase02-final-{alert,system,llm,ups,recovered}.png`.
- Completion criteria: two panels ✅ · `[]` when healthy ✅ (hermetic `healthy_client` + a live gateway-only instance returned `[]`; the real fleet can never be `[]` — its UPSes permanently warn on battery age/self-test) · one alert per dead host with `/system` link ✅ · pytest/live/coverage/ruff/pyright ✅ · no change on `/system`,`/llm`,`/ups` ✅ · phase 01 verdict vocabulary intact ✅.
- Docs: README `web/` bullet now states the dimmed `n/a` row; AGENTS.md invariant 4, `alerts.py`/`api.py` bullets, UI-conventions stale-row + window-chip ranges (5m…30d vs 1h…90d), metric example `llm.decode_tps` → `llm.decode_tokens_per_second` (verified against `/api/metrics`), test count 435 → 439. Env table, Containerfile, agent/install-script claims, fleet facts re-checked and consistent.
- Notable: left the fleet's real hostnames/domain in AGENTS.md's "real environment" section (deliberate, prior-phase content) even though rule 6 discourages personal hostnames in tracked files — worth a separate decision; no live config was changed beyond stopping/starting a local dev ssh forward.
- Next pending phase: `.agents/phases/todo/03_system_per_host_panels/`.
This commit is contained in:
@@ -11,6 +11,7 @@ letting `uv run pytest -m live --live` catch an upstream that renamed something.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
@@ -42,6 +43,7 @@ if str(SRC) not in sys.path:
|
||||
sys.path.insert(0, str(SRC))
|
||||
|
||||
from dashboards.rates import RateEstimator # noqa: E402
|
||||
from dashboards.sources.host import HostCollector, parse_host_snapshot # noqa: E402
|
||||
from dashboards.store import SampleStore # noqa: E402
|
||||
|
||||
FIXTURES = Path(__file__).resolve().parent / "fixtures"
|
||||
@@ -77,6 +79,34 @@ def agent_snapshots() -> dict[str, dict[str, Any]]:
|
||||
return out
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host_scrape() -> Any:
|
||||
"""`host_scrape(snapshot, cpu_busy=10.0, mutate=None)` -> one parsed host payload.
|
||||
|
||||
Two scrapes are needed before any rate exists (the agent reports counters; the
|
||||
dashboard owns the window), and the payload the API serves is the second one. `mutate`
|
||||
edits the snapshot before both scrapes so a test can put a value on a chosen side of
|
||||
`severity.THRESHOLDS`; `cpu_busy` sets the CPU percentage over the window directly.
|
||||
Driven from the real capture so a schema drift breaks here rather than in a browser.
|
||||
"""
|
||||
def scrape(snapshot: dict[str, Any], *, name: str = "driveripper", cpu_busy: float = 10.0,
|
||||
mutate: Any = None) -> dict[str, Any]:
|
||||
snap = copy.deepcopy(snapshot)
|
||||
if mutate is not None:
|
||||
mutate(snap)
|
||||
collector = HostCollector(RateEstimator(span=60.0))
|
||||
parse_host_snapshot(name, snap, collector, now=1_000.0)
|
||||
second = copy.deepcopy(snap)
|
||||
second["ts"] = 1_010.0
|
||||
modes = second["cpu"]["modes"]
|
||||
# busy = (Δtotal - Δidle)/Δtotal, and iowait is left untouched so it rates at 0
|
||||
modes["total"] += 1_000.0
|
||||
modes["idle"] += 1_000.0 - (cpu_busy * 10.0)
|
||||
return parse_host_snapshot(name, second, collector, now=1_010.0).payload
|
||||
|
||||
return scrape
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def metrics_text() -> Any:
|
||||
"""metrics_text('gufo') / metrics_text('vllm') -> the captured exposition verbatim."""
|
||||
|
||||
Reference in New Issue
Block a user