Compare commits
18
Commits
6444a158f2
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
883e36c727 | ||
|
|
219d7a7c3b | ||
|
|
86c8bf4992 | ||
|
|
0271387053
|
||
|
|
51772ffef2 | ||
|
|
25fd74b7c5 | ||
|
|
e366f599f3 | ||
|
|
fa4f38c694 | ||
|
|
44d61d3ab9 | ||
|
|
5d37444fb0 | ||
|
|
9ff3c7889b | ||
|
|
2bf236fd92 | ||
|
|
558a83d0be | ||
|
|
89d9fb9f89 | ||
|
|
5f3ed9bcd4 | ||
|
|
895f309f8e | ||
|
|
5e267737dc | ||
|
|
973b7299fe |
@@ -0,0 +1,34 @@
|
||||
# Phase 25 — Masthead control placement: badges everywhere, right-justified, clock rightmost
|
||||
|
||||
**Source:** `TODO.md` L3–L5 — "hosts disks engines UPS disappear when in the certs/account page" · "I made a mistake earlier, I did want hosts disks engines UPS badges and certs/account/dark right justified don the navbar" · "The time should show up as the farthest right item on the navbar"
|
||||
**Story:** n/a
|
||||
**Context:** phase 20 (complete) — the masthead is three rows at every width (row 1 brand + `#nav-toggle` + the five tabs, row 2 the `#nav-status` per-source dots, row 3 the `.masthead-right` controls), not sticky (`position: relative`, `z-index: 40`), with the ≤640px dropdown sliding on `nav.tabs.is-open`; phase 18 — the hamburger reveal through `--nav-toggle-display`. The dots are built by `Dash.initChrome()` in `src/dashboards/web/static/app.js` (the `if (nav)` block on `#nav-status`, one chip per `system`/`disks`/`llm`/`ups` dashboard from `/api/health`, refreshed every 15 s), and exist in the markup only of the five dashboard pages — `certs.html` and `settings.html` ship no `#nav-status` (the comment in `settings.html` says a session page "has no fleet reachability to report"), which is what L3 names. The served-CSS pins for the bar are in `tests/test_api.py`: `test_the_masthead_is_three_left_justified_rows_at_every_width` (the two `justify-content: flex-start` declarations, the two `flex-basis: 100%` row breaks, the `min-height`/`gap`), `test_the_mobile_masthead_block_declares_no_second_bar_height`, `test_the_masthead_scrolls_away_and_stays_the_dropdown_anchor`; the one-loop control-order audit is `test_every_page_links_to_both_admin_pages` (clock → `⚷ Certs` → `⚙ Account` → theme toggle on all eight signed-in pages), and the two pins that assert the admin pages have **no** dots are in `test_the_settings_page_is_not_one_of_the_dashboards` and `test_the_certs_page_is_a_page_and_not_a_sixth_dashboard`. `tests/test_auth_api.py` asserts `/login` carries no `#nav-status` — unchanged by this phase.
|
||||
|
||||
## Objective
|
||||
|
||||
The four per-source badges (hosts · disks · engines · UPS) render on **every** page that renders the masthead — including `/certs` and `/settings` — and the two right-hand rows (the dots, the controls) are right-justified, with the clock the farthest-right item on the bar. The three-row shape at every width is preserved; only the alignment and the control order move, and the clock's position is what L5 asks for.
|
||||
|
||||
## Dependencies
|
||||
- — (none) — the phase builds on complete phase 20 and phase 18; it is scheduled first of the three appended phases on purpose, because the mobile audit phase (27) measures the masthead in the shape this phase leaves it in.
|
||||
|
||||
## Tasks
|
||||
1. `01_status_dots_on_admin_pages.md` — `#nav-status` on `/certs` and `/settings` (markup only — `initChrome` already builds the dots when the container exists); the two no-dots pins flip to dots-present
|
||||
2. `02_right_justify_masthead_rows.md` — `justify-content: flex-end` on `#nav-status` and `.masthead-right`, the comment block re-worded, the three-rows pin re-pinned to the new alignment
|
||||
3. `03_clock_rightmost.md` — the clock moves to the end of the control row on all eight signed-in pages; the one-loop order pin updated; the phase's real-browser verification (badges on the admin pages, both rows right-justified, clock rightmost, at 1280 px and 360 px)
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the served-byte pins in `tests/test_api.py` — the flipped admin-page pins (task 01), the re-pinned row-alignment pin (task 02), the updated control-order pin (task 03); `tests/test_auth_api.py`'s `/login`-has-no-dots pin must stay green untouched
|
||||
- Coverage: **>90%** on new/modified code (the new code is the pins themselves; the change is CSS and template markup)
|
||||
- Real-browser check (invariant 10): a `data/logs/25-task03-*.cjs` script in the `20-task01-rows.cjs` convention, run against the restarted dev server at 1280 px and 360 px on all seven tabbed pages plus `/login` (unsigned), with screenshots in `data/logs/`
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass updates the now-stale sentences in `AGENTS.md`: "The masthead is three **left-justified** rows at every width" (alignment), "A page with no dots (`/certs`, `/settings`) renders two rows" (gone — all masthead pages carry the dots), "the two links therefore sit on all eight signed-in pages in one order (clock → `⚷ Certs` → `⚙ Account` → theme toggle)" (new order: `⚷ Certs` → `⚙ Account` → theme toggle → clock), and the "left-justified" wording in the `dash.css` header comment. `README.md` is checked for the same sentences; none is expected.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `/certs` and `/settings` render the four badges, populated from `/api/health` (pinned + verified in a browser against the live dev fleet)
|
||||
- [ ] the dots row and the controls row are right-justified at every width; the three-row shape and the ≤640px dropdown are unchanged (pinned + measured in a browser)
|
||||
- [ ] the clock is the farthest-right item on the bar on all eight signed-in pages (pinned + measured)
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the row-break mechanism (`flex-wrap` + the two `flex-basis: 100%`), the non-sticky bar, the dropdown's slide and its four close paths, and the dots' refresh semantics are all untouched
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
# Task 01 — The badges render on the admin pages too
|
||||
|
||||
**Phase:** `25_masthead_control_placement` · **Source:** `TODO.md:3–4` — "hosts disks engines UPS disappear when in the certs/account page" · "I made a mistake earlier, I did want hosts disks engines UPS badges and certs/account/dark right justified don the navbar"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`#nav-status` exists in the `.masthead-right` of `/certs` and `/settings`, exactly as it does in the five dashboard pages, so `Dash.initChrome()` builds the four badges (`hosts`/`disks`/`engines`/`UPS`) there without any change to `app.js` — the "disappear when in the certs/account page" symptom is the missing container, and the fix is the container.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/certs.html` — add `<div id="nav-status" class="gap" aria-live="polite"></div>` as the **first** child of `<div class="masthead-right">` (before the `<span class="clock">`), the same position the five dashboard pages use (`index.html:23` is the reference).
|
||||
2. `src/dashboards/web/pages/settings.html` — the same addition, and delete the `<!-- No `#nav-status`: a session page has no fleet reachability to report, and `Dash.initChrome()` builds the dots only when the container exists. -->` comment — the premise it states is what the operator overruled (the badges report the fleet, and the fleet is the point of the site, on every page).
|
||||
3. `src/dashboards/web/static/app.js` — **no change** (stated so the executor does not "improve" it): `initChrome()`'s `if (nav)` block already builds the four chips and refreshes them from `/api/health` every 15 s whenever `#nav-status` exists; `setDot`'s `idle` state already covers a dashboard the page's config does not run.
|
||||
4. `tests/test_api.py` — flip the two pins, one assertion each: in `test_the_settings_page_is_not_one_of_the_dashboards`, `assert 'id="nav-status"' not in html` becomes `in html` (message re-worded: the session page grew the badges the operator asked for); in `test_the_certs_page_is_a_page_and_not_a_sixth_dashboard`, the same flip, and the docstring sentence "no `#nav-status` (a CA page has no fleet reachability to report)" is re-worded to the new fact. `test_the_certs_page_is_a_page_and_not_a_sixth_dashboard`'s other assertions (zero `aria-current`, the tab strip, the head) are untouched, and `tests/test_auth_api.py`'s assertion that **`/login`** has no `#nav-status` is untouched — `/login` still renders the brand link and the theme toggle only.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the two flipped pins above; `test_the_header_status_dot_picks_up_the_disks_job` and `test_the_disks_tab_is_in_the_masthead_of_every_page` stay green unchanged
|
||||
- Coverage: **>90%** on this task's new/modified code (markup + pins)
|
||||
- The badges populating on the two admin pages is the phase's real-browser check (task 03), run against the live dev fleet so the dots are `ok` green, not grey `idle`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /certs` and `GET /settings` (signed in) both contain `<div id="nav-status" class="gap" aria-live="polite">` as the first child of `.masthead-right` (pinned)
|
||||
- [ ] in a browser, both admin pages show the four badges and they answer the same `ok`/`warn`/`fail` state the dashboard pages show (measured, screenshots in `data/logs/`)
|
||||
- [ ] `/login` (unsigned) still carries no `#nav-status` (`tests/test_auth_api.py` green)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed work: `initChrome` is byte-identical, and the badge refresh cadence and `idle` semantics are untouched
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
# Task 02 — The dots row and the controls row are right-justified
|
||||
|
||||
**Phase:** `25_masthead_control_placement` · **Source:** `TODO.md:4` — "I made a mistake earlier, I did want hosts disks engines UPS badges and certs/account/dark right justified don the navbar"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Phase 20 task 01 made the two right-hand rows of the masthead flush left; this task moves them flush **right**. The mechanism is two declarations — `justify-content: flex-end` on `#nav-status` and on `.masthead-right` — and nothing else: the row breaks (the two `flex-basis: 100%` items inside the `flex-wrap: wrap` bar), the `min-height`, the gaps and the ≤640px block all stay exactly as they are, so the bar is still three rows at every width and the phone's wrapped control row still wraps *within itself*.
|
||||
|
||||
- ASSUMPTION: the three-row shape is preserved at every width and only the alignment changes. A return to the old single-row bar is not the ask — the row breaks are what make the bar fit at 360 px (phase 18/20 measured that), and the operator's word is "right justified in the navbar", not "one row".
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/dash.css` — two declarations: `#nav-status`'s `justify-content: flex-start` becomes `flex-end`, and `.masthead-right`'s `justify-content: flex-start` becomes `flex-end`. `margin-inline-start: auto` stays **absent** from `.masthead-right` (the old single-row right-pinning is gone; `flex-end` on a full-basis row is the alignment now).
|
||||
2. `src/dashboards/web/static/dash.css` — re-word the comments that state the old alignment, so a future reader is not sent to a left-justified bar that no longer exists: the header comment block (the "three left-justified rows at every width" sentence, and its row-2/row-3 descriptions), the `Row 3:` comment above `.masthead-right` (which currently cites the old `TODO.md` word for `flex-start` — the new sentence states the operator overruled it: the badges and the cluster are right-justified), the `Row 2:` comment above `#nav-status`, and the ≤640px block's comment "Row 3 alignment is the base rule's `flex-start`, restated nowhere" (now `flex-end`). The `#nav-toggle { margin-inline-start: auto }` line in the mobile block is **unchanged**: the hamburger sits at the right of row 1 (brand + toggle + tabs), which is where a phone's thumb already is — it was never part of the right-justified cluster.
|
||||
3. `tests/test_api.py` — rename `test_the_masthead_is_three_left_justified_rows_at_every_width` to `test_the_masthead_is_three_rows_right_justified_at_every_width` and change its two alignment assertions from `== "flex-start"` to `== "flex-end"` (the dots line and the cluster line), re-wording the assertion messages and the docstring ("two of them flush left" → "two of them flush right"). Every other assertion in the test stays: `flex-wrap: wrap`, the `flex-direction: row` check, the no-`height`/`min-height: var(--header-h)` pair, the gap, the two `flex-basis: 100%` row breaks, and the "no `margin-inline-start` on the cluster" guard — the row-break mechanism is exactly what must survive the re-justify.
|
||||
4. `tests/test_api.py` — `test_the_mobile_masthead_block_declares_no_second_bar_height` and `test_the_masthead_scrolls_away_and_stays_the_dropdown_anchor` stay green **unchanged** (neither reads `justify-content`); confirm they do not need a word change in their docstrings before moving on.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the re-pinned alignment pin above; the two untouched mobile/sticky pins as the guard that the row mechanism survived
|
||||
- Coverage: **>90%** on this task's new/modified code (CSS + pin)
|
||||
- The measured half (the rows actually render right-justified) is the phase's real-browser check (task 03)
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the served stylesheet reads `justify-content: flex-end` on both `#nav-status` and `.masthead-right`, with `margin-inline-start` still absent from the cluster (pinned)
|
||||
- [ ] the ≤640px block still declares `#nav-status { flex-wrap: nowrap; }` and the wrapping cluster, and still declares no second bar height (existing pins green)
|
||||
- [ ] in a browser at 1280 px and 360 px, the dots row and the controls row are flush with the bar's right padding edge and the bar is still three rows (measured, screenshots in `data/logs/`)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed work: the dropdown slide, the hamburger reveal and the dots' content are untouched
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 03 — The clock is the farthest-right item, and the phase's browser proof
|
||||
|
||||
**Phase:** `25_masthead_control_placement` · **Source:** `TODO.md:5` — "The time should show up as the farthest right item on the navbar"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
On a right-justified control row, the farthest-right item is the **last** one in markup order, so the clock moves from the first control (before `⚷ Certs`) to the last control (after the theme toggle). The new order on all eight signed-in pages is `⚷ Certs` → `⚙ Account` → `☾ Dark` → clock. This is the phase's verification task as well: the layout change is CSS + markup, so the served-byte pins cannot say the rows actually render right-justified — this task measures them in a real browser (invariant 10).
|
||||
|
||||
- ASSUMPTION: "farthest right" is achieved by markup order (the clock is the last child of `.masthead-right`), not by a CSS `order` declaration — the one-loop order pin compares markup, and a `order:` property the pin cannot see is a drift waiting to happen.
|
||||
|
||||
## Work
|
||||
|
||||
1. The seven page templates that render `.masthead-right` — `src/dashboards/web/pages/index.html`, `system.html`, `disks.html`, `llm.html`, `ups.html`, `settings.html`, `certs.html` — move the line `<span class="clock" id="clock">--:--:--</span>` from its current position (first control, before the Certs link) to **after** the `<button … id="theme-toggle">` element, so the control row reads `⚷ Certs` → `⚙ Account` → `☾ Dark` → clock. The `<div id="nav-status">` stays the first child (it breaks onto row 2 before the controls). All seven templates must end byte-identical in the control-row region apart from the page-specific comment — `test_every_page_links_to_both_admin_pages` is a one-loop audit precisely because the eight markups are supposed to be the same.
|
||||
2. `src/dashboards/web/static/dash.css` — re-word the header comment's row-3 line ("row 3 the `.masthead-right` controls (clock, `⚷ Certs`, `⚙ Account`, theme)") to the new order. No declaration changes.
|
||||
3. `tests/test_api.py` — in `test_every_page_links_to_both_admin_pages`: update the docstring's order sentence (clock → Certs → Account → theme toggle ⇒ `⚷ Certs` → `⚙ Account` → theme toggle → clock); keep the two existing `before_theme` assertions (Certs and Account still sit before the theme toggle, Certs before Account — both true in the new order); add the new one: `html.index('id="clock"') > html.index('id="theme-toggle")` on every page (the clock is the last control — on a `flex-end` row that is the farthest-right item). The `/login` half of the test (no links, no toggle) is untouched.
|
||||
4. `data/logs/25-task03-verify.cjs` — the browser proof, in the `20-task01-rows.cjs` convention (`NODE_PATH=<playwright node_modules> node data/logs/25-task03-verify.cjs`, `DASH_BASE`/`DASH_PW` env, `check()` accumulator, non-zero exit on any failure): sign in, then on all seven tabbed pages at **1280 px** and **360 px** — (a) group the bar's children into rows by bounding box and assert the three-row shape; (b) the `#nav-status` row and the controls row are flush with the bar's right padding edge (their right edges equal the bar's content right edge within 1 px, and they are **not** flush left — the old layout's right edge was far from the bar's); (c) within the controls row, the clock's right edge is the rightmost of the four controls and the order reads Certs, Account, theme, clock; (d) `#nav-status` on `/certs` and `/settings` is populated (four chips, none `idle`-grey against the live dev fleet — the task 01 check). `/login` (unsigned) at both widths: brand + theme toggle only, no dots, no clock, no horizontal overflow. Screenshots `data/logs/masthead-25-<page>-<width>.png` (the seven tabbed pages at both widths; the script names them, the pass line is every `check()` green).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the extended `test_every_page_links_to_both_admin_pages` (the order pin), with `test_the_masthead_is_three_rows_right_justified_at_every_width` (task 02) and the two task-01 flipped pins as the surrounding contract
|
||||
- Coverage: **>90%** on this task's new/modified code (markup + pin + script)
|
||||
- Real-browser check: the `.cjs` pass above against the restarted dev server (`./scripts/dev-server.sh restart` first, per invariant 10), the dev fleet answering so the badges are green
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] on all eight signed-in pages the control row reads `⚷ Certs` → `⚙ Account` → `☾ Dark` → clock in the markup, with the clock after the theme toggle (pinned)
|
||||
- [ ] the `.cjs` pass is all-green at 1280 px and 360 px: three rows, both right-hand rows flush right, clock rightmost, badges populated on `/certs` and `/settings` (screenshots in `data/logs/`)
|
||||
- [ ] `/login` (unsigned) is unchanged: no badges, no clock, no links (existing `tests/test_auth_api.py` pin green)
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] `AGENTS.md`'s now-stale masthead sentences are updated (the phase overview's Documentation & Consistency list) and `README.md` checked
|
||||
@@ -0,0 +1,34 @@
|
||||
# Phase 26 — UPS events: power events collapsible, connection errors always visible
|
||||
|
||||
**Source:** `TODO.md` L6 — "The phase I ran earlier collapsed all the events in the UPS tab. That wasn't the point. I wanted the power loss events in a collapsible list, not events like \"NUT server NutError: cannot reach upsd at 10.1.0.126:3051 (timed out after 4s)\" in a collapsible list."
|
||||
**Story:** n/a
|
||||
**Context:** phase 19 (complete) put the **whole** `/ups` event feed behind one collapsed `details.events-box` (no `open` attribute, lazy: `refresh()` fetches the feed only while `state.events.open`, the first expand fetches, a fresh re-expand paints from the `<ul>`'s kept children, `painted`/`fetchedAt` stamped on the success path alone, the `Dash.eventsPager` window of 50 inside the bounded 320px scroll). The operator's complaint is that the classification is wrong, not that a collapse exists: a poll failure like `NUT server NutError: cannot reach upsd … (timed out after 4s)` is hidden behind a summary nobody opens, while the power events it was meant to tuck away are the only ones the operator actually wants collapsed. The feed's **kinds** are fixed and known: the poller (`collectors.py:490` — `Job("nut", "ups", "NUT server", …)`, events under the dashboard name `ups`) writes `unreachable` / `reachable` / `recording` / `store-error` (`poller.py:127/157/164/190` — the quoted `NutError` line is the `unreachable` message); the `NutMonitor` transition bookkeeping writes `status` (a state change lands at `collectors.py:393`) and `test` (the self-test verdict, `collectors.py:409`); `api.py` writes `action` for every command sent through `POST /api/ups/command` (the fixed-text `action` events, `api.py:999/1014`). The served-byte pins for the current shape are in `tests/test_api.py`: `test_ups_power_events_feed_sits_behind_a_collapsed_disclosure` (the `details.events-box`, no `open`, `#events` inside), `test_ups_events_are_fetched_only_while_the_disclosure_is_open` (the lazy gate — **this phase inverts it on purpose**), `test_the_ups_event_rows_keep_the_date_time_and_severity_shape` (the row shape), `test_the_ups_events_disclosure_is_a_quiet_disclosure_not_a_new_component` (the `details.events-box` CSS shape — unchanged by this phase). The page is `src/dashboards/web/static/page-ups.js` (`refresh()` around line 439, `refreshEvents()` around line 445, the disclosure's toggle handler in the `DOMContentLoaded` block) over the card in `src/dashboards/web/pages/ups.html:75-78`.
|
||||
|
||||
## Objective
|
||||
|
||||
`/ups` renders **two** lists from the one unchanged feed fetch: the power events — state transitions (including the power-loss transitions), self-test verdicts and sent commands — stay behind the collapsed `details.events-box` disclosure, and the connection/error events (`unreachable`/`reachable`/`recording`/`store-error`) are a card that is **always visible** (hidden only while it has no rows to show). The split is page-side, keyed on the event's `kind`; `/api/events` gains no vocabulary.
|
||||
|
||||
## Dependencies
|
||||
- — (none) — the phase builds on complete phase 19; it is scheduled before `27_mobile_tables_and_audit` on purpose, because that phase's mobile audit captures `/ups` in the shape this phase leaves it in (two cards, one of them hidden-when-empty).
|
||||
|
||||
## Tasks
|
||||
1. `01_page_side_event_split.md` — the `kind` split in `page-ups.js` (power kinds `status`/`test`/`action`, everything else errors) and the removal of the phase-19 lazy gate: the feed is fetched on every poll, because a visible list that ages is worse than one more 200-event fetch per interval
|
||||
2. `02_render_both_lists_paged.md` — the always-visible `#events-errors` card (above the disclosure, hidden while empty) and both lists paged through the shared `Dash.eventsPager`
|
||||
3. `03_repin_and_browser_verify.md` — the pin re-pins (the disclosure stays, the lazy gate inverts, the row shape extends to the error list, the kind set is pinned so a silent upstream rename breaks a test) and the phase's real-browser verification
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the re-pinned and new pins in `tests/test_api.py` (task 03); the split itself is pinned on the served script (the kind set) and the served markup (the error card, `hidden`, not inside a `<details>`)
|
||||
- Coverage: **>90%** on new/modified code
|
||||
- Real-browser check (invariant 10): a `data/logs/26-task03-*.cjs` script in the `20-task01-rows.cjs` convention, run against the restarted dev server: the disclosure shut by default, the error card's visibility matching the feed's error-kind rows, both lists' rows matching a direct `/api/events?source=ups&limit=200` split, screenshots in `data/logs/`
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass updates the now-stale sentences in `AGENTS.md`: the event-feed paragraph's lazy-fetch block ("`refresh()` fetches the feed only while `state.events.open` is true, so a card nobody is looking at costs the poll loop nothing" and the first-expand/fresh-re-expand sentences) is re-written to the new fact (the one fetch serves two lists, always; the disclosure gates only the power list's visibility), and the pin description of `test_ups_events_are_fetched_only_while_the_disclosure_is_open` is re-pointed at its inverted successor. `README.md` is checked for the same sentence; none is expected.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] a poll failure (e.g. `NUT server NutError: cannot reach upsd …`) is visible on `/ups` without opening anything, and the power events stay behind the collapsed disclosure (pinned + verified in a browser)
|
||||
- [ ] the error card is hidden while it has no rows and appears with its first error row; both lists share the one `/api/events?source=ups` fetch and the shared pager (pinned)
|
||||
- [ ] a failed fetch leaves both lists on their last good rows (the stale-last-good rule) — pinned where phase 19 pinned its
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the `details.events-box` CSS shape, the pager's page/scroll semantics, and the row's time/host/severity shape are untouched
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 01 — The feed splits on `kind`, and the lazy gate comes out
|
||||
|
||||
**Phase:** `26_ups_event_split` · **Source:** `TODO.md:6` — "I wanted the power loss events in a collapsible list, not events like \"NUT server NutError: cannot reach upsd at 10.1.0.126:3051 (timed out after 4s)\" in a collapsible list."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`page-ups.js` splits the one unchanged feed fetch into two lists: the **power** events — `status` (the state transitions, including the power-loss transitions the operator names), `test` (the self-test verdicts) and `action` (the commands sent through the confirm dialog) — and **everything else** (the poller's `unreachable`/`reachable`/`recording`/`store-error`, the quoted `NutError` line included), which is the always-visible error list. The phase-19 lazy gate goes out with the split: a visible list that ages is worse than one more 200-event fetch per interval, so `refresh()` fetches on every poll.
|
||||
|
||||
- ASSUMPTION: "power events" is the **known** set of three kinds (`status`/`test`/`action`); any other kind is an error-row by default, because the operator's complaint is that errors must never be the hidden list, and a new kind landing in the hidden list would be the same bug re-shipped.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-ups.js` — at the top of the file, `const POWER_EVENT_KINDS = new Set(["status", "test", "action"]);` with a comment naming the three writers (`collectors.py:393` `status`, `collectors.py:409` `test`, `api.py` `action` — so a rename on the source side is visible at the site of the drift). In `refreshEvents()`: after the one `fetchJSON(\`/api/events?limit=${Dash.EVENT_FEED_LIMIT}&source=ups\`)` (the call itself is **unchanged** — no `page`/`limit`/`kind` vocabulary, the phase-19 rule), `const power = data.events.filter((e) => POWER_EVENT_KINDS.has(e.kind));` and `const errors = data.events.filter((e) => !POWER_EVENT_KINDS.has(e.kind));`.
|
||||
2. `src/dashboards/web/static/page-ups.js` — in `refresh()`, replace `if (state.events.open) await refreshEvents();` with `await refreshEvents();` and re-word the comment above it (the "a shut `<details>` must cost the poll loop nothing" rationale is what the operator overruled: the error list is visible, so the feed is needed whether or not the disclosure is open). In the `DOMContentLoaded` block, delete the `eventsBox` toggle handler's fetch logic — with an always-fresh list the `toggle` event no longer needs to do anything, and `state.events` (with its `open`/`painted`/`fetchedAt` fields) is deleted entirely: the disclosure now gates only the power list's **visibility** (the native `<details>` behaviour), and a failed fetch leaves both lists on their last good rows through `refreshEvents()`'s existing `try { … } catch (err) { /* next poll */ }` — the success-path stamps go away with the state they stamped, and the stale-last-good rule is the `catch` block, which stays.
|
||||
3. `src/dashboards/web/static/page-ups.js` — the split is the **only** change to the fetch path: the `row` renderer, `eventText` handling and the `runtime_seconds` footnote stay byte-identical (task 02 points them at both lists), and the `state` object loses `events` with no other field touched.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the split and the gate removal are pinned on the served script in task 03 (`tests/test_api.py`) — the kind set, the unconditional `refreshEvents()` call, the deleted `state.events`
|
||||
- Coverage: **>90%** on this task's new/modified code
|
||||
- The behavioural proof (a real `unreachable` row visible without opening the disclosure) is the phase's real-browser check (task 03)
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `page-ups.js` defines `POWER_EVENT_KINDS = new Set(["status", "test", "action"])` and `refresh()` calls `refreshEvents()` unconditionally (pinned on the served script)
|
||||
- [ ] `state.events` and the toggle handler's fetch logic are gone; a failed fetch leaves both lists on their last good rows (pinned where phase 19 pinned its)
|
||||
- [ ] the `/api/events` call is byte-identical to the phase-19 one (no new query vocabulary — pinned)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed work: the poll cadence, the confirm dialog, the hero row and the charts are untouched
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 02 — The error card: always visible, hidden only when empty
|
||||
|
||||
**Phase:** `26_ups_event_split` · **Source:** `TODO.md:6` — "I wanted the power loss events in a collapsible list, not events like \"NUT server NutError: cannot reach upsd at 10.1.0.126:3051 (timed out after 4s)\" in a collapsible list."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The error list is a card of its own, **above** the power-events card, built from the existing vocabulary (`section.card`, `ul.events`, the pager's strip) with no new CSS component: visible from first paint while it has rows, `hidden` while it has none, and paged by the same `Dash.eventsPager` the power list uses (its `WeakMap` state is keyed by the `<ul>`, so the two lists page independently — the pager's existing contract).
|
||||
|
||||
- ASSUMPTION: the error section sits **above** the Power events disclosure — the visibility complaint is about *seeing* the errors, and the collapsible list keeps its current position in the card stack.
|
||||
- ASSUMPTION: the error card is `hidden` while it has zero rows — a permanent empty "Connection & errors" card is the noise the page's empty-state rules exist to avoid, and its first `unreachable` row is what makes it appear.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/ups.html` — above the existing `<section class="card" style="margin-top:.85rem">` (the disclosure card, `ups.html:74-79`), add the error card: `<section class="card" id="ups-errors-card" hidden>` with `<header><h3>Connection & errors</h3><span class="spacer"></span><span class="faint">upsd reachability and this process's own failures</span></header>` and `<div class="body"><ul class="events" id="events-errors"><li class="muted">loading…</li></ul></div>`. The `hidden` attribute is the resting state (before the first successful fetch there are no rows); `style="margin-top:.85rem"` moves to the **error** card's position only in the sense that the disclosure card keeps its existing attribute — a `hidden` card is `display: none`, so the disclosure card's own `margin-top` still gives the gap to whatever is above it when the error card is absent, and no rule is needed for the two-cards case beyond the disclosure card's existing attribute (verify in the browser that the gap is `.85rem` in both states; if the hidden card's own margin would double it, drop the error card's `margin-top` and let the stack's own gap rule do the work — the page's sections carry the shared `.85rem` gap).
|
||||
2. `src/dashboards/web/static/page-ups.js` — in `refreshEvents()`, after the split (task 01): render the power list exactly as today — `Dash.eventsPager($('#events'), power, { label: 'Power events', … })` with its existing `empty` sentence and `row` renderer — and the error list — `Dash.eventsPager($('#events-errors'), errors, { label: 'Connection & errors', empty: '<li class="muted" style="grid-template-columns:1fr">No connection errors — upsd is answering and this process is recording.</li>', row: <the same `row` function as the power list, extracted to one const so the two lists cannot drift> })`; then `$('#ups-errors-card').hidden = errors.length === 0;`. The `loading…` seed row in the error card is overwritten on the first successful fetch and never shows, because the card is `hidden` until that fetch lands with rows (a first fetch with zero error rows keeps it hidden — no flash).
|
||||
3. `src/dashboards/web/static/dash.css` — **no new rules**: the card, the `ul.events` bounded 320px scroll and the `.events-pager` strip are all existing vocabulary (the phase-19 rule a second opinion about the driver's word is the bug — the same applies to a second style for a second list). If step 1's gap verification needs a rule, the only allowed one is a margin on `#ups-errors-card` itself, pinned in task 03.
|
||||
4. The `row` renderer's `runtime_seconds` footnote stays on both lists (an `action` row's runtime is the power list's; an error row simply has no `event.data.runtime_seconds` and prints none — no branch added).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: task 03's pins — the error card in the served markup (`hidden`, not inside a `<details>`), the kind set, the single `row` const, the `hidden = errors.length === 0` line
|
||||
- Coverage: **>90%** on this task's new/modified code
|
||||
- The two-cards layout, the gap in both states and the error card's first-appear are the phase's real-browser check (task 03)
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `/ups` shows the `Connection & errors` card above the Power events disclosure, paged, sharing the row shape (pinned + measured in a browser)
|
||||
- [ ] the card is `hidden` while the feed has no non-power kinds and appears with its first error row, the gap to the disclosure card being `.85rem` in both states (measured)
|
||||
- [ ] both lists page independently (pager state per `<ul>`) and a failed fetch leaves both on their last good rows (pinned)
|
||||
- [ ] no new CSS selector is introduced for the error card (pinned)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed work: the disclosure's CSS shape and the power list's empty sentence are untouched
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — The pins re-pin, and the browser proves the split
|
||||
|
||||
**Phase:** `26_ups_event_split` · **Source:** `TODO.md:6`
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The phase's pins: the disclosure shape stays pinned (the power list is still behind the collapsed `details.events-box`), the lazy-gate pin inverts to its replacement (the fetch is no longer gated on the disclosure), the row-shape pin extends to the error list, and the kind set is pinned on the served script so a silent rename on the source side breaks a test before an operator sees a misclassified row. Then the real-browser verification (invariant 10).
|
||||
|
||||
## Work
|
||||
|
||||
1. `tests/test_api.py` — `test_ups_power_events_feed_sits_behind_a_collapsed_disclosure`: keep every existing assertion (one `details.events-box`, no `open` attribute, `#events` inside it) and extend: `id="events-errors"` is present in the served page and is **not** inside any `<details>` (the error list must not be collapsible — the operator's whole complaint), and the error card carries the `hidden` attribute in the template.
|
||||
2. `tests/test_api.py` — `test_ups_events_are_fetched_only_while_the_disclosure_is_open` becomes its inversion, renamed `test_ups_events_are_fetched_on_every_poll` (the phase-19 lazy gate is what the operator overruled; the old test failing is the point of this phase): the served script's `refresh()` region (the same `boot.index("…")` slice the old test uses) shows `refreshEvents()` called **unconditionally**, and the served script carries no `state.events` and no `toggle`-handler fetch. The stale-last-good pin (a failed fetch leaves the rows on screen) moves onto the `refreshEvents()` `catch` block, the way phase 19 pinned it.
|
||||
3. `tests/test_api.py` — `test_the_ups_event_rows_keep_the_date_time_and_severity_shape`: the existing `#events` assertions stay, and the same shape (the `<time>` element, the host chip with its `sev` class, the `eventText`-cleaned message, the optional runtime footnote) is asserted for `#events-errors` through the shared `row` const. New pin: the served script defines `POWER_EVENT_KINDS` as exactly `status`, `test`, `action` (a set literal read out of the script text — a kind added to or removed from the set is a classification the operator did not make). New pin: `$('#ups-errors-card').hidden = errors.length === 0` (or its exact equivalent line) — the hidden-when-empty behaviour is the one state this page may claim about a list it cannot see.
|
||||
4. `tests/test_api.py` — `test_the_ups_events_disclosure_is_a_quiet_disclosure_not_a_new_component`: unchanged, and extended with one guard — no new CSS selector for the error card in the served stylesheet (the card is `section.card` + `ul.events` + `.events-pager` vocabulary only; a second style for a second list is the phase-19 anti-pattern).
|
||||
5. `data/logs/26-task03-verify.cjs` — the browser proof, in the `20-task01-rows.cjs` convention (`DASH_BASE`/`DASH_PW`, `check()` accumulator, non-zero exit on failure), against the restarted dev server (`./scripts/dev-server.sh restart` first): sign in; fetch `/api/events?source=ups&limit=200` **directly** and split by the three power kinds; load `/ups` and assert — (a) the disclosure is shut by default and the power list renders only on expand; (b) the error card is visible if and only if the direct fetch has non-power-kind rows; (c) each list's DOM row count and first row's text match the direct-fetch split (the `eventText` cleaning applied); (d) a failed-fetch leg is not needed against live hardware — the stale-last-good rule is pinned in step 2's pin instead. Screenshots `data/logs/ups-event-split-<state>.png` (the error card visible with rows, and hidden when the feed has none — if the live feed has no error rows, the hidden state is the one measured, and the visible state is measured by the DOM from the direct-fetch split, not by staging rows).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the pins of steps 1–4
|
||||
- Coverage: **>90%** on this task's new/modified code
|
||||
- Real-browser check: step 5's `.cjs` pass, screenshots in `data/logs/`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the four pin groups above are green, with `test_the_ups_events_disclosure_is_a_quiet_disclosure_not_a_new_component` passing unchanged in its original assertions
|
||||
- [ ] the `.cjs` pass is all-green against the live dev server (screenshots in `data/logs/`)
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] `AGENTS.md`'s now-stale lazy-fetch sentences are updated (the phase overview's Documentation & Consistency list) and `README.md` checked
|
||||
@@ -0,0 +1,33 @@
|
||||
# Phase 27 — Mobile: every table scrolls, and the whole app is audited at phone width
|
||||
|
||||
**Source:** `TODO.md` L7 — "None of the tables are horizontally scrollable on mobile. Fix that and make sure everything works on mobile. Do a full mobile compatible audit."
|
||||
**Story:** n/a
|
||||
**Context:** AGENTS.md gap 12 (measured on 2026-10-20): at 360 px, `/`'s fleet table measures 450–510 px inside a 320 px `section.card`, and `.card { overflow: hidden }` clips the tail columns (`WORST DISK`, `GPU BUSY`, `TEMP`, `UP`) with no scroll container to reach them — `table.data` only scrolls with its `.scroll` modifier (`dash.css:884`: `display: block; overflow: auto; max-height: 320px`), which **no page currently uses** (phase 21's `.scroll` on the `/certs` table is still in `todo/` and has not landed). Gap 12 names the fix as "a responsive-table pass (`.scroll` on the wide tables, or a per-host card layout at ≤640px)" and records it as a phase, not a cleanup, because it touches pinned page shape on `/`. The table inventory is fixed and small: two tables in static markup (`index.html:61` `#fleet-table`, `certs.html:85` `#agents-table`) and five in JS-built markup (`page-system.js:194` the GTt table, `:257` the filesystems table, `:269`/`:274` the two top-process tables, all inside the "More detail" disclosure; `page-ups.js:278` the raw NUT-variables table, inside its disclosure). Phase 18's standing guarantee — the document no longer scrolls sideways at a phone width (360 px where it was 750 px) — must survive this phase, so the fix is **inside** the tables, never the document. The ≤640px breakpoint is the repo's mobile boundary (the hamburger, the bar's row-breaks). The browser-proof convention is the `data/logs/*.cjs` scripts (`20-task01-rows.cjs` is the template: `NODE_PATH=<playwright node_modules> node …`, `DASH_BASE`/`DASH_PW`, a `check()` accumulator, non-zero exit on failure, screenshots in `data/logs/`). Phase 26 (scheduled before this one) is what the audit will capture on `/ups`: two cards, the error one hidden-when-empty.
|
||||
|
||||
## Objective
|
||||
|
||||
Every `table.data` on every page scrolls **horizontally inside its own container** when it is wider than the card (a phone's 320 px), with no new document-level sideways scroll, and the whole app — all eight signed-in pages plus `/login` — is audited at 360 px (1280 px as the control) by a re-runnable script whose pass is the phase's proof.
|
||||
|
||||
## Dependencies
|
||||
- `26_ups_event_split` (todo) — the audit captures `/ups` in the shape 26 leaves it in (the two cards and the hidden-when-empty error card are what the 360 px pass must not clip); running 27's audit before 26 lands would measure the pre-split single card.
|
||||
|
||||
## Tasks
|
||||
1. `01_scrollable_tables_on_mobile.md` — the one ≤640px rule that makes every `table.data` horizontally scrollable (horizontal-only, no height cap), pinned on the served CSS
|
||||
2. `02_mobile_audit_script.md` — `data/logs/27-task02-mobile-audit.cjs`: the re-runnable sweep of all nine pages at 360 px and 1280 px — no horizontal document scroll, no clipped table (the gap-12 failure mode), the masthead intact, screenshots
|
||||
3. `03_fix_mobile_issues_found.md` — run the audit, fix every finding, re-run until every page passes both widths; the final screenshot set is the phase's proof
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the served-CSS pin (task 01); the audit script itself is the integration evidence (tasks 02–03) — the repo has no JS test runner, so a `.cjs` pass with screenshots is the established browser-proof shape (invariant 10)
|
||||
- Coverage: **>90%** on new/modified code (the new code is the pin; the change is one CSS rule)
|
||||
- Real-browser check (invariant 10): the task-02 script, run against the restarted dev server, is the phase's verification — not an add-on to it
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass resolves AGENTS.md **gap 12**: the clipping paragraph becomes history ("the gap-12 clipping is closed by `27_mobile_tables_and_audit`: every `table.data` scrolls horizontally inside its own container at ≤640px, measured by `data/logs/27-task02-mobile-audit.cjs`"), the "a phase rather than a cleanup" sentence goes with it, and the mobile-nav paragraphs that quote the 360 px document width are checked for consistency with the audit's no-document-scroll assertion. `README.md` is checked for the same sentence; none is expected.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] at 360 px, every `table.data` wider than its container scrolls horizontally inside the container — the gap-12 fleet-table clipping is gone (pinned + measured by the audit)
|
||||
- [ ] the audit pass is all-green on all nine pages at 360 px and 1280 px: no horizontal document scroll (phase 18's guarantee intact), masthead intact, screenshots in `data/logs/`
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: at 1280 px the tables render exactly as before (no scrollbar where the table fit, no layout shift)
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 01 — Every table scrolls horizontally at the mobile breakpoint
|
||||
|
||||
**Phase:** `27_mobile_tables_and_audit` · **Source:** `TODO.md:7` — "None of the tables are horizontally scrollable on mobile. Fix that…"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
One ≤640px rule makes **every** `table.data` horizontally scrollable inside its own card when it is wider than the card, with no per-table markup change: the five JS-built tables and the two static ones all carry the class already, so a class-level rule reaches all seven at once. The mechanism is the one the existing `.scroll` modifier proves — `display: block` keeps the table's internal layout in an anonymous box, `overflow-x: auto` gives a scrollbar only where the table overflows.
|
||||
|
||||
- ASSUMPTION: the rule is **horizontal-only** (`overflow-x`, not `overflow: auto`) and carries **no** `max-height` — the `.scroll` modifier's 320px height cap is its vertical-use contract, and a phone getting a height-capped table on top of a horizontal one is two new failure modes bought to fix one.
|
||||
- ASSUMPTION: the rule lives in the existing `@media (max-width: 640px)` block (the repo's mobile boundary, the same block the hamburger and the bar's gap live in), applied to `table.data` generally rather than adding the `.scroll` class per table — the per-table class path is what gap 21 does for one table, and seven hand-placed classes are the drift this phase exists to avoid. The all-widths `table.data.scroll` rule (`.css:884`) is untouched.
|
||||
- ASSUMPTION: the `.scroll`-shape pass is taken over gap 12's alternative (a per-host card layout at ≤640px) — a re-layout of `/`'s fleet table into cards touches pinned page shape far more than an overflow rule does, and the operator's word is "horizontally scrollable", which is the `.scroll` shape.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/dash.css` — inside the existing `@media (max-width: 640px)` block (the "mobile nav" section, beside the `.masthead`/`#nav-status` mobile rules), add: `table.data { display: block; overflow-x: auto; }` with a comment stating the contract: horizontal-only, no height cap, every `table.data` (static or JS-built) at the phone breakpoint, the document itself never scrolling sideways (phase 18's guarantee) and the all-widths `table.data.scroll` rule untouched.
|
||||
2. `src/dashboards/web/static/dash.css` — verify the rule reaches the tables inside disclosures: the "More detail" tables (`page-system.js:194/257/269/274`) and the NUT-variables table (`page-ups.js:278`) sit in `display: none` containers until expanded; `overflow-x: auto` on a `display: block` table inside a revealed `details` is the same box model (no extra rule needed — but the audit's task-02 expanded-disclosure leg is what proves it, so do not skip expanding those two disclosures in the script).
|
||||
3. `tests/test_api.py` — pin the served CSS the way `test_the_mobile_masthead_block_declares_no_second_bar_height` reads its block: in the ≤640px block, `table.data` declares `display: block` **and** `overflow-x: auto`, and declares **no** `max-height` (the height cap must not migrate into the mobile rule). The all-widths `table.data.scroll` rule's declarations are asserted unchanged in the same test.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the served-CSS pin above
|
||||
- Coverage: **>90%** on this task's new/modified code (one rule + pin)
|
||||
- The behavioural proof (the gap-12 table actually scrolls at 360 px) is the task-02 audit
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the ≤640px block declares `table.data { display: block; overflow-x: auto }` with no `max-height`, pinned on the served stylesheet
|
||||
- [ ] the `table.data.scroll` rule (`.css:884`) is byte-identical to before
|
||||
- [ ] at 360 px in a browser, `/`'s fleet table scrolls horizontally inside its card (the tail columns `WORST DISK`/`GPU BUSY`/`TEMP`/`UP` reachable) and the document does not scroll sideways (measured by the task-02 audit)
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean
|
||||
- [ ] no behavior change in completed work: at 1280 px no table shows a scrollbar and no layout shifts (measured by the audit's control width)
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 02 — The re-runnable mobile audit script
|
||||
|
||||
**Phase:** `27_mobile_tables_and_audit` · **Source:** `TODO.md:7` — "…make sure everything works on mobile. Do a full mobile compatible audit."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`data/logs/27-task02-mobile-audit.cjs` — the full mobile compatibility audit as a re-runnable script in the repo's established convention (`20-task01-rows.cjs` is the template: `NODE_PATH=<playwright node_modules> node data/logs/27-task02-mobile-audit.cjs`, `DASH_BASE`/`DASH_PW` env with the dev defaults, a `check(name, ok, detail)` accumulator, non-zero exit on any failure). It sweeps **all nine pages** — the eight signed-in pages (`/`, `/system`, `/disks`, `/llm`, `/ups`, `/settings`, `/certs`) plus `/login` (unsigned, the only anonymous page) — at **360 px** (the phone width gap 12 measured at) with **1280 px** as the control, and its all-green pass is the phase's proof.
|
||||
|
||||
## Work
|
||||
|
||||
1. `data/logs/27-task02-mobile-audit.cjs` — setup: sign in at `DASH_BASE` (the `20-task01-rows.cjs` sign-in pattern, `testpass` default), then per page × width, the check set:
|
||||
- **No horizontal document scroll** (phase 18's standing guarantee — this phase must not break it): `document.documentElement.scrollWidth <= window.innerWidth + 1` and `document.body.scrollWidth <= window.innerWidth + 1`.
|
||||
- **No clipped table** (the gap-12 failure mode, the one this phase exists for): for every `table.data`, if `el.scrollWidth > el.clientWidth + 1` then the computed `overflow-x` is `auto` or `scroll` (the table scrolls **inside** its own container); a wide table whose `overflow-x` is `visible` (clipped by the card's `overflow: hidden`) fails with the table's id. At 1280 px the same check runs as the control (a desktop table that overflows — none should — would fail there too).
|
||||
- **Every element fits the viewport**: a generic overflow scan — no element with `getBoundingClientRect().right > window.innerWidth + 1` other than a `table.data` inner box already accounted for (a hero row, chart or card spilling off the phone is the "make sure everything works on mobile" half the table check does not name).
|
||||
- **Masthead intact** (360 px leg): the three-row shape by bounding-box grouping (the `20-task01-rows.cjs` `PROBE()`), every control's box inside the viewport, and the ≤640px dropdown still opening directly under the bar (open `#nav-toggle`, measure `nav.tabs`'s top against the bar's bottom, close it).
|
||||
- **Expanded-disclosure leg** (360 px): on `/system` expand one host's "More detail" `details` and on `/ups` open the NUT-variables disclosure, re-run the table check on the newly revealed tables (the `display: none` containers of task 01's step 2), then close them.
|
||||
- **`/ups` two-card leg** (360 px, phase 26's shape): the error card (when present) and the disclosure card both fit, the error card's `hidden` state matching the feed as task 26 pinned.
|
||||
- Screenshots: `data/logs/mobile-27-<page>-<width>.png` per page × width (nine × two = eighteen, plus the two expanded-disclosure states), each `check()` line printed and accumulated.
|
||||
2. `data/logs/27-task02-mobile-audit.cjs` — the 1280 px control leg doubles as the "no behavior change" proof: the same element-fits and table checks at desktop width, so a rule that shifts a desktop table (a spurious `display: block` at 1280 px, a layout shift) fails the control as loudly as the phone leg fails the phone.
|
||||
3. The script exits non-zero on any failure and prints the failure lines (the `20-task01-rows.cjs` convention); it is **run twice** as part of this task: once as written (expecting the pre-task-03 findings, if any, to surface) and once after task 03's fixes — the first run's output is the findings list task 03 works from, and both runs' output is the phase's evidence. (The script is in gitignored `data/logs/` — evidence, not a tracked pin; the tracked pin is task 01's served-CSS one.)
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the script's check set is the integration evidence; task 01's served-CSS pin is the tracked half
|
||||
- Coverage: **>90%** on new/modified code (the script is evidence, not shipped code — no coverage line of its own, the pin carries the gate)
|
||||
- Real-browser check: this **is** the real-browser check (invariant 10), run against the restarted dev server (`./scripts/dev-server.sh restart` first) with the live fleet answering so the tables have real rows
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the script exists, runs in the documented one-liner, signs in, sweeps nine pages × two widths, and exits non-zero on any failure
|
||||
- [ ] its check set covers: document scroll, per-table clipping (the gap-12 mode), the generic element-fits scan, the masthead + dropdown at 360 px, the two expanded-disclosure legs, and the `/ups` two-card leg
|
||||
- [ ] the first run's output is the findings list (task 03's input) and its screenshots are in `data/logs/`
|
||||
- [ ] `uv run pytest` green, `ruff` + `pyright` clean (the script is `node --check`-clean — `bash .agents/validate.sh`'s node check covers the `web/static` scripts, and this one is checked by hand as part of the run)
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 03 — Fix every finding until the audit is all-green at both widths
|
||||
|
||||
**Phase:** `27_mobile_tables_and_audit` · **Source:** `TODO.md:7` — "…make sure everything works on mobile. Do a full mobile compatible audit."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The audit's first run (task 02) is the findings list; this task closes it. Every finding is a CSS/markup fix — the data layer, the API and the page scripts' logic are out of scope (a table that does not fit is a layout fact, and the fix is the layout, never the data) — and the phase is done when the audit passes **every page at both widths** with the screenshots in `data/logs/`.
|
||||
|
||||
## Work
|
||||
|
||||
1. Run the task-02 script against the restarted dev server and work the findings list top down. The expected findings, in the order the audit surfaces them:
|
||||
- **`/` fleet table** (gap 12, the one the phase exists for): task 01's rule is the fix — verify the tail columns (`WORST DISK`, `GPU BUSY`, `TEMP`, `UP`) are reachable by scrolling the table, not the document. If the rule is already in place from task 01 and the table still clips, the card's `overflow: hidden` is clipping the table's **scrollbar track**: the fix is `table.data` keeping `display: block; overflow-x: auto` (task 01's pin) — a `card`-level change is the fallback, and it needs a pin of its own if used.
|
||||
- **`/certs` agents table**: same rule, same check; note phase 21 (still `todo/`) is scheduled to add the `.scroll` class to this table — this task's rule makes the class redundant at ≤640px but harmless at all widths, so 21 landing later is a no-op collision, not a conflict (state that in the fix note if the table is already covered).
|
||||
- **`/system` "More detail" tables** (filesystems, top processes, GTt): the expanded-disclosure leg of the audit; a table inside the `details` that still clips gets the same class-level fix (no per-table markup — the class is already there).
|
||||
- **`/ups` raw NUT-variables table**: same, through its disclosure.
|
||||
- **Anything the generic element-fits scan names that is not a table** (a hero row, a chart, a card spilling off the phone): fix the specific element — the fix is named per element in the findings note, and a fix that hides a control to make the width work is a refusal (the phase-18 rule: nothing is hidden to make the arithmetic work).
|
||||
2. `tests/test_api.py` — for every finding whose fix is a **markup** change (an `id` added, a wrapper introduced, a class added to a static table), add the served-byte pin in the same pass; a CSS-only fix is covered by task 01's pin. No pin is added for a fix that only touches `data/logs/` evidence.
|
||||
3. Re-run the task-02 script until **every check on all nine pages passes at 360 px and 1280 px**; the final run's output and its eighteen-plus screenshots are the phase's proof (the final-run screenshot set supersedes the first run's in `data/logs/`).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: task 01's served-CSS pin + any new markup pin from step 2
|
||||
- Coverage: **>90%** on new/modified code
|
||||
- Real-browser check: the final all-green audit run (invariant 10), the screenshots in `data/logs/`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the audit is all-green on all nine pages at 360 px and 1280 px, with the final-run screenshots in `data/logs/`
|
||||
- [ ] every finding from the first run is closed by a named fix (CSS/markup only) or a stated refusal (a control the operator needs, not hidden)
|
||||
- [ ] the gap-12 fleet-table clipping is gone: the tail columns reachable by the table's own scrollbar, the document not scrolling sideways (measured)
|
||||
- [ ] `AGENTS.md` gap 12 is resolved per the phase overview's Documentation & Consistency list, and `README.md` checked
|
||||
- [ ] `uv run pytest` green, coverage not down, `ruff` + `pyright` clean
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the 1280 px control leg is all-green (no desktop layout shift, no spurious scrollbars)
|
||||
@@ -0,0 +1,39 @@
|
||||
# Phase 28 — The agent reads the gateway's proxy configs and its live sockets
|
||||
|
||||
**Source:** `TODO.md` L3 — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs. I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)." (the agent half)
|
||||
**Story:** n/a
|
||||
**Context:** `agent/host_agent.py` (the stdlib-only single-file agent, `Collector._safe` best-effort sections, `snapshot()`, the SCHEMA docstring, `AGENT_VERSION` 1.1.0, the `--no-smart` opt-out and the three-way availability precedent it sets), `tests/test_agent.py` (the in-process harness over a fake `root` tree, the real-`/proc` test, the smartctl-capture fixture pins), `scripts/capture-fixtures.sh` (the ssh-capture convention: plain command first, `sudo -n` fallback, refuse to overwrite good fixtures with unreadable output), `tests/fixtures/` (the verbatim-capture ground-truth directory), the gateway host's live state verified 2026-10-21 (read-only): `nginx` and `haproxy` both `active`, `/etc/haproxy/haproxy.cfg` (6 `frontend` blocks — `bind :::22 v4v6` et cetera — each with one `default_backend`; 6 `backend` blocks, each one `server <name> <domain>:<port> check inter 5s`), `/etc/nginx/nginx.conf` (a catch-all `listen 80` / `server_name _` block plus `include /etc/nginx/conf.d/*.conf;` and `include /etc/nginx/stream.d/*.conf;`), `/etc/nginx/conf.d/http.conf` (server blocks: `listen 4443 ssl proxy_protocol` / `listen 8443 ssl`, `server_name <domain>`, `location / { proxy_pass http://<domain>:<port>; }`), `/etc/nginx/stream.d/stream.conf` (a `map $ssl_preread_server_name $targetBackend { … }` block and `proxy_pass $targetBackend` with `ssl_preread on`), all three files world-readable and readable by the `dashboards-agent` user, `/proc/net/tcp` readable the same way. The dashboard side (source, wiring, page) is phase 29; this phase makes the data exist on the wire.
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` 1.2.0 serves three new **additive** snapshot sections on the gateway host — `haproxy` (frontends + binds, backends + server targets, the frontend→backend routing), `nginx` (server blocks with `listen` + `server_name`, `upstream`, `proxy_pass` including the `$var`-to-`map` join, `map` blocks) and `sockets` (LISTEN + ESTABLISHED from `/proc/net/tcp{,6}`) — each best-effort (a missing config degrades its own section, never the snapshot), `--no-proxy` opting out silently, the real configs captured verbatim as fixtures, the rollout onto the gateway host done, and the sections proven live over mTLS. Nothing changes for token-mode or pre-1.2.0 consumers: an old agent's snapshot is byte-identical in every old field.
|
||||
|
||||
## Dependencies
|
||||
- `27_mobile_tables_and_audit` (todo) — not a code dependency (different files); ordered after it so the fleet rollout in task 05 does not race the mobile-audit's browser passes on the shared dev box
|
||||
- `15_agent_mtls` / `24_dev_mtls_rollout` (complete) — the installed-bundle install path (`deploy/install-agent.sh --cert-bundle`), the `--status` CA-only probe, and the gap-10 re-stage-first rule the rollout follows
|
||||
|
||||
## Tasks
|
||||
1. `01_haproxy_section.md` — the agent reads + parses `haproxy.cfg` into the additive `haproxy` section, three-way availability, `--no-proxy` opt-out, never fails the snapshot
|
||||
2. `02_nginx_section.md` — the agent reads the main config + one level of `include <glob>` (bounded) into the additive `nginx` section, with the `proxy_pass $var` → `map` join
|
||||
3. `03_socket_section.md` — the agent parses `/proc/net/tcp` + `/proc/net/tcp6` (bounded) into the additive `sockets` section: LISTEN + ESTABLISHED with local/remote ip:port
|
||||
4. `04_version_and_capture.md` — `AGENT_VERSION` 1.2.0, the SCHEMA docstring, `scripts/capture-gateway-config.sh` (real configs → `tests/fixtures/`), the live probe that the sections answer on the gateway host
|
||||
5. `05_rollout_gateway.md` — re-stage `/home/ops/agent-install/`, install 1.2.0 on the gateway host, restart the unit, verify the sections over mTLS — read-only live verification, transcript in `data/logs/`
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_agent.py` — every new section over a fake `root` tree (present config → parsed shape; missing file → `available: false` + exactly one errors line; unreadable → `null` + one errors line; `--no-proxy` → sections absent and no errors line; a corrupt config degrades its own section and the snapshot still succeeds with the other sections intact)
|
||||
- Fixture: `tests/fixtures/gateway_haproxy.cfg` + `gateway_nginx_*.conf` + `gateway_proc_net_tcp{,6}.txt` are verbatim captures from the gateway host (invariant 7); the parser tests run against them, so a config drift fails a test before a dashboard does
|
||||
- Coverage: **>90%** on the new/modified code (the three collectors + the parsers)
|
||||
- Live (read-only, invariant 10): task 04's probe over the mTLS dial, and task 05's post-restart verification with its transcript
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass (task 05) updates `AGENTS.md`'s agent bullet (the new sections, `AGENT_VERSION` 1.2.0, the `--no-proxy` line) and the `SCHEMA` docstring is the self-documenting half on the wire. `README.md` is checked for any agent-version sentence.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (the hermetic suite, no network), coverage not down
|
||||
- [ ] a 1.2.0 agent over a captured-config root tree answers all three sections; over a tree without them it answers `available: false`/`null` with one errors line each and the snapshot's other sections are intact
|
||||
- [ ] `AGENT_VERSION` is 1.2.0 and the SCHEMA docstring names the three new sections
|
||||
- [ ] the fixture files are verbatim captures (the capture script's refusal-to-overwrite-with-unreadable pin holds)
|
||||
- [ ] the gateway host runs 1.2.0 (`systemctl is-active dashboards-agent` → `active`, the file's `AGENT_VERSION` reads 1.2.0) and the sections answer over mTLS — transcript in `data/logs/`
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: a 1.2.0 agent's snapshot is identical to 1.1.0's in every pre-existing field for a non-gateway host (the new keys are the only difference, and `--no-proxy` removes even those)
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 01 — The agent reads and parses the haproxy config
|
||||
|
||||
**Phase:** `28_gateway_agent_proxy` · **Source:** `TODO.md:3` — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs…"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` gains a `haproxy` snapshot section: the agent reads the haproxy config (path an argument, default `/etc/haproxy/haproxy.cfg`), parses its `frontend` blocks (`name`, each `bind` address:port, `default_backend`/`use_backend` references) and `backend` blocks (`name`, each `server` line's name + target host:port), and serves them as a new additive section. The section follows the smart precedent exactly: a missing file is `available: false` plus **one** errors line for the pass, a file that cannot be read or parsed is `available: null` plus one errors line, `--no-proxy` opts out silently, and no failure of this section ever fails the snapshot or `/health`.
|
||||
|
||||
- ASSUMPTION: the parsed shape is `haproxy: {available: true|false|null, config: str (the path), frontends: [{name, binds: [{addr, port}], default_backend: str|null}], backends: [{name, servers: [{name, target, port}]}]}` — the frontend→backend routing is a name the **dashboard** joins (sources/gateway.py, phase 29), the agent only reports what the config says.
|
||||
- ASSUMPTION: `bind` lines parse the last token's `addr:port` (the captured shape is `bind :::22 v4v6` — the address token is the one that contains `:port` after the word `bind`; a `bind` line without a parsable address:port is skipped with no error, because haproxy allows non-address bind forms and the section degrades to fewer binds rather than failing).
|
||||
- ASSUMPTION: `server` lines parse `server <name> <target>[:port] <rest>` — `check inter 5s` and the like are the ignored rest; a `target` without a port (a bare upstream name) is kept with `port: null` (absent is null, never 0 — the module's own rule).
|
||||
- ASSUMPTION: `use_backend <name>` lines inside a frontend are collected as `backends: [str]` on that frontend (the captured config uses `default_backend` only; the parser covers the captured shape plus `use_backend` because haproxy answers two names for the same routing fact, the way `resolve_command()` does for NUT verbs); unknown directives are ignored, never errors.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — module docstring: add the three-way availability + `--no-proxy` rule to the design-rules section (the "SMART is a best-effort add-on" shape, adapted: one errors line for a missing binary becomes one errors line for a missing/unreadable config).
|
||||
2. `agent/host_agent.py` — `Collector.__init__`/arguments: a `haproxy_config` path argument (default `/etc/haproxy/haproxy.cfg`) and the `--no-proxy` flag on the CLI (the `--no-smart` precedent: a flag that removes the section entirely, no errors line), threaded into `snapshot()`.
|
||||
3. `agent/host_agent.py` — `Collector.haproxy()` + a module-level `parse_haproxy_config(text) -> dict` (the `parse_smart_output` idiom: a pure function over text so tests feed it fixtures without a filesystem): the block scanner (a `frontend`/`backend` line starts a block, an indented line belongs to it, a top-level line ends it) and the `bind`/`server`/`default_backend`/`use_backend` token parsers per the assumptions above.
|
||||
4. `agent/host_agent.py` — `snapshot()`: `self._safe(self.haproxy, "haproxy", errors)` added to the returned document (additive key; a 1.1.0 consumer sees no difference, a 1.2.0 consumer with `--no-proxy` sees no `haproxy` key at all — the same absence the smart section takes).
|
||||
5. `tests/test_agent.py` — the in-process harness cases: the captured-shape fixture parses to the exact expected frontends/backends (assert the `:::22` → `addr: "::"`, `port: 22` split and the six `default_backend` names); a `root` tree without the file → `available: false` + exactly one errors line naming the missing path (the smartctl "not found" pin's shape — the line must still say the file is missing, `tests/test_agent.py` pins that wording); an unreadable file (a mode-000 file in the fake tree, where the harness allows it) → `available: null` + one errors line; a corrupt file (garbage text) → `available: null` + one errors line and the snapshot's `cpu`/`memory` sections still intact; `--no-proxy` → no `haproxy` key and no errors line.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the five harness cases above + `parse_haproxy_config` over `tests/fixtures/gateway_haproxy.cfg` (task 04 records it; write the test against the captured shape now — the real file's content is known: 6 frontends, 6 backends, the exact `bind :::NNNN v4v6` and `server <name> <domain>:<port> check inter 5s` lines)
|
||||
- Coverage: **>90%** on `parse_haproxy_config` + `Collector.haproxy`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/test_agent.py -q` green, including the new haproxy cases and every pre-existing agent test
|
||||
- [ ] a snapshot over a tree with the config carries the `haproxy` section with all six frontends and six backends; over a tree without it, `available: false` + one errors line; the snapshot never fails because of it
|
||||
- [ ] `--no-proxy` removes the section with no errors line (pinned)
|
||||
- [ ] `uv run ruff check agent tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: every pre-existing snapshot field byte-identical on a host without haproxy (the `haproxy` key is the only addition)
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 02 — The agent reads and parses the nginx configs
|
||||
|
||||
**Phase:** `28_gateway_agent_proxy` · **Source:** `TODO.md:3` — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs…"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` gains an `nginx` snapshot section: the agent reads the main nginx config (path an argument, default `/etc/nginx/nginx.conf`) **and follows its `include <path-or-glob>` lines one level** (the captured main config includes `/etc/nginx/conf.d/*.conf` and `/etc/nginx/stream.d/*.conf`), parses `server` blocks (`listen` address:port pairs, `server_name`), `upstream` blocks, `proxy_pass` directives (a literal `scheme://host:port` **and** a `proxy_pass $var` whose target the agent resolves against the file's `map` blocks), and the `map` blocks themselves. Same three-way availability and best-effort rules as task 01: missing main file → `available: false` + one errors line; unreadable/unparseable → `null` + one line; `--no-proxy` removes the section; the snapshot never fails because of it.
|
||||
|
||||
- ASSUMPTION: the parsed shape is `nginx: {available: true|false|null, config: str, files: [str (main + the included paths actually read)], servers: [{listen: [{addr, port}], server_names: [str], proxy_passes: [{location: str, scheme: str|null, target: {host, port}|null, variable: str|null}]}], upstreams: [{name, servers: [{host, port}]}], maps: [{variable: str, entries: [{key, target: {host, port}|null}]}]}` — a `proxy_pass $targetBackend` whose `$targetBackend` matches a `map` block's variable is served with `variable` set and `target: null` (the map decides per-connection at runtime; the **dashboard** joins the variable to the map, the agent reports both facts and never merges them, because a merged target would be a number the wire did not publish).
|
||||
- ASSUMPTION: include-following is **one level, bounded** — `include` lines in the main config are read (glob expanded), but `include` lines inside an included file are not followed (the captured tree is one level deep; a deeper include is a config the operator changed, and the section degrades to `files` naming only what was read rather than erroring). Bounds: at most 8 included files and 1 MB of included text combined; exceeding a bound drops the excess includes with one errors line (the smart pass-budget shape: the pass reports what it did not cover) and the section stays `available: true`.
|
||||
- ASSUMPTION: `listen` parsing takes the first token after `listen` (the captured shapes: `listen 4443 ssl proxy_protocol;`, `listen [::]:4443 ssl;`, `listen 80;`) — `host:port`, `[host]:port`, or bare `port` (→ `addr: null`); the `ssl`/`proxy_protocol`/`http2` flags are not parsed (they change how the bytes travel, not what the page's pipes show).
|
||||
- ASSUMPTION: `server_name _` (the catch-all) is kept as the literal `_` entry — the dashboard renders it as "any domain", the agent does not interpret it.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — `Collector.__init__`/arguments: a `nginx_config` path argument (default `/etc/nginx/nginx.conf`), threaded through like `haproxy_config`; `--no-proxy` covers this section too (it already exists from task 01).
|
||||
2. `agent/host_agent.py` — a module-level `parse_nginx_config(main_text, included: dict[str, str]) -> dict` (pure over text, the `parse_haproxy_config` idiom) with: a brace-block scanner (nginx directives nest by `{}`, unlike haproxy's indent — the scanner is the task's core); `server`/`upstream`/`map` block extraction; the `listen`/`server_name`/`proxy_pass`/`include`/`server`-inside-upstream token parsers per the assumptions above.
|
||||
3. `agent/host_agent.py` — `Collector.nginx()`: read the main config through the existing `read_text` (which returns `None` on unreadable — the section's `null` arm), expand each `include` line (a glob through `glob.glob`, the literal path otherwise), apply the 8-file/1 MB bounds, call the parser, and return the section dict with `files` naming the main + each included path actually read.
|
||||
4. `agent/host_agent.py` — `snapshot()`: `self._safe(self.nginx, "nginx", errors)` added to the returned document (additive, task 01's shape).
|
||||
5. `tests/test_agent.py` — the in-process harness cases: the captured main + `conf.d/http.conf` + `stream.d/stream.conf` (as a fake `root` tree under `etc/nginx/`) parse to the expected servers (assert the `4443`/`8443` listen ports, the `server_name` list, the `proxy_pass` targets including the `127.0.0.1:8081` literal), the `stream.conf` `map` block (assert the variable name `$targetBackend` and a representative entry), and the `proxy_pass $targetBackend` served with `variable: "targetBackend"` + `target: null`; a tree without the main file → `available: false` + one errors line; a corrupt main file → `null` + one line, snapshot intact; the include bound (a main config including 9+ files) → `available: true`, `files` the first 8, one errors line naming the dropped count.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the five harness cases above + `parse_nginx_config` unit cases for the token parsers (`listen [::]:4443` → `addr: "::"`, bare `80` → `addr: null`, a `proxy_pass http://host:port` with a scheme, an `upstream` block's `server` lines)
|
||||
- Coverage: **>90%** on `parse_nginx_config` + `Collector.nginx`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/test_agent.py -q` green, including the new nginx cases and every pre-existing agent test
|
||||
- [ ] a snapshot over the captured-config tree carries the `nginx` section with every `server` block, the `map`, and the variable-form `proxy_pass` served unmerged
|
||||
- [ ] the one-level include bound is pinned (9th+ include dropped with one errors line, section still `available: true`)
|
||||
- [ ] `uv run ruff check agent tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: every pre-existing snapshot field byte-identical on a host without nginx
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 03 — The agent reads the live sockets
|
||||
|
||||
**Phase:** `28_gateway_agent_proxy` · **Source:** `TODO.md:3` — "I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)." (the live half — the configs say *where* the pipes go; this task says *what is flowing now*)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` gains a `sockets` snapshot section: the agent parses `/proc/net/tcp` and `/proc/net/tcp6` (both readable by the agent user, verified on the gateway host) into the LISTEN sockets (the gateway's open frontends, with their local address:port) and the ESTABLISHED sockets (local and remote address:port — the live end-to-end pairs the dashboard joins to the config). Same best-effort rules: an unreadable `/proc/net/tcp` degrades the section to `null` with one errors line, `--no-proxy` removes it, and the snapshot never fails because of it. The section is a raw fact list — **no** config join happens in the agent (a socket does not know its domain; the dashboard's `sources/gateway.py` does the join, phase 29).
|
||||
|
||||
- ASSUMPTION: the parsed shape is `sockets: {available: true|false|null, listens: [{family: "v4"|"v6", addr: str|null, port: int}], established: [{family, local: {addr, port}, remote: {addr, port}}]}` — `addr` is the dotted/colon form (v4) or the expanded `x:x:…` form (v6, the `/proc` 4-byte-group little-endian spelling unrolled); a `0.0.0.0`/`::` listener keeps its literal wildcard address (it is the fact the dashboard's inbound join keys on — the port, with the wildcard addr marking "any address").
|
||||
- ASSUMPTION: only the two states the page needs are reported — `0A` (LISTEN) into `listens`, `01` (ESTABLISHED) into `established` — every other state is skipped (a TIME_WAIT ocean is noise the pipes do not show, and skipping it keeps the section small).
|
||||
- ASSUMPTION: the read is bounded at 4096 lines per file (the header line excluded from the count); a file longer than the bound reports what it read and one errors line naming the truncation (the smart pass-budget shape), `available` stays `true`.
|
||||
- ASSUMPTION: the hex→address conversion is a pure module-level `parse_proc_net_tcp(text, family) -> (listens, established)` (the `parse_smart_output` idiom), so the captured-`/proc` fixtures test the parser without a live filesystem.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — module docstring: the `sockets` design line (two states only, the 4096-line bound, the best-effort rule) beside the smart one.
|
||||
2. `agent/host_agent.py` — module-level `parse_proc_net_tcp(text: str, family: str) -> tuple[list[dict], list[dict]]`: the column split (the `/proc/net/tcp` columns: `sl local_address rem_address st …`; `local_address`/`rem_address` are `HEXADDR:HEXPORT`, v4 a 32-char hex little-endian word, v6 four 32-char groups in the same order), the `st` filter (`01`/`0A` only), and the address unrolling per the assumptions above.
|
||||
3. `agent/host_agent.py` — `Collector.sockets()`: read `/proc/net/tcp` and `/proc/net/tcp6` through `read_lines` (an unreadable file is `None` → the `null` arm; both unreadable → `null` + one combined errors line), apply the line bound, call the parser, and return the section dict.
|
||||
4. `agent/host_agent.py` — `snapshot()`: `self._safe(self.sockets, "sockets", errors)` added to the returned document (additive, tasks 01/02's shape).
|
||||
5. `tests/test_agent.py` — the parser unit cases over a captured `/proc/net/tcp` fragment (a LISTEN on `:::443` and one on `0.0.0.0:80`, an ESTABLISHED v4 pair, an ESTABLISHED v6 pair, a TIME_WAIT line asserted **absent** from both lists) + the harness cases: a fake tree whose `proc/net/tcp{,6}` carry the fragment → the section's exact lists; a tree without the files → `null` + one errors line, snapshot intact; the 4096-line bound (a 5000-line file) → `available: true`, the first 4095 data rows, one errors line naming the truncation.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the parser unit cases + the three harness cases above
|
||||
- Coverage: **>90%** on `parse_proc_net_tcp` + `Collector.sockets`
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/test_agent.py -q` green, including the new socket cases and every pre-existing agent test
|
||||
- [ ] a snapshot over a tree with the captured `/proc` files carries the `sockets` section with the exact LISTEN/ESTABLISHED lists; the TIME_WAIT line is absent (pinned)
|
||||
- [ ] the 4096-line bound is pinned (truncation line, section still `available: true`)
|
||||
- [ ] `uv run ruff check agent tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: every pre-existing snapshot field byte-identical on a host where `/proc/net/tcp` is unreadable (the section is `null`, nothing else moved)
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 04 — Agent 1.2.0, the SCHEMA, and the gateway-config captures
|
||||
|
||||
**Phase:** `28_gateway_agent_proxy` · **Source:** `TODO.md:3` — the ground-truth half: invariant 7 (verify against the deployed upstream; never guess a metric name) applied to the three new sections
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The agent's version and self-documentation catch up with tasks 01–03 (`AGENT_VERSION` 1.2.0, the SCHEMA docstring names the three sections), and the capture script records the gateway host's real `haproxy.cfg`, nginx configs, and `/proc/net/tcp{,6}` as verbatim fixtures — so the parser tests run against the deployed shape, not a memory of it, and a config drift fails a test before a dashboard does.
|
||||
|
||||
- ASSUMPTION: the fixtures **keep the real service domain names** (e.g. `nextcloud.reeseapps.com`) — unlike the smartctl capture's Serial/WWN scrub, the domains are the data the `/gateway` page exists to show (inbound domain, outbound target) and the page is session-gated; a scrubbed fixture would be a fixture that tests a config the gateway does not run. The capture script therefore scrubs nothing; what it refuses is the other direction (overwriting a good fixture with unreadable output, the `capture-fixtures.sh` rule).
|
||||
- ASSUMPTION: `AGENT_VERSION` goes 1.1.0 → 1.2.0 (a new additive section is a minor bump, the 1.0.0 → 1.1.0 TLS bump's shape); the bundle's `# BUNDLE-REV` is untouched (the agent file is not the bundle — no CA fact moved).
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — `AGENT_VERSION = "1.2.0"` and the SCHEMA docstring: the three new section shapes (the `haproxy`/`nginx`/`sockets` blocks, one line each, the way `smart` is documented) plus the three-way availability + `--no-proxy` note.
|
||||
2. `scripts/capture-gateway-config.sh` — a new capture script (the `capture-fixtures.sh` convention, ssh-alias driven: `GATEWAY_SSH` default `gateway-ops`, `ssh` alias `gateway-ops`): scp/capture `/etc/haproxy/haproxy.cfg` → `tests/fixtures/gateway_haproxy.cfg`; `/etc/nginx/nginx.conf`, `/etc/nginx/conf.d/*.conf`, `/etc/nginx/stream.d/*.conf` → `tests/fixtures/gateway_nginx_main.conf` / `gateway_nginx_conf_d_*.conf` / `gateway_nginx_stream_d_*.conf`; `cat /proc/net/tcp /proc/net/tcp6` → `tests/fixtures/gateway_proc_net_tcp.txt` / `gateway_proc_net_tcp6.txt`. The `capture-fixtures.sh` refusal rule: a capture that comes back empty or with an ssh/permission error is **not** written over an existing fixture (one line naming the failure, exit 1). No sudo (all five reads are world-readable by the agent user — verified 2026-10-21).
|
||||
3. Run the capture once (the operator's box reaches the gateway host) and commit the fixtures; `tests/test_agent.py`'s task 01–03 cases already assert against the captured shape.
|
||||
4. `tests/test_deploy_scripts.py` — pin the script the way `capture-fixtures.sh` is pinned: it dials the ssh alias (not a hardcoded IP), scrubs nothing (a domain grep over its own source asserts no scrub step exists), and the refusal-to-overwrite branch is named.
|
||||
5. Live probe (read-only, invariant 10): `ssh gateway-ops` — `systemctl is-active dashboards-agent`, then a `python3` one-liner (the `--status` probe idiom, CA-only over the installed bundle's CA) that dials the **current** 1.1.0 agent and confirms the old sections still answer (the no-behavior-change control), and confirms the new sections are **absent** from the 1.1.0 snapshot (the three-way `predates` arm's ground truth). The probe writes `data/logs/28-task04-probe.out`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_deploy_scripts.py` pins for the capture script; the existing fixture-pinned parser tests (tasks 01–03) run against the real captures
|
||||
- Coverage: **>90%** on the new script's Python-side branches (the script is bash; its pins are the `test_deploy_scripts.py` shape)
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `AGENT_VERSION` reads `1.2.0` and the SCHEMA docstring names all three new sections (pinned by a `tests/test_agent.py` assert on the constant)
|
||||
- [ ] `bash scripts/capture-gateway-config.sh` (or the operator's run) leaves the seven fixture files in `tests/fixtures/`, verbatim, with the refusal rule holding
|
||||
- [ ] `tests/test_agent.py` green against the real captures (a config drift in the fixture fails a test)
|
||||
- [ ] `data/logs/28-task04-probe.out` records: the gateway's installed agent is 1.1.0, the old sections answer, the three new sections are absent
|
||||
- [ ] `uv run ruff check scripts` (if the script has a Python leg) / `bash -n scripts/capture-gateway-config.sh` clean, `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the 1.1.0 agent's served snapshot is unchanged (the probe proves it)
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 05 — Roll 1.2.0 onto the gateway host, verify over mTLS
|
||||
|
||||
**Phase:** `28_gateway_agent_proxy` · **Source:** `TODO.md:3` — the deployment half: the sections only exist on the wire once the gateway host runs 1.2.0 (the phase-24 rollout's per-host safety order, gap-10's re-stage-first rule)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The gateway host runs `host_agent.py` 1.2.0 with the three new sections live over mTLS, installed by the same granted-path procedure the fleet uses — re-stage `/home/ops/agent-install/` first (the gap-10 rule: a path grant is only as current as the file it points at), run the installer, restart the unit, and verify the sections answer. The install is **read-only on the dashboard's side** (no config change, no `HOST_TARGETS` change — the gateway is already in `HOST_TARGETS` and dials fine on 1.1.0), and the verification transcript lands in `data/logs/`.
|
||||
|
||||
- ASSUMPTION: the gateway host's unit is **unchanged by this rollout** — `--no-proxy` is not passed (the sections are on by default), so the installer's re-run guard sees an identical unit render and stays silent (the phase-17 task-03 silent path); no `--force` needed.
|
||||
- ASSUMPTION: the restart is the `systemctl restart dashboards-agent` the grant's `systemctl restart *` line allows (the gateway's `ops` has `(ALL) NOPASSWD: ALL` — the fleet record's `deploy/agent-sudoers` shape).
|
||||
|
||||
## Work
|
||||
|
||||
1. Re-stage: `ssh gateway-ops` — copy the repo's `agent/host_agent.py` (1.2.0, md5 recorded) and `deploy/install-agent.sh` (the repo's current sha256, recorded) into `/home/ops/agent-install/` (the directory is `ops`-owned, no root needed — the phase-24 task-03 procedure), leaving any existing snapshot beside it as `<name>-<date>-backup` (a file sudo will not run).
|
||||
2. Install: run the granted path `sudo /home/ops/agent-install/deploy/install-agent.sh --bind 10.1.1.27 --port 9120` (the gateway's existing unit arguments — read them first with `systemctl cat dashboards-agent` and reproduce them exactly, plus `--cert-bundle` if the unit has the line) — the re-run guard prints nothing (identical unit) and the agent file is updated in place.
|
||||
3. Restart: `sudo systemctl restart dashboards-agent`; `systemctl is-active dashboards-agent` → `active`.
|
||||
4. Verify (read-only): the `--status` CA-only probe (the `python3` `ssl` one-liner over the installed bundle's CA, `--bind 10.1.1.27 --port 9120`) answers; then a `python3` one-liner that dials over mTLS (the dashboard's client cert, the stored CA — the `tests/test_live_fleet.py` dial idiom) and asserts: `agent_version` is `1.2.0`, the three new keys are present, `haproxy.available` is `true` with 6 frontends, `nginx.available` is `true` with ≥ 9 server blocks and the `$targetBackend` map, `sockets.available` is `true` with a non-empty `listens` (port 443 among them), and the pre-existing sections (`cpu`, `memory`, `network`) still answer.
|
||||
5. `data/logs/28-task05-rollout.md` — the transcript: the md5/sha256 of what was staged where, the `systemctl cat` before/after (diff empty), the probe output, the one-liner's assertions, and the `haproxy`/`nginx`/`sockets` section summaries (counts only, no config text — invariant 6: no config body in the note).
|
||||
|
||||
## Testing & Quality
|
||||
- Live (read-only, invariant 10): the verification one-liner is the phase's proof; no dashboard process is touched, no `HOST_TARGETS` line moves
|
||||
- The hermetic suite is untouched by this task (no code change) — `uv run pytest` is re-run as the gate, not for this task's logic
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the gateway host's installed `host_agent.py` is the repo's 1.2.0 (md5 matches) and `systemctl is-active dashboards-agent` → `active`
|
||||
- [ ] the `--status` CA-only probe answers; the mTLS one-liner asserts all five facts (version, three sections present + available, pre-existing sections intact)
|
||||
- [ ] `data/logs/28-task05-rollout.md` exists with the transcript and no config text
|
||||
- [ ] the dashboard's `host:gateway` / `disks:gateway` jobs still read `ok` in `./scripts/dev-server.sh status` (the 1.2.0 snapshot's new keys cost the old sources nothing)
|
||||
- [ ] `uv run pytest` green, `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the gateway's pre-existing snapshot fields are unchanged (the one-liner's control assertions prove it)
|
||||
@@ -0,0 +1,38 @@
|
||||
# Phase 29 — The gateway dashboard page: animated pipes from config + live sockets
|
||||
|
||||
**Source:** `TODO.md` L3 — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs. I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)." (the dashboard half)
|
||||
**Story:** n/a
|
||||
**Context:** phase 28 (the `haproxy`/`nginx`/`sockets` agent sections, `AGENT_VERSION` 1.2.0, the gateway host rolled out), `src/dashboards/config.py` (`Settings`, `_env`, the `nut_enabled`/`actions_allowed` gate properties), `src/dashboards/sources/disks.py` (the "a source of its own reading the same agents" precedent — the `disks:<host>` job dials the same `/snapshot`), `src/dashboards/poller.py` (job loop, health records, stale/last-good), `src/dashboards/app.py` (`build_hub` — the job list wired once for production and the injected-store test path; the `agent_tls_map` the `host:`/`disks:` jobs share per host), `src/dashboards/api.py` (the snapshot/history route pairs, the `404 {"configured": false}` gate that runs before any dial, `_history_batch`, the `/api/summary` additive blocks), `src/dashboards/alerts.py` (the "an unconfigured source never alerts" rule), `src/dashboards/store.py` (the `sample(source, host, metric, res, ts)` + rollup/retention story — invariant 9), `src/dashboards/web/pages/` + `static/` (the five dashboards, the five-tab masthead pinned by `tests/test_api.py`, `app.js`'s `initChrome` badges, `Dash.hero`/`DashCharts` — no framework, no CDN, invariant 5), `tests/test_api.py` (the served-bytes pins), `tests/test_config.py`, `tests/test_live_fleet.py` (the read-only live suite).
|
||||
|
||||
## Objective
|
||||
|
||||
The fifth dashboard page — `/gateway` — renders the gateway host's connection topology as animated pipes: for every configured frontend (haproxy `bind`, nginx `listen`, the stream `443`) an **inbound** pipe (client → gateway: the port, the `server_name` domains, the live connection count and client addresses) and, fanning out from the gateway, one **outbound** pipe per configured target (haproxy `server`, nginx `proxy_pass`, the stream `map` entries: the target host:port, the live count). The page is a **source of its own** (`gateway`, its own poller job on the same `/snapshot` the `host:` job dials, its own interval/timeout/health/events), its counts are stored as gauges (invariant 9's rollup/retention story) and graphed from `/api/gateway/history`, and it keeps every invariant the fleet already keeps: unconfigured is `404` without a dial (invariant 3), a dead source ages instead of zeroing (invariant 4), no count is a rate (rule 2 — no estimator, no `Δ<window>`), no count carries a verdict (like `net.*`), and an old agent (1.1.0, no sections) reads as "predates gateway support" — dimmed, `n/a`, never red.
|
||||
|
||||
## Dependencies
|
||||
- `28_gateway_agent_proxy` (todo) — the three agent sections on the wire and the gateway host running 1.2.0 (the live half of this phase's verification dials what 28 installed)
|
||||
- `27_mobile_tables_and_audit` (todo) — the page's 360 px behaviour is audited by 27's script shape (the table, if any, scrolls inside its card)
|
||||
|
||||
## Tasks
|
||||
1. `01_config_and_source.md` — `GATEWAY_HOST` in `Settings` + the gate properties; `sources/gateway.py`: the config + sockets join into inbound/outbound pipes, gauges only, the three-way agent support
|
||||
2. `02_wiring.md` — the `gateway` poller job in `build_hub` (30 s), health, the alerts rules, the `/api/summary` block, the `/api/config` entry
|
||||
3. `03_api_routes.md` — `/api/gateway/snapshot` + `/api/gateway/history`: the configured gate before any dial (spy-pinned), history through `_history_batch`, unknown names as empty series
|
||||
4. `04_page.md` — `/gateway` + `page-gateway.js`: the animated pipes (hand-rolled SVG + CSS, no CDN, `prefers-reduced-motion`, `aria-label`), the live counts, the three degradation arms
|
||||
5. `05_nav_and_summary.md` — the sixth tab on every signed-in page (the pinned masthead pins updated + browser-measured), the `/api/summary` pin extended
|
||||
6. `06_docs_and_verify.md` — README/AGENTS (the page row, the `GATEWAY_HOST` row, the fleet fact), the browser screenshot pass against live data, the read-only live verification
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_gateway_source.py` (new — the join over fixture snapshots: an inbound join by listen port, the domain-target port-only join labelled "by port", the raw-IP exact join, the `$var`→`map` join, the 1.1.0-predates arm, the never-answered stub arm); `tests/test_config.py` (`GATEWAY_HOST` default, the gate property); `tests/test_api.py` (the route pins, the summary block, the nav pins)
|
||||
- Coverage: **>90%** on `sources/gateway.py` + the new config field
|
||||
- Real-browser check (invariant 10): task 05's measurement + task 06's screenshots against the live dev server
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- Task 06 updates `AGENTS.md`'s page table (`/gateway` row), the `config.py` bullet (`GATEWAY_HOST`), and the metric-naming paragraph (`gateway.*` gauges); `README.md` gains the page's how-to row. The invariant 9 story: the `gateway.*` series roll up raw → 1 m → 1 h like every other gauge.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (the hermetic suite), coverage not down; `uv run pytest --cov` holds the 94% floor
|
||||
- [ ] `GATEWAY_HOST` unset (or a name not in `HOST_TARGETS`) → `/api/gateway/snapshot` answers `404 {"configured": false}` with no dial (spy-pinned) and the page says "not configured"
|
||||
- [ ] a 1.1.0-style snapshot (no sections) → the page's "agent predates gateway support" arm, dimmed, every number `n/a`, the job still `ok` (the invariant-3 shape, pinned)
|
||||
- [ ] the gateway host (1.2.0, post-phase-28) → the page renders the six haproxy frontends, the nginx server blocks and the stream map, with live counts that match a direct `/proc/net/tcp` read (task 06's verification)
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the nine existing sources read `ok` in `./scripts/dev-server.sh status`, the five existing dashboards render as before, the `/api/summary` pre-existing blocks are byte-identical in shape (the `gateway` block is additive)
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 01 — `GATEWAY_HOST` and the gateway source
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — "…visualizes my incoming connections based on the gateway host's nginx and haproxy configs… data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`Settings` names which `HOST_TARGETS` entry is the gateway (`GATEWAY_HOST`, default `gateway`), and `sources/gateway.py` projects one agent snapshot into the page's data: the **inbound** pipes (each configured frontend — a haproxy `bind` port, a nginx `listen` port, the stream `443` — with its `server_name` domains and the live ESTABLISHED count + client addresses keyed on the listen port) and the **outbound** pipes (each configured target — a haproxy `server`, a nginx `proxy_pass`, a stream `map` entry — with the live ESTABLISHED count). The counts are gauges: **no rate** (no `RateEstimator`, no `window_seconds`, no `Δ<window>` — a count is a count, rule 2), **no verdict** (no `severity` dict on any count — `severity.py` rates no connection count, the `net.*` precedent), and the three-way agent support degrades a pre-1.2.0 agent to "predates gateway support" (invariant 3), never red.
|
||||
|
||||
- ASSUMPTION: **inbound join**: an ESTABLISHED socket belongs to frontend `P` when its **local port** is `P`'s listen port; the client is the socket's **remote** address. A socket whose local port matches no configured frontend is `unmatched` (reported, not dropped — the page shows "N other connections", the operator's fact that the gateway talks to something the configs don't name).
|
||||
- ASSUMPTION: **outbound join**: an ESTABLISHED socket belongs to target `T` when its **remote** address matches — a raw-IP target on an exact (ip, port) match; a **domain target on a port-only match, labelled `by_port: true`** in the payload (the agent never does DNS — invariant 7's "never guess" applied to the join: a port match against a domain is a real socket fact plus a config fact, and the label says which half is which). A socket whose remote matches no target is `unmatched` too.
|
||||
- ASSUMPTION: the payload shape is `{configured: true, supported: true, host: str, haproxy: {frontends: [{name, port, backend: str|null}]}, nginx: {servers: [{port, server_names: [str]}], map_targets: [{key, host, port}]}, inbound: [{port, source: "haproxy"|"nginx", domains: [str], connections: int, clients: [str]}, outbound: [{target: {host, port, name: str|null, by_port: bool}, connections: int, source: str|null}], unmatched: {inbound: int, outbound: int, other: int}, sockets: {listens: int, established: int}}` — `supported: false` (a 1.1.0-style snapshot) carries the same keys as empty/nulls plus `supported: false`, and a never-answered host is the stub the `/api/system` stubs use (`unreachable: true`, every count `null`, invariant 4).
|
||||
- ASSUMPTION: the `$var`→`map` join: a `proxy_pass $targetBackend` whose variable matches a `map` block contributes the map's entries as `map_targets` (key = the domain, the target = the entry's host:port) and **no** literal outbound pipe (the per-connection target is the map's job; the page renders the map entries as the fan-out — the 17-entry `stream.conf` map is exactly the "data going out (domains)" the TODO names).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/config.py` — `Settings.gateway_host: str = _env("GATEWAY_HOST", "gateway")`, `gateway_interval_seconds` (default `30.0`, the `DISKS_INTERVAL_SECONDS` idiom) and `gateway_timeout_seconds` (default `10.0`), and the `gateway_enabled` property (`True` iff `gateway_host` is among the `HOST_TARGETS` names — the `nut_enabled` shape); the `any_source_enabled` property includes it.
|
||||
2. `src/dashboards/sources/gateway.py` — new: `parse(snapshot: dict, host: str) -> dict` (the `sources/disks.py` projection idiom — pure over the snapshot the poller fetched, no I/O of its own): the section readers (an absent section → the `supported: false` arm; `available: false`/`null` → a `why` line in the payload, the invariant-3 arms), the inbound/outbound joins per the assumptions above, the `unmatched` tallies, and the `samples` list: one gauge per distinct listen port (`gateway.in_conns@<port>`) and one per distinct target host (`gateway.out_conns@<host>`; a `by_port` domain target's series is keyed on the host, the port is in the payload not the name — the `@stream` scheme, invariant 9's naming).
|
||||
3. `tests/test_config.py` — `GATEWAY_HOST` default `gateway`, an explicit value winning, `gateway_enabled` true/false over the target list (the `nut_enabled` test shape).
|
||||
4. `tests/test_gateway_source.py` — new: the join over fixture-shaped snapshots (reuse the phase-28 fixtures' parsed shape — the test builds the snapshot dict the agent serves): the inbound join (a socket on local `:443` lands in the stream frontend's pipe with its client address; a socket on an unconfigured port is `unmatched.inbound`); the outbound join (a raw-IP target's exact match; a domain target's port-only match with `by_port: true`; no DNS call anywhere — a `socket.getaddrinfo` spy must record nothing); the `$var`→`map` join (the 17-entry map → 17 `map_targets`, no literal pipe); the `supported: false` arm (a 1.1.0-style snapshot without the sections); the never-answered arm is the API's job (task 03), not the source's.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_config.py` + `tests/test_gateway_source.py` as above
|
||||
- Coverage: **>90%** on `sources/gateway.py` + the new config fields
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GATEWAY_HOST` defaults to `gateway` and `gateway_enabled` is true only for a configured name (pinned)
|
||||
- [ ] `parse()` over a 1.2.0 fixture snapshot yields the exact inbound/outbound lists (counts, clients, `by_port`, the map fan-out) — a rename in the agent's section keys fails a test
|
||||
- [ ] the `getaddrinfo` spy records **nothing** across the whole source (pinned — the join never dials DNS)
|
||||
- [ ] the `supported: false` arm renders the payload's `why` and null counts, never a red verdict (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing `Settings` fields and the `host`/`disks`/`llm`/`nut` sources are byte-identical in behaviour
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 02 — The gateway job: poller, health, alerts, summary, config
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the source is a **job of its own** (the `disks:<host>` precedent): its own interval, timeout, health record, and events, so a wedged config read ages the pipes instead of blinking `/system`
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The `gateway` source runs as its own poller job (the `disks:` shape — one job dials the same `/snapshot` the `host:` job of the gateway host reads, with the gateway's own interval/timeout), so the page ages on its own cadence and a failed poll keeps the last good pipes dimmed instead of zeroing them (invariant 4). The alerts rules keep an unconfigured source silent (invariant 3) and a dead source to exactly one line (the `disks` echo rule — the gateway job dials the same agent, so its failure carries no new information while the `host:` job is the voice; a gateway job failing *while its host answers* — the sections wedged — is the only voice that fault has). `/api/summary` gains the additive `gateway` block, `/api/config` gains the gateway row.
|
||||
|
||||
- ASSUMPTION: the job is `gateway` (one job, the `nut` shape — a single source, not per-host: the page is about the one gateway host), `DISKS`-interval defaults from task 01 (30 s / 10 s), events under the dashboard name `gateway`.
|
||||
- ASSUMPTION: the **no-verdict alert rule**: the source's payload carries no `severity`, so `alerts.py` gains no threshold for `gateway` — the only alerts a `gateway` job raises are reachability (stale > 3× its 30 s interval → `warn`, stopped answering → `fail`, the `disks`-echo suppression applied: a dead gateway job while its `host:` job is also dead adds no line).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/app.py` — `build_hub`: the `gateway` job (one, not per host) wired the `disks:` job is wired — the same `AgentTls` entry the gateway host's `host:`/`disks:` jobs use (so a revocation fails all three with one sentence, the task-03 `agent_tls_map` shape), its `parse` the `sources.gateway.parse` + `samples` pass-through, no `RateEstimator` (the source takes none — task 01). The job exists **only** when `settings.gateway_enabled` (the `disks` job's configured gate: no job at all when the source is unconfigured, invariant 3).
|
||||
2. `src/dashboards/alerts.py` — the `gateway` block: the reachability rules per the assumption above; the `HREF` table gains `/gateway`; **no** verdict rule (asserted: a `gateway` payload with a fabricated `severity` dict still produces no verdict alert — the module may not have a threshold of its own, the phase-09 disk rule's shape).
|
||||
3. `src/dashboards/api.py` — `/api/summary`: the additive `gateway` block (`{configured, supported, host, inbound: int (Σ connections), outbound: int, map_targets: int, unmatched: int, states: {gateway: str}}` — a configured job with no payload yet is a stub, the `system.hosts` stub shape); `/api/config`: the gateway row (`GATEWAY_HOST`'s name + the `Target.display_url` of the entry it names, redacted — invariant 6's boundary).
|
||||
4. `tests/test_api.py` — the `build_hub` wiring pins (the job present iff `gateway_enabled`; the shared `AgentTls` entry — a revocation written into `meta` fails the `gateway` job **before any dial**, the `test_a_revoked_agent_fails_both_of_its_sources_without_a_dial` shape extended to three sources); the `/api/summary` `gateway` block pin (with and without the source configured — unconfigured: the block is `configured: false` and **absent no dial happens**, the spy-pinned shape); the alerts pins (unconfigured → no `gateway` alert; dead job + dead host → one line, not two; a wedged-sections failure while the host answers → one line naming `gateway`).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the wiring/alerts/summary pins above
|
||||
- Coverage: **>90%** on the modified `app.py`/`alerts.py`/`api.py` branches
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] a configured build carries exactly one `gateway` job on the gateway host's shared TLS entry; an unconfigured build carries none (pinned)
|
||||
- [ ] a revocation in `meta` fails the `gateway` job before any dial (spy-pinned, the `host:`/`disks:` pair's shape)
|
||||
- [ ] `/api/summary`'s `gateway` block is pinned with and without configuration; the pre-existing blocks are unchanged
|
||||
- [ ] the alerts pins hold: unconfigured silent, dead-source one line, no verdict rule
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing jobs, alerts, and summary blocks behave exactly as before
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — `/api/gateway/snapshot` and `/api/gateway/history`
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the page's data routes, in the app's established shape: the configured gate before any dial (invariant 3), history through the one batched door (invariant 9 / phase-14 task 02)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`/api/gateway/snapshot` answers the source's payload (task 01's shape: the config, the joined inbound/outbound pipes, the unmatched tallies, `supported`) for the `GATEWAY_HOST` entry — with a configured-but-never-answered host as the `unreachable` stub (every count `null`, invariant 4's page-side half) and `GATEWAY_HOST` unset/unknown as `404 {"configured": false}` **before anything that could reach the network** (no client, no fetch, no job — spy-pinned both ways, the `tests/test_api.py` disks-pair shape). `/api/gateway/history` serves the `gateway.in_conns@<port>` / `gateway.out_conns@<host>` series through the same `_history_batch()` → `store.history_many()` door the other four history routes use (phase-14 task 02 — one batched decision, the same `{source, host, metric, range_seconds, points}` document, unknown names as empty series).
|
||||
|
||||
- ASSUMPTION: the route pair is read-only — a fourth data page grows no write (invariant 1's shape: the app's only POST that dials an upstream stays `/api/ups/command`, and the `tests/test_api.py` route-verb enumeration gains the two GETs to the read-only list).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/api.py` — the `gateway` router: `GET /api/gateway/snapshot` (the configured gate first — `settings.gateway_enabled` false → `404 {"configured": false}` with **no** client construction and **no** `fetch_snapshot` call, spy-pinned; configured → the job's last payload through `sources.gateway.parse`'s never-answered-stub arm) and `GET /api/gateway/history` (the `hosts=`/`metrics=`/`range=` query shape the other three history routes take, answered through `_history_batch`).
|
||||
2. `src/dashboards/api.py` — the route-verb enumeration test's read-only list gains the two paths (the task-01 assumption's pin).
|
||||
3. `tests/test_api.py` — the pins, the disks pair's shape: (a) `GATEWAY_HOST` unset → `404 {"configured": false}`, the `httpx.AsyncClient.get` spy and the `collectors.fetch_snapshot` spy record **nothing**; (b) `GATEWAY_HOST` a name not in `HOST_TARGETS` → the same refusal, the same no-dial proof; (c) configured, the job's payload present → the snapshot's exact keys (the task-01 shape — a rename fails a test); (d) configured, no payload yet → the `unreachable` stub (every count `null`); (e) the history route answered through `_history_batch` — points read straight out of the store with `query()` (a third path, in neither the loop nor the batch, fails the test — the phase-14 task-02 pin shape) for `gateway.in_conns@443` and a `gateway.out_conns@<host>` series, with an unknown name answering as an empty series.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the five pins above
|
||||
- Coverage: **>90%** on the new routes
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] unconfigured (unset or unknown name) → `404 {"configured": false}` with the two spies recording nothing (pinned both arms)
|
||||
- [ ] the never-answered stub is pinned (`unreachable: true`, every count `null`)
|
||||
- [ ] the history route's points equal the store's rows for the `gateway.*` series; unknown names → empty series (pinned)
|
||||
- [ ] the two GETs are in the read-only enumeration; no new POST exists (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing snapshot/history routes answer byte-identical shapes
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 04 — The `/gateway` page: animated pipes
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — "I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The `/gateway` page renders the joined payload (task 01) as animated pipes: one **inbound** pipe per configured frontend — `client → gateway` — labelled with the listen port, the `server_name` domains (haproxy's L4 frontends label their port only), the live connection count, and up to a few client addresses; and, fanning out from the gateway node, one **outbound** pipe per configured target — `gateway → target` — labelled with the target host:port (and `· by port` on a `by_port` match), its live count, and its source (the haproxy backend / nginx location / stream-map key it came through). The animation is **decorative at a fixed speed** — moving dashes whose direction is the data flow — because a count is not a rate (rule 2): the page never claims a tok/s-shaped number from it. No count is coloured by a verdict (the `net.*` precedent: the stylesheet section carries no `ok`/`warn`/`fail` token), and the three degradation arms render their own sentence: unconfigured ("not configured — set `GATEWAY_HOST`"), a 1.1.0 agent ("agent predates gateway support — install 1.2.0", dimmed, `n/a`), a never-answered host (dimmed `.stale`, `n/a`, the row kept, invariant 4).
|
||||
|
||||
- ASSUMPTION: the render is **one card per source family** (haproxy / nginx-https / nginx-stream / unmatched), each card a hand-rolled SVG: a left column of inbound pipe stubs, the gateway node mid-card (one rounded rect, its label the host name), a right column of outbound pipe stubs — the pipe is a `<path>` whose `stroke-dasharray`/`stroke-dashoffset` CSS animation runs only while the pipe has ≥ 1 live connection (a zero-connection pipe is a static faint line + the count `0`, because animating a pipe nobody is using would be a shape claiming a flow the count denies — the sparkline's "a shape is not a verdict" rule applied to motion); `prefers-reduced-motion: reduce` renders every pipe static (no animation at all, the repo's standing rule).
|
||||
- ASSUMPTION: the labels are `textContent`-only (no `innerHTML` for anything upstream-provided — a config domain is upstream text; the `DashCharts.escapeHtml` idiom via the page's own escape), and every SVG pipe carries `role="img"` + an `aria-label` built from its inputs (the battery-glyph convention: the pipe carries information the markup must say in words) — e.g. `"inbound pipe, port 4443, nextcloud.reeseapps.com, 3 connections"`.
|
||||
- ASSUMPTION: the page is the app's standing chrome — the masthead's sixth tab (task 05), the four `#nav-status` badges (the `gateway` job is the fifth dot, `app.js`'s one-dot-per-job rule), the theme toggle, the window chips **absent** (the page's history is the count trend — one `LineChart` per card, the `Charts` disclosure's `/system` shape: collapsed, lazy-built, one `/api/gateway/history` request naming every series the card could draw, `ariaLabel` on the plot), and the page's single `faint` footnote: the `by port` join sentence (the only why-line the operator needs to read a count).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/gateway.html` — the page: the shared `<head>`/masthead (task 05's tab slot), the four badge container, the card grid (the four cards' skeletons with their empty-state sentences), the `#nav-status`, and no `id` the script does not reference (the orphan-`id` audit's shape).
|
||||
2. `src/dashboards/web/static/page-gateway.js` — the page script (the `page-disks.js` idiom): `state` keyed per card; `renderPipes(card, pipes, direction)` — the SVG builder (the `DashCharts` escape discipline, the `aria-label` per the assumption, the zero-connection static arm, the `by port` sub-label); the degradation arms (the unconfigured / predates / never-answered sentences, the `.stale` dim, `n/a` for every count); the `Charts` disclosure's one-history-request rule (the `/system` shape: collapsed default, lazy build, a cached response from another window re-fetched); the `unmatched` card (the "N other connections" sentence — the operator's fact that the gateway talks to something the configs do not name).
|
||||
3. `src/dashboards/web/static/dash.css` — the pipe rules: `.pipe` (the `<path>` stroke, the `stroke-dasharray`), `@keyframes pipe-flow` (the dashoffset drift), `.pipe.still` (the zero-connection arm), and the `@media (prefers-reduced-motion: reduce)` block naming **both** the animating and the still states (the phase-18 reduced-motion shape — naming only one leaves the other animating).
|
||||
4. `tests/test_api.py` — the served-bytes pins: the pipe `@keyframes` + the reduced-motion rule (both states, inside the file's one such block); the zero-connection `.still` arm; the page's absence of any `ok|warn|fail` token in the gateway card section (the no-verdict pin, the phase-06 GPU shape); the `aria-label` on every `LineChart` the page builds (the standing chart-audit pin, extended); the orphan-`id` audit over the served page; the CDN-reference absence (invariant 5).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the served-bytes pins above (the repo has no JS test runner — a `.cjs` browser pass is the behavioural proof, task 06)
|
||||
- Coverage: **>90%** on the new Python-side branches (the pins); the JS is pinned on served bytes + the browser pass
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the page renders the four cards with their pipes, labels (port / domains / host:port / `· by port` / counts / clients), and the `unmatched` card — against a fixture-driven dev store
|
||||
- [ ] a zero-connection pipe is static (`.still`), a ≥ 1-connection pipe animates, `prefers-reduced-motion` renders all static (pinned + measured by task 06's browser pass)
|
||||
- [ ] the three degradation arms print their own sentence, dimmed where the invariant says, every count `n/a` (pinned)
|
||||
- [ ] no verdict token in the gateway card section; no CDN reference; every chart labelled (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean, `node --check` on the new script
|
||||
- [ ] no behavior change in completed work: the pre-existing pages render byte-identical
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 05 — The sixth tab, and the `/api/summary` pin
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the page is one of the dashboards: the masthead's tab row gains `Gateway` on every signed-in page, and the overview's fleet probe names the new source
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The masthead's tab row — five tabs on all seven signed-in pages (`/`, `/system`, `/disks`, `/llm`, `/ups` + `/settings`/`/certs`'s no-`aria-current` five) — gains the sixth, `Gateway`, in one order on every page, and the pinned markup audits move from a five-tab assertion to a six-tab one (the pins are the contract; they move on purpose, with the browser measurement as the proof). `app.js`'s one-dot-per-job rule makes the `gateway` job the fifth `#nav-status` badge with no JS change (the container already exists; the job is what the badge reads). The `/api/summary` pin (task 02's additive block) is completed here with the page-side half: the overview renders the `gateway` block's one line (the "N inbound · M outbound" count, linked to `/gateway` via the block's own href), and the no-behavior-change control is the five pre-existing dashboards rendering as before.
|
||||
|
||||
- ASSUMPTION: the tab order is `Overview · System · Disks · LLM · UPS · Gateway` (the new tab last — a sixth tab inserted mid-row would move the `aria-current` targets the five pinned pages assert, and a moved tab is a broken-button read on every page at once).
|
||||
- ASSUMPTION: the tab is `aria-current="page"` on `/gateway` only (the standing rule: the five dashboards carry it on their own page, `/settings`/`/certs` carry the five tabs with **no** `aria-current` — the sixth tab inherits the same shape on the two admin pages).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/*.html` — the six tab pages (`index`, `system`, `disks`, `llm`, `ups`, `gateway`) gain the `Gateway` tab (the `aria-current` on `gateway.html` only); `settings.html` and `certs.html` gain the same sixth tab with no `aria-current` (the two admin pages' shape — the phase-20 task-04 "both links on both admin pages" rule extended to the tab row).
|
||||
2. `tests/test_api.py` — the tab pins move: the five-tab assertion becomes the six-tab one (the `MARKUP_PAGES` loop's tuple gains `Gateway`, the order pin per the assumption, the `aria-current` pin per the assumption); the `test_every_page_links_to_both_admin_pages`-shape loop stays one loop over the pages (the phase-25 task-03 pin's shape — one order, one loop).
|
||||
3. `src/dashboards/web/static/page-index.js` — the overview's `gateway` line: the block's one line in the fleet panel's shape (count + the `/gateway` link via the block's `href`), hidden-when-unconfigured (the `disks` block's invariant-3 shape — an unconfigured source never renders a row, never a red one).
|
||||
4. `tests/test_api.py` — the `/api/summary` pin completed: the `gateway` block's keys pinned (task 02) **and** the overview line's rendering pinned on the served `page-index.js` (the line reads the block's keys — a rename fails a test, the `SLOT_FIELDS` device shape).
|
||||
5. `data/logs/29-task05-tabs.cjs` — the browser measurement (the `25-task03-verify.cjs` shape): eight signed-in pages × two widths (1280 / 360) — six tabs present in order, the `aria-current` on `/gateway` only, the five pre-existing tabs' `aria-current` unchanged, the sixth `#nav-status` badge present and populated on `/gateway`, the 360 px dropdown lists all six, no document-level sideways scroll (phase 18's guarantee), screenshots `data/logs/masthead-29-*.png`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the served-markup pins (the moved tab audit, the `aria-current` pin, the overview-line pin)
|
||||
- Real-browser check (invariant 10): the task-05 script is the proof — eight pages × two widths
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] six tabs on all eight signed-in pages in the pinned order, `aria-current` on `/gateway` only (pinned + measured)
|
||||
- [ ] the `gateway` job is the fifth `#nav-status` badge (measured on `/gateway`, populated from `/api/health`)
|
||||
- [ ] the overview renders the `gateway` line when configured and hides it when unconfigured (pinned)
|
||||
- [ ] the 360 px dropdown lists all six tabs; no document-level sideways scroll (measured)
|
||||
- [ ] `uv run pytest` green, `node --check` on the touched scripts, screenshots in `data/logs/`
|
||||
- [ ] no behavior change in completed work: the five pre-existing dashboards' tab row is byte-identical apart from the added tab (the served-bytes pin proves it)
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 06 — Docs, and the live verification of the rendered pipes
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the definition of done: UI changes verified in a real browser with a screenshot against **live** data (invariant 10), and the docs updated when a page, a config row, or a fleet fact changed
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The docs carry the new page (the `AGENTS.md` table row, the `config.py` bullet's `GATEWAY_HOST`, the metric-naming paragraph's `gateway.*` gauges, the `README.md` how-to row), and the phase's proof is the read-only live pass: the dev server (1.2.0 gateway host, post-phase-28) renders `/gateway` with the real topology — the six haproxy frontends, the nginx server blocks, the stream map — and the page's live counts **match a direct `/proc/net/tcp` read** on the gateway host (the join is proven end-to-end, not just against fixtures). The pass writes its transcript and screenshots to `data/logs/`.
|
||||
|
||||
- ASSUMPTION: the verification is **read-only** (the `tests/test_live_fleet.py` rule): the pass dials the dashboard's public routes over the session, reads `/proc/net/tcp` on the gateway host via the ssh alias, and compares — it starts no workload, sends no command, writes nothing to the fleet.
|
||||
- ASSUMPTION: the count comparison is **port-bucketed, not exact-timestamped**: the pass reads the page's inbound count for port `P` and the host's ESTABLISHED-on-`P` count within one poll window, and asserts they are equal **or** differ by ≤ 1 (a connection can close between the two reads — the one-socket tolerance is the honest bound; a divergence > 1 fails the pass).
|
||||
|
||||
## Work
|
||||
|
||||
1. `AGENTS.md` — the page table row (`/gateway` — "which domains are coming in on which port, and where the traffic goes — the gateway host's nginx + haproxy configs joined to its live sockets" — source: "the same agents, `haproxy`/`nginx`/`sockets` sections of `GET /snapshot` (agent 1.2.0)"); the `config.py` bullet's `GATEWAY_HOST` line (default `gateway`, the gate property); the metric-naming paragraph's `gateway.in_conns@<port>` / `gateway.out_conns@<host>` gauge line (no rate, no verdict — the `net.*` sentence applied).
|
||||
2. `README.md` — the page's how-to row (the `GATEWAY_HOST` setup, the not-configured arm, the 1.2.0 agent requirement) in the standing page-row shape.
|
||||
3. `data/logs/29-task06-live.cjs` — the read-only live pass (the `24-task04-verify.cjs` shape): sign in, `GET /api/gateway/snapshot` (assert `configured: true`, `supported: true`, the six haproxy frontends, the nginx server blocks, the stream map's 17 entries), `GET /gateway` (the six tabs' page, the four cards rendered, the pipes' `aria-label`s present), then `ssh gateway-ops` → `cat /proc/net/tcp /proc/net/tcp6` → the port-bucketed count comparison per the assumption (inbound on each configured listen port; the `unmatched.inbound` tally against the port complement), and the `by_port` spot-check (one domain target's count equals the ESTABLISHED-on-that-port count).
|
||||
4. `data/logs/29-task06-live.out` + screenshots `data/logs/29-task06-{gateway,card-haproxy,card-stream,charts-expanded}.png` — the transcript (the assertion table, the count comparison, **no** config body or token in it — invariant 6) and the screenshots (the phase's proof, invariant 10).
|
||||
5. The 1.1.0 arm's live proof is the **dev-store** one (not a fleet rollback — a fleet arm is the operator's call): the dev store pointed at a 1.1.0-style fixture snapshot renders the "predates" arm, screenshot `data/logs/29-task06-predates.png`.
|
||||
|
||||
## Testing & Quality
|
||||
- Live (read-only, invariant 10): the task-06 pass is the phase's proof; the 1.1.0 arm is dev-store-driven
|
||||
- The hermetic suite is re-run as the gate (`uv run pytest --cov`), not for this task's logic
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `AGENTS.md`'s page table, the `config.py` bullet, and the metric-naming paragraph name `/gateway` + `GATEWAY_HOST` + the `gateway.*` gauges (a grep over both files finds each)
|
||||
- [ ] `README.md` has the page's how-to row
|
||||
- [ ] the live pass's transcript records: the six frontends, the nginx blocks, the 17 map entries, the port-bucketed count comparison (divergence ≤ 1 on every compared port), the `by_port` spot-check
|
||||
- [ ] the four screenshots + the predates-arm screenshot are in `data/logs/`, and the transcript carries no config body and no token (invariant 6, grepped)
|
||||
- [ ] `uv run pytest --cov` holds the 94% floor, `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the nine pre-existing sources read `ok` in `./scripts/dev-server.sh status` and the five pre-existing dashboards render as before (the pass's control leg)
|
||||
@@ -0,0 +1,64 @@
|
||||
# Phase 30 — The gateway connection map: two config-derived tables, joined lines, green = in the config
|
||||
|
||||
**Source:** operator request, 2026-10-23 (post-phase-29 review of `/gateway`)
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/sources/gateway.py` (the join this phase rewrites), `agent/host_agent.py` **1.2.0** (its `haproxy` / `nginx` / `sockets` sections — the *only* upstream this phase reads, and it needs **no agent change**), `tests/fixtures/gateway_haproxy.cfg` + `gateway_nginx_main.conf` + `gateway_nginx_conf_d_http.conf` + `gateway_nginx_stream_d_stream.conf` + `gateway_proc_net_tcp{,6}.txt` (the verbatim captures the graph is pinned to), `src/dashboards/api.py` (`/api/gateway/*`, the `/api/summary` `gateway` block), `src/dashboards/alerts.py` (the reachability-only rule this phase changes by decision), `src/dashboards/web/pages/gateway.html` + `static/page-gateway.js` + `static/dash.css` (the pipe vocabulary), `tests/test_gateway_source.py`, `tests/test_api.py` (the served-bytes pins this phase deliberately rewrites).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway` becomes the **connection map** it was meant to be: **two tables — nginx and haproxy** — in which every row is a connection **that exists in a config file**, drawn as a line that physically joins its inbound end to its outbound end (haproxy straight across; nginx branching out of `:443` to the services the stream `map` routes to), each line **green because the config says it exists**, each line carrying the **switch** shape from the operator's sketch (built, visible, not yet wired). The socket table is demoted to a number beside the line instead of deciding its colour or its existence; the `stream map` card and the `unmatched` card are gone; anything the two configs do not describe is **not reported at all**, and the warnings that remain are **config-integrity** problems — a backend no frontend routes to, a `proxy_pass $var` with no `map` — surfaced as a warnings panel on the page and one alert row on `/`.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Green means "in the config."** | Every configured connection draws green. Colour is an **existence state**, not a verdict: `severity.py` still has no `gateway.*` row and no payload key carries `severity`. Red is the **off** state (a route switched out of the config), styled now, unreachable until the write path exists. |
|
||||
| 2 | **Motion means "a socket is open this instant."** | The dash animates while that pipe's live count is ≥ 1 and sits still otherwise. No rolling/recent-peak derivation, no speed tied to volume. |
|
||||
| 3 | **The switch is built, not wired.** | Real `<button role="switch">` per connection, `aria-checked="true"` (in config = on), `aria-disabled="true"` (not `disabled`, which would drop it from the a11y tree), no click handler, a `title` naming why. The UPS controls are the precedent for a gated control; the write path is a **later phase**. |
|
||||
| 4 | **Live counts stay, demoted.** | The inbound trunk carries the real ESTABLISHED count + client addresses; the 24 h per-card chart stays; the header keeps tallies. A leaf endpoint written as a **name** shows `—` (never `0`, never a fake `n/a` column). |
|
||||
| 5 | **Only the unambiguous socket join survives.** | Inbound = the socket's local port is a configured listen port. Outbound = the socket's remote `(address, port)` **equals a target the config wrote as an address**. `by_port`, `unmatched.inbound`, `unmatched.outbound`, `unmatched.groups`, `partial`, `unmatched_listeners` are **deleted**. |
|
||||
| 6 | **Anything outside the two configs is ignored.** | `9120` (this app's own agent), `2222`, DHCP/mDNS/whatever else listens is neither drawn, nor counted, nor warned about. `sockets.listens` / `sockets.established` stay in the payload as the kernel's raw totals for programmatic consumers only. |
|
||||
| 7 | **The warnings list is config integrity.** | Seven kinds (`03_config_warnings.md`) plus three explicit refusals where the payload cannot tell broken from merely unreadable; all derived from config text, all actionable. Panel on `/gateway`, and **one** `warn` row in `/api/summary`'s `alerts` — which changes the "a `gateway` alert is reachability only" rule in `alerts.py` and in `AGENTS.md`. |
|
||||
|
||||
**Out of scope:** any change to `agent/host_agent.py` (1.2.0 already publishes everything needed, including `upstreams` and `maps`), any write path or config edit, any new dependency, any new sample family (the existing `gateway.in_conns@<port>` / `gateway.out_conns@<host>:<port>` names stay, so stored history keeps its meaning).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `28_gateway_agent_proxy` (complete) — agent 1.2.0's `haproxy` / `nginx` / `sockets` sections and the captured fixtures.
|
||||
- `29_gateway_dashboard_page` (complete) — the `gateway` poller job, `/api/gateway/*`, the page's chrome, the six-tab masthead. This phase **reshapes** that payload and page; it does not remove the job, the routes, the tab, or the gauges.
|
||||
- `27_mobile_tables_and_audit` (complete) — the 360 px audit script `data/logs/27-task02-mobile-audit.cjs` must still exit 0 with `/gateway`'s new rows in it.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_connection_graph.md` — `sources/gateway.py`: the config-derived graph — `listeners[]` + `connections[]` (haproxy `frontend → backend → server`, nginx vhost `server_name → proxy_pass`, stream `proxy_pass $var → map` entries, `proxy_pass <pool> → upstream` servers), with no counts yet.
|
||||
2. `02_the_join_simplified.md` — the surviving socket join (inbound by local port, outbound by exact address only), the shared-endpoint rule, the deletion of `outbound` / `by_port` / `unmatched` / `partial`, the gauges re-pointed, and `/api/summary`'s `gateway` block plus the overview's one line reshaped in the same commit.
|
||||
3. `03_config_warnings.md` — `warnings()`: the seven config-integrity kinds (and three refusals), empty on this fleet's captured configs (a pinned assertion), plus one `warn` overview row from `alerts.py`.
|
||||
4. `04_page_two_tables.md` — `gateway.html` + `page-gateway.js` + structural CSS: exactly **two** panels (nginx, haproxy), trunk → leaf rows, the warnings panel on the shared `ul.alerts` vocabulary, four header tallies, the degradation arms kept.
|
||||
5. `05_lines_and_switches.md` — `dash.css` + `page-gateway.js`: the connecting line (straight across for haproxy, branched for nginx), the green `--gate-on`/`--gate-off` existence pair, `live` motion, the switch built to the sketch and pinned as unwired. The `ok|warn|fail`-in-the-pipe-section pin is **restated**, not deleted.
|
||||
6. `06_docs_and_verify.md` — `AGENTS.md` (the `sources/gateway.py`, `api.py`, `alerts.py`, metric-naming, UI-conventions and gap-13 passages) + `README.md`, the browser pass at 1440 px and 360 px with screenshots, the mobile-audit re-run, and the read-only live verification against the real ingress.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_gateway_source.py` is **rewritten around the graph** — every connection the captured configs declare is present and no other; the join identity is restated as *"the inbound counts sum to the ESTABLISHED sockets whose local port a config names"* (the old `inbound + outbound + unmatched == the table` identity dies with `unmatched`); `socket.getaddrinfo` stays spied and **never called**; the six warning kinds each get a synthetic broken config and the captured configs pin **zero** warnings. `tests/test_api.py`'s gateway pins are rewritten in the same tasks that change the payload/page — including `GATEWAY_PAGE_FIELDS`, `GATEWAY_FIELDS`, the stub/answer key-parity pin, the no-verdict/no-window walk, and the colour pin (restated per decision 1).
|
||||
- Integration: the page renders from a fixture-driven dev store; the warnings panel appears only when a warning exists; the chart disclosure stays collapsed-lazy-one-request-per-card.
|
||||
- Coverage: **>90%** on new/modified Python (`sources/gateway.py`, `alerts.py`, `api.py`).
|
||||
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's) and `bash .agents/validate.sh` is green, coverage floor included.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For this phase that specifically means: `AGENTS.md`'s `sources/gateway.py` bullet (the payload keys, the join, the deleted `unmatched`/`by_port`/`partial`), its `api.py` `/api/gateway/*` paragraph and the `/api/summary` `gateway` block field list, its `alerts.py` bullet (the gateway alert is no longer reachability-only), the `/gateway` UI-conventions paragraph (two tables, joined lines, green = in the config, the unwired switch), the metric-naming paragraph, and **gap 13** (the clipped-pipe-label gap, which the new row layout either resolves or restates honestly).
|
||||
- Tasks update `README.md`/`AGENTS.md` immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/gateway` serves and renders **exactly two** cards, `nginx` and `haproxy`; no `stream map` card, no `unmatched` card, no third table.
|
||||
- [ ] Every haproxy row draws its line **straight across** from `:<bind port>` to its `server` target; the nginx `:443` trunk **branches** to each `map` entry (16 on this fleet) and each http vhost trunk to its `proxy_pass` target — measured in the browser pass, not asserted in prose.
|
||||
- [ ] Every configured line is **green**; it animates only while its live count ≥ 1; both states still static under emulated `prefers-reduced-motion: reduce`. No `severity` key at any depth of the payload; no `severityFrom`/`severityFor` in the page script.
|
||||
- [ ] One switch per connection, `role="switch"` + `aria-checked="true"` + `aria-disabled="true"`, no click handler bound, `title` naming the missing write path — and a test that fails if it becomes clickable without a handler or stops saying it is unwired.
|
||||
- [ ] `POST` nothing: the verb-enumeration pin still finds `/api/ups/command` as the app's only upstream write, and `/gateway` dials only `/api/gateway/snapshot` + `/api/gateway/history`.
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress: `warnings == []`, every one of the 6 haproxy frontends connected to its `server`, `:443` connected to the 17 `map` entries, `:4443`/`:8443` connected to their `proxy_pass` targets, and no `unmatched` / `by_port` / `partial` key anywhere.
|
||||
- [ ] A `gateway` `warn` row appears in `/api/summary.alerts` **iff** `warnings` is non-empty, and disappears when they are; a stale/stopped `gateway` job still alerts exactly as before.
|
||||
- [ ] `uv run pytest` (incl. `--cov` ≥ 94 floor), `uv run ruff check src tests agent`, `uv run pyright`, `node --check` on the changed scripts, `bash .agents/validate.sh` — all green.
|
||||
- [ ] `data/logs/30-*.png` screenshots at 1440 px and 360 px; `data/logs/27-task02-mobile-audit.cjs` still exits 0 with `/gateway` in its sweep.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards and `/settings`/`/certs` render as before; `host:*`, `disks:*`, `llm:*`, `nut` still read `ok` in `./scripts/dev-server.sh status` (10 sources, `gateway` included).
|
||||
@@ -0,0 +1,78 @@
|
||||
# Task 01 — The connection graph: `listeners` and `connections`, derived from config text only
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** operator request 2026-10-23 — "the dotted line connections need to connect to their counterparts… for nginx you'll need to branch the lines from `:443` to the services they connect to… this is just 'exists in config therefore it is a connection'"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`sources/gateway.py` learns the thing it never published: **which outbound target belongs to which inbound listener**. It gains `connections[]` — one entry per connection the two configs actually declare, each carrying its inbound port, its outbound endpoint, and the config's own selector for it (a haproxy backend name, an nginx `server_name`, a stream `map` key) — and renames `inbound` to `listeners`, because "inbound" was a direction and this is a thing the page draws a trunk for. Task 01 adds the graph and renames; task 02 deletes the guessing socket join. **No agent change**: everything needed is already in agent 1.2.0's `haproxy` / `nginx` sections, including `upstreams` and `maps`.
|
||||
|
||||
## The graph, stated exactly
|
||||
|
||||
**`listeners[]`** — one per configured listen port, same claim rule as today's `inbound`, same `metric` string:
|
||||
|
||||
```
|
||||
{"port": 443, "source": "nginx", "domains": ["_"], "connections": 0|int|None,
|
||||
"clients": [...], "metric": "gateway.in_conns@443"}
|
||||
```
|
||||
|
||||
**`connections[]`** — one per `(source, in_port, out_host, out_port)`, selectors merged, in config order:
|
||||
|
||||
```
|
||||
{"id": "nginx:443->reeseapps-homepage.reeselink.com:443",
|
||||
"source": "nginx", # the card it belongs to: "haproxy" | "nginx"
|
||||
"in": {"port": 443},
|
||||
"out": {"host": "reeseapps-homepage.reeselink.com", "port": 443,
|
||||
"name": None, # the haproxy `server` name when there is one
|
||||
"kind": "map", # "backend" | "server_name" | "pool" | "map"
|
||||
"selectors": ["homepage.reeseapps.com"], # the config's own word(s) that reach this target
|
||||
"address": False}, # True when the config wrote an IP literal
|
||||
"connections": None, # filled by task 02; task 01 always None
|
||||
"metric": "gateway.out_conns@reeseapps-homepage.reeselink.com:443"}
|
||||
```
|
||||
|
||||
Derivation, per config family:
|
||||
|
||||
1. **haproxy** — `frontend → routes → backend → server`. For every frontend row (one per `bind` port, `routes` = `default_backend` then each `use_backend` name, which `haproxy_frontends()` already builds — note the agent's own `frontend.backends` holds **only** the `use_backend` names and keeps `default_backend` separate, which is why that insert exists) and every `server` of each routed backend block: a connection with `in.port = the bind port`, `out = {host: server.target, port: server.port, name: server.name, kind: "backend", selectors: [backend name]}`. This needs `haproxy_targets()` to stop dropping the backend it was declared in — change its rows to `{name, host, port, backend}`.
|
||||
2. **nginx http vhost** — for every `server` **block**, for every `proxy_passes` entry with a concrete `target` and a port: one connection **per `listen` port of that block**, `out.selectors = the block's server_names` verbatim (the catch-all `_` included), `kind: "server_name"`.
|
||||
3. **nginx pool** — a `proxy_pass http://<name>` with **no port** names an `upstream` block. When an `upstreams[]` entry has that name: one connection **per upstream `server` host:port**, `kind: "pool"`, `selectors: [<pool name>]`. When none does: **no connection** (that is task 03's `nginx_pool_undeclared`).
|
||||
4. **nginx stream map** — a `proxy_pass` with `variable` set: find the `maps[]` block whose `variable` matches after `lstrip("$")`. For each of its `entries` with a target, one connection **per `listen` port of the block that wrote the `proxy_pass`**, `kind: "map"`, `selectors: [entry.key]`. The key `default` is kept **verbatim** — it is the config's own word for "everything nobody mapped", not a guess. A map no `proxy_pass` names contributes nothing (the `$connection_upgrade` rule, unchanged), and an entry with a null target contributes nothing (task 03 warns about it).
|
||||
5. **Never** invent a connection for what the payload cannot decide: a `proxy_pass` with neither `target` nor `variable` (a unix socket, a bare word) yields nothing and **no warning**; a haproxy frontend whose only routing is a fetch expression (`use_backend %[req.hdr(host),map_dom(...)]`, which the agent reports as no name) yields its **listener** and **no connections** — the page draws that trunk alone, and `AGENTS.md` rule 2 is why it is not a warning: "no name" and "no routing directive" are one value in this payload.
|
||||
|
||||
Merge rule: identical `(source, in_port, host, port)` is **one** connection whose `selectors` are the union in first-seen order (one vhost's two names arriving at one pool is one line, not two). Never merge across `in_port` — `:4443 → keycloak:8080` and `:8443 → keycloak:8080` are two pipes on two socket spaces (the `minecraft` 25565/25566 lesson).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- `haproxy_targets()`: rows gain `backend`; docstring says the backend name is what makes the frontend→server join possible.
|
||||
- New `nginx_server_blocks(section)` returning one row per **server block** — `{ports: [int], names: [str], proxies: [entry]}` — so the block's `listen`, `server_name` and `proxy_pass` stay associated (today's `nginx_listen_ports()` merges names across blocks and loses exactly the fact this task needs; keep it if the echo still needs it, or derive the echo from `nginx_server_blocks()` — do not keep two readers of the same block).
|
||||
- New `config_connections(snapshot_haproxy, snapshot_nginx) -> list[dict]` implementing rules 1–5 + the merge, pure and clock-free, with the module docstring gaining a **"The connection graph"** section stating the four derivations, the merge rule, and the two refusals (rule 5).
|
||||
- `parse()`: rename the `inbound` payload key to `listeners` (its rows keep every field they had, including `metric`), add `payload["connections"] = config_connections(...)`, and keep `outbound` and `unmatched` **untouched for this task** (task 02's business). `connections[].connections` stays `None` here — say so in a comment so task 02's owner sees where the count lands.
|
||||
- `_skeleton()`: add `"listeners": []` and `"connections": []` (replacing `"inbound": []`) so `stub()` grows the same key set — the key-parity test fails otherwise.
|
||||
2. `src/dashboards/api.py` — the mechanical rename in every reader of the payload: `/api/gateway/snapshot` (it passes the payload through plus `states`), the `/api/summary` `gateway` block (its `inbound`/`outbound` totals now read `payload["listeners"]` / `payload["outbound"]`; **keep the block's field names as they are** — task 04 reshapes that block deliberately, and this task must not change `/api/summary`'s shape).
|
||||
3. `src/dashboards/web/static/page-gateway.js` — the minimum that keeps the page honest while task 05 rewrites it: `familyPipes()` reads `snapshot.listeners` instead of `snapshot.inbound`, and `renderHeader()`'s inbound tally does too. Nothing else about the drawing changes.
|
||||
4. `tests/test_gateway_source.py` — keep every existing assertion working under the rename, and **add the graph pins** (this task's real product):
|
||||
- `test_every_connection_the_captured_configs_declare_is_drawn` — against the real `gateway_snapshot` fixture, **32** connections total, asserted as the **set** of `(in_port, out.host, out.port)` so a missing or extra one fails by name:
|
||||
- **6** haproxy, one `server` each: `:22 → reeseapps-gitea.reeselink.com:2222` (`gitea`), `:2000 → stackexpected-gitea.reeselink.com:2222`, `:5201 → web.reeselink.com:5201`, `:8080 → unifi-external.reeselink.com:8080`, `:25565 → minecraft.reeselink.com:25565`, `:25566 → minecraft.reeselink.com:25566`
|
||||
- **17** on nginx `:443` from the stream map — the 16 `*.reeselink.com:443` entries **plus** `default → 127.0.0.1:4443`
|
||||
- **6** on nginx `:4443` (nextcloud `:11000`, gitea `:3000`, jellyfin `:8096`, bricktracker `:3333`, keycloak `:8080`, emmaleaf `:80`) and **3** on `:8443` (nextcloud `:11001`, `127.0.0.1:8081`, keycloak `:8080`), each with its own `server_name` in `selectors`
|
||||
- `:80` (nginx, `server_name _`) is a **listener with no connection** — it declares no `proxy_pass`. That is this fleet's live instance of refusal rule 5 and it gets its own assertion: the trunk exists, `connections` names port 80 nowhere.
|
||||
- `test_a_haproxy_connection_carries_the_backend_and_the_server_name` — `out.name == "gitea"`, `out.kind == "backend"`, `selectors == ["gitea-2222"]`.
|
||||
- `test_the_stream_default_entry_is_kept_verbatim` — a connection whose `selectors == ["default"]` to `127.0.0.1:4443`, `out.address is True`.
|
||||
- `test_a_pool_names_its_upstream_servers_and_an_undeclared_pool_draws_nothing` — synthetic nginx config with `upstream backend_pool { server 10.0.0.1:3000; server 10.0.0.2:3000; }` + `proxy_pass http://backend_pool;` → two connections `kind == "pool"`; and `proxy_pass http://no_such_pool;` → no connection at all (and **no** invented host).
|
||||
- `test_two_names_one_target_one_port_is_one_connection` (selectors merged, first-seen order) and `test_the_same_target_on_two_listen_ports_is_two_connections` (the `admin.keycloak` / `keycloak` pair from the real `http.conf`).
|
||||
- `test_a_frontend_that_routes_by_expression_is_a_listener_with_no_connection` and `test_a_proxy_pass_with_neither_target_nor_variable_draws_nothing` — the two refusals.
|
||||
- `test_no_connection_is_ever_invented_for_a_port_no_config_names` — walk every `connections[].in.port` and assert it is in `listeners`.
|
||||
5. `tests/test_api.py` — the rename only: `GATEWAY_PAGE_FIELDS`' `inbound…` paths → `listeners…`, the stub/answer key-parity pin's expected key set, `test_the_gateway_stub_and_the_answer_share_one_key_set`, `test_the_gateway_snapshot_carries_no_verdict_and_no_window` (walk `connections` too), `test_gateway_snapshot_answers_the_pipes_the_projection_joined`, and `/api/summary`'s `GATEWAY_FIELDS` **unchanged in names** (only its derivation moved).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the pins above; `uv run pytest tests/test_gateway_source.py tests/test_api.py -q` then the full suite. Coverage **>90%** on `sources/gateway.py`'s new branches (all four derivations, both refusals, the merge, the undeclared-pool arm).
|
||||
- `uv run ruff check src tests agent && uv run pyright` clean; `node --check src/dashboards/web/static/page-gateway.js`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/gateway/snapshot` on the dev server (live ingress) returns `listeners` (10 rows today), `connections` (**32** on the captured configs — 6 haproxy + 17 stream-map + 9 vhost; the *test* asserts the captured fixture's set, the live read is a recorded observation, not an assertion, because `/etc/nginx` on a router changes), and `outbound` still present.
|
||||
- [ ] Every connection's `in.port` appears in `listeners`; every haproxy connection names its backend; every stream connection names its map key; `default` survived.
|
||||
- [ ] `stub()` and `parse()` answer the same key set (pinned).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; ruff + pyright + `node --check` clean.
|
||||
- [ ] `README.md`/`AGENTS.md` — no user-visible change yet (the page still draws the old cards); note nothing here, task 04/05 do the doc updates.
|
||||
- [ ] No behaviour change in completed work: the other pages byte-identical, the 10 sources `ok`.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Task 02 — Delete the guessing socket join: inbound by port, outbound by exact address, nothing else counted
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** operator request 2026-10-23 — "This isn't live tracking, this is just 'exists in config therefore it is a connection'… I don't know what the unmatched section is" + the two follow-up decisions: *"exact-address match stays"* and *"ignore anything not in haproxy or nginx configs"*
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The socket table stops deciding the page's shape. `sources/gateway.py` keeps **exactly two** sound attributions — a socket's **local port** names a configured listener (inbound), and a socket's remote `(address, port)` **equals a target the config wrote as an address** (outbound) — and ignores every other socket: not counted, not totalled, not warned about. `outbound[]`, `unmatched`, `unmatched.groups`, `partial`, `unmatched_listeners`, `by_port` and `MAX_GROUP_HOSTS` leave the payload; `AGENTS.md`'s "the join is the whole honesty of the module" passage is rewritten in task 06 against this file.
|
||||
|
||||
**Why the deleted arms went** (state this in the module docstring, because the reason is the rule): a config writes `reeseapps-brain.reeselink.com:443` and `/proc` reports `10.1.1.106:443`. Nothing in this stack resolves a name (pinned by the `socket.getaddrinfo` spy), and 16 map entries share port 443, so the old `by_port` arm could only credit a port — which the module itself refused to attribute and dumped in `unmatched.outbound`. On the live router that residual was **mostly this app talking to itself**: the "listeners no config names" were `9120` (the dashboards agent's own port) and `2222`. A residual that is mostly other software is not a signal, and a card of it teaches the operator to ignore the panel.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- `join_sockets(listeners, endpoints, established)` — rewrite to the two arms above and **return nothing**; it stamps counts in place. Signature docs state the new identity: *"every socket is counted at most once; inbound + outbound ≤ `established`; a socket that matches neither is not counted at all, by decision."* Keep the `continue`-after-count discipline (a loopback hop is two `/proc` rows and must not be drawn down one pipe twice).
|
||||
- Build `endpoints: dict[(host, port), {"connections": int, "routes": int}]` from the **address-written** `connections` only (`out.address is True`); `routes` counts how many connections claim the endpoint. After the pass, stamp `conn["connections"] = endpoint["connections"]` and `conn["shared"] = endpoint["routes"] > 1`. A name-written connection keeps `connections: None` — the truth, not a zero.
|
||||
- Delete: `outbound` / `_outbound_pipe()` / `_groups()` / `MAX_GROUP_HOSTS` / `live_listen_ports()` (its only job was the unmatched direction test) / the `unmatched` key from `_skeleton()`. Add `payload["endpoints_live"]`? **No** — nothing needs it; the page reads the count off each connection.
|
||||
- `samples()`: gauges from `listeners[].metric` (`gateway.in_conns@<port>`) and `connections[].metric` (`gateway.out_conns@<host>:<port>`) whenever the value is a number. Consequence to state in the docstring **and** in task 06's `AGENTS.md` pass: **name-keyed `gateway.out_conns@<domain>:<port>` series stop being written**, because the number they carried was the port's, not the target's; existing rows age out under the unchanged retention policy, so no schema or prune change (invariant 9).
|
||||
- Module docstring: replace "The join, and where its honesty lives" with the two surviving arms + the ignore rule + the loopback-hop note; keep "No DNS, ever" and "No verdict" verbatim (they are unchanged rules).
|
||||
2. `src/dashboards/api.py`
|
||||
- `/api/gateway/snapshot`: passes the payload through plus `states` — update to the new key set (drop `unmatched`), nothing else.
|
||||
- `/api/summary`'s `gateway` block becomes exactly: `{configured, host, href, supported, unreachable, listeners, connections, live_in, live_out, map_targets, states}` — `listeners`/`connections` are the row counts, `live_in`/`live_out` are the sums of the counts that answered (**`None`, never `0`,** when the socket table did not answer or nothing answered), `map_targets` stays the stream-map entry count, `href` still comes from `alerts.HREF`. `unmatched`, `unmatched_listeners` and `partial` are **deleted**. (`warnings` joins this block in task 03.)
|
||||
- `/api/config`'s `gateway` row is unchanged (it never named a pipe field).
|
||||
3. `src/dashboards/web/static/page-index.js` — the one line under the fleet table reads the new block: `gateway <listeners> ports · <connections> routes · <live_in> live`, every figure `n/a` when the block says `null`, the shared `.stale` dim when `unreachable`, and **nothing at all** when `configured: false` (invariant 3, unchanged). `href` comes from the block; the page invents no route and no number.
|
||||
4. `src/dashboards/web/static/page-gateway.js` — **compatibility only**, so the page stays truthful while task 04 rebuilds it: the outbound column reads `snapshot.connections` filtered by `source` (map each to the shape the existing row renderer wants, or inline a small renderer — do not add new visual vocabulary here), the `by port` chip and its `title` are **deleted**, the `unmatched` card is deleted from the JS and its `<article>` from `gateway.html`, the header's `unmatched` tally cell is deleted, and `sumConnections()` reads `listeners` / `connections`. Any deleted id must leave no orphan container (the orphan-`id` audit).
|
||||
5. `tests/test_gateway_source.py` — rewrite the join pins:
|
||||
- **delete** `test_a_domain_target_is_matched_on_port_alone_and_labelled_by_port`, `test_a_port_shared_by_several_names_resolves_none_of_them`, `test_the_group_host_list_is_bounded_and_reports_the_real_total`, `test_a_listening_port_no_config_claims_stays_visible`, `test_a_connection_on_a_port_no_config_names_is_reported_not_dropped`.
|
||||
- **add** `test_a_name_target_is_never_credited_for_a_socket_on_its_port` (the 16-way `:443` fan-out keeps `connections: None`, and the sockets are not counted anywhere — assert the sum over `connections` is unchanged by their presence), `test_a_port_no_config_names_is_neither_counted_nor_reported` (add a LISTEN row on 9120 to a synthetic table: nothing in the payload mentions it), `test_an_endpoint_written_as_an_address_is_matched_exactly_and_carries_its_count` (no `by_port` key exists any more — assert its absence), `test_two_connections_sharing_an_endpoint_say_so_and_repeat_the_count_never_split_it` (`shared: true`, both carry the endpoint's number), `test_the_loopback_hop_counts_once_on_each_side` (`:4443` inbound and `127.0.0.1:4443` outbound each take their own `/proc` row, and no row is counted twice).
|
||||
- **restate** `test_the_live_counts_reconcile_with_the_socket_table` as: every inbound count's port is a configured listener, `sum(listeners) + sum(endpoint counts) <= established`, and `sockets.listens` / `sockets.established` equal the fixture's raw row counts. Keep `test_no_join_ever_asks_a_resolver` exactly as it is.
|
||||
6. `tests/test_api.py` — `GATEWAY_PAGE_FIELDS` and `GATEWAY_FIELDS` to the new key sets; the stub/answer key-parity pin; `test_the_gateway_snapshot_carries_no_verdict_and_no_window` walks `connections` too (recursive: no `severity`, no `window_seconds` at any depth); `test_the_summary_gateway_block_is_the_payloads_sums_not_the_pages_arithmetic`, `…_is_the_stub_of_a_source_that_never_answered`, `…_of_the_captured_ingress_reads_the_live_join`; `test_a_dead_gateway_host_is_one_overview_line_not_three` (the `disks`/`gateway` echo-suppression rule is unchanged — a dead host's `gateway` job still adds nothing new); `test_the_gateway_counts_grow_no_verdict_alert_even_with_verdicts_in_the_payload` now injects `severity` into `listeners` **and** `connections`; the overview-line pin `test_the_overview_line_reads_the_gateway_block_it_claims` resolves `GATEWAY_FIELDS` against a live document and `page-index.js`'s readers; `/gateway`'s markup pins for the deleted unmatched card and `by port` chip (the **absence** of both, now).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the pins above; the full suite plus the **live** read (`uv run pytest -m live --live`) confirms `gateway` still reads `ok` and the snapshot's `listeners` counts match `ss -tan` on the router for configured ports.
|
||||
- Coverage: **>90%** on `sources/gateway.py` (both arms, the shared-endpoint arm, the sockets-silent arm), `api.py`'s changed block, `page-index.js` pinned on served bytes.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `grep -rn "unmatched\|by_port\|MAX_GROUP_HOSTS" src/ tests/` finds nothing outside a comment explaining what was removed and why.
|
||||
- [ ] `GET /api/gateway/snapshot` (live): no `unmatched`, no `outbound`, no `by_port`; `connections[].connections` is a number **only** for the address-written endpoints (`127.0.0.1:4443`, `127.0.0.1:8081` today) and `null` for every name.
|
||||
- [ ] `GET /api/summary` → `gateway` block has exactly the 11 fields listed above; `unmatched` / `unmatched_listeners` / `partial` are gone; the overview page's one line prints ports / routes / live and vanishes when unconfigured.
|
||||
- [ ] Nothing in the payload mentions port `9120` or any other port the two configs do not name (live check + pinned).
|
||||
- [ ] `uv run pytest` **100% passing** incl. `--cov` ≥ 94 floor; ruff + pyright + `node --check` on both changed scripts.
|
||||
- [ ] `AGENTS.md`'s `sources/gateway.py` / `api.py` `/api/gateway/*` / `/api/summary` passages no longer describe `unmatched`/`by_port`/`partial` (full rewrite lands in task 06; this task must not leave a sentence that **contradicts** the shipped payload).
|
||||
- [ ] No behaviour change in completed work: the five other dashboards and the two admin pages unchanged, 10 sources `ok`.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Task 03 — Config-integrity warnings: seven kinds, from the config text, on the page and one row on `/`
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** operator request 2026-10-23 — "I don't know what the unmatched section is, there should be nothing unmatched, but if there is that should be called out in a **list of warnings to address** like other things in this dashboard project" + the follow-up decision *"Ignore anything not in haproxy or nginx configs"* and *"Config integrity, yes"*
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
With everything outside the two configs ignored (task 02), the only warning left worth an operator's attention is a config that **does not connect to itself**. `sources/gateway.py` gains `warnings(snapshot)` — seven kinds, each derived from config text alone, each actionable — the payload carries them, `/api/summary` counts them, and `alerts.py` raises **one** `warn` row for them. That row is the deliberate change to the documented "a `gateway` alert is reachability only" rule, and task 06 rewrites that rule in `AGENTS.md` against this file.
|
||||
|
||||
## The seven kinds
|
||||
|
||||
Each warning is `{"id", "kind", "subject", "message"}`; `id = f"{kind}:{subject}"` is the page's stable DOM key; `message` is operator-facing prose written in this file's voice, carrying the config's own words verbatim.
|
||||
|
||||
| kind | condition | message shape |
|
||||
|------|-----------|---------------|
|
||||
| `haproxy_route_dangling` | a frontend's `default_backend` or any `use_backend` name no `backend` block declares | `frontend haproxy-22 routes to backend "gitea-2222", which no backend block declares` |
|
||||
| `haproxy_backend_orphan` | a backend that declares ≥ 1 `server` and appears in no frontend's routes | `backend "old-tunnel" has 2 servers and no frontend routes to it` |
|
||||
| `haproxy_backend_empty` | a backend block with no `server` line at all | `backend "gitea-2222" declares no server — it routes nowhere` |
|
||||
| `nginx_server_unlistened` | an nginx `server` block with `server_name` and/or `proxy_pass` and **no** `listen` | `nginx server "nextcloud.example.com" proxy_passes but never listens` |
|
||||
| `nginx_pool_undeclared` | a `proxy_pass http://<name>` with no port and no matching `upstream` block (task 01 draws no connection for it) | `nginx listen 4443 proxy_passes to pool "backend_pool", which no upstream block declares` |
|
||||
| `nginx_variable_unmapped` | a `proxy_pass $var` with no `map` block defining that variable | `nginx listen 443 proxy_passes $targetBackend, which no map block defines` |
|
||||
| `nginx_map_route_missing` | an entry in a **referenced** map whose target is null | `map $targetBackend entry "x.example.com" names no address to route to` |
|
||||
|
||||
**Three deliberate non-warnings** (each gets a comment where the check would have gone, because the absence is the rule):
|
||||
|
||||
- an nginx `server` block with a `listen` and no `proxy_pass` — it may be a static or redirect vhost, and this payload cannot tell the two apart, so a warning would invent a fault;
|
||||
- a haproxy `frontend` whose only routing is a fetch expression (`use_backend %[req.hdr(host),map_dom(…)]`, which agent 1.2.0 reports as **no name**) — "no name" and "no routing directive" are one value on the wire, so warning about it blames a config this agent cannot read; making this distinguishable is an agent change (a `routes_dynamic` flag), not a source-level guess;
|
||||
- a `proxy_pass` with neither `target` nor `variable` (a unix socket, a bare word) — legitimate, undrawable, not broken.
|
||||
|
||||
A section is judged **only** when its own `available` is `True`. `predates`, `no file`, and `unreadable config` produce **no** warnings — a host that cannot be read has not been found guilty of anything (invariant 3).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- New pure `warnings(haproxy_section, nginx_section) -> list[dict[str, str]]` implementing the table above with the three refusals, deterministic order (haproxy kinds in config order, then nginx kinds in config order), deduplicated by `id`.
|
||||
- `parse()`: `payload["warnings"] = warnings(haproxy, nginx)` when supported, `[]` in the predates arm and whenever the socket/config halves did not answer; `_skeleton()` gains `"warnings": []` so `stub()` matches (the key-parity pin).
|
||||
- Docstring: a short **"What this module warns about"** section — the seven kinds, the three refusals, and the sentence *"a warning here is a config that does not connect to itself; nothing outside the two configs is reported at all."*
|
||||
2. `src/dashboards/alerts.py`
|
||||
- `build(...)` gains `gateway_payload: Mapping[str, Any] | None = None` (keyword, last, defaulted, so every existing caller and test keeps working).
|
||||
- One row, emitted **only when the payload exists, the job is currently answering, and `warnings` is non-empty**: `{id: "gateway:config", severity: "warn", source: "gateway", message: f"{n} proxy config {problem/problems} to address — {first['message']}", href: HREF["gateway"], at: <payload ts when it has one>}`. When the `gateway` job is failing or stale the reachability row it already produces is the **only** line (the existing "a dead source gets exactly one line" rule), so a dead agent never yields two.
|
||||
- This module still holds **no thresholds** and no `severity.py` row: a config warning is a structural fact, not a graded number. Update this module's docstring rule list — it currently says the gateway source has "no verdict rule and none is wanted"; it now says: *reachability plus config-integrity, no graded number, ever.*
|
||||
3. `src/dashboards/api.py` — pass the gateway payload to `alerts.build(...)` from `/api/summary`; add `warnings` (the count) to the `gateway` block; `GATEWAY_FIELDS` pin extended in the same commit.
|
||||
4. `src/dashboards/web/static/page-index.js` — the overview's one gateway line appends ` · N to address` **only when `warnings > 0`**, so a clean ingress's line does not grow a permanent zero.
|
||||
5. `tests/test_gateway_source.py` — one test per kind, each on a **synthetic** config written into the fake `--root` tree the `gateway_snapshot` fixture uses (or a hand-built section dict where the agent's parser would only be in the way, which is legitimate for a pure function over sections — say which you used in the test docstring), plus:
|
||||
- `test_the_captured_configs_produce_no_warnings` — the real fixture asserts `warnings == []` **and** the exact reason per kind (this is the pin that keeps the seven checks from being a lie about this fleet);
|
||||
- `test_an_unreadable_or_absent_section_warns_about_nothing` (`no_haproxy` / `no_nginx` / `predates` modes),
|
||||
- `test_a_listen_with_no_proxy_pass_is_not_a_warning` and `test_a_frontend_routing_by_fetch_expression_is_not_a_warning` (the refusals, pinned in the direction that they produce **no** warning while still producing their listener),
|
||||
- `test_warnings_are_deterministic_deduplicated_and_json_serialisable`.
|
||||
6. `tests/test_alerts*.py` (whichever file holds the `alerts.build` suite today) — the row appears for one warning and for seven; **one** row, not one-per-warning; `severity == "warn"`; `href == "/gateway"`; a failing/stale `gateway` job yields its reachability row and **not** the config row; an unconfigured gateway yields nothing (invariant 3); the `disks`/`gateway` echo-suppression table is unchanged.
|
||||
7. `tests/test_api.py` — `/api/summary` with a mutated gateway payload (the `self_test_app` precedent: swap the collector so a poll cannot overwrite what the test set) yields exactly one new `warn` row linking to `/gateway`, and the same mutation with an empty `warnings` list yields none. **Both directions**, per the project's testing policy.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: as listed; the alert row's two directions are mandatory.
|
||||
- Coverage: **>90%** on `warnings()`, the `alerts.build()` branch, and the `api.py` wiring.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress answers `warnings: []`, and `GET /api/summary` carries **no** `gateway:config` alert row.
|
||||
- [ ] Injecting one synthetic dangling `use_backend` into the fixture produces exactly one `haproxy_route_dangling` warning, one `warn` row on `/api/summary` whose `href` is `/gateway`; removing it produces none (both pinned).
|
||||
- [ ] A `gateway` job that is failing produces its reachability row and **no** config row (pinned).
|
||||
- [ ] `test_the_captured_configs_produce_no_warnings` is green and names why each of the seven kinds cannot fire on this fleet.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; ruff + pyright clean.
|
||||
- [ ] `AGENTS.md`'s `alerts.py` bullet updated in this task (it documents a rule this task changes) — task 06 re-checks the whole file.
|
||||
- [ ] No behaviour change in completed work: no other source's alert rows change shape or count, the overview's two-panel structure is intact.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Task 04 — Two panels, not four: the nginx and haproxy connection tables, grouped trunk → leaves
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** operator request 2026-10-23 — "the stream map shouldn't be its own table, it's part of nginx. There should be only 2 tables: nginx and haproxy"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway` becomes two panels and one warnings list. The `stream map` card is gone — a `map` is part of nginx, so its 17 entries are drawn as the leaves of nginx's `:443` trunk — and the `unmatched` card is gone (task 02 deleted the data behind it). Each card lists its **listeners**, and under each listener the **connections** that listener's config declares, so the structure of the page *is* the structure of the file. This task ships the markup and the render pass with the connectors present but plainly styled; task 05 gives them the green line, the elbows and the switches.
|
||||
|
||||
> **On the word "table":** these are `.card` panels with a header row and one row per connection, **not** `<table>` elements. A `<table>` cannot draw an elbow from `:443` to sixteen rows, and `AGENTS.md` pins `table.data` to the horizontal-scroll rules (phase 27) which a graph row does not want. The visual reading is a table — heading, aligned rows, a right-hand number column — and the orphan-`id` / every-`<table>`-carries-`data` pins stay green because no `<table>` is introduced.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/gateway.html`
|
||||
- `#gate-cards` holds **exactly two** `<article class="card gate-card">`: `data-gate="nginx"` then `data-gate="haproxy"` (the operator named nginx first, and it is the bigger fan-out). Delete the `map` and `unmatched` articles and every id/class only they used.
|
||||
- Each card body: `<div class="gate-map" data-k="map">` (groups injected here) + the `data-k="empty"` sentence + the existing `details.charts-box` block, unchanged in shape.
|
||||
- New warnings panel, **above** the cards, shipped `hidden`:
|
||||
```html
|
||||
<section class="card" id="gate-warnings" hidden>
|
||||
<header><h3>Warnings to address</h3><span class="chip" id="warnings-count"></span></header>
|
||||
<div class="body"><ul class="alerts" id="warnings-list"></ul></div>
|
||||
</section>
|
||||
```
|
||||
Its rows use the overview's vocabulary verbatim — `<li class="warn"><span class="src">haproxy</span><span class="msg">…</span></li>` — because "like other things in this dashboard project" is the instruction and `ul.alerts` *is* that thing. Clean means **hidden**, not an empty card.
|
||||
- `#tallies` becomes four cells: `ports` (configured listeners), `connections` (config-declared connections), `live` (ESTABLISHED sockets on configured ports), `warnings`. `#t-listeners` / `#t-established` / `#t-unmatched` and their JS are deleted; `#sub` keeps `host · agent · N connections`.
|
||||
- `<footer class="pagefoot">`: delete the `by port` sentence. Keep/rewrite three, each one sentence: the animation rule; **"only an endpoint the config wrote as an address can be matched to a socket — a name is never resolved, so its line carries no number"**; and **"the switch shows what the config says; toggling a route needs a config write path and is not wired up yet."**
|
||||
2. `src/dashboards/web/static/page-gateway.js`
|
||||
- `CARDS = [{key: 'nginx', …}, {key: 'haproxy', …}]` and no third entry; `pruneCards` unchanged in behaviour.
|
||||
- New `groupsFor(snapshot, key)` → `[{port, listener, connections: [...]}]`, ordered: ports ascending, connections in payload order. A listener with **no** connections still produces a group (this fleet's `:80` vhost with no `proxy_pass` is the live case) and its leaf list renders the one faint sentence `no target named in the config` — a refusal, never a blank row.
|
||||
- `renderGroup(card, group)` builds once and patches afterwards (the `page-disks.js` / slot-grid discipline: keyed `Map` per card, `appendChild` reorders rather than rebuilds, so a 17-row fan-out is not re-created every 30 s). Row structure:
|
||||
```
|
||||
<li class="gate-link" data-id="{connection.id}">
|
||||
<span class="gate-elbow" aria-hidden="true"></span>
|
||||
<span class="gate-switch-slot"></span> <!-- task 05 fills this -->
|
||||
<svg class="pipe on">…</svg> <!-- task 05 styles it -->
|
||||
<span class="gate-target">
|
||||
<span class="pipe-name mono">{host}:{port}</span>
|
||||
<span class="gate-selector faint">{selectors joined} · {kind}</span>
|
||||
</span>
|
||||
<span class="pipe-count">{count or '—'}</span>
|
||||
</li>
|
||||
```
|
||||
Trunk: `:{port}` + the live count (`14 live` / `n/a`) + up to `CLIENTS_SHOWN` client addresses + the domains count (`16 names`, full list in `title`). Every upstream string (`server_name`, `proxy_pass`, map key, client address) goes out through `textContent`/`title` only, and the row's `title` + `aria-label` carry the **whole** endpoint even when CSS ellipsises it (gap 13's rule, re-measured in task 06).
|
||||
- A count that is `null` prints `—` with `title` = `not attributed: the config names this endpoint as a name, and nothing here resolves names` — never `0`, and not a repeated `n/a` column of twelve identical words. A number prints through `countText`.
|
||||
- `renderWarnings(snapshot)`: fills `#warnings-list` with one `li.warn` per payload warning (`src` = the config family word, `msg` = `warning.message`), sets `#warnings-count` to `N` , and keeps `#gate-warnings` `hidden` when the list is empty. The page renders `warnings` and never re-derives one.
|
||||
- `renderHeader()` → the four tallies; the `.stale` dim and the three degradation arms (`why` banner, `unreachable` dim, `404 {configured:false}` → `#gate-empty` naming `GATEWAY_HOST` and agent **1.2.0**) keep their exact current behaviour. A failed poll still re-draws nothing (last good map stays).
|
||||
- `cardStreams()` (the chart series list) reads `listeners` + `connections` for that source; one `/api/gateway/history` request per card per refresh, plot built only while the disclosure is open, `ariaLabel` on every plot — the phase-29 discipline, unchanged.
|
||||
3. `src/dashboards/web/static/dash.css` — the **structure** only in this task: `.gate-map` (grid, `min-width: 0`), `.gate-group` (two columns: trunk `minmax(0, 9.5rem)` + leaves `minmax(0, 1fr)`), `.gate-leaves` (list-style none, `display: grid`, gap), `.gate-link` (`display: grid`, `align-items: center`, columns for elbow / switch / pipe / target / count), `.gate-target` (`min-width: 0`), `.gate-selector` (11 px, ellipsis), `.gate-trunk` (`min-width: 0`), `@media (max-width: 640px)` stacking the group to one column with the trunk above its leaves. No colours, no dashed borders, no switch rules yet — task 05's job, so the two are reviewable separately.
|
||||
4. `tests/test_api.py` — rewrite the page-shape pins in the same commit:
|
||||
- `test_the_gateway_page_is_a_card_per_source_family_with_a_reconciliation_row` → **`test_the_gateway_page_is_exactly_two_connection_panels_and_a_warnings_list`**: exactly two `[data-gate]` articles and they are `nginx` + `haproxy` (assert the list, ordered, so a third card fails); no `[data-gate="map"]`, no `[data-gate="unmatched"]`; `#gate-warnings` + `#warnings-list` exist and ship `hidden`; the four tally ids exist and the three deleted ones do not; every id on the page is referenced by the script or the stylesheet (the standing orphan audit must stay green by inspection of its own list).
|
||||
- `test_the_gateway_page_prints_every_field_the_projection_publishes` → `GATEWAY_PAGE_FIELDS` resolves against a live payload, now including `connections[].in.port`, `connections[].out.host/port/name/kind/selectors/address`, `connections[].connections`, `warnings[].kind/subject/message`, `listeners[].domains/clients/connections`.
|
||||
- `test_the_gateway_page_renders_each_degradation_arm_in_the_payloads_own_words` — still all four arms, on the new markup.
|
||||
- `test_the_gateway_page_keeps_its_last_drawing_when_a_poll_fails` — unchanged behaviour, new selectors.
|
||||
- **new** `test_the_stream_map_is_rendered_inside_nginx_and_nowhere_else` — the served script contains no `map` card key and the live payload's map-sourced connections are drawn under `source: "nginx"`; and `test_a_listener_with_no_declared_target_says_so_instead_of_drawing_nothing` (the `:80` arm), `test_warnings_render_in_the_overviews_alert_vocabulary_and_hide_when_clean` (both directions), `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero`.
|
||||
- `test_no_page_narrates_its_own_pipeline` — the new footer sentences must pass the existing deleted-sentence list (they are operator-facing why-lines, not pipeline narration) and no removed sentence comes back.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: served-bytes pins above (no JS runner in this repo — task 06's browser pass is the behavioural proof).
|
||||
- Coverage: the changed Python is nil-or-minimal here; the pins are the product. `node --check src/dashboards/web/static/page-gateway.js` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `/gateway` renders **two** panels; the nginx panel's `:443` group lists 17 leaves on the live config, `:4443` six, `:8443` three, haproxy six groups of one, and `:80` shows its refusal sentence.
|
||||
- [ ] No `stream map` card, no `unmatched` card, no `by port` chip anywhere (grep + pin).
|
||||
- [ ] `#gate-warnings` is hidden with the live (clean) configs and appears with rows when a warning is injected into a fixture payload.
|
||||
- [ ] No sideways document scroll at 360 px with the 17-row fan-out open (measured now, re-measured in task 06).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; ruff + pyright + `node --check` clean.
|
||||
- [ ] README/AGENTS: the `/gateway` UI paragraph gains "two panels, nginx + haproxy, map is nginx's" in this task if the sentence already contradicts the shipped page; task 06 does the full pass.
|
||||
- [ ] No behaviour change in completed work: the other five dashboards + `/settings` + `/certs` byte-identical, 10 sources `ok`.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Task 05 — The line and the switch: green because it is in the config, joined trunk to leaf, unwired by design
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** operator request 2026-10-23 — "the orange dashes look great but they should be **green if the connection is active**. 'Active' means it exists in the nginx config… the dotted line connections need to connect to their counterparts. This is easy for haproxy — just draw the line straight across. For nginx you'll need to branch the lines from `:443` to the services they connect to" + the sketch (`/tmp/pi-clipboard-…png`: green boxes + horizontal pill = on, red boxes + vertical pill = off) + "build the switches in now but don't wire them up"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Every connection row draws a **green dashed line from its inbound end to its outbound end** — straight across for a haproxy frontend with one `server`, **branched** from the `:443` trunk into one tick per map entry for nginx — with the **switch** from the sketch sitting in the middle of it. Colour is an **existence state** taken from the config (green = in the config, red = out of it), never a verdict on a number; motion still means "a socket is open this instant" (decision 2). The switch is real markup, real ARIA, red-and-green styled, **and unwired**: `aria-disabled="true"`, no handler, a `title` that says so.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/dash.css`
|
||||
- In the `:root` block that declares `--ok`/`--fail`, add the **existence pair** with a comment naming what it is *not*:
|
||||
```css
|
||||
/* The gateway's existence pair — deliberately NOT the severity vocabulary. Green means the
|
||||
route is written in a proxy config; red means it is out of the config. No gateway figure
|
||||
is graded (`severity.py` has no `gateway.*` row), so nothing here may reach for --ok,
|
||||
--warn or --fail directly: the pair is one indirection whose name says what it answers. */
|
||||
--gate-on: var(--ok);
|
||||
--gate-off: var(--fail);
|
||||
```
|
||||
(The standing "no rule reaches for a token `:root` does not declare" pin sees both new tokens; per-theme `--ok` substitution happens on the root element, so light/dark both follow without a second declaration.)
|
||||
- Rewrite the `gateway pipes` section's colour half:
|
||||
```css
|
||||
.pipe.on .pipe-path { stroke: var(--gate-on); }
|
||||
.pipe.off .pipe-path { stroke: var(--gate-off); }
|
||||
.pipe.live .pipe-path { animation: pipe-flow 1.2s linear infinite; }
|
||||
.pipe.still .pipe-path { stroke-dasharray: 4 8; } /* a route with nothing on it right now */
|
||||
```
|
||||
`.pipe-path`'s base rule keeps **no** `animation` and **no** verdict colour; the `live`/`still` classes stay (their meaning is now "a socket is open this instant" — restate that in the comment, because the class name no longer says "idle route").
|
||||
- The connectors, all of it pure CSS (no JS coordinate maths, no `ResizeObserver`, nothing measured at runtime):
|
||||
```css
|
||||
.gate-group { position: relative; display: grid; column-gap: 0;
|
||||
grid-template-columns: minmax(0, 9.5rem) minmax(0, 1fr); }
|
||||
.gate-leaves { position: relative; padding-inline-start: 26px; }
|
||||
.gate-link { position: relative; }
|
||||
/* each row carries its own half of the branch, so the spine starts at the first branch and
|
||||
stops at the last without a mask, a magic height or a background-coloured eraser */
|
||||
.gate-link::before { content: ""; position: absolute; left: -26px; top: 0; height: 50%;
|
||||
border-left: 2px dashed var(--border-strong); }
|
||||
.gate-link::after { content: ""; position: absolute; left: -26px; top: 50%; bottom: 0;
|
||||
border-left: 2px dashed var(--border-strong); }
|
||||
.gate-link:first-child::before, .gate-link:last-child::after { display: none; }
|
||||
.gate-elbow { align-self: center; height: 0; border-top: 2px dashed var(--border-strong); }
|
||||
.gate-trunk { position: relative; } /* its own tick reaches the spine at right: 0 */
|
||||
/* straight across: a group with one leaf is not a branch, so it draws no vertical at all */
|
||||
.gate-group.solo .gate-link::before, .gate-group.solo .gate-link::after { display: none; }
|
||||
```
|
||||
`.gate-group.solo` is added by the JS when a group has exactly one connection (every haproxy group on this fleet). The `.gate-link` grid columns are `26px auto minmax(2rem, 4.5rem) minmax(0, 1fr) auto` — elbow, switch, the animated run, the target, the count.
|
||||
- The switch, keyed on `[aria-checked]` and **never** on `:enabled` (a not-yet-wired switch must still look green when it is on), and tinted from the same pair so the gateway owns one colour vocabulary:
|
||||
```css
|
||||
.gate-switch { width: 34px; height: 18px; padding: 2px; border-radius: 999px;
|
||||
border: 1px solid var(--gate-on); display: inline-flex; align-items: center;
|
||||
background: color-mix(in srgb, var(--gate-on) 14%, transparent);
|
||||
cursor: not-allowed; }
|
||||
.gate-switch-thumb { width: 12px; height: 12px; border-radius: 50%;
|
||||
background: var(--gate-on); transform: translateX(16px);
|
||||
transition: transform .16s ease; }
|
||||
.gate-switch[aria-checked="false"] { border-color: var(--gate-off);
|
||||
background: color-mix(in srgb, var(--gate-off) 14%, transparent); }
|
||||
.gate-switch[aria-checked="false"] .gate-switch-thumb { transform: translateX(0); }
|
||||
```
|
||||
`color-mix` (already the sheet's idiom) is why no `--ok-soft` / `--fail-soft` appears in a gateway rule: the soft fill derives from the existence pair rather than borrowing the verdict tints.
|
||||
- The existing `@media (prefers-reduced-motion: reduce)` block gains `.gate-switch-thumb { transition: none; }` beside its two `.pipe` rules (it must name every animated thing on the page — naming one and leaving the thumb sliding is the phase-18 lesson).
|
||||
- `@media (max-width: 640px)`: the group stacks (task 04's rule) and the spine's `top` reaches up into the group gap so the trunk above still reads as connected; the switch and its row do not shrink and nothing scrolls sideways.
|
||||
2. `src/dashboards/web/static/page-gateway.js`
|
||||
- `gateLink()` builds the row in task 04's order and fills `.gate-switch-slot`:
|
||||
```html
|
||||
<button class="gate-switch" type="button" role="switch" aria-checked="true" aria-disabled="true"
|
||||
title="Toggling a route is not wired up yet — it needs a config write path. This switch shows what the config says.">
|
||||
<span class="gate-switch-thumb"></span>
|
||||
</button>
|
||||
```
|
||||
`aria-checked` is `true` because the row exists at all (drawn ⇒ in the config); the `false` styling is present and unreachable until the write phase. `aria-label` names the endpoint, the selector and the config family (`"nextcloud.reeseapps.com → nextcloud.reeselink.com:11000, in the nginx config, on"`), since a switch with no accessible name is silence. **No click handler is bound, anywhere.**
|
||||
- `renderGroup()` toggles `.solo` on the group from its own leaf count, and toggles `live`/`still` **and** `on`/`off` on the row's `<svg class="pipe">` (`on` always today; `off` is a one-line seam for the write phase).
|
||||
- `pipeAria()` gains the state word and keeps every existing fact (port / target / domains / clients / count), so an unattributed count is still announced as unknown and never as zero.
|
||||
3. `tests/test_api.py` — **restate, do not delete**, the colour pin:
|
||||
- `test_no_gateway_count_is_coloured_by_a_verdict` keeps every current assertion (no `--ok|--warn|--fail|.ok|.warn|.fail|severity` inside the `gateway pipes` section, no `severityFrom`/`severityFor`/`.severity` in the script, no verdict class in the markup) and its docstring is rewritten to say the pipes/switches colour from `--gate-on`/`--gate-off`, that the pair is an **existence** state and not the severity vocabulary, and that the one place `.warn` legitimately appears is the warnings panel's `ul.alerts` rows (which are `alerts.py`'s verdict, not a number's).
|
||||
- **new** `test_a_connection_is_green_because_it_is_in_the_config` — `.pipe.on .pipe-path` stroke is `var(--gate-on)`, `.pipe.off .pipe-path` is `var(--gate-off)`, `:root` declares both, and `.pipe-path`'s base rule declares no `stroke` verdict; the script sets `on` unconditionally for a drawn connection.
|
||||
- **new** `test_haproxy_draws_straight_across_and_nginx_branches` — served CSS: `.gate-group.solo` hides `.gate-link::before`/`::after`; the branch halves and `.gate-elbow` exist; and the script's `.solo` toggle reads the group's own leaf count.
|
||||
- **new** `test_the_switch_is_built_and_explicitly_unwired` — `role="switch"`, `aria-checked`, `aria-disabled="true"` (and **not** `disabled`, which would drop it from the a11y tree), the `title` names the missing write path, and no `addEventListener` in the file names `gate-switch` / binds a click on it. This is the pin that fails the day someone ships a dead-looking-live control.
|
||||
- `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` — both pipe states **and** `.gate-switch-thumb` are named in the file's one reduced-motion block.
|
||||
- `test_a_pipe_moves_only_while_it_carries_a_connection` — unchanged claim, new class pairs (`live` animates, `still` does not, colour comes from `on`/`off`).
|
||||
- The orphan-`id` audit and the CDN-reference pin stay green.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: served-CSS + served-script pins above; the behavioural proof is task 06's browser pass (computed stroke colour, animation running, elbow geometry, 360 px).
|
||||
- Coverage: no new Python; `node --check` clean; `bash .agents/validate.sh` (which runs `node --check` over `web/static`) green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data: every haproxy row's line runs **straight** from `:22`-style trunk to its `server` with no vertical segment; the nginx `:443` trunk has a spine reaching every one of its 17 ticks; each row has exactly one switch.
|
||||
- [ ] Every drawn pipe computes to the green of `--gate-on` in both themes; a row with ≥ 1 live connection animates and one with 0 or `null` does not; with `prefers-reduced-motion: reduce` emulated, nothing moves (pipes and thumb).
|
||||
- [ ] `grep -n "disabled" src/dashboards/web/static/page-gateway.js` shows `aria-disabled` and no `disabled` attribute; no click listener exists for the switch; the `title` sentence is on every switch.
|
||||
- [ ] The severity vocabulary is still absent from the pipes section, the script and the markup — and its absence is pinned with the restated docstring, not removed.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; ruff + pyright + `node --check` clean.
|
||||
- [ ] No behaviour change in completed work; the other pages byte-identical.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Task 06 — Documentation, the browser pass, and the live read-only verification
|
||||
|
||||
**Phase:** `30_gateway_connection_map` · **Source:** `AGENTS.md` definition of done (browser screenshot against live data; README/AGENTS updated when a rule, command, gap or fleet fact changed)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Make the record match the shipped page: `AGENTS.md` and `README.md` rewritten where phase 30 changed a rule (two panels, config-derived connections, green = in the config, the unwired switch, the deleted `unmatched`/`by_port`/`partial`, the new `gateway` config-integrity alert, name-keyed outbound gauges no longer written), and the whole thing proven in a real browser at 1440 px and 360 px plus a read-only pass against the actual router.
|
||||
|
||||
## Work
|
||||
|
||||
1. `data/logs/30-connection-map.cjs` — the browser pass (Playwright, the repo's idiom: `NODE_PATH=/var/home/ducoterra/.agents/skills/interactive-browser/node_modules node data/logs/30-connection-map.cjs`, `DASH_BASE`/`DASH_PW` from `.env.dev`, transcript to `data/logs/30-connection-map.out`, **fails non-zero on any FAIL**). Checks, in this order, each with a message that names what broke:
|
||||
- exactly two `[data-gate]` panels are visible, `nginx` then `haproxy`; no `map`/`unmatched` panel;
|
||||
- **straight across:** every `.gate-group.solo` leaf has `display: none` on `::before`/`::after` (read through `getComputedStyle(el, '::before')`) and its trunk, elbow, switch, pipe and target share one horizontal centre line (`top` of each within 2 px) — haproxy's rows are the case;
|
||||
- **branched:** nginx's `:443` group has a spine and one tick per leaf, the ticks are 26 px wide and land at each row's centre, and the first/last row's halves are suppressed (so the spine starts and stops at a branch);
|
||||
- **green from the config:** every `.pipe-path` computes to the theme's `--ok` rgb, in `dark` **and** after clicking the theme toggle to `light`; the pipe never computes the accent orange;
|
||||
- **motion:** `getAnimations()` (or `animation-name`) is `pipe-flow` on exactly the rows whose printed count is ≥ 1 and `none` on the rest; with `page.emulateMedia({ reducedMotion: 'reduce' })` every pipe and every `.gate-switch-thumb` reports no animation;
|
||||
- **switch:** one `.gate-switch` per connection row, all `role=switch` + `aria-checked="true"` + `aria-disabled="true"` + a non-empty `title` + a non-empty accessible name; clicking one changes nothing on screen and issues **no** network request (assert on the page's request log);
|
||||
- **warnings:** intercept `/api/gateway/snapshot` with `page.route` to inject two synthetic `warnings` entries — the panel appears with two `li.warn` rows in the `ul.alerts` vocabulary — and with the unmutated response it is `hidden`;
|
||||
- **degradation:** intercept to `404 {configured:false}` → `#gate-empty` names `GATEWAY_HOST` and `1.2.0`, no cards; intercept `supported:false` → the `why` banner and every count `—`/`n/a`, nothing red;
|
||||
- **360 px:** document `scrollWidth === clientWidth` on every state above with the 17-row `:443` group expanded, no element painted past the viewport that is not inside a horizontal scroll container, and every ellipsised label carries its full text in `title`;
|
||||
- screenshots: `data/logs/30-connection-map-{desktop,light,phone,warnings,predates}.png`.
|
||||
2. `data/logs/27-task02-mobile-audit.cjs` — re-run as-is and require **exit 0** with `/gateway` in the sweep (its per-table and per-disclosure claims must hold for the new rows). Transcript to `data/logs/30-final-mobile-audit.out`.
|
||||
3. `AGENTS.md` — the full pass, task by task's own edits included. Sections that are now wrong and must be rewritten against the shipped code (read each one, do not patch a phrase):
|
||||
- the **`/gateway` row of the page table** (what question it answers, and its source),
|
||||
- the **`sources/gateway.py`** bullet — the payload's real key set (`listeners`, `connections`, `warnings`), the two surviving join arms, the ignore rule, the seven warning kinds and the three refusals, the shared-endpoint rule, and **the deletion of `unmatched` / `by_port` / `partial` / `MAX_GROUP_HOSTS`** (the bullet currently describes all four as the module's honesty),
|
||||
- the **metric-naming paragraph** — `gateway.in_conns@<port>` and `gateway.out_conns@<host>:<port>` stay, and the paragraph must say name-keyed outbound gauges are **no longer written** and why, with the rows aging out under unchanged retention (invariant 9 — no schema change),
|
||||
- the **`api.py`** bullet's `/api/gateway/*` paragraph and the `/api/summary` `gateway` block field list,
|
||||
- the **`alerts.py`** bullet — reachability **and** config-integrity now, still no thresholds and no graded number,
|
||||
- the **UI conventions** `/gateway` paragraph — two panels, trunk→leaf rows, the line joins its endpoints (straight across / branched), green = in the config via `--gate-on`/`--gate-off` and *why that is not the severity vocabulary*, motion = a socket open this instant, the switch built-and-unwired, the dash-for-unattributed rule,
|
||||
- **gap 13** (the clipped pipe labels): re-measure with `data/logs/graph-probe.py` + the browser pass and either close it or restate it with today's numbers (the row layout gives a long `server_name → target` string a whole column now, so the residual is probably smaller — write the measured count, not a hope),
|
||||
- any other inconsistency the full read turns up, including pre-existing ones.
|
||||
4. `README.md` — the `/gateway` row and the "what each page answers" prose; the `GATEWAY_HOST` configuration row is unchanged (no new env setting was added by this phase — say so in the pass note); the rollout/agent sections are unchanged (no agent version change).
|
||||
5. **Live, read-only** verification against the real ingress (no writes, no config edits, no battery-tests, nothing restarted):
|
||||
- `GET /api/gateway/snapshot`: `warnings == []`; no `unmatched` / `outbound` / `by_port` / `partial` key anywhere; every haproxy frontend connected to exactly its `server`; `:443` connected to the 17 map entries (`default → 127.0.0.1:4443` among them); `:4443`/`:8443` connected to their `proxy_pass` targets; `:80` present as a listener with no connection; no figure mentions `9120` or `2222`.
|
||||
- Reconcile the inbound counts with the kernel: `ssh gateway-ops "cat /proc/net/tcp /proc/net/tcp6"` (the same world-readable read the agent makes, recorded read-only) and assert `sum(listeners[].connections)` equals the ESTABLISHED rows whose local port is a configured listen port, allowing for the seconds between the two reads — record both numbers in the note.
|
||||
- `./scripts/dev-server.sh status` → **10 sources `ok`**, `gateway` among them; `GET /api/summary` → no `gateway:config` alert row on a clean config, and one after a fixture mutation (the suite pins both).
|
||||
- `git diff --stat agent/ deploy/` → **empty**: this phase touched no agent and no installer.
|
||||
- Write the transcript to `data/logs/NOTES-30-connection-map.md` (the numbers, the screenshots, the reconciliation, and the two facts the next phase will need: that the switch markup exists and where its one `off` seam is).
|
||||
6. Commit the phase with the repo's convention (`--no-gpg-sign`, `LLM-Generated: true` trailer) once everything below is green.
|
||||
|
||||
## Testing & Quality
|
||||
- The browser pass and the mobile audit are the phase's behavioural proof; the hermetic suite remains the contract. No new Python is expected in this task, so coverage is verified, not raised.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `data/logs/30-connection-map.out` ends in a 0-exit with every check named above PASS, and `data/logs/30-connection-map-{desktop,light,phone,warnings,predates}.png` exist.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.out`-style re-run (`data/logs/30-final-mobile-audit.out`) exits 0 with `/gateway` swept at both widths.
|
||||
- [ ] `AGENTS.md` contains no sentence that contradicts the shipped payload or page (grep for `unmatched`, `by_port`, `partial`, `stream map` card, "four cards" — none may survive outside a historical note), and `README.md` matches.
|
||||
- [ ] Live read: `warnings: []`, the connection sets above, and the inbound reconciliation recorded in `data/logs/NOTES-30-connection-map.md`.
|
||||
- [ ] `./scripts/dev-server.sh status` → 10 sources `ok`; `git diff --stat agent/ deploy/` empty.
|
||||
- [ ] `uv run pytest` **100% passing**, `uv run pytest --cov` ≥ 94 floor, `uv run ruff check src tests agent` + `uv run pyright` clean, `bash .agents/validate.sh` green, `uv pip list --outdated` empty (no dependency moved this phase — verify, don't assume).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs` and `/login` render as before; `host:*`, `disks:*`, `llm:*`, `nut` unchanged; `/api/health` shape untouched.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Phase 31 — The gateway route is green end to end, and the row says *which domain*
|
||||
|
||||
**Source:** operator request, 2026-10-23 (post-phase-30 review of `/gateway`) — "The only animated line is `:443 -> 127.0.0.1:4443`. I can't tell which `reeseapps.com` or `stackexpected.com` addresses flow to their reeselink counterparts (very important). Why are the dotted lines on the left grey? Shouldn't those be green too? They're active."
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/web/static/dash.css` (the `gateway pipes` section: `.pipe-path`, `.gate-link::before`/`::after`, `.gate-elbow`, `.gate-trunk`, `.gate-target`, `.gate-selector`, `.pipe-name`), `src/dashboards/web/static/page-gateway.js` (`leafRow`/`renderLeaf`/`groupElement`/`renderTrunk`/`selectorText`/`leafTitle`/`leafAria`/`endpointText`), `src/dashboards/web/pages/gateway.html` (the two panels, the four tallies, the three footer paragraphs), `src/dashboards/sources/gateway.py` (**read-only for this phase** — `connections[].out.{host,port,name,kind,selectors,address}`, `listeners[].{connections,clients,domains,metric}`), `tests/test_api.py` (the gateway page pins: `GATEWAY_PAGE_FIELDS`, `test_a_clipped_pipe_label_keeps_its_full_text_on_the_element`, `test_a_pipe_moves_only_while_it_carries_a_connection`, `test_the_pipe_animation_stills_in_both_states_under_reduced_motion`, `test_no_gateway_count_is_coloured_by_a_verdict`, `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero`, `test_every_pipe_is_an_accessible_image_built_from_the_payload`), `data/logs/27-task02-mobile-audit.cjs` (the nine-page sweep that must still exit 0).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway` answers the operator's actual question — *"which inbound domain flows to which counterpart?"* — from the row itself, and **every route a config declares is green from its trunk to its endpoint** instead of grey for two thirds of its length. The spine and the tick stop being structural ink and take the same existence pair as the run and the switch; the leaf row leads with the config's **inbound** word (`brain.reeseapps.com`) and puts the counterpart (`reeseapps-brain-of-reese.reeselink.com:443`) under it; the trunk draws its own short line that moves on the one live number the kernel genuinely reports for that **port** (`listeners[].connections`), so the page is not dead ink while a name target waits for phase 32; and one filter box finds a domain among the 33 routes without inventing a taxonomy.
|
||||
|
||||
This phase touches **no Python**: the payload already carries every string and every number these four changes need. `sources/gateway.py` is deliberately left byte-identical so phase 32's join rewrite (`todo/32_gateway_resolved_target_liveness`) lands on an unchanged surface.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Colour is still existence, and the whole route carries it.** | `.gate-link::before`, `.gate-link::after` and `.gate-elbow` take `--gate-on` / `--gate-off`, not `--border-strong`. `on`/`off` moves onto the `<li class="gate-link">` so **one** class paints every segment of one row (spine halves, tick, run, switch) and a row can never show a green run hanging off a grey tick. The base rules declare `transparent`, so a row that is built and never rendered still draws nothing — the `.pipe-path`-declares-no-`stroke` rule, restated for borders. `severity.py` still has no `gateway.*` row; no verdict token enters this section. |
|
||||
| 2 | **Motion stays the per-row run's claim and nothing else.** | The spine segment between two rows is **shared** by both leaves, so a dash travelling down it cannot be attributed to one route — animating it would put a per-route claim on a per-list structure. Motion therefore remains "≥ 1 ESTABLISHED socket on this pipe right now", carried by `.pipe.live`, and the CSS comment says why the connectors get colour and not motion. |
|
||||
| 3 | **The row reads inbound → counterpart, both verbatim.** | `.gate-target`'s two lines swap rank: line 1 is the config's inbound word (`out.selectors`, falling back to `out.kind`), line 2 is `endpointText(out)` with a leading `→`. Neither string is edited, truncated in data, or reordered by the page; `title` and the pipe's `aria-label` carry **both** in full (gap 13). |
|
||||
| 4 | **The trunk moves on the port's own real number.** | The trunk draws a short `.pipe.pipe-trunk` that is `live` when `listeners[].connections >= 1` and `still` when it is `0`, and gets **no** motion class when the socket table did not answer (`null` — never treated as `0`, never printed as `0`). Its `aria-label` says `N connections arrived on :PORT in this reading`: it is a fact about the port, and it never implies which leaf carries them. |
|
||||
| 5 | **The filter is a client-side eye, not a query.** | One `<input type="search" id="gate-filter">` over rows **already in the payload**: matches on the inbound word(s), the target, the port and the haproxy `server` name; hides non-matching rows and the groups left empty; re-applies inside the existing patch pass (no rebuild, no re-fetch, no new endpoint); Escape clears it; the warnings panel and the header tallies are **never** filtered. |
|
||||
| 6 | **The page draws no zones.** | No grouping, sorting or colouring derived from a domain suffix. `reeseapps.com` vs `stackexpected.com` is the operator's own typing into the filter, not a payload key — a page that split by suffix would be inventing a fact the two config files never stated (rule 2). |
|
||||
| 7 | **No Python, no payload, no agent, no write.** | Phase 31 is CSS + page JS + markup + tests + docs. The verb-enumeration pin still finds `/api/ups/command` as the app's only upstream write, `routeInConfig()` still answers `true` unconditionally, and the switch stays unwired — **phase 33 is what wires it**, and this phase must not half-start it. |
|
||||
| 8 | **The `—` keeps its meaning and loses its mystery.** | The footer's second paragraph says what a dash is *in the operator's terms* (this endpoint is written as a name and the page matches sockets by address) instead of restating the join's internals. Phase 32 removes the reason; until then one honest sentence, not a column of unexplained dashes. |
|
||||
|
||||
**Out of scope:** any change to `sources/gateway.py`, `api.py`, `poller.py`, `alerts.py` or `agent/host_agent.py`; DNS resolution and any live count on a **name** endpoint (phase 32); any config write, toggle handler, `POST` route or privileged helper (phase 33); the `--gate-*` pair's definition (it stays a substitution of `--ok`/`--fail` in `:root`); window chips (the page keeps one fixed 24 h window); any new dependency.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `30_gateway_connection_map` (complete) — the two panels, the trunk→leaf rows, the `--gate-on`/`--gate-off` pair, the unwired switch, the `connections[]`/`listeners[]` payload this phase re-labels and re-colours. This phase **corrects** that phase's decision-1 rendering (the connectors were left `--border-strong`) and restates its colour pins; it deletes no claim.
|
||||
- `29_gateway_dashboard_page` (complete) — the page shell, the tallies, the chart disclosures, the degradation arms.
|
||||
- `27_mobile_tables_and_audit` (complete) — `data/logs/27-task02-mobile-audit.cjs` must still exit 0 with `/gateway` in its nine-page sweep, at 360 px and 1280 px, after the row re-rank and the filter's arrival.
|
||||
- **Forward, not blocking:** `32_gateway_resolved_target_liveness` (name → address resolution, so the leaf lines animate) and `33_gateway_route_switching` (the config-write path that gives the switch a handler). Both are written after this one; nothing here pre-judges their shape beyond decision 7.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_whole_route_is_green.md` — `dash.css` + `page-gateway.js`: `on`/`off` on the row, the spine/tick/run/switch coloured from one class, base borders `transparent`, the verdict-vocabulary pin restated.
|
||||
2. `02_the_inbound_word_leads_the_row.md` — `page-gateway.js` + `dash.css` + `gateway.html`: the leaf's two lines swap rank (inbound word first, `→ counterpart` second), `title`/`aria-label` carry both, the `—` footer sentence rewritten, `GATEWAY_PAGE_FIELDS` and the clip pin updated rather than relaxed.
|
||||
3. `03_the_trunk_moves_on_a_real_number.md` — `page-gateway.js` + `dash.css`: the trunk's own short pipe, `live`/`still`/neither from `listeners[].connections`, its accessible sentence naming the port, reduced-motion named.
|
||||
4. `04_find_a_domain.md` — `gateway.html` + `page-gateway.js` + `dash.css`: `#gate-filter`, client-side match set, groups emptied by the filter collapse to their own "no route matches" state, an `N of M` chip, Escape to clear, no fetch.
|
||||
5. `05_docs_and_live_verify.md` — `AGENTS.md` (the `/gateway` UI-conventions paragraph, the metric-naming/`sources/gateway.py` passages if their wording about colour and labels went stale, gap 13) + `README.md`, the browser pass at 1440 px and 360 px with screenshots, the mobile-audit re-run, and a read-only live check against the real ingress.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Served-byte pins in `tests/test_api.py`, **restated rather than deleted** wherever a class or line-rank moves: `test_no_gateway_count_is_coloured_by_a_verdict` keeps every "no verdict vocabulary here" assertion and gains "the connectors are coloured from `--gate-on`/`--gate-off` and their base colour is `transparent`"; `test_a_pipe_moves_only_while_it_carries_a_connection` is restated to cover the trunk pipe's *different* source of truth (`listeners[].connections`) without letting it imply a leaf; `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` gains the trunk pipe; `test_a_clipped_pipe_label_keeps_its_full_text_on_the_element` is strengthened (both ends now, not just the target); `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero` is unchanged in claim.
|
||||
- **New** pins: `test_the_whole_route_takes_one_existence_class` (the `<li>` carries `on`/`off`; CSS colours spine halves, tick, run and switch from it; no connector rule reads `--border-strong`), `test_the_leaf_row_leads_with_the_inbound_word` (line 1 is `selectors`-derived, line 2 is the endpoint; the served field table resolves both), `test_the_trunk_pipe_is_an_accessible_image_of_its_port`, `test_the_filter_matches_only_payload_text_and_costs_no_request` (no `fetchJSON` in the filter path, warnings panel untouched, `null` counts never match a query as text `"null"`).
|
||||
- No new Python, so coverage cannot rise: the gate is that it does **not** fall (`--cov-fail-under=94` green) and that `node --check` passes on the changed script (`.agents/validate.sh` runs it).
|
||||
- Integration: the page renders from a fixture-driven dev store in every degradation arm the payload can produce (`unreachable`, `supported: false`, per-section `available: false/null`), with the filter empty, matching, and matching nothing.
|
||||
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's) and `bash .agents/validate.sh` is green, coverage floor included.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For this phase that specifically means: the `/gateway` UI-conventions paragraph in **both** files (the row now reads inbound word → counterpart; the connectors are existence-coloured; the trunk's line moves on the port's own count; the filter exists and what it may not do), the claim that the page's footer is three paragraphs (only if task 02 changes the count — it must not silently), and **gap 13** (the clipped-endpoint-label gap: task 02 shortens the primary line to the inbound word and task 04 makes long targets findable, so the gap is either closed with measurements or restated honestly with the new clip counts).
|
||||
- Tasks update `README.md`/`AGENTS.md` immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] On live data at 1440 px: **every** drawn route — spine halves, tick, run, switch — computes to the `--gate-on` colour in dark *and* light theme; no grey dashed segment remains anywhere in the connection map; measured, not asserted in prose.
|
||||
- [ ] The nginx `:443` fan-out reads as `brain.reeseapps.com` on line 1 and `→ reeseapps-brain-of-reese.reeselink.com:443` on line 2 for all of its leaves, and typing `stackexpected` into `#gate-filter` leaves only the five `stackexpected*` routes visible with the panel tallies unchanged.
|
||||
- [ ] The trunk pipe moves on a port the kernel reports ≥ 1 ESTABLISHED socket for and sits still on one it reports `0` for; with the socket-table arm stubbed to `null` it draws green, motionless, and prints no `0`. `prefers-reduced-motion: reduce` stops every animated thing on the page including the trunk pipe and the switch thumb.
|
||||
- [ ] `grep -n "border-strong" src/dashboards/web/static/dash.css` shows no gateway connector rule; `grep -n "severity\|--ok\|--warn\|--fail" ` over the gateway CSS section and `page-gateway.js` still finds only the warnings-panel `.warn` row class that `alerts.py` owns.
|
||||
- [ ] No `addEventListener` for the switch exists; `routeInConfig()` still returns `true`; `git diff` for this phase touches **no** file under `src/dashboards/` except `web/` — the verb-enumeration pin still lists `/api/ups/command` as the only upstream write.
|
||||
- [ ] `uv run pytest` (incl. `--cov` ≥ 94 floor), `uv run ruff check src tests agent`, `uv run pyright`, `node --check` on the changed script, `bash .agents/validate.sh` — all green.
|
||||
- [ ] `data/logs/31-*.png` screenshots at 1440 px and 360 px (dark, light, filtered, and each degradation arm) and `data/logs/31-*.out` transcripts; `data/logs/27-task02-mobile-audit.cjs` still exits 0 across all nine pages with 0 FAIL / 0 FINDING.
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress unchanged byte-for-byte from before the phase (proof the phase was page-only), and `./scripts/dev-server.sh status` still reads all ten sources `ok`.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards and `/settings`/`/certs` render as before; the gateway charts, warnings panel, staleness dim and `404 {configured:false}` arm behave exactly as phase 29/30 left them.
|
||||
@@ -0,0 +1,72 @@
|
||||
# Task 01 — One existence class paints the whole route: spine, tick, run and switch go green together
|
||||
|
||||
**Phase:** `31_gateway_route_readability` · **Source:** operator request 2026-10-23 — "Why are the dotted lines on the left grey? Shouldn't those be green too? They're active."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Every segment of a connection row — the vertical spine halves, the tick out to the switch, the animated run, and the switch pill — is coloured by the **same** existence class on the same row. Phase 30 shipped the `--gate-on`/`--gate-off` pair and applied it only to `.pipe-path`, leaving `.gate-link::before`, `.gate-link::after` and `.gate-elbow` on `var(--border-strong)`, so a route that the config declares green arrived as two thirds grey plumbing. This task closes that gap and pins it so the pair cannot drift back apart.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/dash.css` — the `gateway pipes` section only.
|
||||
- Base rules: **geometry, no colour.** A connector that has not been given a state draws nothing, which is the `.pipe-path`-declares-no-`stroke` rule restated for borders (an SVG with no `stroke` is invisible; a border's default colour is `currentColor`, so `transparent` has to be written down).
|
||||
```css
|
||||
.gate-link::before {
|
||||
content: ""; position: absolute; left: 0;
|
||||
top: calc(var(--gate-leaf-gap) / -2); height: calc(50% + var(--gate-leaf-gap) / 2);
|
||||
border-left: 2px dashed transparent;
|
||||
}
|
||||
.gate-link::after {
|
||||
content: ""; position: absolute; left: 0; top: 50%;
|
||||
height: calc(50% + var(--gate-leaf-gap) / 2);
|
||||
border-left: 2px dashed transparent;
|
||||
}
|
||||
.gate-elbow { align-self: center; height: 0; border-top: 2px dashed transparent; }
|
||||
```
|
||||
- State rules, one pair per family, driven by the **row's** class so one paint pass covers the row:
|
||||
```css
|
||||
/* The whole route is one claim, so one class paints all of it. `page-gateway.js` writes
|
||||
`on`/`off` on the <li>, and every segment of the row — spine halves, tick, the run inside
|
||||
its <svg>, the switch — reads the existence pair from that one class. A rule here may
|
||||
never reach for --border-strong (structure ink) or for --ok/--warn/--fail (a verdict):
|
||||
phase 30 decision 1 says green means "written in a proxy config", and a grey dashed
|
||||
segment beside a green run contradicts it on every row of the page. */
|
||||
.gate-link.on { --gate-line: var(--gate-on); }
|
||||
.gate-link.off { --gate-line: var(--gate-off); }
|
||||
.gate-link.on::before, .gate-link.off::before,
|
||||
.gate-link.on::after, .gate-link.off::after { border-left-color: var(--gate-line); }
|
||||
.gate-link.on .gate-elbow, .gate-link.off .gate-elbow { border-top-color: var(--gate-line); }
|
||||
```
|
||||
The `--gate-line` indirection is the mechanism: `.gate-switch` / `.gate-switch-thumb` / `.pipe.on` keep their existing declarations (so phase 30's pins on those selectors stay green), and the connectors get the same value from the row without a second pair of tokens in `:root`. If the executor prefers four explicit rules to one custom property, that is fine — the pinned contract is "the row's class colours the connectors", not the technique.
|
||||
- The **`live`/`still`** rules are untouched, and the section comment gains one sentence for decision 2: motion is the per-row run's claim because the spine segment between two rows is *shared* by both leaves — animating it would put a per-route claim on a per-list structure. Colour is what the shared segment is allowed to say.
|
||||
- `@media (max-width: 640px)`: the first row's restored `::before` inherits the colour from the same rule; the block gains nothing (no colour is restated per width — that is how the pair drifts).
|
||||
2. `src/dashboards/web/static/page-gateway.js`
|
||||
- `renderLeaf(li, conn)` writes the state onto the row as well as the `<svg>`:
|
||||
```js
|
||||
li.classList.toggle('on', on);
|
||||
li.classList.toggle('off', !on);
|
||||
```
|
||||
beside the existing four `svg.classList.toggle` calls, in the same pass, from the same `on` — one reading, two targets, no chance for a green run under a grey tick.
|
||||
- `leafRow()` builds the `<li class="gate-link">` with **neither** class, and its comment keeps `buildPipe()`'s sentence: a row built and never rendered draws nothing rather than drawing a confident green line.
|
||||
- `renderGroups()` must clear the classes wherever it clears a row: rows are removed wholesale (`rows.get(id).remove()`), so no extra pruning is needed — verify by reading, and say so in the comment rather than adding dead code.
|
||||
3. `tests/test_api.py`
|
||||
- **Restate** `test_no_gateway_count_is_coloured_by_a_verdict`: keep every existing assertion (no `--ok|--warn|--fail|.ok|.warn|.fail|severity` inside the `gateway pipes` section, no `severityFrom`/`severityFor`/`.severity` in the script, no verdict class in the markup, the one legitimate `.warn` being the warnings panel's `ul.alerts` row from `alerts.py`) and add that the connectors read `--gate-on`/`--gate-off` (directly or through `--gate-line`) and that **no connector rule names `--border-strong`** — with a message quoting the offender.
|
||||
- **New** `test_the_whole_route_takes_one_existence_class`:
|
||||
- served CSS contains rules selecting `.gate-link.on`/`.gate-link.off` that reach `::before`, `::after` and `.gate-elbow`;
|
||||
- the base `.gate-link::before` / `::after` / `.gate-elbow` rules declare `transparent` (so an unrendered row is invisible, not `currentColor`);
|
||||
- served JS sets `on`/`off` on the row element and on the `<svg>`, in the same function, off the same value;
|
||||
- `.gate-switch`'s colour still comes from `--gate-on` / `[aria-checked="false"]` (phase 30's pin, asserted again here so the two halves of the row cannot drift to two vocabularies).
|
||||
- `test_the_gateway_page_is_exactly_two_connection_panels_and_a_warnings_list`, `test_a_pipe_moves_only_while_it_carries_a_connection`, `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` and the orphan-`id` audit must pass **unchanged** — this task adds no `id` and removes no element.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the served-CSS/served-JS pins above; the behavioural proof is task 05's browser pass (computed `border-left-color` / `border-top-color` of a spine, a tick and a run, in both themes).
|
||||
- Coverage: no Python changes; `node --check src/dashboards/web/static/page-gateway.js` clean; the coverage floor must not fall.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `grep -n "border-strong" src/dashboards/web/static/dash.css` returns no line inside the `gateway pipes` section.
|
||||
- [ ] In a browser on live data, at 1440 px: a `.gate-link::before`, a `.gate-link::after`, a `.gate-elbow` and a `.pipe-path` all compute to the same green (`--gate-on`) on the same row, in dark **and** light theme; a row given `off` in devtools turns all four red together.
|
||||
- [ ] A row built without a state class (temporarily strip `on` in devtools) shows no dashed ink at all.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated for anything user-visible this task changes (the `/gateway` conventions sentence about what the colour covers — the phase's final pass re-checks both project-wide).
|
||||
- [ ] No behaviour change in completed work: switch stays unwired, `routeInConfig()` still returns `true`, no Python touched, other pages byte-identical.
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# Task 02 — The row leads with the inbound domain and puts its counterpart under it
|
||||
|
||||
**Phase:** `31_gateway_route_readability` · **Source:** operator request 2026-10-23 — "I can't tell which `reeseapps.com` or `stackexpected.com` addresses flow to their reeselink counterparts (very important)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Both ends of every route are legible on the row, in the operator's reading order: **line 1 is the config's inbound word** — the `server_name`/`map` key the operator types into a browser (`brain.reeseapps.com`), or the haproxy backend name, or the literal `default` — and **line 2 is the counterpart it flows to** (`→ reeseapps-brain-of-reese.reeselink.com:443 · map`). Phase 30 drew the target on the primary line and demoted the inbound domain to an 11px faint qualifier, which is exactly backwards from the question the page exists to answer.
|
||||
|
||||
No payload change: both strings are already in `connections[].out` (`selectors`, `kind`, `host`, `port`).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- Replace `selectorText(out)` with two pure text helpers, each returning the config's own text and nothing derived:
|
||||
```js
|
||||
/** Line 1: the config's own word(s) for who reaches this endpoint, verbatim and in full. */
|
||||
function inboundText(out) {
|
||||
const words = ((out || {}).selectors || []).join(' ');
|
||||
return words || (out || {}).kind || null;
|
||||
}
|
||||
|
||||
/** Line 2: the endpoint the config routes to, marked as the destination, plus its kind. */
|
||||
function counterpartText(out) {
|
||||
const kind = (out || {}).kind;
|
||||
return `${ARROW}${endpointText(out)}${kind ? ` · ${kind}` : ''}`;
|
||||
}
|
||||
```
|
||||
`ARROW = '→'` is a page constant. All selectors stay on line 1 joined by a space exactly as phase 30 joined them — a merged connection (two vhosts claiming one `(in_port, endpoint)`) therefore loses nothing, and the page never picks a "best" name (rule 2: the config's list is the fact, one of them is not).
|
||||
- `leafRow()` keeps the DOM shape — `.gate-target` with two spans — but the second span becomes `.pipe-end mono` (from `.gate-selector faint`) and keeps `hidden` while there is nothing to print. `.pipe-name` remains the *primary line of its row* (the trunk prints its port there, the leaf prints the inbound word), so every layout rule phase 27/30 earned about ellipsis and `max-content` keeps working untouched.
|
||||
- `renderLeaf()`:
|
||||
- `.pipe-name` ← `inboundText(conn.out)`;
|
||||
- `.pipe-end` ← `counterpartText(conn.out)`, `hidden = inbound === null` handling unchanged in spirit (a row with no selector still shows its endpoint as line 1 rather than printing `null` — state which arm that is in the comment: `connections[]` always carries `host`, so line 1 falling back to `kind` is the only empty case);
|
||||
- `leafTitle(conn)` returns the whole pair — `"<all selectors> → host:port"` (plus the kind and, when `shared`, the sharing sentence) — so whatever CSS clips, `title` carries both ends in full;
|
||||
- `leafAria(conn, on)` gains the same pair in words (`"connection from :4443 to nextcloud.reeselink.com:11000, reached by nextcloud.reeseapps.com, …"`) and keeps every existing fact: selectors, `server` name, `shared`, the state word, and `connections not attributed` vs `N live connections` (never `0`).
|
||||
- `endpointText()` is unchanged and stays the single place a target is spelled.
|
||||
2. `src/dashboards/web/static/dash.css`
|
||||
- Add `.pipe-end` beside `.gate-selector`, with the same truncation trio (`min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap`) and `font-size: 11px`, **mono** because both ends are hostnames and they should align down a fan-out. `.gate-selector` keeps its rule — the trunk's `N names` line still uses it.
|
||||
- `.pipe-name` keeps its `flex: 1 1 max-content` behaviour unchanged (gap 13's measurement: the name claims the line; a longer-than-column value alone gets the whole column). Update its comment: the leaf's name is now the inbound domain, which is the shorter of the two strings on this fleet — say so, and say the counterpart below it is the one that clips, with `title`/`aria-label` carrying it whole.
|
||||
- Nothing else changes: no new tokens, no new widths, and the `minmax(0, …)` tracks stay exactly as they are (phase 27's rule that a hostname may not widen the card).
|
||||
3. `src/dashboards/web/pages/gateway.html`
|
||||
- Footer paragraph 2 is rewritten to the operator's terms and nothing else changes in the file. It must say: a dash means this endpoint is written in the config as a **name** and the page matches sockets by **address**, so the route is drawn and counted nowhere — not that the route is idle. One sentence, no pipeline narration:
|
||||
> `a dash means the endpoint is written in the config as a name; the page matches sockets by address and resolves nothing, so this route is drawn but not counted.`
|
||||
- The card comments above `.gate-map` gain one line: the leaf's first line is the inbound name, its second is the counterpart (so the next reader does not re-swap them).
|
||||
4. `tests/test_api.py`
|
||||
- **Strengthen** `test_a_clipped_pipe_label_keeps_its_full_text_on_the_element`: the `title` assertion now demands **both** ends appear in full in the row's `title` *and* in the pipe's `aria-label` (inbound selector and `host:port`), not just the target. Its message names gap 13.
|
||||
- **New** `test_the_leaf_row_leads_with_the_inbound_word`:
|
||||
- served JS assigns `inboundText(...)` to `.pipe-name` and `counterpartText(...)` to `.pipe-end`, and neither call site passes `endpointText(out)` to the primary line;
|
||||
- `inboundText` reads `out.selectors` and falls back to `out.kind`, never to `out.host`;
|
||||
- every path the two helpers index resolves against a live `GET /api/gateway/snapshot` connection row (the `GATEWAY_PAGE_FIELDS` device, extended with `connections[].out.selectors`, `connections[].out.kind`, `connections[].out.host`, `connections[].out.port` so the table resolves in **both** directions and a rename on either side fails by name).
|
||||
- `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero` stays green unchanged (the dash lives in `.pipe-count`, which this task does not touch).
|
||||
- `test_the_gateway_page_prints_every_field_the_projection_publishes` and the "footer is three paragraphs" claim stay true (paragraph count unchanged; only wording moved).
|
||||
- The flavour-text audit (`test_no_page_narrates_its_own_pipeline`) must still pass: the new footer sentence names an operator-facing cause and consequence, not the app's design — if the audit objects, shorten the sentence rather than adding the test to an allow-list.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: served-bytes pins above; a fixture-driven render (`gateway_client`) asserting a `:443` map row's line 1 is `brain.reeseapps.com` and line 2 contains `reeseapps-brain-of-reese.reeselink.com:443` for the captured configs — the actual complaint, asserted on the actual capture.
|
||||
- Integration: `unreachable` / `supported: false` / `available: null` arms still render, with the `:80` refusal sentence untouched.
|
||||
- Coverage: **>90%** on modified paths (no new Python); `node --check` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data: every nginx `:443` leaf's first line is a `reeseapps.com` / `stackexpected.com` / `wellsfamilycounseling.reeseapps.com` / `default` value from `stream.conf`, and its second line is the `*.reeselink.com:443` counterpart; every haproxy leaf's first line is its backend name and its second is `minecraft.reeselink.com:25565`-style.
|
||||
- [ ] `reeseapps-brain-of-code.reeselink.com:443` (42 characters, this fleet's longest) is either fully visible or ellipsised with the whole string in `title` **and** in the pipe's `aria-label` — measured at 1440 px and 360 px.
|
||||
- [ ] The `:80` trunk still prints `no target named in the config`; no row prints `null`, `undefined` or an empty primary line.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated for the row's reading order (the `/gateway` conventions paragraph in both) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: counts, dash semantics, chart disclosure, warnings panel and the switch are untouched.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Task 03 — The trunk draws its own line and moves on the one live number the kernel really reports for that port
|
||||
|
||||
**Phase:** `31_gateway_route_readability` · **Source:** operator request 2026-10-23 — "The only animated line is `:443 -> 127.0.0.1:4443`."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Each group's **trunk** carries a short line of its own that animates while its port has an open connection, because `listeners[].connections` **is** a real measurement — the ESTABLISHED sockets whose *local* port that config names — and it is the only live figure this page can honestly print until phase 32 resolves name endpoints. Today the trunk prints `9 live` as text while drawing no line and moving nowhere, so on the live ingress exactly one pipe animates and the page reads as dead ink beside a busy router.
|
||||
|
||||
This is a **port** claim and never a leaf claim: the trunk's line says "connections are open on `:4443` right now", never "these are `nextcloud`'s". That distinction is carried in its `aria-label`, its `title`, and a pin.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- `buildPipe(trunk = false)` gains one argument and, when true, adds `pipe-trunk` to the `<svg>`'s class list and nothing else — one path, `M0 5 H100`, `viewBox="0 0 100 10"`, `preserveAspectRatio="none"`, `vector-effect="non-scaling-stroke"`, **no** state class at build time (the `buildPipe` comment's rule: a pipe built and never rendered draws nothing).
|
||||
- `groupElement(port)` keeps its `innerHTML` template and appends the trunk pipe as the **last** child of `.gate-trunk`, after `.pipe-count` — the count already carries `margin-inline-start: auto`, so `9 live` and its stub land together at the trunk's right edge, which is the same x the leaf spine starts at (`column-gap: 0`). Appending via DOM (not the template string) is required and commented: an `<svg>` needs `createElementNS`, which is why `buildPipe()` builds it.
|
||||
- `renderTrunk(el, group)`:
|
||||
```js
|
||||
const svg = el.querySelector('.pipe-trunk');
|
||||
const count = listener.connections;
|
||||
const measured = typeof count === 'number' && isFinite(count);
|
||||
// The trunk's `live` comes from the port, not from any leaf: `/proc` states how many
|
||||
// ESTABLISHED sockets have this listen port as their local end, and that is the only
|
||||
// live figure this page can claim about a trunk. It never says which route carries them.
|
||||
svg.hidden = group.listener == null; // no listener row ⇒ no reading ⇒ no line
|
||||
svg.classList.toggle('on', group.listener != null);
|
||||
svg.classList.toggle('off', false);
|
||||
svg.classList.toggle('live', measured && count > 0);
|
||||
svg.classList.toggle('still', !measured || count === 0);
|
||||
svg.setAttribute('role', 'img');
|
||||
setTxt(..., trunkAria(group)); // written only through the helper below
|
||||
svg.title = <the same sentence>;
|
||||
```
|
||||
`on` is honest here for the same reason it is on a leaf: `listeners[]` contains **only** ports a config names (`sources/gateway.py` builds it from haproxy `bind`s and nginx `listen`s), so a trunk row existing *is* the existence fact. The `group.listener == null` arm is the one case where a group was made from a connection whose port no listener carries — hide the pipe rather than draw a claim about a port nobody read (that arm cannot happen with the captured configs; say so in the comment so the next reader does not delete the guard).
|
||||
- **New** `trunkAria(group)` — one sentence, every word from the payload, three arms and no number invented:
|
||||
- `":4443 — 9 connections are open on this port right now"`
|
||||
- `":4443 — no connections are open on this port right now"`
|
||||
- `":4443 — connections on this port are unknown: the socket table did not answer"`
|
||||
Never `"0"` in the unknown arm, and never a leaf's endpoint in any arm.
|
||||
- `renderGroups()` needs no change beyond calling the existing `renderTrunk`; `.solo` stays exactly as it is (it governs the leaf spine, not the trunk stub).
|
||||
2. `src/dashboards/web/static/dash.css`
|
||||
- One rule beside `.pipe`:
|
||||
```css
|
||||
/* The trunk's own stub: the port's live count drawn as a line instead of only as words. It is
|
||||
short and `flex: none` because it is a reading, not a run — the trunk has no endpoint to
|
||||
travel to, and a full-width line here would read as a route. Colour and motion come from the
|
||||
same classes the leaf run uses, so a port with traffic moves and an idle one does not. */
|
||||
.pipe-trunk { width: 26px; height: 10px; flex: none; }
|
||||
```
|
||||
No new token, no new keyframe: `.pipe.on .pipe-path` colours it and `.pipe.live .pipe-path` animates it, which is also why the file's one `@media (prefers-reduced-motion: reduce)` block covers it for free — assert that rather than restating selectors.
|
||||
- The `@media (max-width: 640px)` block gains nothing: the trunk is already a full-width stacked row there, and `flex: none` keeps the stub 26 px at every width.
|
||||
3. `tests/test_api.py`
|
||||
- **New** `test_the_trunk_pipe_is_an_accessible_image_of_its_port`: served JS builds a `.pipe.pipe-trunk` inside `.gate-trunk` through `buildPipe(true)`; `trunkAria` reads `listeners[].connections` and **only** that; its three arms print the strings above (asserted from the literal, so a paraphrase that drops "on this port" fails); `null` yields the unknown arm and no `live` class; the pipe is `hidden` when the group has no listener row.
|
||||
- **Restate** `test_a_pipe_moves_only_while_it_carries_a_connection` to cover both pipes and say out loud that they read from **two different payload paths**: a leaf from `connections[].connections`, a trunk from `listeners[].connections`, and `null` moves neither. Its message must state the trunk's claim is about a port and never about a leaf — that sentence is the reason the test exists.
|
||||
- `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` gains the assertion that the reduced-motion rule is written as `.pipe.live .pipe-path` / `.pipe.still .pipe-path` (a descendant selector, which is what covers `.pipe-trunk` without a second declaration), plus that `.pipe-trunk` is created by `buildPipe` so it can never grow a different class shape.
|
||||
- `test_every_pipe_is_an_accessible_image_built_from_the_payload` now finds **two** pipe builders and must assert both set `role="img"` and a non-empty `aria-label`/`title` derived from payload text — extended, not skipped.
|
||||
- `test_no_port_the_two_configs_do_not_name_appears_anywhere` (payload pin) stays green untouched.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above, plus a fixture render asserting `:443` animates and a port whose payload count is `0` does not, and that the unknown arm prints `n/a` in `.pipe-count` **and** the unknown sentence in `aria-label` — both directions of "never `0`".
|
||||
- Integration: with `sockets.available: null` stubbed into the fixture, every trunk is green, motionless, and reads `n/a`; no trunk anywhere on the page prints `0 live`.
|
||||
- Coverage: **>90%** on modified paths (no new Python); `node --check` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data: every port the kernel reports ≥ 1 ESTABLISHED socket on (`:443`, `:4443`, `:8443`, `:22`, … on the day) has a **moving** green stub at its trunk's right edge, and every port reporting `0` has a static one. Measured at 1440 px and 360 px with a screenshot each.
|
||||
- [ ] The stub's `aria-label` names the port and never an endpoint; reading three trunks aloud with a screen reader (or dumping `aria-label`s) yields the three pinned sentences and nothing else.
|
||||
- [ ] Stubbed `sockets.available: null`: stubs are green, static, and print `n/a` — **0** occurrences of `0 live` in the rendered page (asserted with a `grep` of the DOM in the browser pass).
|
||||
- [ ] The stub does not push the trunk's `:port` text or its `N names` line off the trunk: no clipping, no sideways document scroll at 360 px, and the leaf spine still starts at the trunk's right edge (measured).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated for the trunk's line and what it may claim (the `/gateway` conventions paragraph) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: leaf counts, the dash semantics, `.solo`, the charts and the switch are untouched.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Task 04 — One filter box finds a domain among the routes, client-side, and queries nothing
|
||||
|
||||
**Phase:** `31_gateway_route_readability` · **Source:** operator request 2026-10-23 — "I can't tell which `reeseapps.com` or `stackexpected.com` addresses flow to their reeselink counterparts (very important)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
A search field above the connection map narrows the two panels to the routes whose **payload text** contains what the operator typed — `brain`, `stackexpected`, `4443`, `minecraft`, `keycloak.reeselink.com` — because 33 rows in config order is not a lookup, and the operator's question is always about one domain. It is an eye and not a query: it hides rows the payload already sent, fires no request, re-derives no fact, and never touches the tallies or the warnings panel.
|
||||
|
||||
This is also the phase's answer to "group by zone, then": the page does **not** sort, split or colour by domain suffix (`reeseapps.com` vs `stackexpected.com` is a fact no config file states — rule 2). The operator types the suffix; the page hides the rest.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/gateway.html` — the "Connection map" `.page-head` only:
|
||||
```html
|
||||
<div class="page-head">
|
||||
<h2 style="font-size:.95rem;margin:0">Connection map</h2>
|
||||
<div class="spacer"></div>
|
||||
<input type="search" id="gate-filter" class="gate-filter" autocomplete="off" spellcheck="false"
|
||||
placeholder="filter by domain, endpoint or port"
|
||||
aria-label="Filter routes by domain, endpoint or port">
|
||||
<span class="chip faint" id="gate-filter-count" hidden aria-live="polite"></span>
|
||||
</div>
|
||||
```
|
||||
and one `<div class="pipe-empty faint" data-k="no-match" hidden></div>` inside each card's `.body.stack`, above the `Charts` disclosure, for the panel-level empty result. The card comments gain one sentence: the filter hides rows and is not a fetch (so the next reader does not "optimise" it into a query parameter).
|
||||
2. `src/dashboards/web/static/dash.css`
|
||||
- Add `input[type="search"]` to the existing shared rule (`input[type="text"], input[type="password"]` in the buttons/inputs section) rather than writing a one-off box: the control is then the same field the `/settings` and `/login` forms draw. No `type="search"` exists on any other page today, so the addition is inert everywhere else — pin that.
|
||||
- `.gate-filter { max-width: 18rem; min-width: 0; }` — `min-width: 0` because it sits in a flex `.page-head` and phase 27's rule (a flex/grid item may not be wider than its track) is exactly what stops a phone-width overflow; add `width: 100%` inside the ≤640 px block so the field is full-width there and the document still measures 0 px of sideways scroll.
|
||||
- `.gate-filter-count` needs nothing beyond `.chip`; no new colour token anywhere (a filter is not a verdict).
|
||||
3. `src/dashboards/web/static/page-gateway.js`
|
||||
- `state.filter = ''`, and a single pure matcher over **payload strings only** — never `textContent`, because a label the CSS ellipsised must still match its whole string:
|
||||
```js
|
||||
/** One row's haystack: every string the payload sent about it, and nothing else. */
|
||||
function haystack(conn) {
|
||||
const out = conn.out || {};
|
||||
return [String((conn.in || {}).port ?? ''), out.host, out.name, out.kind,
|
||||
...(out.selectors || [])].filter((s) => typeof s === 'string' && s)
|
||||
.join(' ').toLowerCase();
|
||||
}
|
||||
```
|
||||
`filter` + the `typeof s === 'string'` test are the guard that a `null` host or a missing `kind` never becomes the literal text `"null"` and matches a query for `null`.
|
||||
- The trunk matches on its own payload (`:${port}`, `domains[]`, `clients[]`); a group is visible when the trunk matches **or** any leaf matches; when only the trunk matched, **all** its leaves stay visible (typing `4443` asks for the port, not for one route).
|
||||
- Visibility is applied **inside the existing patch pass** (`renderLeaves()` sets `li.hidden`, `renderGroups()` sets the group's `hidden` and the card's `[data-k="no-match"]`), so:
|
||||
- a poll cannot undo the filter (the patch re-applies it every refresh),
|
||||
- clearing it repaints from `state.snapshot` with no network round trip,
|
||||
- phase 30's build-once/patch-after rule holds: rows are hidden, never removed, so `state.rows` keeps its identity and a row that reappears keeps its `<svg>` and its `aria-label`.
|
||||
- The `input` handler is `state.filter = value.trim(); repaintFromSnapshot();` where `repaintFromSnapshot()` re-runs the render pass on `state.snapshot` and issues **no** `fetchJSON`. A `keydown` handler for `Escape` sets `state.filter = ''`, empties the field, and repaints (a `type="search"` clear button fires `input` in Chrome; Escape-on-some-browsers does not, and a filter you cannot leave with the keyboard is a trap on a page whose other shortcuts are all keys).
|
||||
- `#gate-filter-count` reads `12 of 33 routes` **only while the filter is non-empty** (`hidden` otherwise — a permanent tally is the ink the empty-state rules exist to avoid) and counts **leaf routes**, not groups. `[data-k="no-match"]` prints `no route matches “<query>”` and only when the panel's groups are all hidden by the filter — never when the panel is hidden for `sections` reasons.
|
||||
- `renderHeader()` and `renderWarnings()` are untouched and must not gain a single reference to `state.filter`: `ports`, `connections`, `live` and `warnings` describe the ingress, not the operator's current view of it.
|
||||
4. `tests/test_api.py`
|
||||
- **New** `test_the_filter_matches_only_payload_text_and_costs_no_request`:
|
||||
- served JS: `haystack` indexes `in.port`, `out.host`, `out.name`, `out.kind`, `out.selectors` and nothing else; it drops non-strings (so `null` cannot match `"null"`);
|
||||
- the filter path contains no `fetchJSON` and no call to `refresh()`;
|
||||
- `renderHeader` and `renderWarnings` contain no `state.filter` reference, and `#gate-filter-count` is the only tally the filter moves;
|
||||
- visibility is set on rows/groups inside `renderLeaves`/`renderGroups` (the patch pass), not by removing nodes from `state.rows`.
|
||||
- **New** `test_the_filter_field_is_pinned_markup_and_the_shared_input_rule_covers_it`: `#gate-filter` exists on the served page with its `aria-label`, `input[type="search"]` is in the shared input rule's selector list, and **no other served page** contains `type="search"` (so widening that rule cannot restyle another page by accident).
|
||||
- `test_no_page_carries_an_id_nothing_reads` (and the same audit run unsigned on `/login` in `tests/test_auth_api.py`) must stay green — both new `id`s (`#gate-filter`, `#gate-filter-count`) are read by the served script, and the `[data-k="no-match"]` element is reached through `querySelector`, not an `id`.
|
||||
- `test_the_gateway_page_is_exactly_two_connection_panels_and_a_warnings_list` stays green: the filter sits in the `Connection map` head, and the pin's `<section>` count must not change (assert it in the task's own run, not by editing the pin's count).
|
||||
- The flavour-text audit stays green: `placeholder` and the no-match sentence are functional text; nothing on the page narrates that a filter exists or how it works.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; a fixture render asserting `brain` leaves exactly the `brain.reeseapps.com` + `brain-of-code.reeseapps.com` rows visible with their `:443` trunk shown, `4443` leaves that trunk's routes all visible, `zzzz` hides every group and shows both panels' no-match sentences, and clearing restores all 33 rows without a second history request (spy on the client).
|
||||
- Integration: the filter survives a poll that adds a route (the new row is filtered on the pass that creates it), and survives the panel-level arms (`supported: false` → no groups → filter shows nothing to filter, no crash).
|
||||
- Coverage: **>90%** on modified paths (no new Python); `node --check` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data at 1440 px: typing `stackexpected` leaves only the `stackexpected.com`/`gitea.stackexpected.com`/`keycloak…`/`nextcloud…`/`hedgedoc…` map leaves and the two haproxy/`stackexpected-gitea` routes visible, the tallies still read the unfiltered ingress (`ports`, `connections`, `live`, `warnings` unchanged), and the chip reads `N of 33 routes`.
|
||||
- [ ] Clearing the field (button or Escape) restores every row with **no** new request — verified by an empty network log between the keystroke and the repaint.
|
||||
- [ ] A query that matches nothing prints the no-match sentence in each visible panel and never a blank card, never a stray `:80` trunk.
|
||||
- [ ] At 360 px: the field is full-width, the document measures 0 px of sideways scroll, and no group is clipped (the mobile audit stays at 0 FAIL / 0 FINDING).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated: the `/gateway` paragraph names the filter and states the refusal (no zone grouping by suffix) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: tallies, warnings panel, charts, switch, dash semantics untouched.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Task 05 — Docs, the browser measurement, and a read-only pass over the real ingress
|
||||
|
||||
**Phase:** `31_gateway_route_readability` · **Source:** phase completion pass (AGENTS.md definition of done: UI changes are verified in a real browser with a screenshot, and `README.md`/`AGENTS.md` are re-read whole).
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove tasks 01–04 on live data with measurements rather than prose, carry the new rules into `AGENTS.md` and `README.md`, restate or close gap 13 with the phase's own clip counts, and confirm the phase really was page-only by showing `GET /api/gateway/snapshot` byte-identical and every other source still `ok`.
|
||||
|
||||
## Work
|
||||
|
||||
1. `data/logs/31-route-readability.cjs` — a new Playwright pass (same house shape as `data/logs/30-connection-map.cjs`: `NODE_PATH=<playwright node_modules>`, `DASH_BASE`/`DASH_PW` overridable, signs in with `DASH_ADMIN_PASSWORD`, exits non-zero on any FAIL). Checks, each printed with the number it measured:
|
||||
- **existence colour**: on the nginx and haproxy panels, for a sample of at least 5 rows including a `.solo` haproxy row and a branched `:443` leaf — computed `borderLeftColor` of `::before`/`::after` (via `getComputedStyle(el, '::before')`), `borderTopColor` of `.gate-elbow`, computed `stroke` of `.pipe-path`, and the switch's `borderTopColor` all equal each other and equal `--gate-on`'s computed value; and **no** element in `.gate-map` computes a dashed border in `--border-strong`'s value. Both themes (toggle through `#theme-toggle`).
|
||||
- **row order**: every leaf's line 1 resolves to a value that appears in that row's payload `out.selectors` and line 2 contains `out.host` + `out.port`; the two strings are both present in `title` and in the pipe's `aria-label`. Reports the clip counts (which lines are ellipsised) at 1440 px and 360 px — these are gap 13's new numbers.
|
||||
- **trunk motion**: for each group, `getComputedStyle(trunkPath).animationName` is `pipe-flow` iff `listeners[].connections >= 1`, and `none` for a `0`/`null` port; assert the rendered DOM contains **zero** occurrences of `0 live`; assert every `.pipe-trunk[aria-label]` names its own port and contains no hostname.
|
||||
- **filter**: type `stackexpected` → assert the visible leaf set equals the payload rows whose haystack contains it, the four header tallies are unchanged from before the keystroke, and **no** network request leaves the page (CDP `request` listener empty between keystroke and repaint); press Escape → full set back, still zero requests.
|
||||
- **reduced motion**: with `prefers-reduced-motion: reduce` emulated, nothing in `.gate-map` animates (leaf runs, trunk stubs, switch thumb).
|
||||
- **degradation arms**: inject/stub each and check the page still says the payload's own words and colours nothing red — `unreachable: true`, `supported: false` with `why`, `sections: {nginx: false}`, `sockets.available: null`, and `404 {configured: false}` (the setup block).
|
||||
- **no sideways scroll**: `document.scrollingElement.scrollWidth === clientWidth` at 1440 px and 360 px with the widest fan-out expanded and both charts open.
|
||||
- Screenshots: `data/logs/31-gateway-{1440,360,light,filtered,no-match,trunk-live,predates,phone}.png`; transcript `data/logs/31-route-readability.out`.
|
||||
2. Re-run `data/logs/27-task02-mobile-audit.cjs` (the nine-page sweep, `/gateway` included) and record the tally in the transcript — it must be **0 FAIL / 0 FINDING**, exit 0. The `#gate-filter` field and the trunk stub are the two new things it measures; if it finds a clipped fact, fix the layout rather than the audit.
|
||||
3. `AGENTS.md` — update, in full, everything this phase made stale:
|
||||
- the `/gateway` UI-conventions paragraph: the row's reading order (inbound word → `→ counterpart`), the fact that **one** `on`/`off` class on the `<li>` paints spine, tick, run and switch, why motion stays the per-row run's claim (the spine segment is shared between two leaves), what the trunk stub is and the exact limit of its claim (a port, never a leaf), and the filter — what it matches, that it costs no request, and that it never moves the tallies or the warnings panel.
|
||||
- the sentence about the branch ("The branch is four CSS rules and no JavaScript…") now has to say the connectors are existence-coloured from `--gate-on`/`--gate-off` with a `transparent` base, not structural ink.
|
||||
- **gap 13**: restate it with this phase's measured clip counts at both widths. If the inbound-first line means nothing clips at 1440 px any more, close the gap with the measurement and say what remains at 360 px; do not delete it on the strength of the filter.
|
||||
- the `countText()` / dash sentence: the footer now explains the dash in operator terms — reflect that, and note that phase 32 removes the reason for it.
|
||||
- any claim that the page's footer is three paragraphs (task 02 changed wording, not count — verify, don't assume).
|
||||
4. `README.md` — the `/gateway` table row and the gateway prose section: the page answers "which inbound domain flows where", the filter exists, the trunk line is a port reading. Keep the "unconfigured is not red" and "agent 1.2.0" statements intact (this phase changes neither).
|
||||
5. The read-only live proof:
|
||||
- `GET /api/gateway/snapshot` over a signed-in session, before the phase (stash it at task 01's start) and after task 04 — **byte-identical**, printed as a `diff` that must be empty. This is the pinned proof the phase is page-only and cannot have moved a payload contract.
|
||||
- `./scripts/dev-server.sh status` → all ten sources `ok`, `gateway` included.
|
||||
- `git diff --stat` shows changes under `src/dashboards/web/` and `tests/` only — no `src/dashboards/*.py`, no `agent/`, no `scripts/`.
|
||||
6. Record the follow-on intent in `AGENTS.md`'s gap list **without** pre-writing their design: one line each that name-endpoint liveness (resolution, agent 1.3.0) and route enable/disable (a second gated upstream write, which amends invariant 1 and needs the operator's `PLAN.md` sign-off) are the next two phases. No implementation detail beyond what this phase's decisions 2 and 7 already state.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: this task **is** the integration pass — every claim in tasks 01–04 gets a measured line in the transcript, and the transcript plus the screenshots are the artifacts the phase's Completion Criteria name.
|
||||
- Coverage: no code change expected; if the browser pass forces a fix, that fix goes through the same pins as the task it belongs to, and the affected test is named in the transcript.
|
||||
- The whole suite, `ruff`, `pyright`, `node --check` and `bash .agents/validate.sh` green at the end of the phase.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `data/logs/31-route-readability.out` exists, exits 0, and contains the computed-colour line for at least 5 rows in both themes, the clip counts at both widths, the trunk-motion truth table, the filter's zero-request proof, and the reduced-motion pass.
|
||||
- [ ] The eight `data/logs/31-gateway-*.png` screenshots exist, including one `filtered` showing a `stackexpected` result set and one `phone` at 360 px.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING and its tally recorded (`/gateway` in the sweep).
|
||||
- [ ] The before/after `diff` of the live `/api/gateway/snapshot` is empty, and `./scripts/dev-server.sh status` lists ten `ok` sources.
|
||||
- [ ] `git diff --stat` for the phase touches only `src/dashboards/web/`, `tests/`, `README.md`, `AGENTS.md`, and `data/logs/` (gitignored, present on the box).
|
||||
- [ ] `AGENTS.md` and `README.md` describe the page as it now renders, gap 13 restated with measurements, and a full read of both files finds no other inconsistency (fix it in the same commit, including pre-existing ones).
|
||||
- [ ] `uv run pytest` **100% passing** (every test in the project), coverage floor green, `uv run ruff check src tests agent` + `uv run pyright` clean, `bash .agents/validate.sh` green.
|
||||
- [ ] No behaviour change in completed work: `/`, `/system`, `/disks`, `/llm`, `/ups`, `/settings`, `/certs`, `/login` render as before.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Phase 32 — Names become addresses where they are actually dialled: every route gets a real live count
|
||||
|
||||
**Source:** operator request, 2026-10-23 — "The only animated line is `:443 -> 127.0.0.1:4443`." + the decision taken in that conversation: **the agent resolves the names it reads**, because the resolution nginx's `resolver 10.1.0.1` and haproxy's `default-server init-addr last,libc,none` perform happens *on the gateway*, and a laptop- or dashboard-side lookup is a different vantage answering a different question.
|
||||
**Story:** n/a
|
||||
**Context:** `agent/host_agent.py` (1.2.0 → **1.3.0**: the `haproxy`/`nginx` parsers that already produce every target name, the `sockets` reader, the `AGENT_VERSION` + `/schema` contract, the `_smart_lock`/`_smart_refreshing`/`_refresh_smart` cache-and-background-pass precedent, `--no-proxy`/`--no-smart`/`--root`), `src/dashboards/sources/gateway.py` (`_connection()`'s `address`/`joinable`/`metric` rule, `join_sockets()`, `endpoint_of()`, `samples()`), `src/dashboards/collectors.py` (`ADDON_ERROR_PREFIXES` — the cross-module contract AGENTS.md says a new add-on section must satisfy **in the same commit**), `scripts/capture-gateway-config.sh` (the six fixtures + its "no privilege, no copy tool, a bad read is never a fixture" rules + the RFC 5737/3849 address scrub), `tests/test_gateway_source.py`, `tests/test_agent.py`, `tests/fixtures/gateway_*`, `data/logs/verify-28-final.py` (the side-by-side additivity control this phase copies).
|
||||
|
||||
## Objective
|
||||
|
||||
`agent/host_agent.py` **1.3.0** resolves every target name the two proxy configs declare — with `socket.getaddrinfo`, i.e. the same libc resolution haproxy itself uses, on the machine that opens the connections — and publishes the address set, the vantage and the timestamp per name. `sources/gateway.py` then joins the kernel's ESTABLISHED sockets through those addresses, so `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443` carries a **number** and its line **moves**, instead of a permanent em dash on 31 of 33 routes.
|
||||
|
||||
This is a measurement with provenance, not a guess with better marketing: the payload says *which* addresses, *how* they were obtained (`via: "libc"`), and *when*; a name that does not resolve right now publishes no addresses, gets no gauge, and prints the dash with the reason. `sources/gateway.py` still never calls a resolver — that spy pin survives this phase on purpose, because a dashboard resolving through its own network would be measuring a different box's DNS than the one routing the traffic.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Resolution happens on the gateway, in the agent.** | `host_agent.py` gains a `resolver` section. `sources/gateway.py` keeps its `socket.getaddrinfo` spy pin (fails if the module ever resolves), and `tests/test_gateway_source.py` gains the matching assertion that the only resolver in the system is the agent's. The dashboard's own network vantage is not the routing vantage; a lookup from here would be a fact about *this* box's `resolv.conf`. |
|
||||
| 2 | **libc, and the vantage is printed.** | `via: "libc"` on every entry (what `getaddrinfo` is), plus the nginx config's own `resolver` directive captured verbatim as **`resolver.config_directive`** (`"10.1.0.1"` on this fleet) so the page can name both and an operator can see a divergence. It lives inside the new section rather than inside `nginx` so that `nginx` stays byte-stable and the phase's additivity control remains a strict top-level key comparison with no exception list. The agent does **not** hand-roll UDP DNS queries to that resolver: stdlib only, no new dependency, and a divergence is a config finding for the operator rather than a second resolver implementation to maintain. |
|
||||
| 3 | **Only names the configs wrote are ever resolved.** | The name set is derived from the parsed `haproxy` `server` targets and the parsed `nginx` `proxy_pass`/`upstream`/`map` targets — nothing else, ever. Not a name from `/proc`, not a client address, not anything a request supplied. Pinned by asserting the spied `getaddrinfo` calls are exactly the config's target set. |
|
||||
| 4 | **Bounded, cached, never on the request path after the first pass.** | `RESOLVE_MAX_NAMES = 64` (the excess is **one** line naming the count — the include/smart budget shape), `RESOLVE_CACHE_SECONDS = 300`, one single-flight background refresher (the `_smart_refreshing` idiom), a per-pass budget so a black-holing resolver cannot hold `/snapshot`. A wedged lookup must never blow `HOST_TIMEOUT_SECONDS` and turn `/system` red over an add-on it never draws — the exact lesson SMART learned. |
|
||||
| 5 | **Three-way `available`, and `--no-resolve` removes the key in silence.** | `true` resolved what it was asked, `false` no proxy sections to read from, `null` the pass could not run — each non-`true` arm exactly one `resolver:` line (the pinned-wording contract shared with `smartctl:`/`haproxy:`/`nginx:`/`sockets:`). `--no-resolve` is `--no-smart`'s precedent: the key is **absent** ("not collected"), which stays distinguishable from "collected and empty". |
|
||||
| 6 | **A resolved name is joinable; an unresolved one is not.** | `_connection()`'s `metric`/`joinable` test becomes `address is True or resolved-address-set is non-empty`. Name-keyed gauges (`gateway.out_conns@<host>:<port>`) come back **because they now have measurements** — the AGENTS.md passage that retired them said a name-keyed series "would be a chart of a number that is always `null`", and this phase is the reason that stopped being true. The port stays in the name (`:25565`/`:25566` on one host are two services). |
|
||||
| 7 | **The shared-endpoint rule now has a real case and stays exactly as written.** | Two names resolving to the same `(address, port)` are one count **repeated** on both lines with `shared: true` — never halved, never attributed to one of them. `samples()` keeps de-duplicating by metric, and `live_out` in `/api/summary` keeps summing **distinct endpoints** (`api._endpoint_values()`). |
|
||||
| 8 | **Nothing moves that the operator did not ask about.** | `warnings()` is unchanged (resolution is not a config-integrity question); `/api/summary`'s `gateway` block gains **no** thirteenth key; `listeners[]` and the inbound join are untouched; the switch stays unwired and unwire-able (phase 33), and `routeInConfig()` keeps returning `true`. |
|
||||
|
||||
**Out of scope:** any config write, any new sample *family* (only the meaning of `gateway.out_conns@…` widens), any change to `/api/health` or `/api/config`, any `severity` (a count is not graded), the enable/disable path (phase 33), and re-bundling or re-issuing certificates (the agent moves in place on its existing bundle, exactly as phase 28 moved the gateway 1.1.0 → 1.2.0).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `31_gateway_route_readability` (todo, **must complete first**) — the row that reads inbound → counterpart and the green end-to-end route are what a live count paints. Its payload-purity guarantee (`sources/gateway.py` untouched) is what makes this phase's rewrite land cleanly.
|
||||
- `30_gateway_connection_map` (complete) — `connections[]`, the join, `endpoint_of()`, `shared`, the gauge names, `warnings()`.
|
||||
- `28_gateway_agent_proxy` (complete) — the parsers whose target set this phase resolves, the fixture capture script and its scrub rule, and `tests/test_agent.py`'s fake-tree `--root` machinery the resolver tests reuse.
|
||||
- `24_dev_mtls_rollout` / the fleet record — the rollout is phase 28 task 05's procedure on one host (`gateway`), on its existing bundle, with `systemctl restart` and a read-only probe first.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_agent_resolves_the_names_it_reads.md` — agent 1.3.0: the `resolver` section (`available`, `via`, `config_directive`, entries), `RESOLVE_MAX_NAMES`/`RESOLVE_CACHE_SECONDS`/pass budget, the single-flight background pass, three-way `available`, `--no-resolve`, `/schema`, `AGENT_VERSION`.
|
||||
2. `02_resolution_ground_truth.md` — `capture-gateway-config.sh` records the resolver document and the two config files' `resolver` line; the RFC-5737/3849 scrub extended to it; the 1.2.0-vs-1.3.0 **additivity control** (`data/logs/verify-32-additive.py`, copied from `verify-28-final.py`).
|
||||
3. `03_join_through_the_address_the_name_became.md` — `sources/gateway.py`: `out.resolved`/`via`/`resolved_ts`, the widened `joinable`/`metric` rule, the address-set match, `shared` unchanged, `null` refusals, and `collectors.ADDON_ERROR_PREFIXES` gaining `resolver:` **in this commit**.
|
||||
4. `04_the_page_says_which_unknown_it_is.md` — `page-gateway.js` + `gateway.html`: the dash splits into "the name did not resolve in this reading" vs "resolved and nothing is open", the `libc`/timestamp provenance on `title`, footer paragraph 2 rewritten, tallies unchanged.
|
||||
5. `05_roll_the_gateway_to_1_3_0.md` — read-only probe first, then the phase-28 procedure on `gateway` (re-stage, install, restart, verify both ways), the live join numbers recorded, and the fleet record updated with the new version spread.
|
||||
6. `06_docs_and_live_verify.md` — `AGENTS.md` (the agent bullet, `sources/gateway.py`, `collectors.py`, the metric-naming passage, the `/gateway` UI paragraph, the fleet record, gap 13) + `README.md`, the browser pass with screenshots, the mobile-audit re-run, the full gate.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **Hermetic by construction.** No test dials real DNS: `tests/test_agent.py` monkeypatches `socket.getaddrinfo` and asserts (a) it is called with **exactly** the parsed target names and ports and nothing else, (b) the cache means a second `/snapshot` costs zero lookups, (c) a pass over `RESOLVE_MAX_NAMES` reports one line and keeps `available: true`, (d) a raising/timeout-simulating resolver yields `status`-bearing entries and never fails `/snapshot` or `/health`.
|
||||
- **Real captures for shape.** `tests/fixtures/gateway_resolver.json` (documentation addresses, verbatim names) plus the existing six fixtures drive `tests/test_gateway_source.py`: every captured name route gets a count when the recorded `/proc/net/tcp{,6}` holds a socket to its recorded address; `minecraft.reeselink.com:25565` and `:25566` stay two distinct gauges; two names → one endpoint is one count twice with `shared: true`; an entry with no addresses publishes `connections: null`, **no** metric, and no sample.
|
||||
- **The spy pins are restated, not relaxed.** `socket.getaddrinfo` stays forbidden in `sources/gateway.py` (spy) and `0` becomes "no sample", never "an empty chart" — a pipe whose name resolved to nothing publishes no point.
|
||||
- **Live (`-m live --live`)**: one new test reads the real gateway and asserts every `map` entry whose `resolver` status is `ok` carries a finite count **or** `0`, and that none of them is `null` — the operator's complaint, pinned against the box. Skips when `GATEWAY_HOST` is not a target.
|
||||
- Coverage **>90%** on `agent/host_agent.py`, `sources/gateway.py`, `collectors.py`; the whole suite passes 100% and `bash .agents/validate.sh` is green, coverage floor included.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase, fixing every inconsistency **including pre-existing ones**. Specifically: AGENTS.md's agent bullet gains the `resolver` section (and its `--no-resolve`, its bounds, its three-way `available`), its `sources/gateway.py` bullet's join paragraph is rewritten (the join now has two address sources: written-as-address and resolved), the **"no name-keyed `gateway.out_conns@…` name is ever inserted"** sentence in the metric-naming passage is replaced with the widened rule and *why it changed*, the `collectors.ADDON_ERROR_PREFIXES` list gains `resolver` (the "prefix in the same commit" rule), the `/gateway` UI paragraph's dash sentence splits in two, and the **fleet record** moves `gateway` to 1.3.0 with the deskwork/driveripper spread and what the spread costs.
|
||||
- `README.md`: the `/gateway` table row and the gateway prose now describe counted name routes, the resolution vantage, and the capture script's new outputs.
|
||||
- Tasks update both files immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `GET /schema` on the gateway agent answers `agent_version` **1.3.0** with a `resolver` block (its `available` flag named) and `--no-resolve` listed; `tests/test_agent.py` pins the constant and the schema text.
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress: **0** `connections[]` rows with `connections: null` among those whose target resolved; `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443` carries a finite count; `socket.getaddrinfo` is still never called anywhere under `src/dashboards/` (spied) and every address in `out.resolved` names `via: "libc"` with a `resolved_ts`.
|
||||
- [ ] Two names resolving to one `(address, port)` draw one count repeated with `shared: true` and **one** stored sample; `/api/summary`'s `gateway.live_out` still equals the sum over **distinct** endpoints (pinned before and after).
|
||||
- [ ] A stubbed `nxdomain` for one name publishes `out.resolved: null`, no `metric`, no sample, and the page prints the "did not resolve in this reading" sentence — never `0`, never a silent blank.
|
||||
- [ ] `--no-resolve` on a loopback agent removes the `resolver` key entirely and `/api/gateway/snapshot` behaves exactly as phase 30/31 (names uncounted, `supported: true`); a 1.2.0 agent on any host behaves identically (the spread is harmless — pinned by a fixture without a `resolver` key).
|
||||
- [ ] `/system` is not `degraded` on a host whose agent reports `resolver: nxdomain …` — the `ADDON_ERROR_PREFIXES` pin (invariant 3), asserted both ways.
|
||||
- [ ] `data/logs/verify-32-additive.py` proves the 1.3.0 document equals the 1.2.0 document key-for-key **plus** `resolver`, and 1.3.0 with `--no-resolve` equals it exactly.
|
||||
- [ ] `uv run pytest` (incl. `--cov` ≥ 94), `uv run pytest -m live --live`, `uv run ruff check src tests agent`, `uv run pyright`, `node --check`, `bash .agents/validate.sh` — all green; `uv pip list --outdated` empty (no dependency moved, and this phase must not add one).
|
||||
- [ ] `data/logs/32-*.png` at 1440 px and 360 px (counted routes, an unresolved arm, both themes) + `data/logs/32-*.out`; `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING.
|
||||
- [ ] `./scripts/dev-server.sh status` lists all ten sources `ok`, `gateway` included, with `host:gateway`/`disks:gateway` unaffected by their host's agent upgrade.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs`, `/login`, the warnings panel, the unwired switch and every non-gateway source render and behave as phase 31 left them.
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Task 01 — The agent resolves the names its own configs declare, off the request path
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** operator decision 2026-10-23 — "the agent resolves the names it reads"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`agent/host_agent.py` goes **1.2.0 → 1.3.0** and gains one additive top-level section, `resolver`: for every `(name, port)` the two parsed proxy configs write as a target, the libc address set, its status, the vantage and a timestamp — obtained on the machine that dials, bounded, cached, and refreshed on a background thread so a slow or black-holing resolver can never hold `/snapshot` open. Everything the agent published before is byte-identical; `--no-resolve` removes the key in silence.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py`
|
||||
- Constants beside the proxy section's bounds, each with a comment naming what it protects:
|
||||
```python
|
||||
AGENT_VERSION = "1.3.0"
|
||||
RESOLVE_MAX_NAMES = 64 # a gateway's config-declared targets, with room to grow
|
||||
RESOLVE_CACHE_SECONDS = 300.0 # a LAN target moves when the operator moves it, not every poll
|
||||
RESOLVE_PASS_BUDGET_SECONDS = 3.0 # a black-holing resolver must not hold /snapshot open
|
||||
RESOLVE_VIA = "libc" # getaddrinfo, i.e. the resolution haproxy's libc path performs
|
||||
```
|
||||
- **New pure helper** `target_names(haproxy: Mapping | None, nginx: Mapping | None) -> list[tuple[str, int | None]]`
|
||||
Derived **only** from the parsed sections: haproxy `backends[].servers[].target` (in config order), then nginx `upstreams` members, `server`-block `proxy_pass` targets, and `maps[].entries[].target`, deduplicated keeping first-seen order. An IP literal is kept as a target with `addresses` set without any lookup (it is already an address, and the lookup would be a round trip to say so) and marked `status: "literal"`. A name from `/proc`, from a request, or from anywhere else must never enter this list — that is decision 3 and it is pinned by asserting the spied call set equals this function's output.
|
||||
- **The pass, following the SMART idiom exactly** (`_smart_lock` / `_smart_cache` / `_smart_refreshing` / `_refresh_smart`):
|
||||
`_resolver_lock`, `_resolver_cache: dict[str, dict[str, Any]]` keyed on `f"{name}:{port}"` (a name on two ports is two targets — `minecraft.reeselink.com` on `25565` **and** `25566` is on this fleet today), `_resolver_refreshing: bool`, `_resolver_errors: list[str]`.
|
||||
```python
|
||||
def resolver_pass(self, targets) -> tuple[dict[str, dict[str, Any]], list[str]]:
|
||||
"""One bounded libc lookup per declared target. Never raises out of this function."""
|
||||
```
|
||||
For each target: `socket.getaddrinfo(name, port, proto=socket.SOCK_STREAM)`, collect the unique `sockaddr[0]` values, keep both families, sort address-family-stable (v4 ascending then v6 ascending) so the payload is deterministic across polls. `socket.gaierror` with `EAI_NONAME`/`EAI_NODATA` → `status: "nxdomain"`, any other `gaierror`/`OSError` → `status: "error"`, both with `addresses: []`. After each lookup compare elapsed against `RESOLVE_PASS_BUDGET_SECONDS`; on exhaustion stop and append **one** line — `resolver: pass budget 3.0s exhausted, 41 of 58 targets looked up` — mirroring the smart pass-budget wording the agent already prints (phase 16 pins that family of tests; match its shape so the next reader has one idiom, not two).
|
||||
- Entry shape (and the whole section):
|
||||
```python
|
||||
"resolver": {
|
||||
"available": True | False | None,
|
||||
"via": "libc",
|
||||
"config_directive": "10.1.0.1" | None, # nginx's own `resolver` line, verbatim arguments
|
||||
"ts": <wall clock of the pass>,
|
||||
"entries": [{"name": …, "port": …, "status": "ok|nxdomain|error|literal",
|
||||
"addresses": […], "ts": <per-entry, when it was answered>}],
|
||||
"errors": [ …at most one line per arm… ],
|
||||
}
|
||||
```
|
||||
`config_directive` lives **here** rather than inside the `nginx` section on purpose: it keeps the `nginx` document byte-stable so phase 32's additivity control stays a strict top-level key comparison instead of an exception list. Read it from the nginx config text the agent already has (`resolver` directive, arguments joined by a space, `None` when the file has none) — no second file read.
|
||||
- `available` is the three-way answer the other add-ons use: `true` the pass ran, `false` there is nothing to resolve because the proxy sections were not read, `null` a pass that could not run — and **exactly one `resolver:` line per non-`true` arm**, the missing/no-proxy case saying **not found** (the pinned cross-module wording the dashboard reads).
|
||||
- Cache policy: the **first** pass runs inline (the SMART precedent, bounded by the budget); an expired cache afterwards refreshes on a single-flight daemon `threading.Thread` while the request is served the previous result; a failed refresh keeps the previous entries and retries on the next expiry instead of hammering DNS. `resolver.entries[].ts` is always the time that entry was actually answered, so a served-from-cache entry cannot pretend to be current.
|
||||
- `--no-resolve` (argparse, no env default — `--no-smart`/`--no-proxy` precedent): the key is **absent** from `snapshot()` ("not collected", which stays distinguishable from "collected and empty"), silently, and `--once` prints the same absence. `--no-proxy` also removes it (nothing to resolve on a box that collects no proxy) — say so in one comment.
|
||||
- `AGENT_VERSION` bump, the module docstring's `/schema` block gains the `resolver` section with its `available` flag and `--no-resolve`, and the status page's path list is unchanged (this task adds no path — task 02 of phase 33 is what adds a POST).
|
||||
2. `tests/test_agent.py`
|
||||
- `test_the_agent_version_is_1_3_0_and_the_schema_names_the_resolver_section` (the constant + the `/schema` text names `resolver`, its `available`, and `--no-resolve`).
|
||||
- `test_only_the_targets_the_configs_declare_are_resolved` — monkeypatch `socket.getaddrinfo`, run a snapshot against the fake-tree configs, assert the `(name, port)` pairs it was called with are **exactly** `target_names(...)`' output for the same tree, in config order, with no duplicates, and that no call carries an address from `/proc/net/tcp`.
|
||||
- `test_the_resolver_cache_means_one_lookup_per_target` — two snapshots, one call set; expiry triggers a background refresh and the first of the two responses is served from cache with its original per-entry `ts`.
|
||||
- `test_a_slow_resolver_cannot_hold_the_snapshot_open` — a `getaddrinfo` stub that sleeps: the pass stops at the budget, prints the one budget line naming the counted and total figure, `available` stays `true`, `/snapshot` and `/health` still answer.
|
||||
- `test_nxdomain_is_a_status_and_never_a_failure` — one raising name yields `status: "nxdomain"`, `addresses: []`, other entries intact, snapshot `ok`.
|
||||
- `test_no_resolve_removes_the_key_in_silence` and `test_a_host_with_no_proxy_sections_says_so_once`.
|
||||
- `test_the_resolver_lines_never_fail_the_snapshot` — every arm, plus the existing "no section can fail `/snapshot`" family extended to cover the new one.
|
||||
- An IP-literal target is reported `status: "literal"` with its address and **no** lookup.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; hermetic by monkeypatching `socket.getaddrinfo` — **no test resolves a real name**.
|
||||
- Coverage **>90%** on the new agent code; `uv run ruff check agent` + `uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Run against a fake tree built the way `tests/test_agent.py::nginx_tree()` builds one (the captured `gateway_nginx_main.conf` + `conf.d` + `stream.d` under `<tmp>/etc/nginx/`, with `--haproxy-config tests/fixtures/gateway_haproxy.cfg`): `python3 agent/host_agent.py --once --no-smart --root <tmp> --haproxy-config tests/fixtures/gateway_haproxy.cfg --nginx-config <tmp>/etc/nginx/nginx.conf` prints a `resolver` block with `available`, `via: "libc"`, `config_directive: "10.1.0.1"` and one entry per declared target — and prints **no** `resolver` key with `--no-resolve` added.
|
||||
- [ ] `grep -n 'AGENT_VERSION = ' agent/host_agent.py` reads `1.3.0` and the `/schema` docstring names `resolver`.
|
||||
- [ ] Two consecutive `--once` runs with the stubbed resolver perform one lookup set, not two.
|
||||
- [ ] `uv run pytest -k agent` **100% passing**; the whole suite green; coverage not down.
|
||||
- [ ] README.md/AGENTS.md updated for the new agent section and flag (this task's own user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: every pre-1.3.0 section is byte-identical for the same `--root` (formally proven in task 02's control).
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 02 — Ground truth for the resolution, and a control that proves 1.3.0 is only additive
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md invariant 7 ("verify against the deployed upstream; never guess a metric name") + the phase-28 additivity precedent
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Two things this phase must be able to point at: a **captured** resolution document from the real gateway, recorded through the same no-privilege, never-a-fixture-on-a-bad-read discipline `scripts/capture-gateway-config.sh` already keeps; and a **control** that proves agent 1.3.0's snapshot equals 1.2.0's key-for-key plus `resolver` — so the upgrade of a live ingress is provably additive rather than "the page still looks fine".
|
||||
|
||||
## Work
|
||||
|
||||
1. `scripts/capture-gateway-config.sh`
|
||||
- After the existing config captures, derive the target set **from the repo's own parser**, never by re-implementing one:
|
||||
```python
|
||||
# The name list is what the agent will ask for, from the configs we just captured — one
|
||||
# source of truth. If `target_names()` changes, this capture follows it, and a fixture
|
||||
# that lists a name the agent no longer resolves is a fixture that fails a test.
|
||||
from agent.host_agent import target_names
|
||||
```
|
||||
(`bash` calls `uv run python - <<'PY'`, same interpreter the tests use.)
|
||||
- One **world-readable, unprivileged** read per name, over the same `$GATEWAY_SSH` alias, which is libc resolution *on the box that dials* — the exact vantage `via: "libc"` claims:
|
||||
`ssh "$GATEWAY_SSH" getent ahostsv4 <name>` then `getent ahostsv6 <name>`. No `sudo`, no `dig`/`host` assumption, no copy tool.
|
||||
- `config_directive` is read from the **already captured** nginx text (the `resolver` directive's arguments) — no extra round trip, and the fixture then matches what the agent will read out of the same bytes.
|
||||
- Write `tests/fixtures/gateway_resolver.json` in the agent's own entry shape, sorted in the agent's order, with `ts` set to a fixed capture constant so the fixture is byte-stable, and:
|
||||
- **names verbatim** — the domains are the ingress map, exactly as the config fixtures keep them;
|
||||
- **addresses scrubbed by the socket table's rule**: keep RFC 1918 / ULA / loopback, rewrite any address `ipaddress` calls globally routable into documentation space (RFC 5737 / RFC 3849), first-seen order, same spelling, **idempotent** — invariant 6, the same reason `/proc/net/tcp` is scrubbed (a global address is not this repo to publish);
|
||||
- a `nxdomain` is recorded as a **status**, not dropped.
|
||||
- Failure discipline, unchanged in spirit and pinned: a failed `ssh`, a missing `getent`, or **every** name failing → exit 1 having written nothing (a broken vantage is not a fixture); a half-written file is removed. One name failing among successes is a fact and is kept.
|
||||
- `--help`/usage comment updated in the same edit (the script's header is the documentation `README.md` points at).
|
||||
2. `tests/fixtures/gateway_resolver.json` — the capture itself, committed.
|
||||
3. `data/logs/verify-32-additive.py` (copy the shape of `data/logs/verify-28-final.py`)
|
||||
- `git stash`-safe pattern: load `agent/host_agent.py` from `git show HEAD:agent/host_agent.py` into a temp file and import it side by side with the working tree's, via `importlib.util.spec_from_file_location` under distinct module names.
|
||||
- Build one fake tree (`nginx_tree`-shaped) plus the haproxy capture, monkeypatch `socket.getaddrinfo` in **both** modules to the same deterministic stub, call `snapshot()` on both with identical arguments, and demand:
|
||||
- `new == old | {"resolver": …}` — every pre-1.3.0 key equal, `resolver` the only addition;
|
||||
- `new --no-resolve == old` **exactly**;
|
||||
- no key removed, renamed, or reordered-meaning-changed anywhere, printed as a diff on failure.
|
||||
- `config_directive` lives inside `resolver` (not inside `nginx`) precisely so this comparison stays a strict top-level key test with no exception list — record that reason in the script's header so nobody later "tidies" the field into the nginx section and weakens the control.
|
||||
- Transcript: `data/logs/32-additive.out`, exit non-zero on any difference.
|
||||
4. `tests/test_deploy_scripts.py` (or the capture-script test module that already covers this script)
|
||||
- Extend the existing pins: the script names `getent` and no `sudo`; it derives the name list by importing `target_names` (so the capture cannot out-ask the agent); a forced-`ssh`-failure run exits 1 and leaves the existing fixture byte-identical; a re-run over the same inputs is idempotent; the new fixture carries no globally routable address (`test_the_gateway_fixtures_carry_no_routable_endpoint`-style walk, extended to JSON).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; `tests/test_agent.py` and `tests/test_gateway_source.py` both consume `gateway_resolver.json` (task 03 wires the second).
|
||||
- Coverage: shell + fixtures, so the gate is the existing script pins plus no Python regression; `bash -n scripts/capture-gateway-config.sh` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `./scripts/capture-gateway-config.sh` run on a box with the `gateway-ops` key writes `tests/fixtures/gateway_resolver.json` and exits 0; the file has one entry per name `target_names()` derives from the captured configs, `via: "libc"`, `config_directive: "10.1.0.1"`, and no globally routable address anywhere in it.
|
||||
- [ ] Killing the alias (or `GATEWAY_SSH=nope`) exits non-zero with every existing fixture byte-identical (`sha256sum` before/after, printed).
|
||||
- [ ] `uv run python data/logs/verify-32-additive.py` exits 0 and its transcript shows the two documents equal plus `resolver`, and the `--no-resolve` arm equal exactly.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md's capture section and AGENTS.md's invariant-7/fixture prose name the new capture output (this task's user-visible change) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: the six existing fixtures are unchanged unless the live router's config genuinely moved, in which case that drift is recorded in the transcript and `tests/test_gateway_source.py`'s tallies are re-counted in the same commit (never edited to match a live read).
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Task 03 — The join counts a name route through the address the gateway itself dials
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** operator complaint — "the only animated line is `:443 -> 127.0.0.1:4443`"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`sources/gateway.py` reads the agent's `resolver` section and widens the one rule that decided a route could never be counted: an endpoint is joinable when the config wrote an **address** *or* when the gateway's own libc resolution published one for the name. `connections[]` grows three provenance fields, name-keyed gauges start carrying measurements, and the two rules that made phase 30 honest stay exactly where they were — the dashboard still never resolves, and a pipe nobody measured still publishes **no** sample.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- New pure reader `resolution_index(snapshot) -> dict[tuple[str, int | None], Mapping[str, Any]]`, keyed on `(name, port)` from `snapshot["resolver"]["entries"]`, ignoring entries with no `addresses` (a `nxdomain` contributes an empty set, never a `None` address). Absent `resolver` section (a 1.2.0 agent, or `--no-resolve`) → `{}`, which is exactly today's behaviour and must be pinned with a fixture that has no `resolver` key at all.
|
||||
- New pure pass `apply_resolution(connections, index, via, ts)` — called between `config_connections()` and the socket join — writing on each connection's `out`:
|
||||
- `resolved`: the address list, or `null` when nothing published one (never `[]`, so the page's "did not resolve" arm is one `is null` test rather than a length check that confuses empty with unknown);
|
||||
- `via`: `"libc"` (the agent's string, copied, never invented here) and `resolved_ts`: that entry's own timestamp;
|
||||
- `joinable`-driven `metric`, widened:
|
||||
```python
|
||||
# A line is countable when the kernel can name its far end: either the config wrote an
|
||||
# address, or the gateway's own libc resolution — the one that opens this connection —
|
||||
# published one. Phase 30 retired name-keyed gauges because they could only ever be
|
||||
# empty; this is the fact that made that untrue. The NAME keeps the config's spelling so
|
||||
# the stored series is the same one phase 29/30 would have written.
|
||||
```
|
||||
`metric` stays `gateway.out_conns@<host>:<port>` with the **config's** host spelling (history continuity, and `:25565`/`:25566` stay two gauges), and stays `null` for a line whose address is unknown.
|
||||
- `endpoint_of(connection)` → returns a **list** of `(address, port)` keys: the written address, or every resolved address. `join_sockets()` builds its `endpoints` map by pointing **every** key at the **same** counter row, so a multi-homed name is one endpoint with several spellings in the kernel's table, and two lines reaching one endpoint still share one counter (`shared: true`, unchanged).
|
||||
- **Address comparison is normalised, spelling is not.** Match with `ipaddress.ip_address(...)` on both sides so `/proc`'s `inet_ntop` output, an IPv4-mapped `::ffff:10.1.2.64`, and a resolver's `10.1.2.64` are the same address; the **payload** keeps publishing whatever the config and the agent wrote, verbatim. No string equality anywhere in the match path — that is the bug class where a route resolves and still never counts.
|
||||
- `_skeleton()` gains `"resolver": {"available": None, "via": None, "entries": []}` so `parse` and `stub` keep one key set (the parity pin), and `parse()` fills `available`/`via` from the section while `entries` carries the entries the index was built from — the page reads provenance from the document it was drawn from, not from a second call.
|
||||
- `samples()`: unchanged logic, now wider in effect (a resolved name route emits its gauge). A `null` count still publishes nothing. De-duplication by metric stays, so a shared endpoint costs one insert.
|
||||
- `supported()` — a 1.3.0 snapshot must answer `supported: true` (its three proxy sections are what the check reads); add the assertion rather than assuming it.
|
||||
- `warnings()` untouched. The `/api/summary` block grows **no** field.
|
||||
2. `src/dashboards/collectors.py` — **the same commit**, per AGENTS.md's cross-module rule for a new add-on section:
|
||||
- `ADDON_ERROR_PREFIXES` gains `resolver`, so a `resolver: … nxdomain …` or `resolver: pass budget … exhausted` line is an *unconfigured* report on `/system`, not `degraded`, and never an overview WARN. Invariant 3, both directions pinned: a host with that line stays out of `alerts`, and a `nginx:`/`smart:` line still behaves as before.
|
||||
3. `tests/test_gateway_source.py`
|
||||
- `test_a_name_route_counts_through_the_address_the_gateway_published` — captured configs + `gateway_resolver.json` + the recorded `/proc/net/tcp{,6}`: a socket whose remote is `10.1.2.64:443` lands on `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443`'s line, with `out.via == "libc"` and a `resolved_ts`.
|
||||
- `test_the_match_is_an_address_not_a_string` — the same address in `::ffff:` / leading-zero / uppercase spellings on either side still matches; the payload keeps verbatim spelling.
|
||||
- `test_two_names_one_endpoint_is_one_count_shared` and `test_two_ports_on_one_host_are_two_gauges` (`minecraft` `25565`/`25566`).
|
||||
- `test_an_unresolved_name_publishes_no_metric_and_no_sample` — `connections: null`, `metric: null`, zero samples, never `0`.
|
||||
- `test_the_dashboard_still_never_resolves` — `socket.getaddrinfo` spied and asserted uncalled through the whole `parse()` path, **and** a new comment naming where the resolver lives and why (vantage). This pin is restated, not relaxed.
|
||||
- `test_a_snapshot_without_a_resolver_section_is_phase_30_verbatim` — the 1.2.0 fixture: identical payload to today's except `resolver`/`out.resolved`/`via`/`resolved_ts` being absent/`null`, and `supported` still `true`.
|
||||
- The counted-once identity and the `sum(listeners) + sum(endpoints) <= established` inequality are restated with the widened match set, including the `127.0.0.1:4443` loopback hop counting once inbound **and** once outbound on two different pipes.
|
||||
- `tests/test_api.py`: the "no name-keyed `gateway.out_conns@…` series is ever inserted" pin is **rewritten** to the new rule (an endpoint whose address is known — written or resolved — gets its gauge; an unresolved name never does), and `test_the_summary_gateway_block_is_the_payloads_sums_not_the_pages_arithmetic` gains the resolved case so `live_out` still sums **distinct endpoints** (`api._endpoint_values()`), never lines.
|
||||
- `tests/test_live_fleet.py` (marked `live`): against the real ingress, every `connections[]` row whose resolver status is `ok` has a finite `connections` (a number, `0` allowed) — never `null`. Skips when `GATEWAY_HOST` is not a configured target.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit + fixture as above; `tests/test_gateway_source.py` keeps running the **real** `host_agent` parsers over the captures so the join is compared with the agent's document, not with a copy of its field names.
|
||||
- Coverage **>90%** on `sources/gateway.py` and `collectors.py`; whole suite 100%; `uv run ruff check src tests agent` and `uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Against the captures: the number of `connections[]` rows with a non-null `connections` rises from 2 (the two loopback targets) to every row whose name resolved in `gateway_resolver.json`, and `tests/test_gateway_source.py` prints that count so the change is visible rather than implied.
|
||||
- [ ] `grep -rn "getaddrinfo" src/dashboards/` finds nothing; `grep -n "resolver" src/dashboards/collectors.py` shows `ADDON_ERROR_PREFIXES` carrying it.
|
||||
- [ ] `GET /api/gateway/snapshot` on a fixture-driven store shows `out: {…, address: false, resolved: ["10.1.2.64"], via: "libc", resolved_ts: …, …}` with a finite `connections` on that row, and `null`/`null` on an injected `nxdomain` row.
|
||||
- [ ] The stored series for one shared endpoint is **one** insert per poll (asserted by counting rows), and `/api/summary`'s `gateway.live_out` equals the sum over distinct endpoints.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated for the widened join rule and the metric-naming passage in the same commit (the "no name-keyed series" sentence is replaced, with the reason) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `warnings()`, `/api/summary`'s key set, the inbound join, `/system`'s `degraded` for the other add-on prefixes, and every non-gateway source are untouched.
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Task 04 — The dash now says *which* unknown it is, and the resolution rides on the tooltip
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** phase-31 decision 8 ("the `—` keeps its meaning and loses its mystery") meeting this phase's new fact
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Two facts that used to share one glyph separate: **"the gateway could not resolve this name in this reading"** and **"resolved, and nothing is open on it right now"**. The first keeps the em dash with a sentence that names the cause; the second prints its `0`. Every counted line carries where its address came from (`libc`) and when, on `title` and in the `aria-label` — provenance the operator needs to explain a number, kept off the ink because the page's copy rule is *would you act differently without it?*
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- Replace the single `NO_COUNT_WHY` with the two sentences the payload can now distinguish, chosen from data and never guessed:
|
||||
```js
|
||||
const UNRESOLVED_WHY = 'no number: the gateway could not resolve this name in this reading';
|
||||
const IDLE_TITLE = (n) => `${n} open connection(s) on this endpoint`
|
||||
+ (resolved ? ` · address from ${via}, resolved ${age}` : '');
|
||||
```
|
||||
`age` comes from `Dash.duration(Dash.ageSeconds(conn.out.resolved_ts))`; `null`/absent prints nothing rather than `NaN`, and `via` is the payload's word (`libc`) copied, not paraphrased.
|
||||
- `leafAria(conn, on)`: the unknown arm becomes `'connections not attributed: the gateway could not resolve this name in this reading'`; the counted arm keeps its existing facts and adds nothing else. A measured `0` keeps its own sentence (`0 open connections`) — never the unresolved wording, because it is a measurement.
|
||||
- `countText()` and the `—` glyph are unchanged; a `0` continues to print `0`.
|
||||
- **Filter amendment** (phase 31 task 04's `haystack`): include `conn.out.resolved` addresses, so `10.1.2.64` finds `brain.reeseapps.com`. They are payload strings, the filter is still an eye, and this is the one case where matching an address helps the operator — pinned with the rest of the haystack contract.
|
||||
- `cardStreams()` is untouched: a line with no `metric` still draws no series, and the chart legend still names only pipes that have points.
|
||||
2. `src/dashboards/web/pages/gateway.html`
|
||||
- Footer paragraph 2 is rewritten to the new truth and stays **one** paragraph (the file's three-paragraph shape is pinned in `tests/test_api.py`):
|
||||
> `a dash means the gateway could not resolve that name in this reading — hover a counted line for the address it matched and when that answer was taken.`
|
||||
- The card comment above `.gate-map` gains one line: `out.resolved` is the agent's libc answer and is why a name line can carry a number.
|
||||
- The `#gate-empty` setup block keeps naming agent **1.2.0** as the minimum (a 1.3.0 agent is not required for the page to work, only for name routes to count) — do not bump it here; the version floor is `sources/gateway.supported()`.
|
||||
3. `tests/test_api.py`
|
||||
- `GATEWAY_PAGE_FIELDS` gains `connections[].out.resolved`, `connections[].out.via`, `connections[].out.resolved_ts`, resolved against a live document in both directions (the existing device), so a rename on the Python side fails by name here.
|
||||
- **New** `test_an_unresolved_name_and_an_idle_route_say_two_different_sentences`: served JS picks the sentence from `out.resolved === null`, the two strings differ, neither is `n/a`, and a `0` is described as a measurement.
|
||||
- `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero` stays green with its claim intact and its docstring amended to name the new cause (restated, not deleted).
|
||||
- **New** `test_a_counted_line_names_where_its_address_came_from`: the `title` builder reads `out.via` and `out.resolved_ts` through the shared age helpers, and a missing `resolved_ts` yields no age fragment rather than `NaN`/`undefined`.
|
||||
- The filter pin from phase 31 (`test_the_filter_matches_only_payload_text_and_costs_no_request`) is extended: the haystack includes `out.resolved` and still contains no `fetchJSON`, no DOM text, and no `"null"` matchable value.
|
||||
- `test_no_page_narrates_its_own_pipeline` must still pass on the new footer sentence (it names an operator-facing cause and a place to look, not the app's design); if it objects, shorten the sentence rather than touching the audit's allow-list.
|
||||
- The degradation-arm pin gains a case: `resolver: {available: null}` — no red, no extra ink, `why` still the page's one sentence.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: served-bytes pins above + a fixture render for each of the three arms (resolved+live, resolved+idle, unresolved) asserting the glyph, the `title`, and the `aria-label` differ exactly as specified.
|
||||
- Integration: with the resolver section stubbed to `available: null`, every name line keeps the unresolved sentence and the page stays uncoloured; no line anywhere prints `NaN`, `undefined` or `null`.
|
||||
- Coverage: no new Python in this task; `node --check` clean; floor does not fall.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data: a counted name line's `title` reads `… · address from libc, resolved <age>`; an unresolved line (stub one) prints the em dash with the resolution sentence; a resolved line with no socket prints `0`.
|
||||
- [ ] Reading five lines' `aria-label`s aloud (or dumping them) yields three distinct claims and no invented number.
|
||||
- [ ] Typing `10.1.2.64` (or any address from the live `resolver` section) into `#gate-filter` reveals exactly the route(s) that resolve to it, with zero network requests.
|
||||
- [ ] `grep -c "<p" ` on the served footer is still 3 and `test_no_page_narrates_its_own_pipeline` is green.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` dash sentence updated (this task's user-visible change) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: tallies, warnings panel, chart disclosure, switch, phase 31's colour and row order untouched.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Task 05 — Roll the gateway to 1.3.0, read-only first, exactly as phase 28 moved it to 1.2.0
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md's fleet record (the rollout discipline that kept `host:gateway`/`disks:gateway` `ok` through phase 28) + the standing rule that a path grant is only as current as the file it points at
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
One host moves: `gateway` runs agent **1.3.0** on its existing bundle and its existing unit, and the live page gets real counts on its name routes. Nothing else moves — no `HOST_TARGETS` edit, no certificate, no URL flip, no other host (a 1.1.0 `deskwork`/`driveripper` is unaffected, and a 1.2.0 gateway left running after a failed attempt is a working gateway). The transcript is the artifact.
|
||||
|
||||
## Work
|
||||
|
||||
1. **Read-only probe first** — `data/logs/verify-32-task05-probe.py` (copy `data/logs/verify-28-task04.py`), printing and asserting:
|
||||
- the CA-only `--status`-style probe against `https://10.1.1.27:9120` answers `version` **1.2.0** on `/health`, and still refuses `/snapshot` with `client certificate required`;
|
||||
- the dashboard's own mTLS dial (`collectors.fetch_snapshot` with `certs.agent_tls_map` over the stored authority — the `verify-28-task05.py` idiom, never a hand-rolled `client.get`) reads every pre-1.3.0 section with live numbers and **no** `resolver` key: the control that makes the install provably additive before anything is touched;
|
||||
- `/api/health` reads `host:gateway`, `disks:gateway` `ok` before the change, and the `gateway` job's payload counts how many rows carry a number (the before figure).
|
||||
2. **Re-stage, then install** over the `gateway-ops` key, as `ops`:
|
||||
- `scp agent/host_agent.py deploy/install-agent.sh → /home/ops/agent-install/…` (the directory is `ops`-owned; no root), leaving the previous files beside the grant as `host_agent.py.20261023-backup` / `install-agent.sh.20261023-backup`, which sudo will not run;
|
||||
- read the live unit back (`systemctl cat dashboards-agent`) and run the **same** arguments it carries:
|
||||
`sudo /home/ops/agent-install/deploy/install-agent.sh --bind 10.1.1.27 --port 9120`
|
||||
— no `--cert-bundle` (the bundle on disk, not the flag, is what puts the TLS lines into the render — the phase-28 finding, quoted in the transcript), no `--as-root`/`--caps` (that is `driveripper`'s shape, not this host's);
|
||||
- expect the re-run guard to stay silent (identical render). `hashsum.sha256` the unit before and after and record both; a differing unit means the run did something this phase did not plan, so stop and report rather than `--force`ing;
|
||||
- `sudo systemctl restart dashboards-agent` (the file changed; `enable --now` would not restart it), then `systemctl is-active`.
|
||||
3. **Verify both ways, read-only**:
|
||||
- the CA-only probe answers `version 1.3.0` and still refuses `/snapshot`;
|
||||
- the dashboard's mTLS read shows `agent_version: "1.3.0"`, `resolver.available: true` with an entry per declared target, `haproxy`/`nginx`/`sockets` and every pre-1.3.0 section intact with live numbers, and `/api/gateway/snapshot` answering a **finite** `connections` on every row whose resolver status is `ok` (the after figure, printed beside the before figure);
|
||||
- `./scripts/dev-server.sh status` → all ten sources `ok`; `/system` shows no new `degraded` on any host (the `resolver:` prefix, invariant 3);
|
||||
- a failure at any step is a rollback, not a fix-forward: copy `host_agent.py.20261023-backup` back to `/usr/local/lib/dashboards-agent/host_agent.py` and restart — the unit, the bundle and the URL never changed, so that *is* the whole rollback. Record which arm happened.
|
||||
4. `data/logs/NOTES-32-rollout.md` — sha256 of what was staged where, the unit hash before/after, the before/after "rows with a number" counts, the live `resolver` entry table, the `nxdomain` list if any, the screenshots' names, and the rollback command. The AGENTS.md fleet record's version spread is task 06's edit; this note is the evidence it cites.
|
||||
5. **Do not** touch `deskwork` or `driveripper`. Their 1.1.0 agents have no proxy sections and nothing to resolve; the spread costs nothing because 1.3.0 is additive, and task 02's control is the proof of that claim.
|
||||
|
||||
## Testing & Quality
|
||||
- This task changes no repo code; its quality gate is that the suite is still green afterwards and the two probe scripts exit 0 with their transcripts saved (`data/logs/32-task05-probe.out`, `data/logs/32-task05-verify.out`).
|
||||
- `uv run pytest -m live --live` afterwards: the new live pin (every resolved name route carries a number, never `null`) passes rather than skips.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `systemctl is-active dashboards-agent` is `active` on `gateway`, and the CA-only probe answers **1.3.0**.
|
||||
- [ ] The unit's sha256 is identical before and after the install run, and the guard printed no diff.
|
||||
- [ ] `/api/gateway/snapshot` on the live ingress: `resolver.available: true`, and **0** `connections[]` rows with `connections: null` among the rows whose resolver status is `ok`; the before/after counts are in the transcript.
|
||||
- [ ] `host:gateway` and `disks:gateway` read `ok` across the whole exercise, and no other host changed `degraded` state.
|
||||
- [ ] `data/logs/NOTES-32-rollout.md` exists with the hashes, the two probe transcripts, and the rollback command named.
|
||||
- [ ] `uv run pytest` **100% passing**; `uv run pytest -m live --live` green (the gateway pin passing, not skipping).
|
||||
- [ ] No repo file changed except `data/logs/` (gitignored).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Task 06 — Docs that match the counted page, the browser measurement, and the full gate
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md definition of done (browser verification against live data, plus a full `README.md`/`AGENTS.md` consistency pass)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the counted page in a browser on the upgraded ingress, carry the resolution facts into the two documents that outlive this conversation, and close the phase with every gate green — including the ones that only bite if something drifted (dependency freshness, the mobile audit, the live suite).
|
||||
|
||||
## Work
|
||||
|
||||
1. `data/logs/32-counted-routes.cjs` — a Playwright pass over the live page (house shape: `NODE_PATH=<playwright node_modules>`, `DASH_BASE`/`DASH_PW`, sign in with `DASH_ADMIN_PASSWORD`, exit non-zero on FAIL, print every measured number):
|
||||
- **motion where the complaint said there was none**: count the animating elements on load (`getComputedStyle(path).animationName === 'pipe-flow'`) and assert the set is non-empty **and** that at least one of them is a **name** endpoint (a `connections[]` row whose `out.address` is `false`); record the counts per panel;
|
||||
- **provenance**: for 5 sampled rows, `title` contains `address from libc` and an age fragment, and `aria-label` states a number where the payload has one and the resolution sentence where it does not;
|
||||
- **no invented zero**: with a stubbed `resolver: {available: null}` and with `sockets.available: null`, no line prints `0` that the payload did not measure, and the trunk prints `n/a`;
|
||||
- **the filter still costs nothing**: query by an IP from the `resolver` section and by a domain, asserting zero network requests and unchanged tallies;
|
||||
- **colour is still existence**: every drawn route — spine, tick, run, switch — computes to `--gate-on` in both themes, and a row given `off` in devtools turns all four together (phase 31's contract, re-asserted after this phase's DOM/provenance changes);
|
||||
- **reduced motion**: nothing animates under emulated `prefers-reduced-motion: reduce`;
|
||||
- **degradation arms**: `unreachable`, `supported: false`, `sections {nginx: false}`, `resolver {available: null}`, `404 {configured:false}` — each in the payload's own words, none red;
|
||||
- **no sideways scroll** at 1440 px and 360 px with the widest fan-out expanded and both charts open;
|
||||
- screenshots `data/logs/32-gateway-{1440,360,counted,light,unresolved,filtered,reduced,phone}.png`, transcript `data/logs/32-counted-routes.out`.
|
||||
2. Re-run `data/logs/27-task02-mobile-audit.cjs` (nine pages, both widths) — exit 0, **0 FAIL / 0 FINDING**, tally recorded in the transcript.
|
||||
3. `AGENTS.md` — update every passage this phase made stale, in full:
|
||||
- the agent bullet: the `resolver` section (its `available` three-way and one-line-per-arm rule, `via: "libc"`, `config_directive`, `RESOLVE_MAX_NAMES`/`RESOLVE_CACHE_SECONDS`/pass budget, the single-flight background pass and *why* it is off the request path, `--no-resolve` removing the key in silence, and the never-resolve-anything-but-config-targets rule), `AGENT_VERSION` **1.3.0**, and the `/schema` addition;
|
||||
- `sources/gateway.py`'s join paragraph: two address sources (written-as-address and resolved), `out.resolved`/`via`/`resolved_ts`, normalised-not-string matching, the shared rule's now-real case, and the sentence that the dashboard **still** never resolves with the vantage reason;
|
||||
- the metric-naming passage: replace "no name-keyed `gateway.out_conns@…` name is ever inserted" with the widened rule and the reason it changed, keeping the note that an old name-keyed series stops growing rather than being rewritten;
|
||||
- `collectors.py`'s `ADDON_ERROR_PREFIXES` list gains `resolver`, beside the "a new add-on section adds its prefix here in the same commit" rule;
|
||||
- the `/gateway` UI-conventions paragraph: the dash now has two meanings and one glyph, provenance lives on `title`, the filter matches resolved addresses;
|
||||
- the **fleet record**: `gateway` runs **1.3.0**, `deskwork`/`driveripper` remain **1.1.0**, what the spread costs (nothing, with the additivity control named), the ten sources `ok` re-verified, and a pointer to `data/logs/NOTES-32-rollout.md`;
|
||||
- **gap list**: add the two follow-on lines (name-route switching = phase 33, amends invariant 1 and needs `PLAN.md` sign-off) and re-check gap 13's clip counts against this phase's measurements.
|
||||
4. `README.md` — the `/gateway` table row (name routes are counted through the gateway's own libc resolution), the gateway prose (the resolver section, its bounds, `--no-resolve`, what a dash means now), the capture-script section (its new output and its scrub rule), and the agent-version references wherever the page's minimum and the fleet's actual are named — kept as two different facts, not one number.
|
||||
5. Freshness and the whole gate:
|
||||
`uv sync --python 3.14 && uv pip list --outdated` → empty (this phase adds **no** dependency; if `cryptography` or anything else moved, that is a separate decision);
|
||||
`uv run pytest`, `uv run pytest --cov`, `uv run pytest -m live --live`, `uv run ruff check src tests agent`, `uv run pyright`, `bash .agents/validate.sh`.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: this task **is** the live proof of the phase's headline claim; every number in the Completion Criteria is printed by `32-counted-routes.out` or a probe transcript, never asserted in prose.
|
||||
- Coverage: the floor is met on the phase's modified Python (`sources/gateway.py`, `collectors.py`, `agent/host_agent.py`); the run is recorded.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `data/logs/32-counted-routes.out` exits 0 and prints: animating-element counts per panel with at least one **name** endpoint among them, the five provenance `title`s, the zero-invention arms, the filter's zero-request proof, the both-themes colour equality, the reduced-motion pass, and 0 px sideways scroll at both widths.
|
||||
- [ ] The eight `data/logs/32-gateway-*.png` screenshots exist, including `counted` (name routes with numbers) and `phone`.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING, tally recorded.
|
||||
- [ ] `uv pip list --outdated` is empty; `uv run pytest` and `uv run pytest --cov` green above the floor; `uv run pytest -m live --live` green; `ruff` + `pyright` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] `grep -n "1.3.0" AGENTS.md` shows the agent bullet **and** the fleet record; `grep -n "no name-keyed" AGENTS.md README.md` finds nothing (the retired sentence is gone, not contradicted); `resolver` appears in `ADDON_ERROR_PREFIXES`' documented list.
|
||||
- [ ] A full read of `README.md` and `AGENTS.md` finds no inconsistency anywhere in either file, including pre-existing ones — fixed in this phase's commits.
|
||||
- [ ] No behaviour change in completed work: `/`, `/system`, `/disks`, `/llm`, `/ups`, `/settings`, `/certs`, `/login` render as before and the unwired switch stays unwired (phase 33 wires it).
|
||||
@@ -0,0 +1,79 @@
|
||||
# Phase 33 — The switch is wired: turning one route off, and back on, without ever weakening the box
|
||||
|
||||
**Source:** operator request, 2026-10-23 — "Also write the phase to disable and enable services. I want to be able to turn off and on connections as needed in the event of security concerns or upgrades." + the two decisions taken with them in that conversation: **enforcement is a config edit, validated and reloaded, by a narrow root helper on the gateway** (so a disabled route stays disabled across a reload *and* a reboot), and **one switch per connection line, as already drawn**.
|
||||
**Story:** n/a
|
||||
**Context:** `deploy/install-agent.sh` (the unit template, whose `NoNewPrivileges=true` / `ProtectSystem=strict` / `SystemCallFilter=@system-service` / empty `CapabilityBoundingSet` this phase **does not touch**, the `--force` re-run guard, `deploy/agent-sudoers` as the precedent for a path-pinned grant), `agent/host_agent.py` (1.3.0 → **1.4.0**: `do_GET`-only `Handler`, `_certificate_allows`, the `haproxy`/`nginx` parsers that must learn the disabled marker), `src/dashboards/sources/gateway.py` (`connections[]`, `config_connections`, `warnings()`' seven kinds), `src/dashboards/api.py` (`/api/ups/command`'s four-gate shape and the `certs_router()`'s audit-event rule), `src/dashboards/collectors.py` (`AgentTls`, `fetch_gateway_snapshot`, the revoke-before-dial rule), `src/dashboards/config.py` (`GATEWAY_*` block), `src/dashboards/web/static/page-gateway.js` (`buildSwitch`, `routeInConfig`, `renderLeaf`), `tests/test_api.py` (the verb-enumeration pins), `tests/test_auth_api.py` (the served-route walk), `tests/test_deploy_scripts.py` (the fake-root sandbox the helper's tests extend).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway`'s switch becomes a control: **disable** `brain.reeseapps.com` and the nginx `stream` map entry stops routing it, persistently, with the config validated and the proxy reloaded and the page showing that exact route **red** until it is re-enabled. One new write route, `POST /api/gateway/route`, gated seven times, with the write itself performed off-box by a **root helper on the gateway that the agent reaches over a systemd-activated unix socket** — because the agent's own unit forbids privilege (`NoNewPrivileges=true`) and this phase is not going to weaken the sandbox of a read-only collector to give it a root shell.
|
||||
|
||||
This amends AGENTS.md invariant 1 from *one* exception to **two**, and that is the phase's real risk: a bug here is an outage on the fleet's only ingress. Everything below exists to make the blast radius one line of one file, and to make the app refuse rather than improvise whenever it cannot see what it is doing.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Invariant 1 becomes "two exceptions".** | `POST /api/ups/command` and `POST /api/gateway/route`. Both: session, CSRF/origin on a cookie write, an env gate (`NUT_ALLOW_ACTIONS` / `GATEWAY_ALLOW_WRITES`, default **off**), a typed `DANGER`, and every gate re-checked server-side on every request. `PLAN.md`'s anchor table records a read-only app, so this phase's overview carries the sign-off request; the phase never edits `PLAN.md`. |
|
||||
| 2 | **`DANGER` is required for both verbs.** | Enabling is not the safe direction: applying either change re-validates the file and **reloads the proxy**, and a reload touches every route on the box. A word that means "I know this affects the ingress" has to mean it both ways. |
|
||||
| 3 | **The agent keeps its sandbox; the privilege lives outside it.** | No sudo from the agent, no setuid, no change to `NoNewPrivileges`/`ProtectSystem`/`SystemCallFilter`. The write peer is `dashboards-routectl.socket` + `dashboards-routectl@.service` (systemd socket activation, `Accept=yes`, `User=root`, `SocketUser=dashboards-agent`, `SocketGroup=dashboards-agent`, `SocketMode=0600`): the agent connects to a socket it is *filemode-permitted* to connect to, and no other uid on the box can even open it, and the helper proves the peer uid with `SO_PEERCRED`. Access control is a file mode plus the mTLS client certificate that already authorises the agent's other paths — not a capability the agent process can be tricked into. |
|
||||
| 4 | **Only an id crosses a wire. Never a path, never a line, never text.** | The body is `{id, action, danger?, dry_run?}` where `id` is the payload's own `connections[].id`. The helper re-parses the configs itself, maps that id to the one line it declared, and refuses anything it cannot map. There is no `file`, no `line`, no `pattern`, no `command` anywhere in the contract, and the argv to the helper is fixed with no shell. |
|
||||
| 5 | **A route is off in the way that actually stops traffic for its proxy.** | haproxy: the `server` line is commented with a `# dashboards:off … was: <verbatim original>` marker. nginx vhost: the `proxy_pass` line is commented the same way. nginx **stream `map` entry**: commenting it would fall through to the map's `default` (`127.0.0.1:4443`) and **mis-route** the domain instead of blocking it, so the helper rewrites that entry's value to a sink (`127.0.0.1:44450`) and keeps the original in the marker; the sink is a `server` block the helper maintains in `/etc/nginx/stream.d/00-dashboards-sink.conf`, which the existing `stream { include /etc/nginx/stream.d/*.conf; }` glob already loads — so **no hand edit of the main config is ever required**. The map's `default` entry is **refused**: it is the fallback for every unmapped name, not a service. |
|
||||
| 6 | **Validate, restore on failure, reload — never restart.** | Write the new bytes, run `nginx -t` / `haproxy -c -f …`; on failure put the previous bytes back, do **not** reload, and answer `applied: false, validated: false, restored: true`. On success `systemctl reload <unit>` — reload, because a restart cuts every Minecraft session and every in-flight upload on an ingress that is doing its job. A `reload` failure is reported with the unit's own status and the file is left in the applied state (the config is correct; the reload is the operator's next action, said out loud). |
|
||||
| 7 | **Self-describing marker + drift refusal, not silent trust.** | The disabled line carries the original verbatim after `was:`, so a human can restore it by eye and the helper can restore it mechanically. A sidecar `/var/lib/dashboards-routectl/state.json` (`0600`, root) records actor, timestamp and the sha256 of what was written. Enabling restores only if the current line still matches what the helper wrote; otherwise it **refuses** and reports `drift` — a file an operator hand-edited is theirs to fix, not the app's to guess at. |
|
||||
| 8 | **Refuse rather than act blind.** | The write is refused when: writes are not enabled; the id is not in the **current** payload; the route is already in that state (`changed: false`, no reload); either config section is not `available: true`; the `gateway` job is `ok: false` or stale beyond 3× its interval; the `host:<GATEWAY_HOST>` system job is not `ok`; the agent is revoked (refused before any socket, the existing rule); the throttle is tripped. That is the UPS "refused while on battery" rule, translated: do not rewrite a config you cannot currently read end to end. |
|
||||
| 9 | **`off` finally means something the payload said.** | The agent's parsers learn the marker and report the marked `server`/`proxy_pass`/`map` entry with `disabled: true`, so the route stays in the graph as a declaration and `connections[].enabled` becomes **`false`**. `routeInConfig()` stops being a literal, the `off` styling phase 30 built becomes reachable, and `warnings()` learns the marker: a backend emptied by a marker is not `haproxy_backend_empty` (the operator chose that; the page shows it red), while a backend with no `server` line at all still is. |
|
||||
| 10 | **A live route's count is never stamped.** | Disabling does not write `0` into the payload. The sockets drain on the reload and the next poll reports what `/proc` says. Invariant: no code path in this phase writes a count. |
|
||||
| 11 | **The audit is one event, fixed in shape, with no config text in it.** | `gateway`-source events `route-disabled` / `route-enabled` / `route-denied`: verb, id, the session's actor name, `applied`/`validated`/`reloaded`/`drift`, and the proxy's own one-line error when there was one. The unified diff is returned to **the caller's response** (it is what the confirm dialog shows) and never stored — events are served to a browser and `/api/events` is a feed, not a config archive. |
|
||||
| 12 | **The overview is told, because "something is switched off" is not a page-only fact.** | `/api/summary`'s `gateway` block gains **one** field, `disabled` (an int count of `enabled is false` rows, `null` in the arms that did not measure) — the thirteenth key, restated in the pinned key set with its reason. A notifier has to learn about a deliberate outage without scraping a page, and it prints nothing at `0` (the `warnings` rule: a permanent `0 switched off` is ink that trains the eye). |
|
||||
| 13 | **One route, one file, one reload per request.** | A request touches exactly one config file and reloads exactly one proxy. There is no batch verb, no "disable all to this host", no `dry_run` that writes. If a security incident needs five routes off, that is five reviewed requests, each with its own event. |
|
||||
|
||||
**Out of scope:** editing anything a marker cannot express (adding/removing vhosts, certificates, `bind` lines, the map's `default`), haproxy runtime `disable server` via a stats socket (a runtime-only state that a reload silently undoes — decision 5 is persistent instead), disabling an entire **host** or **port**, per-user authorisation (one admin, gap 6), a scheduler ("off at 02:00"), and any change to the agent's unit security directives or to another host's agent (`gateway` alone runs 1.4.0; `deskwork`/`driveripper`'s 1.1.0 agents have no routes and are unaffected).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `32_gateway_resolved_target_liveness` (todo, **must complete first**) — agent 1.3.0's resolver section and the counted name routes are what a switched-off line reads against: `off` + a draining count to `0` is the proof the switch worked, and without resolution a disabled route and a never-counted route look identical.
|
||||
- `31_gateway_route_readability` (todo) — the row that names the inbound domain, the green end-to-end route, and the `off` styling this phase finally reaches.
|
||||
- `30_gateway_connection_map` (complete) — `connections[].id` as the vocabulary, the switch markup and its a11y contract, `routeInConfig()` as the named seam.
|
||||
- `15_agent_mtls` / `24_dev_mtls_rollout` (complete) — the verified-client-certificate gate the new POST path inherits, and the revoke-before-dial rule this phase's collector reuses.
|
||||
- `11_admin_login` (complete) — session, CSRF, origin checks, `LoginThrottle` (the `route:<address>` bucket is its third key shape).
|
||||
- **Operator actions, not repo work:** the `dashboards-agent`-to-helper socket grant lands through the units (no sudoers line needed), but installing units on a host and running the first live drill are the operator's — and the drill **stops** unless they are present. `PLAN.md`'s anchor sign-off for invariant 1 is theirs too.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_root_helper_edits_exactly_one_route.md` — `deploy/dashboards-routectl.py`: `show`/`list`/`diff`/`disable`/`enable`/`verify`, the marker with `was:`, the sink file it manages, validate-restore-reload, drift refusal, `--serve` over a systemd-passed socket, `--root` for tests.
|
||||
2. `02_the_privileged_peer_is_a_socket_not_a_sudo_line.md` — the two systemd units + `install-agent.sh --with-routectl`; the proof that the agent's `NoNewPrivileges=true`/`ProtectSystem=strict` unit is unchanged and can still reach the socket.
|
||||
3. `03_the_agent_accepts_one_write.md` — agent **1.4.0**: `POST /route` behind the verified client certificate **and** `--allow-writes`, body validation, one fixed peer call, plus the parsers reporting `disabled` for a marked line; `/schema` and the additivity control.
|
||||
4. `04_the_source_publishes_enabled_and_the_warnings_learn_the_marker.md` — `connections[].enabled`, `off` reachable in the payload, `warnings()` marker-aware, stub/skeleton parity, the one new `/api/summary` field (`gateway.disabled`), and no count ever written.
|
||||
5. `05_the_api_gates_the_write_ten_times.md` — `GATEWAY_ALLOW_WRITES`, `DASH_READ_ONLY`, `collectors.post_route`, `POST /api/gateway/route`, the ten-gate order, the throttle, the events, `/api/config.gateway.writes_enabled`, and the verb-enumeration + served-route pins restated (this is where invariant 1 is amended in `AGENTS.md`).
|
||||
6. `06_the_switch_obeys.md` — `page-gateway.js` + `gateway.html` + `dash.css`: `DANGER` confirm with the diff from a `dry_run` first, patch from the response, `routeInConfig()` reading `enabled`, the writes-disabled arm that keeps the switch honest, footer paragraph 3.
|
||||
7. `07_rehearse_then_drill_on_the_live_ingress.md` — the fake-tree rehearsal, then the live install on `gateway` and a drill on the one named-safe route, restore, verify, `data/logs/NOTES-33-switching.md`. Stops, correctly, without the operator present.
|
||||
8. `08_docs_and_live_verify.md` — `AGENTS.md` (invariant 1, the agent bullet, `sources/gateway.py`, `api.py`, `collectors.py`, `config.py`'s block, the `/gateway` UI paragraph, the fleet record, gap list) + `README.md` (including the operator's sudo/units steps), the browser pass, the mobile audit, the full gate.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **The helper is tested unprivileged, against a fake tree, with the proxy commands stubbed.** `tests/test_routectl.py` runs the real helper in a `tmp_path` tree (`--root`) with `nginx`/`haproxy`/`systemctl` faked on `PATH`, and pins: one line changed and nothing else in the file (byte-diff assertion); `was:` restores verbatim; a failing validator restores the exact previous bytes and reloads nothing; `default` refused; an unknown id refused; a second disable `changed: false`; the sink file created on first use, left alone if an operator's file is there, removed only when it wrote it and nothing is disabled; drift refused rather than restored; `SO_PEERCRED` refused for a non-`dashboards-agent` uid in the fake-peer test; no `shell=True` anywhere (`grep` pin, the installer's precedent).
|
||||
- **The agent's write path is proven against a fake peer, not a real one.** `tests/test_agent.py` runs the real handler over a TLS loopback port (the phase-15 harness) plus a unix-socket stub: `POST /route` without a client certificate is 403 `client certificate required`; with `--allow-writes` absent it is 403 `writes are not enabled on this agent`; a body with an unexpected key, a bad id, or a missing `danger` is refused **before** the peer is touched (asserted by the stub logging nothing); `GET` behaviour is byte-identical to 1.3.0 (the additivity control re-run).
|
||||
- **The API gates are proven twice, as the UPS ones are.** `tests/test_api.py` walks every gate with spies: writes-disabled refuses without dialling; a stale `gateway` job refuses; a revoked agent refuses without a socket; `DANGER` is required for `disable` **and** `enable`; a right request produces one peer call, one event, and the response's `applied`/`validated`/`reloaded` triple. `tests/test_auth_api.py`'s served-route walk demands a session for `/api/gateway/route`, and the CSRF pin is extended to it on the **wire** (a scripted helper records nothing without the header, once with it) — a 403 after the write is an outage with an error message attached.
|
||||
- **Live, opt-in, and reversible:** `-m live --live` gains `test_the_switch_round_trip_on_one_named_route` which is **skipped unless `--allow-live-writes` is passed as well**, names the route in the skip reason, and always restores in a `finally`. The default live suite stays read-only (AGENTS.md's rule survives this phase; the drill belongs to task 07 with the operator watching).
|
||||
- Coverage **>90%** on `deploy/dashboards-routectl.py`, `agent/host_agent.py`, `sources/gateway.py`, `collectors.py`, `api.py`; the whole suite 100%; `bash .agents/validate.sh` green.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it will stand after this phase and fixes every inconsistency found, **including pre-existing ones**. This phase's specific list: **invariant 1** rewritten to two exceptions with the seven gates named (and the sentence "hiding a button is not a safety control" kept, because it is the reason the switches are gated server-side rather than greyed in JS); the agent bullet's `POST /route` and the `disabled` marker; the `sources/gateway.py` bullet's `enabled`/`off` rules and the marker-aware `warnings()`; the `api.py` bullet's write lists (fleet writes are now two); the `collectors.py` bullet (`post_route`, the revocation check it shares); `config.py`'s `GATEWAY_ALLOW_WRITES` row and README's configuration table; the `/gateway` UI-conventions paragraph (the switch is wired, what `off` looks like, what the writes-disabled arm says); the fleet record (gateway **1.4.0**, its unit's security directives unchanged, the two new systemd units, `NOTES-33-switching.md`); and the gap list (name-route switching **closed**, plus the honest limits: one admin, no scheduler, no per-host kill).
|
||||
- `README.md` gains the operator's install steps for the helper units and the exact `--with-routectl` invocation, plus the recovery command for the state file (`systemctl status dashboards-routectl.socket`, `routectl list`, `routectl verify`).
|
||||
- Tasks update both files immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] With `GATEWAY_ALLOW_WRITES=1`, `/gateway` disables `brain.reeseapps.com`: the confirm dialog shows the **diff from a `dry_run`**, asks for `DANGER`, and the row turns **red** (`off`) on the poll after the reload while its live count drains to `0` on its own — no code wrote that zero.
|
||||
- [ ] `ssh gateway-ops 'sudo /usr/local/sbin/dashboards-routectl.py list'` names exactly that route; `verify` reports no drift; the marked line contains the original verbatim after `was:`; the map entry points at the sink and `stream.d/00-dashboards-sink.conf` exists and is owned by the helper's header.
|
||||
- [ ] Enabling restores the original bytes exactly (byte-compare against a pre-drill copy), the row turns green, and `list` is empty afterwards. **The live ingress ends the phase in the state it started in.**
|
||||
- [ ] Every gate refuses with its own sentence and **without** touching a peer: writes disabled, no session, no CSRF, wrong `Origin`, missing `DANGER`, unknown id, already-in-state, section unreadable, `gateway` job stale, `host:gateway` not `ok`, agent revoked. Each is a test, and each test asserts the stub peer logged nothing.
|
||||
- [ ] The agent's unit is unchanged: `systemctl cat dashboards-agent` still shows `NoNewPrivileges=true`, `ProtectSystem=strict`, `SystemCallFilter=@system-service`, and the same sha256 the phase recorded before installing the units — the sandbox was not traded for the feature.
|
||||
- [ ] A failing validator (`nginx -t` stubbed to exit 1) restores the previous bytes, reloads nothing, answers `applied: false, validated: false, restored: true`, and the page's line never turned `off`.
|
||||
- [ ] `POST` inventory: the app now has exactly two upstream writes (`/api/ups/command`, `/api/gateway/route`) and the enumeration pin lists them as such; `AGENTS.md` invariant 1 says **two** and names both gates; `/api/health` gained nothing.
|
||||
- [ ] `uv run pytest` (with `--cov` ≥ 94), `uv run ruff check src tests agent deploy`, `uv run pyright`, `node --check`, `bash .agents/validate.sh` all green; `uv run pytest -m live --live` green **without** `--allow-live-writes` (the round-trip test skipped by default, its reason naming the route it would touch).
|
||||
- [ ] `data/logs/33-*.png` (switch off in both themes, the `DANGER` dialog with its diff, the writes-disabled arm, 360 px) and `data/logs/33-*.out`; `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING; `data/logs/NOTES-33-switching.md` records the drill, the hashes and the restore.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass), including the operator's steps for undoing all of this (`--without-routectl`, remove the units, re-enable every route).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs`, `/login`, the UPS command path and its four gates, and every non-gateway source behave exactly as phase 32 left them.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Task 01 — The root helper edits exactly one line, proves it validates, and reloads or gives up
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** operator decision 2026-10-23 — "Edit the config + validate + reload, via a narrow root helper on the gateway"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`deploy/dashboards-routectl.py` — a stdlib-only root program that turns one `connections[].id` off or back on in one of the two proxy configs, validates the file with the proxy's own checker, reloads that one proxy, and refuses everything else. It is the phase's only writer, it has no network listener of its own (task 02 hands it a systemd-activated socket), and it can be run by hand on the gateway by the operator to see exactly what the dashboard would do.
|
||||
|
||||
The helper **imports the agent's config parser by file path** and never re-implements a rule: two parsers reading one config is two opinions about the same file, which is the failure mode this repo has pinned against since phase 28. `agent/host_agent.py` is world-readable at `/usr/local/lib/dashboards-agent/host_agent.py`, so the helper imports it with `importlib` and refuses to run at all if it cannot.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — the shared grammar only (no version bump here; `AGENT_VERSION` moves in task 03):
|
||||
```python
|
||||
ROUTE_MARKER = "# dashboards:off" # the only marker this app writes into a proxy config
|
||||
ROUTE_WAS = "was:" # `<marker> <id> was: <original line verbatim>`
|
||||
def marker_for(route_id: str, original: str) -> str: ...
|
||||
def split_marker(line: str) -> tuple[str, str] | None: # (route_id, original) or None
|
||||
def config_line_index(haproxy_text, haproxy_path, nginx_docs) -> dict[str, dict[str, Any]]:
|
||||
"""route id -> {"file", "line", "original", "kind", "disabled"} for every declared target."""
|
||||
```
|
||||
`config_line_index` walks the parsed declarations **with their source lines**, so the id → (file, line, verbatim text) map is derived by the same parser that produced the graph, and a marked line is reported as the same id with `disabled: true`. It is pure over text; the file reading stays in the callers (`Collector` for the agent, the helper for the writes), both resolved under `--root`.
|
||||
2. `deploy/dashboards-routectl.py` (new, `0755`, stdlib, no third-party import, no `shell=True`)
|
||||
- **Arguments** (all default to the deployed paths, all overridable for the fake tree):
|
||||
`--root /` · `--haproxy-config /etc/haproxy/haproxy.cfg` · `--nginx-config /etc/nginx/nginx.conf` · `--state /var/lib/dashboards-routectl/state.json` · `--sink-conf /etc/nginx/stream.d/00-dashboards-sink.conf` · `--sink-address 127.0.0.1:44450` · `--agent-module /usr/local/lib/dashboards-agent/host_agent.py` · `--nginx-unit nginx` · `--haproxy-unit haproxy` · `--apply` (without it every write verb is a dry run).
|
||||
- **Verbs**, one JSON object to stdout, non-zero exit on refusal:
|
||||
`list` · `show --id ID` · `diff --id ID --action disable|enable` · `disable|enable --id ID [--apply]` · `verify` (marker/hash vs file → drift report) · `--serve` (read one JSON request from the systemd-passed socket, answer one JSON line).
|
||||
- **Per-kind edit, exactly as decision 5** (the three arms and the `default` refusal are this file's core; a fourth arm is a new decision, not a fall-through):
|
||||
| id kind | action | why |
|
||||
|---|---|---|
|
||||
| haproxy `server` | comment the line with `ROUTE_MARKER` + `was:` | the backend stops receiving; the marker keeps the original restorable by eye and by machine |
|
||||
| nginx vhost `proxy_pass` | comment the line the same way | the vhost keeps `listen` + `server_name` and gains no target — a shape `warnings()` already treats as quiet (the `:80 return 301` arm), so "off" does not read as "broken" |
|
||||
| nginx stream `map` entry | **rewrite the value** to the sink address, marker + `was:` on the line, ensure the sink file exists | commenting would fall through to the map's `default` (`127.0.0.1:4443`) and **mis-route the domain instead of blocking it** |
|
||||
| nginx `map` `default` | **refuse** — `{"applied": false, "refused": "map-default"}` | it is the fallback for every unmapped SNI, not a service |
|
||||
- **The sink file**: created on first need with a `# managed by dashboards-routectl — do not edit` header and a stream `server` block that listens on `127.0.0.1:44450` and returns a short refusal string; loaded by the existing `stream { include /etc/nginx/stream.d/*.conf; }` glob, so **the main config is never edited**. If a file already exists at that path without the header, the helper refuses (an operator's file is not its own to overwrite). It deletes the file only when it wrote it **and** nothing is disabled, and never removes it while any map entry still points at the sink.
|
||||
- **Write protocol** (order is the safety property, so it is written as numbered steps in the file):
|
||||
1. re-read the file; index it; map `id` → line (unknown id → refuse, nothing written);
|
||||
2. already in the requested state → `{"applied": false, "changed": false}` and **no reload**;
|
||||
3. `--apply` absent → answer with the unified `diff` only;
|
||||
4. write to `mkstemp` in the same directory, `fsync`, `os.replace`;
|
||||
5. run `nginx -t` or `haproxy -c -f <that file>`; on failure `os.replace` the previous bytes back and answer `{"applied": false, "validated": false, "restored": true}` with the checker's own first line — **and reload nothing**;
|
||||
6. `systemctl reload <unit>`; a failure is reported as `reloaded: false` with the unit's status line, leaving the applied file in place (the config is right; the reload is the operator's next action and the answer says so);
|
||||
7. re-read and re-index to prove the marker is live; record state.
|
||||
- **Never `restart`** — the docstring says why in one sentence: a restart cuts every live connection through the fleet's only ingress, and a reload does not.
|
||||
- **State sidecar**: `0600`, root, `{id: {"actor", "at", "file", "sha256", "action"}}`, written with `os.open(..., O_CREAT, 0o600)` so the mode is its birth mode (the `dev-certs-export.py` idiom). `enable` restores **only** when the current line hashes to what the helper wrote; otherwise `{"applied": false, "drift": true, "hint": "…"}` and no byte changes.
|
||||
- **`--serve`**: `socket.fromfd(3, AF_UNIX, SOCK_STREAM)` (or the `LISTEN_FDS`/stdin idioms, chosen once and documented), read one newline-delimited JSON request, `getsockopt(SO_PEERCRED)` and **refuse any uid that is not the agent's** (`--allow-uid`, default `dashboards-agent`, resolved with `pwd.getpwnam`), dispatch one verb, write one JSON line, exit. One request per activation: systemd spawns the instance per connection (`Accept=yes`), so there is no accept loop, no thread, no timeout policy to invent.
|
||||
- **Nothing in this file takes a path, a line number, a pattern or a command from the wire.** `--serve`'s accepted keys are exactly `{id, action, dry_run}` and every other key is a refusal. Assert it structurally in the tests.
|
||||
- Every subprocess call is a fixed argv list (`subprocess.run([...])`, no `shell=True`), and the checker/unit names come from **flags the operator installed**, never from the request.
|
||||
3. `tests/test_routectl.py` (new) — real helper, fake tree (`--root`), fake `nginx`/`haproxy`/`systemctl` scripts on `PATH` that record their argv and answer to order:
|
||||
- `test_disabling_one_route_changes_exactly_one_line` — unified diff of the whole file is one hunk, one line; every other byte identical (assert on the bytes, not on a count).
|
||||
- `test_a_disabled_map_entry_points_at_the_sink_and_keeps_its_original` — the sink file exists with its header, the marked line carries `was: <verbatim>`, and re-enabling restores the file byte-identical to the pre-drill capture.
|
||||
- `test_the_map_default_is_refused_and_nothing_is_written`, `test_an_unknown_id_is_refused`, `test_disabling_a_disabled_route_changes_nothing_and_reloads_nothing` (the fake `systemctl` log stays empty).
|
||||
- `test_a_failing_validator_restores_the_previous_bytes` — byte-identical restore, `validated: false`, `restored: true`, zero reload calls, non-zero exit.
|
||||
- `test_a_failing_reload_says_so_and_leaves_the_file_applied`, and `test_reload_is_never_a_restart` (grep the source for `restart` in a call position, the installer's `--status` style pin).
|
||||
- `test_the_helper_imports_the_agents_parser_and_refuses_without_it` — point `--agent-module` at a non-file: exit non-zero, no write, and the message names the path (a helper that silently skipped the parser would be a second opinion about the config).
|
||||
- `test_state_is_0600_and_drift_is_refused` — mode at birth; hand-edit the marked line and `enable` refuses with `drift: true` and no byte change.
|
||||
- `test_serve_refuses_a_peer_that_is_not_the_agent_uid`, `test_serve_accepts_exactly_id_action_dry_run`, `test_no_shell_and_no_path_from_the_wire` (structural greps over the file, the `certs.py`/`auth.py` precedent).
|
||||
- `test_the_sink_file_is_left_alone_when_someone_elses_wrote_it`, `test_the_sink_is_removed_only_when_it_is_empty_of_our_routes`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the whole matrix above, hermetic and unprivileged, over the **captured** `gateway_haproxy.cfg` / nginx tree so the edits are pinned against real config bytes rather than a toy file.
|
||||
- Coverage **>90%** on `deploy/dashboards-routectl.py` and the new `agent/host_agent.py` functions; `uv run ruff check agent deploy` + `uv run pyright` clean; `python3 -m py_compile` on the helper (it must run on a host's Python 3.10, so no 3.14-only syntax — pin that with a `# requires: python >=3.10` header and a test that greps for `match `/`typing.Self`-style 3.11+ syntax if the repo has such a pin; otherwise assert `ast` parses with `feature_version=(3,10)`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run python deploy/dashboards-routectl.py --root <tmp> … diff --id nginx:443->reeseapps-brain-of-reese.reeselink.com:443` prints a one-hunk diff and writes nothing; with `--apply` it edits, validates, reloads (against the fakes) and answers `{"applied": true, "validated": true, "reloaded": true}`.
|
||||
- [ ] Re-enabling leaves the tree byte-identical to its pre-drill `sha256sum` for **both** config files.
|
||||
- [ ] `grep -n "shell=True\|restart" deploy/dashboards-routectl.py` finds neither in a call position; `grep -n "def \|argparse" ` shows no `--file`/`--line`/`--pattern` option anywhere.
|
||||
- [ ] `uv run pytest tests/test_routectl.py -q` 100% passing; whole suite green; coverage not down.
|
||||
- [ ] README.md/AGENTS.md note the helper's existence and its verbs (this task's user-visible surface — the operator can run it by hand) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `/snapshot`'s document is unchanged by this task (the `AGENT_VERSION` bump and the `disabled` field are task 03's, with the additivity control).
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Task 02 — The privileged peer is a socket systemd activates, not a privilege the agent is given
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the agent unit's `NoNewPrivileges=true` + `ProtectSystem=strict` (read from `deploy/install-agent.sh`), which this phase must not weaken to get a write
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Install the helper on the gateway as a **socket-activated root service** the agent may connect to and nothing else: `dashboards-routectl.socket` owned by the agent user at `0660`, `dashboards-routectl@.service` running as root with `Accept=yes`. The agent's own unit keeps every one of its security directives — the phase's central safety claim is that a read-only collector stayed read-only in its sandbox and talked to a peer outside it.
|
||||
|
||||
## Work
|
||||
|
||||
1. `deploy/dashboards-routectl.socket` (new)
|
||||
```ini
|
||||
[Socket]
|
||||
ListenStream=/run/dashboards-routectl.sock
|
||||
SocketUser=dashboards-agent
|
||||
SocketGroup=dashboards-agent
|
||||
SocketMode=0600
|
||||
NoDelay=yes
|
||||
[Install]
|
||||
WantedBy=sockets.target
|
||||
```
|
||||
`0600` + `SocketUser=dashboards-agent` is the whole access control at the file layer: only that uid can connect, everyone else gets `EACCES` before any code of ours runs. The peer check in task 01's `--serve` is the **second** look, not the first.
|
||||
2. `deploy/dashboards-routectl@.service` (new)
|
||||
```ini
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/sbin/dashboards-routectl.py --serve \
|
||||
--allow-uid dashboards-agent --state /var/lib/dashboards-routectl/state.json
|
||||
User=root
|
||||
StandardInput=socket
|
||||
StandardOutput=socket
|
||||
StateDirectory=dashboards-routectl
|
||||
ProtectSystem=read-only # it writes two config files by absolute path; see ReadWritePaths
|
||||
ReadWritePaths=/etc/haproxy /etc/nginx/stream.d
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
NoNewPrivileges=true
|
||||
SystemCallFilter=@system-service
|
||||
```
|
||||
Three notes the file must carry as comments: the helper needs `/etc/haproxy` and `/etc/nginx/stream.d` writable and **nothing else** (`/etc/nginx/conf.d` is deliberately absent — vhost edits land there, so the *vhost* arm is listed in `ReadWritePaths` too and that addition is a decision recorded in the file, not smuggled in); `NoNewPrivileges=true` here as well, because it gets its privilege from being started by systemd as root, not from escalation; and `Accept=yes` on the template means systemd spawns one instance per connection with the socket on stdin, which is why the helper handles exactly one request and exits.
|
||||
- **`ReadWritePaths` must list every directory whose file the helper replaces** (`os.replace` needs write on the directory): `/etc/haproxy`, `/etc/nginx/conf.d`, `/etc/nginx/stream.d`. Getting this wrong is a `EROFS` in production, so the live drill in task 07 is the test that matters and the units file names it.
|
||||
3. `deploy/install-agent.sh` — one new flag, `--with-routectl`, off by default, and its inverse `--without-routectl`:
|
||||
- installs `dashboards-routectl.py` to `/usr/local/sbin/` (`0755`, root) and the two units to `/etc/systemd/system/`, `daemon-reload`, `systemctl enable --now dashboards-routectl.socket`;
|
||||
- **changes nothing about the agent's own unit** — assert that structurally: the rendered `dashboards-agent.service` bytes are identical with and without the flag (`tests/test_deploy_scripts.py`), because the temptation to "help" the agent reach the socket by relaxing it is exactly what this task exists to close;
|
||||
- verifies reachability from the agent's own uid before reporting success: `sudo -u dashboards-agent python3 -c 'import socket; s=socket.socket(socket.AF_UNIX); s.connect("/run/dashboards-routectl.sock")'` — a socket the agent cannot connect to is a feature that fails at 03:00 during an incident, so this check is part of the install, and its output is printed;
|
||||
- `--without-routectl` disables and removes the units and the helper, leaves `/etc` and the state directory alone, and reminds the operator to re-enable any route still off (`routectl list` output printed first).
|
||||
- The re-run guard applies as it does to the unit: a differing installed helper is a diff + exit 1 unless `--force`.
|
||||
4. `deploy/gateway-routectl-README.md` (new, short) — the operator's page: what the two units are, how to see them (`systemctl status dashboards-routectl.socket`, `journalctl -u 'dashboards-routectl@*'`), how to run the helper by hand on the box, how to undo it all, and the sentence that **no sudoers line is involved**: the grant is a file mode systemd enforces.
|
||||
5. `tests/test_deploy_scripts.py` — extend the fake-root sandbox:
|
||||
- `test_with_routectl_installs_helper_and_units_and_leaves_the_agent_unit_byte_identical`;
|
||||
- `test_the_socket_is_0600_and_owned_by_the_agent_user` (the fake `install`/`chown`/`chmod` log, the precedent that proved the bundle's `0640`);
|
||||
- `test_the_helper_is_installed_0755_root_and_its_state_dir_is_created`;
|
||||
- `test_without_routectl_removes_the_units_and_not_the_config_or_the_state`;
|
||||
- `test_no_new_sudoers_line_exists` (the `deploy/agent-sudoers` file is unchanged, and nothing in the new units grants sudo);
|
||||
- `test_the_install_checks_the_agent_can_reach_the_socket` — a fake where that connect fails makes the install exit non-zero rather than reporting a success nobody can use.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; the units are also parsed (`systemd-analyze verify` is not available in the sandbox, so the assertions are on the directive lines, and task 07 runs the real `systemctl` on the gateway).
|
||||
- Coverage: the installer is shell; the gate is the sandbox tests plus `bash -n`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `bash -n deploy/install-agent.sh` clean; `grep -n "NoNewPrivileges\|ProtectSystem\|SystemCallFilter" deploy/install-agent.sh` shows the agent unit's values unchanged from before this task (the guard against "fixing" reachability by weakening the sandbox), and `git diff` for this task shows no edit inside the agent unit template block.
|
||||
- [ ] `uv run pytest tests/test_deploy_scripts.py -q` 100% passing, including the byte-identical agent-unit assertion; whole suite green; coverage not down.
|
||||
- [ ] The two unit files list exactly the paths the helper writes and no others, with the `os.replace`-needs-directory reason stated in a comment.
|
||||
- [ ] README.md/AGENTS.md gain the two unit names and the "no sudoers line" sentence (this task's user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: an install **without** `--with-routectl` is byte-identical to an install before this phase, on every host.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Task 03 — Agent 1.4.0: one POST path, behind the certificate and a flag, plus the marker read back as `disabled`
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the agent's `do_GET`-only `Handler` and `_certificate_allows` (phase 15), and phase 30's `routeInConfig()` seam
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` goes **1.3.0 → 1.4.0** with two additions and no other change: the two config parsers report a marker-authored line as a declared target with **`disabled: true`**, and one new path, `POST /route`, which is reachable only with a verified client certificate **and** an agent started with `--allow-writes`, forwards exactly `{id, action, dry_run}` to the task-02 socket, and answers with the helper's JSON. Every `GET` behaviour stays byte-identical, proven by the additivity control.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py`
|
||||
- Parsers: each haproxy `server` node and each nginx `proxy_pass` / `upstream` member / `map` entry gains `"disabled": <bool>` — `false` for an ordinary line, `true` for one carrying `ROUTE_MARKER` (read with `split_marker`). Additive: no existing key changes meaning, and a route that is off **stays in the document** because "the config declares this and the operator switched it off" is a different fact from "the config never said it" (rule 2).
|
||||
- **New path** in `Handler`:
|
||||
```python
|
||||
def do_POST(self) -> None: # the BaseHTTPRequestHandler API name — keep it
|
||||
if self.path.split("?", 1)[0].rstrip("/") != "/route":
|
||||
self._send(405, {"ok": False, "detail": "no such write path"})
|
||||
return
|
||||
# the certificate check first, exactly as do_GET does it, revoked serial included;
|
||||
# then the flag; then the body; then the peer. Nothing else in this method writes.
|
||||
```
|
||||
Order, and each refusal is its own sentence: no bundle / no verified certificate → `403 client certificate required` (and `403 client certificate revoked` for a withdrawn serial, on this path too); agent started **without** `--allow-writes` → `403 writes are not enabled on this agent`; body not a JSON object, longer than `_ROUTE_MAX_BODY = 4096`, carrying a key outside `{id, action, dry_run}`, an `action` outside `{disable, enable}`, or an `id` longer than 256 chars → `422` with a fixed message and **no peer call**; no `--routectl-socket` configured → `503 no route helper on this host`.
|
||||
- Peer call: one newline-delimited JSON request over `AF_UNIX`, `settimeout(ROUTE_TIMEOUT_SECONDS = 30)` (the helper may run `nginx -t` and a reload), one JSON line back, oversized or unparsable replies are `502 route helper answered something unreadable` and **never** retried — a retry of a write is a second reload. Any `OSError` → `502` with the errno word, no traceback text, no path echoed beyond the flag's own configured value.
|
||||
- **No token path ever gets this**: on the plain-`http` bearer-token arm, `POST /route` is refused with `403 writes require a certificate` — the same reasoning phase 15 applied to reads (the handshake is the identity), and this app's fleet is mTLS anyway.
|
||||
- `--allow-writes` (default off) and `--routectl-socket /run/dashboards-routectl.sock`; both appear in the `/schema` doc block. `--status` and `--uninstall` are unaffected.
|
||||
- `log_message`: the POST line prints the path and the status only — never the body, never the id (an id is a hostname, and the agent's log line convention is that query strings stay out).
|
||||
- `AGENT_VERSION = "1.4.0"`; `/schema` names `POST /route`, `--allow-writes`, `--routectl-socket`, and the `disabled` field on the proxy sections; the status page lists the write path with its refusal sentence so a human with a browser is not surprised by a 405.
|
||||
2. `tests/test_agent.py` — the TLS loopback harness (phase 15) plus a unix-socket stub peer that logs what it received:
|
||||
- `test_the_agent_version_is_1_4_0_and_the_schema_names_the_write_path`;
|
||||
- `test_a_post_without_a_client_certificate_is_refused` and `test_a_revoked_serial_is_refused_on_the_write_path_too` (the denylist is not a read-side rule);
|
||||
- `test_writes_off_refuses_before_the_peer_is_touched` (stub logs nothing);
|
||||
- `test_the_body_vocabulary_is_id_action_dry_run_exactly` — a key added, a wrong `action`, a 5 KB body, a non-object: all refused, all with the peer idle;
|
||||
- `test_a_good_request_reaches_the_peer_once_and_the_answer_is_passed_through` (including `applied:false` staying `false` — the agent invents no success), `test_the_peer_is_called_once_even_when_it_times_out`, `test_an_unreadable_peer_answer_is_502_and_is_not_retried`;
|
||||
- `test_token_mode_cannot_write`; `test_every_other_path_is_405`;
|
||||
- `test_a_marked_server_is_reported_disabled_and_stays_in_the_document` (haproxy + all three nginx arms), `test_an_unmarked_config_has_disabled_false_everywhere`.
|
||||
- **Additivity control re-run**: `data/logs/verify-33-additive.py` (from task 02 of phase 32) extended to demand 1.4.0's `GET /snapshot` equals 1.3.0's with `disabled` added at the target level and nothing else, and equal *exactly* on an unmarked config.
|
||||
3. `src/dashboards/collectors.py` — a `post_route(tls, target, body)` beside `fetch_gateway_snapshot`, reusing the same `AgentTls` material and the **same revoke-before-socket check**, and redacting the URL in every error string it produces (invariant 6: those strings become events).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the matrix above; hermetic (a stub unix socket, no helper, no root, no proxy binaries).
|
||||
- Coverage **>90%** on the new agent code and `collectors.post_route`; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `python3 agent/host_agent.py --help` lists `--allow-writes` and `--routectl-socket`; `/schema` names `POST /route` and `disabled`.
|
||||
- [ ] Against the fake peer: the full refusal matrix answers without a peer call, and the success path calls it **exactly once** and passes the JSON through unaltered.
|
||||
- [ ] `uv run python data/logs/verify-33-additive.py` exits 0: 1.4.0 equals 1.3.0 plus `disabled`, and equals it exactly on unmarked configs.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md note the write path and its two agent-side gates (this task's user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `GET /snapshot`, `/health`, `/info`, `/schema` and the TLS matrix behave exactly as 1.3.0; `deskwork`/`driveripper` are untouched.
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Task 04 — `enabled` joins the payload, `off` becomes reachable, and the warnings stop calling a chosen outage a fault
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** phase 30 decision 1 ("red is the **off** state, styled now, unreachable until the write path exists")
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`connections[].enabled` carries the config's own state (`true` declared and active, `false` declared and marker-off), so `routeInConfig()` stops being a literal and the red styling phase 30 built has a referent. And `warnings()` learns the marker: an empty backend, a targetless vhost or a sink-pointed map entry **caused by the operator's own switch** is not a config-integrity fault, while the same shape caused by a hand-deleted line still is. No count is ever written by this phase.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- `_connection()` gains `"enabled": True`; `config_connections()` sets it from the agent's per-target `disabled` flag (`enabled = not disabled`), and the merge rule keeps its meaning: **a line is enabled only if every claim that merged into it is enabled** — a union of selectors with one disabled line is two lines, so if a merge would mix states the source must not silently union them; state which arm wins and test it (recommended: split into two rows keyed by `enabled`, because "one of these selectors is off" is two facts).
|
||||
- `_skeleton()`/`stub()` gain the key so the parity pin keeps holding; `enabled` is `None` in the stub (nobody read the config), `true`/`false` in an answer — never `false` in an arm that did not read it.
|
||||
- `warnings()` marker-awareness, per kind, and each is a decision to write down rather than a suppression to hope nobody notices:
|
||||
| kind | new rule |
|
||||
|---|---|
|
||||
| `haproxy_backend_empty` | not raised when the backend's only servers are `disabled: true` (the operator emptied it from `/gateway`); still raised when it has no `server` line at all |
|
||||
| `haproxy_route_dangling` / `haproxy_backend_orphan` | unchanged — a switch never renames or removes a `backend`/`use_backend`, so a dangling name is still a broken config |
|
||||
| `nginx_server_unlistened` | unchanged (a disabled `proxy_pass` does not remove a `listen`) |
|
||||
| `nginx_pool_undeclared` / `nginx_variable_unmapped` | unchanged |
|
||||
| `nginx_map_route_missing` | a map entry pointing at the **sink address** is not a missing route; it is a switched-off route, and the sink address is recognised from the payload's own `resolver`/config text rather than hard-coded in the source (a literal in two places is how this drifts) |
|
||||
- The **socket join** is unchanged in code and in ethics: a disabled haproxy `server` has no live sockets after the reload because they drain, and a disabled map entry's sockets to the sink are the sink's own loopback pair. Nothing stamps `0`, nothing zeroes a count, and the `shared` rule is untouched.
|
||||
- `samples()` publishes the gauge for an `enabled: false` line only if the join measured a real count for it (usually `0`, and `0` measured is a legitimate measurement — the distinction is *measured*, never *assumed*).
|
||||
- No `severity` anywhere, still: `off` is a state, not a verdict. The recursive no-verdict walk in `tests/test_api.py` must stay green without an exception for `enabled`.
|
||||
2. `src/dashboards/api.py` — `/api/gateway/snapshot` is otherwise unchanged, but the `gateway` block in `/api/summary` is the one place this phase **does** add a field, because the overview's one line is where "something is switched off" belongs and a page-only fact would be invisible to a notifier: `disabled` (int, count of `enabled is false` rows, `null` in the arms that did not measure). That makes thirteen keys — restate the pinned key set in the same commit, with the reason, and keep `warnings` as a count beside it.
|
||||
3. `tests/test_gateway_source.py`
|
||||
- `test_a_marker_off_route_is_declared_and_disabled` — a config tree with one marked haproxy server + one marked vhost `proxy_pass` + one sink-pointed map entry yields three rows, each `enabled: false`, all their selectors intact, every other row `enabled: true`.
|
||||
- `test_the_source_never_calls_a_route_broken_because_the_operator_disabled_it` and `test_the_same_shape_without_a_marker_is_still_a_warning` — both directions for each of the two affected kinds, run over the **captured** configs plus a marker-injected copy of them (never a toy file).
|
||||
- `test_the_sink_address_is_recognised_not_hard_coded` — a config that writes a different sink port still reads as switched-off, because the sink is identified from the marker+config, not from `44450`.
|
||||
- `test_a_disabled_line_still_publishes_only_a_measured_count` — with no sockets, `connections: 0` only when the table answered; `null` when it did not, and no sample in the second case.
|
||||
- `test_the_stub_and_the_answer_share_one_key_set` gains `enabled`; `test_the_gateway_snapshot_carries_no_verdict_and_no_window` unchanged (extended walk).
|
||||
4. `tests/test_api.py` — the `/api/summary` `gateway` block key set grows `disabled` with its reason in the test docstring; `test_the_overview_line_reads_the_gateway_block_it_claims` learns the new field (and that `0` prints nothing — a permanent `0 switched off` is the ink the empty-state rules exist to avoid, the `warnings` rule restated).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: every rule above from real captures plus injected markers; both directions for each suppression.
|
||||
- Coverage **>90%** on `sources/gateway.py` and `api.py`; whole suite 100%.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/gateway/snapshot` on a fixture with three markers shows three rows with `enabled: false`, their selectors and endpoints intact, and `warnings == []` for the marker-caused shapes.
|
||||
- [ ] Removing the markers from the same tree brings the corresponding warnings back (the suppression is marker-caused, not shape-caused) — asserted as one test pair.
|
||||
- [ ] `/api/summary`'s `gateway.disabled` is `3` on that fixture, `0` on the clean capture, `null` on the stub, and the overview line prints nothing for `0`.
|
||||
- [ ] No `severity` key at any depth, still (the recursive walk passes unmodified).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` paragraph names `enabled`/`off` and the marker-aware warnings rule — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: the socket join, inbound counts, chart gauges and the phase-32 provenance fields are untouched.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 05 — One route, ten gates: `POST /api/gateway/route` and the amendment of invariant 1
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** operator request — "turn off and on connections as needed in the event of security concerns or upgrades" · **Shape borrowed from:** `/api/ups/command`'s four gates and `/api/certs/*`'s fixed-text audit events
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The app's **second and last** upstream write. `POST /api/gateway/route {id, action, danger?, dry_run?}` refuses in nine places before it dials anything, and in the tenth it forwards one reviewed request through `collectors.post_route` and answers with the helper's own triple. `GATEWAY_ALLOW_WRITES` defaults to **off**, `DASH_READ_ONLY` still refuses everything, and `AGENTS.md` invariant 1 is rewritten in this commit to name two exceptions — because a change to a written invariant that leaves the document describing the old one is the bug, not the omission.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/config.py` — two settings beside the existing `GATEWAY_*` block, each with the reason in its comment:
|
||||
- `GATEWAY_ALLOW_WRITES: bool` (default `False`, the same truthy rule as `NUT_ALLOW_ACTIONS`);
|
||||
- `GATEWAY_TIMEOUT_SECONDS` **unchanged** (reads), plus `GATEWAY_WRITE_TIMEOUT_SECONDS: float = 35.0` — deliberately longer than the agent's own 30 s peer timeout so the dashboard never times out first and leaves the operator guessing whether the reload happened.
|
||||
2. `src/dashboards/collectors.py` — `post_route(tls, target, body, timeout)` beside `fetch_snapshot`: the same `AgentTls` client (built per dial, the phase-15 rule), the same **revocation check before any socket opens**, `Authorization` never used on the certificate path, and every error message run through `strip_userinfo` (those strings reach events and `/api/health`-adjacent payloads — invariant 6).
|
||||
3. `src/dashboards/api.py` — the route, and the gate order is the design, so it is written in the function as numbered comments and mirrored in the tests:
|
||||
| # | Gate | Refusal |
|
||||
|---|---|---|
|
||||
| 1 | session (middleware) | `303`/`401`, unchanged |
|
||||
| 2 | `Origin` / `Sec-Fetch-Site` | `403`, unchanged |
|
||||
| 3 | this session's `X-CSRF-Token` on a cookie write | `403`, unchanged |
|
||||
| 4 | `DASH_READ_ONLY` | `403 "this dashboard is read-only (DASH_READ_ONLY)"` |
|
||||
| 5 | `GATEWAY_ALLOW_WRITES` | `403 "route switching is disabled — set GATEWAY_ALLOW_WRITES=1 to enable it"` |
|
||||
| 6 | throttle `route:<address>` on `ctx.login_throttle` | `429` + `Retry-After` (its third key shape; the per-process caveat stays gap 7) |
|
||||
| 7 | body: `id` non-empty string ≤ 256, `action` in `{disable, enable}`, `dry_run` optional bool, **no other key** | `422`, and it is a `422` rather than a `400` for the same reason the login body is |
|
||||
| 8 | `id` exists in the **current** payload (`409` if the job has never answered) and is not already in the requested state (`409 {changed: false}`) | never a guess about a route it cannot see |
|
||||
| 9 | freshness: `gateway` job `ok` and not stale beyond 3× its interval, `host:<GATEWAY_HOST>` `ok`, `sections.haproxy`/`sections.nginx` both `true`, the agent's `version >= 1.4.0` | `409` naming which one — "do not rewrite a config you cannot currently read end to end" |
|
||||
| 10 | `danger == "DANGER"` for **both** verbs | `422` naming the word, the UPS wording's sibling |
|
||||
Then one `collectors.post_route`, and the response is the helper's object plus `host`, `id`, `action`, `dry_run`, and `hint` sentences owned by `api.py` (the `restart_required` precedent: the prose lives in the route, so the page prints what the route said).
|
||||
- **Events**, one per call, `source="gateway"`, `subject=<the route's host>`, `kind` in `{route-disabled, route-enabled, route-denied}`, severity `warn` for a change and `error` for a refusal-that-matters, message fixed-shape: `disable brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443: applied=true validated=true reloaded=true (admin)`. **No diff, no file path, no config text** — `/api/events` is a feed a browser reads, and a config's bytes are not a log line. A `route-denied` from gates 4–10 writes the event with the gate named and **nothing** about which check the caller came closest to passing.
|
||||
- `/api/config`'s `gateway` row gains `writes_enabled` (bool) so the page can render the honest arm instead of assuming (rule 2), and nothing else in that body changes.
|
||||
4. `tests/test_api.py`
|
||||
- `test_the_gateway_surface_is_gets_only_and_the_write_stays_the_ups_one` is **rewritten**, not deleted: the verb enumeration now lists exactly two fleet writes — `/api/ups/command` and `/api/gateway/route` — beside the session writes and the certificate-authority writes, and the test still fails if a third appears anywhere.
|
||||
- `test_each_gate_refuses_without_touching_the_peer`, parameterised over all ten with spies: `collectors.post_route` swapped for a recorder, and each refusal asserts the recorder is empty **and** the event is what the gate named. The CSRF half is proven on the wire in `tests/test_auth_api.py` the way the UPS one is: a scripted helper records nothing without the header and once with it.
|
||||
- `test_danger_is_required_to_enable_as_well_as_disable`, `test_read_only_refuses_and_names_the_variable`, `test_a_route_that_is_already_off_says_so_and_dials_nothing`, `test_a_stale_gateway_or_dead_host_job_refuses_the_write`, `test_a_revoked_agent_refuses_the_write_without_a_socket`, `test_the_write_timeout_is_longer_than_the_agents_peer_timeout` (the ordering, not the numbers).
|
||||
- `test_no_diff_or_config_text_reaches_an_event` — a successful write whose helper answer carries a `diff` field: the event message contains no `diff`, no `--- `/`+++ ` line, no path, and no `was:` marker.
|
||||
- `test_the_writes_enabled_flag_reaches_the_page_config`.
|
||||
5. `tests/test_auth_api.py` — the served-route walk (`walk_routes`) now sees `POST /api/gateway/route` and demands a session; the public-surface list stays exactly four entries; the CSRF-on-the-wire test gains this endpoint.
|
||||
6. `AGENTS.md` — **in the same commit**: invariant 1 rewritten from "exactly one exception" to "exactly two, and both are gated on purpose", naming the two endpoints and the gate list, keeping the sentences that make it work ("Hiding a button is not a safety control", "the UI confirmation … `DANGER` for anything that can cut power" now extended: *or that can cut a route through the fleet's only ingress*). Also: `api.py`'s bullet (its three write lists become four facts and now include the gateway route), `collectors.py`'s bullet (`post_route`), `config.py`'s paragraph (the two new settings), and the `/gateway` UI-conventions sentence about the unwired switch is **removed** — it becomes the wired description in task 06, and the two must not both exist.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: every gate in both directions; the wire-level CSRF proof; the event-shape proof; the verb enumeration restated.
|
||||
- Coverage **>90%** on `api.py` and `collectors.py`; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `curl` against the dev server with writes off answers the `GATEWAY_ALLOW_WRITES` sentence; with it on and no `DANGER` answers the `DANGER` sentence; with both and a live fake agent it answers `applied: true` and the event line is in `/api/events?source=gateway`.
|
||||
- [ ] `grep -n "DASH_READ_ONLY" src/dashboards/api.py` shows it refusing on **both** write paths.
|
||||
- [ ] The verb-enumeration test lists exactly two fleet writes and fails on a third by name; `walk_routes` covers the new POST; `PUBLIC_PATHS` is still four entries.
|
||||
- [ ] `/api/config` serves `gateway.writes_enabled` and the page's own arm is task 06's.
|
||||
- [ ] `AGENTS.md` invariant 1 names two exceptions and their gates in the same commit as the code that made it true.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] No behaviour change in completed work: `/api/ups/command`'s four gates behave exactly as before (the suite's existing UPS pins are untouched), and no read route changed shape.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Task 06 — The switch works, asks for `DANGER` with the diff on screen, and never claims a change it didn't get
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the operator's sketch (phase 30) + this phase's decision that a route off is a config edit with a reload behind it · **Shape borrowed from:** `page-ups.js`'s typed-`DANGER` confirm and `dash.css`'s `dialog` + `input[type="text"].confirm`
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Clicking a switch on `/gateway` shows the **exact one-line diff** the gateway would apply, asks for `DANGER`, applies it, and paints the row from the **answer** — not from hope. Where writes are disabled the switch stays visibly unwired and says which variable would change that, because `/gateway` must look the same on a deployment that chose not to have this feature as it did before the feature existed.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- `routeInConfig(conn)` becomes a reading, not a literal: `return conn.enabled !== false;` — with the comment rewritten: `off` is now reachable, it means "declared in the config, switched off by the operator", and `enabled === undefined` (a pre-1.4.0 agent, or the stub arm) still means "on", so an old agent's page never goes red.
|
||||
- `buildSwitch(conn)` splits into two arms, driven by `/api/config`'s `gateway.writes_enabled` (never inferred):
|
||||
- **writes off** → today's markup exactly: `aria-disabled="true"`, `cursor: not-allowed`, `title` = `Route switching is off — set GATEWAY_ALLOW_WRITES=1 on the dashboard to enable it.` (the sentence from `api.py`, copied into a constant that `tests/test_api.py` compares against the route's refusal so the two cannot drift);
|
||||
- **writes on** → a real control: no `aria-disabled`, `aria-checked` from `enabled`, a full accessible name (`endpoint, reached by <selectors>, in the <source> config, on|off, activate to switch it off`), `title` naming what activating does (edits the file, validates, reloads the proxy).
|
||||
- **The flow**, one async handler, and the order is the safety:
|
||||
1. `POST /api/gateway/route` with `{id, action, dry_run: true}` → `diff` + `hint`;
|
||||
2. open a real `<dialog>` (`showModal()`) whose body is: the row's inbound word and endpoint, the config's own `selectors`, the diff in a labelled `<pre>`, the route's `hint`, and a `input[type="text"].confirm` asking for `DANGER`;
|
||||
3. submit only when the typed word is exactly `DANGER` (client-side check of the *client's* convenience; the server re-checks, always);
|
||||
4. switch `aria-busy="true"`, `POST` with `danger: "DANGER"`, `dry_run: false`;
|
||||
5. paint from the **response**: `applied: true` → the row's `enabled` flips on the next poll (and immediately, optimistically **only** for the switch's own `aria-checked`, with `aria-busy` cleared and a note that the config is applied and the page re-reads); `applied: false` → nothing flips, the refusal prints verbatim through `textContent`;
|
||||
6. `await refresh()` so the numbers come from the agent's next reading rather than from the click.
|
||||
- No optimistic colour, ever: the page does not turn a line green or red because a button was pressed. `pending` is the only state a click may create, and `aria-busy` is how it says so.
|
||||
- A `429`/`409`/`403`/`422` prints the server's `detail` + `hint` in the dialog (`role="alert"`), leaves the dialog open so the operator can read it, and returns focus to the confirm field — the `/login` lesson about a form that eats its own error.
|
||||
2. `src/dashboards/web/pages/gateway.html`
|
||||
- One `<dialog id="gate-switch-dialog" aria-labelledby="gate-switch-title" aria-modal="true">` with the four slots the script fills, shipped `hidden`/closed; no per-row dialogs (the same rows are patched, so one dialog is the whole surface, and a dialog per row is 33 focus targets in the tab order).
|
||||
- Footer paragraph 3 becomes the truth and stays **one** paragraph:
|
||||
> `the switch edits that route in its own config file, validates it with the proxy's own checker and reloads that proxy — nothing is restarted, and no route goes off unless GATEWAY_ALLOW_WRITES is set on the dashboard.`
|
||||
- The `#gate-empty` setup block is untouched.
|
||||
3. `src/dashboards/web/static/dash.css`
|
||||
- `#gate-switch-dialog pre` — mono, `overflow-x: auto`, `max-height` bounded, added/removed lines tinted from the **existing** tokens only (`.diff-add`/`.diff-del` deriving from `--gate-on`/`--gate-off`, the phase-30 existence pair, so no new colour enters the map and the diff reads as the same red/green the rows do);
|
||||
- `button.gate-switch:not([aria-disabled="true"]):hover` and `cursor: pointer` — added **only** under that selector, so a writes-off switch never grows a hover affordance it cannot honour;
|
||||
- any new transition (dialog, knob while `aria-busy`) is named in the file's one `@media (prefers-reduced-motion: reduce)` block — naming one and missing the other is the phase-18 lesson, and there is a test for it;
|
||||
- at ≤640 px the dialog stays inside the viewport (`width: calc(100% - 2rem)` is already the shared rule) and the diff scrolls in its own box, never the document (phase 27's rule).
|
||||
4. `tests/test_api.py`
|
||||
- `test_the_switch_is_built_and_explicitly_unwired` is **replaced** by `test_the_switch_is_wired_only_when_the_deployment_enables_writes`: with `writes_enabled: false` the served script keeps `aria-disabled="true"`, the `GATEWAY_ALLOW_WRITES` sentence, and no listener; with it true the listener exists, `aria-disabled` is gone, and the accessible name states the state and the consequence. The old test's *intent* survives: a dead-looking-live control still fails.
|
||||
- `test_a_toggle_asks_for_a_preview_then_the_word_then_sends_once` — the served script issues `dry_run: true` before any `DANGER` send, requires the exact word, and sends exactly once per activation (no retry loop in the handler).
|
||||
- `test_the_row_is_painted_from_the_answer_not_the_click` — no code path sets `off`/`on` other than the payload/`applied` pass; `applied: false` leaves `aria-checked` untouched.
|
||||
- `test_the_switch_sentence_matches_the_routes_refusal` — the writes-off `title` and `api.py`'s `GATEWAY_ALLOW_WRITES` refusal are the same string, read from both files.
|
||||
- `test_no_gateway_count_is_coloured_by_a_verdict` gains the reachable-`off` fact and keeps every assertion: the diff tints are `--gate-*`, no `.ok/.warn/.fail` or `--ok/--warn/--fail` enters the gateway section, and the warnings panel's `.warn` stays the one exception.
|
||||
- `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` extended to the new transitions (knob busy state, dialog) — and it still asserts there is **one** such block in the file.
|
||||
- `test_the_gateway_page_is_exactly_two_connection_panels_and_a_warnings_list` and the footer-paragraph pin stay green (three paragraphs, one dialog, `<section>` count unchanged).
|
||||
- The orphan-`id` audit must find `#gate-switch-dialog` referenced by the script; every slot inside it is `[data-k=…]`, not an `id`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; a fixture render for each arm (writes on/off, `enabled: false` from the payload, `applied: false` from the response) asserting class, `aria-checked`, `aria-busy`, `title` and the accessible name.
|
||||
- Integration: the dev server against a stub agent (the `tests/conftest.py` fake) answering `applied: true` then a payload with the row `enabled: false` — the row turns red **because of the payload**, and the second read of the switch's name says `off`.
|
||||
- Coverage: `node --check` clean; floor does not fall.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With `GATEWAY_ALLOW_WRITES=0` the page is indistinguishable from phase 32's: `aria-disabled`, the `GATEWAY_ALLOW_WRITES` sentence, no click effect, no hover affordance.
|
||||
- [ ] With it `=1` (and the fake agent): clicking shows the one-line diff and the `DANGER` field; the wrong word sends nothing; the right word sends once; the row's colour changes on the payload that follows and never before.
|
||||
- [ ] A stubbed `applied: false, validated: false, restored: true` leaves every row exactly as it was and prints the route's own sentence.
|
||||
- [ ] Tab order: dialog opens with focus in the confirm field, Escape closes it without a request, focus returns to the switch. At 360 px the dialog and its diff stay inside the viewport and the document scrolls no sideways pixels.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` and `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` paragraph describes the wired switch and both arms (replacing the "unwired" sentence task 05 deleted — the two must not coexist) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: tallies, warnings panel, charts, filters, colour semantics, and the other seven pages.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 07 — Install the helper, prove it from both sides, and drill one named route with the operator watching
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** AGENTS.md's rollout discipline (phases 24 and 28) applied to the one change in this repo that can take the fleet's ingress down
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`gateway` runs agent **1.4.0** with the socket-activated helper, and one route is switched off and back on **for real**, with the live ingress ending the task in the state it started in. The drill is not a formality: it is the only proof that `ReadWritePaths`, `SocketMode`, `SO_PEERCRED`, `nginx -t` and the reload all line up on a machine, which no sandbox can tell us. **If the operator is not present in the session, this task stops after step 3 and reports — that is the correct outcome, not a failure**, and ntfy is not a way to ask (it is outbound-only).
|
||||
|
||||
## Work
|
||||
|
||||
1. **Backups first, hashed.** On `gateway-ops` as root:
|
||||
`cp -a /etc/haproxy /root/dashboards-33-backup-<utc>/haproxy && cp -a /etc/nginx /root/dashboards-33-backup-<utc>/nginx`, then `sha256sum` every file that the helper can edit (`haproxy.cfg`, `nginx.conf`, `conf.d/*.conf`, `stream.d/*.conf`) into the transcript. This is the restore point for every later step.
|
||||
2. **Sandbox rehearsal on the box** (writes nothing real): run the helper with `--apply` against a copy tree (`--root /tmp/dashboards-33-rehearsal`) with the *real* `nginx -t` and `haproxy -c` binaries on `PATH`, for each of the three marker arms and the `default` refusal. Record the three diffs. A checker that fails on a copied tree for path reasons is expected and is *not* the rehearsal's point — the arms and the restore byte-equality are.
|
||||
3. **Install, read-only-verified first**:
|
||||
- re-stage `/home/ops/agent-install/` (repo `agent/host_agent.py` **1.4.0**, `deploy/install-agent.sh`, `deploy/dashboards-routectl.py`, the two unit files), leaving `*.20261023-backup` copies sudo will not execute;
|
||||
- `systemctl cat dashboards-agent` → record its sha256 and the four security directives verbatim;
|
||||
- `sudo /home/ops/agent-install/deploy/install-agent.sh --bind 10.1.1.27 --port 9120 --allow-writes --with-routectl` (arguments read back off the unit; **no** `--cert-bundle`, the bundle on disk is what drives the TLS lines — the phase-28 finding);
|
||||
- the guard's diff is inspected before `--force`: the only legitimate change to the agent unit is the added `--allow-writes` in `ExecStart` — if `NoNewPrivileges`, `ProtectSystem`, `PrivateDevices`, `SystemCallFilter` or `CapabilityBoundingSet` moved, **stop** and report: the sandbox was not traded for this feature;
|
||||
- `sudo systemctl restart dashboards-agent`; `systemctl is-active` for it and for `dashboards-routectl.socket`;
|
||||
- reachability both ways: as `dashboards-agent`, connect to `/run/dashboards-routectl.sock` and call `list` (must answer `[]`); as any other non-root uid, connect and get `EACCES` (the mode is the first gate and the peer check is the second);
|
||||
- read-only probes: CA-only `--status` answers **1.4.0**; the dashboard's mTLS read shows `agent_version 1.4.0`, every section intact with live numbers, `disabled: false` everywhere, and `/api/gateway/snapshot` rendering identically to phase 32's.
|
||||
4. **Preconditions for the drill — all checked, all recorded, any failure stops it:**
|
||||
- the operator has said go, in this session, in words;
|
||||
- `nut` is not on battery and no UPS event is in the last hour (the same "don't do the risky thing during a power event" rule the UPS commands keep);
|
||||
- `gateway`, `host:gateway`, `disks:gateway` all `ok` and fresh; `warnings == []`;
|
||||
- `nginx -t` and `haproxy -c -f /etc/haproxy/haproxy.cfg` pass on the box before anything is touched;
|
||||
- `routectl list` is empty (no half-finished state from an earlier attempt);
|
||||
- the drill target is **`iperf3-5201`** (`haproxy:5201->web.reeselink.com:5201`) — a benchmark service with no human dependency. **Never** the auth, git, alerts or file routes: not `keycloak*`, `gitea*`, `forgejo*`, `ntfy*`, `nextcloud*`, `immich*`, `brain*`, `keycloak`, or any `map` entry an operator would notice within a minute. If even `iperf3` is in doubt, the drill does not run and the task reports that instead.
|
||||
5. **The drill, in order, with the page open:**
|
||||
1. `dry_run` through `POST /api/gateway/route` → the diff, and the page's dialog shown;
|
||||
2. apply with `DANGER`; the helper answers `applied/validated/reloaded`;
|
||||
3. prove it four independent ways: `routectl list` names the route; the marked line is in `haproxy.cfg` with `was:`; `haproxy -c` still passes; `curl -sv --resolve <host>:25565:127.0.0.1` (or the drill port) from the box connects and is refused/closed rather than proxied;
|
||||
4. the page: the row is **red** within one poll, its live count drains to `0` on its own, no other row changed, and `/api/summary`'s `gateway.disabled` is `1`;
|
||||
5. re-enable through the page; `routectl list` empty; `sha256sum` of every file equal to step 1's capture, byte-for-byte;
|
||||
6. the page green again, `disabled` back to `0`, and `journalctl -u 'dashboards-routectl@*'` showing exactly the two activations.
|
||||
6. **Abort rules, written where they can be followed:** if any step fails, restore from the step-1 backup (`cp -a` back, `haproxy -c`, reload), re-read the page, and write down what happened. Never leave a marked line behind without saying so in the transcript's first line. Never run two drills at once, never run one while the `gateway` job is stale, and never re-run an apply to "make sure" — the helper is the place idempotence lives, and the answer tells you which state you are in.
|
||||
7. `data/logs/NOTES-33-switching.md`: the hashes before and after, the unit diff that was guarded, the socket's owner/mode, both probe transcripts, the four proofs, the `journalctl` window, the screenshots' names, and — if the drill did not run — **exactly** which precondition stopped it, so the next operator can finish it deliberately.
|
||||
8. `tests/test_live_fleet.py`: `test_the_switch_round_trip_on_one_named_route`, marked `live` **and** gated on a separate `--allow-live-writes` flag, skipped by default with a reason naming the route it would touch, restoring in a `finally`. The default live suite stays read-only.
|
||||
|
||||
## Testing & Quality
|
||||
- This task's code change is the one opt-in live test; everything else is procedure plus transcript.
|
||||
- After it: `uv run pytest`, `uv run pytest -m live --live` (green, round-trip **skipped**), `./scripts/dev-server.sh status` (ten sources `ok`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `systemctl cat dashboards-agent` shows the four security directives **identical** to step 3's capture, with `--allow-writes` as the only added argument, and its pre/post sha256 pair is in the note.
|
||||
- [ ] `dashboards-routectl.socket` is `active`, owned `dashboards-agent:dashboards-agent` mode `0600`; a non-agent uid gets `EACCES`; the agent's own uid gets a JSON answer.
|
||||
- [ ] The drill's four proofs are in the transcript, plus the byte-identical `sha256sum` restore of every editable config file, and `routectl list` is empty at the end.
|
||||
- [ ] `/gateway` showed the row red and its count draining, and returned to green — screenshotted at both states.
|
||||
- [ ] `uv run pytest -m live --live` passes with the round-trip **skipped** by default; `uv run pytest -m live --live --allow-live-writes` passes too, if and only if the drill ran.
|
||||
- [ ] `data/logs/NOTES-33-switching.md` exists and its first line states the final state of the ingress (`all routes enabled`, or exactly what is off and why).
|
||||
- [ ] No repo file changed except the opt-in live test and `data/logs/`.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Task 08 — Docs that describe a dashboard which can change the fleet, the browser pass, the full gate
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** AGENTS.md definition of done + the fact that this phase changed an invariant, so the documents are part of the change rather than a summary of it
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`README.md` and `AGENTS.md` describe the write path honestly — what it touches, what refuses, how to turn it off entirely, and how to recover a route by hand at 03:00 when the dashboard is the thing that is broken. Then the browser pass measures the wired page in both arms, the mobile audit re-runs, and every gate is green.
|
||||
|
||||
## Work
|
||||
|
||||
1. `AGENTS.md` — the full consistency pass, with these named as the passages this phase makes stale if left:
|
||||
- **invariant 1** — two exceptions, both gated, the gate list, and the reason a typed word survives in the design (`Hiding a button is not a safety control`);
|
||||
- `agent/host_agent.py` bullet — **1.4.0**, the marker grammar (`ROUTE_MARKER`, `split_marker`, `config_line_index`), `disabled` on every target, `POST /route` behind certificate + `--allow-writes`, the single peer call and its no-retry rule;
|
||||
- the new deploy surface — `deploy/dashboards-routectl.py` (verbs, marker + `was:`, validate-restore-reload, never restart, the sink file, drift refusal), the two units, `--with-routectl`/`--without-routectl`, and the sentence that the agent's unit lost no security directive and gained no sudoers line;
|
||||
- `sources/gateway.py` bullet — `enabled`, the merge rule with mixed states, the marker-aware warnings table, the `/api/summary` `disabled` field;
|
||||
- `api.py` bullet — the two fleet writes, `/api/gateway/route`'s gate order, the fixed-shape audit events with no config text, `/api/config.gateway.writes_enabled`;
|
||||
- `collectors.py` bullet — `post_route` and the shared revoke-before-socket rule;
|
||||
- `config.py` paragraph + README's table — `GATEWAY_ALLOW_WRITES` (default off) and `GATEWAY_WRITE_TIMEOUT_SECONDS` (and why it exceeds the agent's own timeout);
|
||||
- the `/gateway` UI-conventions paragraph — the switch is wired, the preview-then-`DANGER`-then-paint-from-the-answer flow, `off`'s appearance, the writes-disabled arm's sentence, the diff box scrolling in its own container;
|
||||
- the **fleet record** — `gateway` at **1.4.0** with the helper units installed, the unit's directives re-read, the drill result and `NOTES-33-switching.md`;
|
||||
- **gap list** — name-route switching **closed**, and the honest limits written as gaps: one admin (gap 6 unchanged, so anyone with the session can switch a route), no scheduler, no per-host kill switch, no runtime haproxy state (a hand reload cannot undo a marker), and `GATEWAY_ALLOW_WRITES` off by default means the feature is invisible on a fresh deployment until it is deliberately enabled.
|
||||
2. `README.md` + both tracked env templates — `.env.example` and `.env.dev.example` gain `GATEWAY_ALLOW_WRITES` with its default and a comment naming exactly what enabling it authorises, and `tests/test_auth_docs.py` gains the pin (the same device that keeps the auth settings declared in the templates), because a write gate no template names is a write gate nobody finds — and never a value, only the declaration. Then the operator-facing half: the `/gateway` table row (routes can be switched off **and on**), a short *Switching a route* subsection (what the dialog does, what `DANGER` means here, that the edit is persistent across reload and reboot, and the three recovery commands: `routectl list`, `routectl verify`, `routectl enable --id …` on the box), the *Enabling it* subsection (`GATEWAY_ALLOW_WRITES=1`, `--with-routectl`, the two units, how to undo everything), and the configuration table's two new rows. Also: the sentence that a route off is a **deliberate outage** and appears in the event feed, so an operator does not mistake it for an incident.
|
||||
3. `data/logs/33-switching.cjs` — Playwright pass (house shape, exit non-zero on FAIL, prints what it measured):
|
||||
- writes-**off** arm: `aria-disabled`, the `GATEWAY_ALLOW_WRITES` sentence, a click does nothing (no request on the wire), no hover affordance;
|
||||
- writes-**on** arm against the stubbed agent: the dialog opens with the diff and focus in the confirm field; the wrong word sends nothing; the right word sends exactly one request; the row's colour changes only after the payload says `enabled: false`; `applied: false` leaves the row untouched and prints the route's sentence; Escape closes with no request and returns focus;
|
||||
- `off` colour contract: spine halves, tick, run, switch and diff tint all compute to the same `--gate-off` value in dark **and** light, and an `on` row is fully `--gate-on` — no fifth colour, no verdict token anywhere in the section;
|
||||
- reduced motion: nothing animates, including the dialog and the busy knob;
|
||||
- 360 px: the dialog and its `<pre>` stay inside the viewport, the diff scrolls in its own box, and the document scrolls zero sideways pixels;
|
||||
- screenshots `data/logs/33-gateway-{writes-off,dialog,danger,off,off-light,refused,phone,phone-dialog}.png`, transcript `data/logs/33-switching.out`.
|
||||
4. Re-run `data/logs/27-task02-mobile-audit.cjs` (nine pages, both widths): exit 0, **0 FAIL / 0 FINDING**, tally recorded — the dialog is new markup and the audit is what catches a fact painted past a card edge.
|
||||
5. The whole gate, recorded in the transcript:
|
||||
`uv run pytest`, `uv run pytest --cov`, `uv run pytest -m live --live`, `uv run ruff check src tests agent deploy`, `uv run pyright`, `bash .agents/validate.sh`, `uv pip list --outdated` (empty — no dependency is added by this phase, and `deploy/dashboards-routectl.py` is stdlib on a host's Python 3.10).
|
||||
6. `./scripts/dev-server.sh status` with writes **off** → ten sources `ok` and no change; with `GATEWAY_ALLOW_WRITES=1` and the fake agent → still ten `ok`, and a write through `curl` round-trips.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: this task is the phase's proof that the documents and the shipped page agree; every completion-criteria number comes from `33-switching.out`, `NOTES-33-switching.md` or a gate's own output.
|
||||
- Coverage: floor green with the phase's new modules included; no `# pragma: no cover` added to reach it.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `grep -n "one exception\|exactly one write" AGENTS.md` finds nothing; invariant 1 names **two** endpoints and their gates, and the `/gateway` paragraph no longer says the switch is unwired (the two statements must not coexist).
|
||||
- [ ] `grep -n "GATEWAY_ALLOW_WRITES" README.md AGENTS.md .env.example .env.dev.example` — declared in both templates with its default and a comment saying what enabling it authorises, and documented in both documents.
|
||||
- [ ] `data/logs/33-switching.out` exits 0 with the both-arms results, the `--gate-off`/`--gate-on` equalities in both themes, the reduced-motion pass and 0 px sideways scroll at 360 px; the eight screenshots exist.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.cjs` exits 0, 0 FAIL / 0 FINDING, tally recorded.
|
||||
- [ ] All five gates green, `uv pip list --outdated` empty, and the live suite green with the round-trip skipped by default.
|
||||
- [ ] A full read of `README.md` and `AGENTS.md` finds no inconsistency anywhere, including pre-existing ones, fixed in this phase's commits.
|
||||
- [ ] No behaviour change in completed work: the UPS write path and its four gates, the other five dashboards, `/settings`, `/certs`, `/login`, and every read route's payload are exactly as phases 31–32 left them.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Phase 34 — Radiance is a first-class engine: a third `/llm` panel that says what its /metrics and /stats publish, nothing invented
|
||||
|
||||
**Source:** operator request, 2026-10-24 — "add support for a new llm engine 'radiance' which is running on driveripper now. Check it out at http://driveripper.reeselink.com:8000/ and come up with a plan." + the two decisions taken with them in that conversation: **radiance replaced vLLM at driveripper:8000** (the dev config's `VLLM_BASE_URL` points at the engine that is now radiance, so the dev config moves to `RADIANCE_BASE_URL`; vLLM *code* stays — the API shape is additive-only and other deployments may run vLLM), and **the per-position spec-acceptance curve is in** (seven positions, each its own acceptance ratio).
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/sources/llm.py` (the gufo/vLLM field maps, the rule-2 throughput precedence, `HISTORY_METRICS`, `_sessions()`, `fleet_totals()`, `SEVERITY_CELLS`), `src/dashboards/config.py` (the `*_BASE_URL`/`*_API_KEY` engine settings, `any_source_enabled`), `src/dashboards/collectors.py` (`fetch_engine`, the best-effort `/slots` fetch, `llm_jobs`), `src/dashboards/api.py` (the `llm_router` 404 gate, `/api/config`'s llm block, the summary's llm block, the history defaults), `src/dashboards/alerts.py` (the cell-driven `_engine_alerts`), `src/dashboards/severity.py` (the only threshold table), `src/dashboards/web/static/page-llm.js` + `pages/llm.html` (the `only:` scoping, strip cells, slot grid), `tests/test_llm_source.py` / `tests/test_collectors.py` / `tests/test_api.py` / `tests/test_config.py` / `tests/test_live_fleet.py`, `scripts/capture-fixtures.sh`, `.env.dev` / `.env.dev.example`, `README.md`.
|
||||
|
||||
## Objective
|
||||
|
||||
The dashboard treats radiance (a vLLM-derived engine, v1.15.1, model `juggernaut`, on `driveripper:8000`) as a **first-class third engine** — `llm:radiance` — with its own field map pinned by a verbatim capture, its native windowed throughput gauges winning per rule 2, its `/stats` endpoint as the session source (radiance's answer to gufo's `/slots`), and a radiance strip on `/llm` (queue cap, version/uptime, per-card VRAM, the 7-bar spec-acceptance curve) — while gufo and vLLM keep their exact payloads, history names, and behaviour.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Radiance replaced vLLM at driveripper:8000.** | `.env.dev`'s `VLLM_BASE_URL` line becomes `RADIANCE_BASE_URL` (same address). The vLLM engine **code and env var stay** — `VLLM_BASE_URL` still configures a vLLM engine for other deployments; only the dev fleet's config moves. `AGENTS.md`'s fleet table says `vLLM :8000` for driveripper; that is the one stale fact the final pass fixes. |
|
||||
| 2 | **The per-position spec-acceptance curve is in.** | `radiance:draft_position_acceptance_ratio{position="0".."6"}` renders as a 7-bar hand-rolled SVG on the radiance strip, `role="img"` + `aria-label`, pinned in `tests/test_api.py` beside every other chart. Snapshot-only: the seven per-position ratios do **not** get history series (a bounded store, rule 9); the overall `draft_acceptance_ratio` does. |
|
||||
| 3 | **Engine internals the operator cannot act on are out.** | `link`/`passes`/`steps*`/`ckpt*`/`hit_cuts`/`released`/`kv_stream*`/`lookup*`/`linear*`/`attn_cached`/`client_disconnect`/`stream_overflow` stay out of the payload (flavour text is not helpful information). What the payload carries: native + measured throughputs, KV usage + max tokens/bytes, queue depth/cap/admission-rejected, preemptions, the prefix hit-rate gauge, overall + per-position spec acceptance, the decode-step gauges, `requests[]` sessions, per-card VRAM, `model`/`version`/`uptime_s`/`startup_s`, lifetime `tokens_total`. |
|
||||
| 4 | **No new thresholds, no new write path.** | The only rated radiance cell is KV pressure, via the **existing** `llm.kv_cache_usage_pct` row (radiance publishes the same 0-1 ratio the vLLM map already feeds it). Spec acceptance is an efficiency number, not a verdict — uncoloured. No radiance endpoint accepts writes; the app's only upstream write stays `POST /api/ups/command` (invariant 1 unchanged). No new dependency, no new `/api` route — the served-route walk and the public-surface pin are untouched. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `23_llm_rate_responsiveness` (complete) — the per-cadence `RateEstimator` and the refused-window → `n/a` + `Δ<window>` semantics this phase reuses for the measured Δ of radiance's vLLM-shaped counters.
|
||||
- `13_llm_page_and_decode_rate` (complete) — the engine-panel/strip shape, the `only:` scoping, the served-byte pins this phase extends.
|
||||
- `24_dev_mtls_rollout` (complete) — the direct-dial fleet; radiance is dialled the same plain way the other engines are (no token, no certificate — the operator's address answers unauthenticated).
|
||||
- Independent of `33_gateway_route_switching` (todo, in progress) — orthogonal module; no shared file besides `api.py`'s already-present llm routes, which this phase does not touch.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_capture_radiance_ground_truth.md` — `capture-fixtures.sh` records radiance's `/metrics`, `/stats` and `/v1/models` verbatim into `tests/fixtures/`, and the fixture name-set is pinned hermetically.
|
||||
2. `02_config_and_the_engine_registry.md` — `RADIANCE_BASE_URL` / `RADIANCE_API_KEY`, one `llm_engines()` registry replacing the five hardcoded gufo/vllm enumeration sites, the dev config moves to radiance.
|
||||
3. `03_the_radiance_field_map.md` — `RADIANCE_SERIES`/`RADIANCE_HISTOGRAMS`, `parse_radiance_metrics()`, the radiance `_sessions()` arm, the additive history series, `fleet_totals`' seams copy.
|
||||
4. `04_the_radiance_job.md` — `fetch_engine` learns the best-effort `/stats` fetch, the `llm:radiance` job.
|
||||
5. `05_the_llm_page_learns_radiance.md` — the radiance strip (queue cap, version/uptime, per-card VRAM, the 7-bar spec curve), the `only:` scopes, `requests[]` rows, the empty-state text.
|
||||
6. `06_docs_and_live_verify.md` — the live read-only test, the fixture re-capture, README/AGENTS.md pass, the live-data screenshots.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the radiance parse against the verbatim fixtures (name-set drift pin, rule-2 precedence in both directions, absent → `n/a`, the refused-window arm, severity pass-through, the per-position shape, both session arms, `fleet_totals` with a radiance payload), the collector's best-effort `/stats` (a dead `/stats` keeps the `/metrics` poll healthy — invariant 4), the registry (one place lists the engines), the API surface (`/api/config.llm.radiance`, the 404 gate, the history defaults, the summary), and the live read-only test.
|
||||
- Coverage: **>90%** on new/modified code; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's — pre-existing failures included) and achieves **>90%** code coverage on new/modified code.
|
||||
- Every UI change is verified in a real browser against live data, screenshot in `data/logs/` (AGENTS.md rule 10) — with gufo and radiance both answering, the cross-engine fleet block finally renders on the dev fleet, which is the phase's own proof that the sum is honest.
|
||||
|
||||
## Documentation & Consistency
|
||||
- The final pass reviews `README.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For `AGENTS.md` it fixes only *stale statements* (a dead command, a changed rule, an outdated fact) and **never adds or expands its content** (a file that states its own update policy, e.g. "updated rarely, if ever", wins).
|
||||
- Tasks update `README.md` immediately when their own changes are user-visible (setup, commands, configuration, features, conventions), and `AGENTS.md` only when a statement in it became stale.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `RADIANCE_BASE_URL` set and `GUFO_BASE_URL` set: `curl -s http://127.0.0.1:8090/api/llm/snapshot | jq .engines | jq 'map(.engine)'` lists `gufo`, `radiance` (and `vllm` only if configured), and `fleet.engines_live` is 2; `/api/config.llm.radiance.enabled` is `true`.
|
||||
- [ ] `RADIANCE_BASE_URL` unset: `/api/llm/snapshot` answers `404 {"configured": false}` with no dial (spy-pinned), and no `llm:radiance` job exists in the hub.
|
||||
- [ ] `tests/fixtures/metrics_radiance.txt` and `tests/fixtures/stats_radiance.json` exist, are verbatim captures, and a hermetic test fails if the fixture's `radiance:*`/`vllm:*` name set drifts from `RADIANCE_SERIES`/`RADIANCE_HISTOGRAMS`.
|
||||
- [ ] The radiance panel on `/llm` shows the native `Δ<window>`-labelled throughputs when the gauge is absent, the KV bar with its severity colour, the 7-bar spec curve with its `aria-label`, the `requests[]` rows when a request is live and the counts-otherwise arm at rest — screenshot in `data/logs/`.
|
||||
- [ ] `uv run pytest -m live --live` includes a radiance test that passes read-only against the running engine.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this phase's), coverage >90%; `ruff` + `pyright` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md is accurate and consistent with the whole project; AGENTS.md has no stale statements (verified in the final pass — the `vLLM :8000` fleet fact included).
|
||||
- [ ] no behaviour change in completed work: the gufo and vLLM payloads, history names, alerts, and served pages are byte-identical where this phase does not touch them (the suite's existing pins untouched).
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 01 — Ground truth first: radiance's /metrics, /stats and /v1/models recorded verbatim
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`scripts/capture-fixtures.sh` records radiance's three endpoints verbatim into `tests/fixtures/`, driven by the same env the app reads (AGENTS.md rule 7: field maps are captured from the running engine, never trusted to a README or a memory of an upstream), and a hermetic test pins the recorded `radiance:*`/`vLLM-compat` name set so a metric rename fails the suite before the dashboard does.
|
||||
|
||||
## Work
|
||||
|
||||
1. `.env.dev` — the engine block: `VLLM_BASE_URL=http://driveripper.reeselink.com:8000` becomes `RADIANCE_BASE_URL=http://driveripper.reeselink.com:8000` with a comment line (the operator's decision of 2026-10-24: radiance replaced vLLM at that address; the vLLM engine code stays for other deployments). `.env.dev` is gitignored — this edit is what makes the live capture below dial the right thing.
|
||||
2. `scripts/capture-fixtures.sh` — the engine section learns radiance, the same way it reads the engine URLs (nothing hard-coded, so a capture can never come from a different topology than the dashboard polls):
|
||||
- `[ -n "${RADIANCE_BASE_URL:-}" ] && ENGINES+=("radiance:${RADIANCE_BASE_URL%/}/metrics")` (the existing `--with-llm` gate stays the only switch).
|
||||
- The radiance entry captures **three** files, not one: `metrics_radiance.txt` (the `/metrics` body, verbatim), `stats_radiance.json` (the `/stats` body, verbatim JSON), `models_radiance.json` (the `/v1/models` body — small, and the model's `capabilities`/`max_model_len` are the only place the engine says what it can do). Each gets the same verbatim-header comment the other fixtures carry: recorded by the script, volatile values move between captures, the contract asserts **shape and name set, never the numbers** (the `nut_*.vars` precedent).
|
||||
- The `requests[]` shape: the capture at rest records `requests: []` (the engine was idle at first contact). The header comment says so, and the script prints a hint line (`requests[] was empty — re-run with a request in flight to record its shape`) when it sees `in_flight == 0` in the stats body. Task 03's session arm must handle the empty shape; if the operator runs a re-capture with live traffic before task 03 lands, the fixture carries the real rows and task 03's defensive arm is tested against both.
|
||||
- `--with-llm`'s `ENGINES` loop currently appends `<name>:<url>/metrics` only; restructure it so each engine entry is `<name> <url>` and the loop writes the `<name>.txt` metrics file plus, for radiance only, the two JSON sidecars (gufo keeps its separate `/slots` pass exactly as today).
|
||||
3. `tests/test_llm_source.py` — a fixture-loader following the existing skip pattern (`tests/conftest.py` skips a contract test when its fixture is missing, naming the capture command in the skip message — the new test skips the same way). With the fixtures present it asserts, against `metrics_radiance.txt`:
|
||||
- the sorted set of series names in the file equals the sorted set `RADIANCE_SERIES` names + `RADIANCE_HISTOGRAMS` names that the fixture actually carries (the drift pin; a renamed `radiance:*` series fails this test, the way the gufo/vLLM captures are pinned);
|
||||
- every name in the map that the fixture does not carry is listed in a module-level `RADIANCE_OPTIONAL` set (names that exist on other radiance builds), so "absent here" and "absent everywhere" are different things in the test (rule 2's absence arms need to be tested against real absence, not a typo).
|
||||
- a second loader for `stats_radiance.json`: the top-level key set is pinned as a frozenset (shape, not values), `cards[]` is a non-empty list of dicts with `index`/`name`/`vram_used`/`vram_total`, `requests` is a list, and `draft_positions` is a list of dicts with `reached`/`accepted`/`rate`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the fixture-loader + name-set + shape tests above (they are the unit — the fixtures are the input contract).
|
||||
- Coverage: **>90%** on this task's new/modified code (the script is shell; its behaviour is pinned by `tests/test_deploy_scripts.py`'s existing capture-script tests — extend one of them to assert the radiance branch exists and writes the three files when `RADIANCE_BASE_URL` is set, using the script's existing sandbox/fake-curl pattern).
|
||||
- The live run itself (operator's machine, this dev fleet): `set -a; . ./.env.dev; set +a && ./scripts/capture-fixtures.sh --with-llm` produces the three files with the recorded headers.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `./scripts/capture-fixtures.sh --with-llm` (with `.env.dev` sourced) writes `tests/fixtures/metrics_radiance.txt`, `stats_radiance.json`, `models_radiance.json`, each with a verbatim header; the `requests[]`-empty hint printed at rest.
|
||||
- [ ] `tests/fixtures/metrics_radiance.txt` carries both families on one file: `grep -c '^radiance:' tests/fixtures/metrics_radiance.txt` > 0 and `grep -c '^vllm:' tests/fixtures/metrics_radiance.txt` > 0 (the vLLM-derived shape is part of the contract).
|
||||
- [ ] The hermetic name-set/shape tests pass (and skip with the capture command named when the fixtures are absent).
|
||||
- [ ] `tests/test_deploy_scripts.py`'s capture-script pin covers the radiance branch.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's).
|
||||
- [ ] README.md updated for anything user-visible this task changes (the capture script's `--with-llm` line now records three engine endpoints); AGENTS.md touched only if a statement in it became stale (the phase's final pass re-checks both)
|
||||
- [ ] no behavior change in completed work (gufo's `/slots` pass and the agent/UPS captures behave exactly as before)
|
||||
@@ -0,0 +1,38 @@
|
||||
# Task 02 — One registry: `RADIANCE_BASE_URL` and the five hardcoded engine lists become one function
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Radiance is configured the way the other two engines are (a base URL, an optional API key, unset by default → "not configured", rule 3), and the **five** places that enumerate the gufo/vllm pair by hand — `collectors.llm_jobs`, the `llm_router` 404 gate, the history-route defaults, `/api/config`'s llm block, and the summary's `llm.enabled` — all read one `llm_engines()` registry, so the next engine is a one-line addition and no enumeration site can silently keep a two-engine world.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/config.py`
|
||||
- Beside the existing engine settings: `radiance_url: str = _base_url("RADIANCE_BASE_URL")` and `radiance_api_key: str = _env("RADIANCE_API_KEY")` — the same shapes, the same comment idiom (unset = "this deployment does not run it", never a red panel).
|
||||
- `any_source_enabled` gains `or bool(self.radiance_url)`.
|
||||
- `def llm_engines(self) -> list[tuple[str, str, str, str]]` — ordered `(key, display, url, api_key)` for the three engines, in **stable display order** `gufo, vllm, radiance` (the UI's card order and the history defaults follow it; the order is part of the contract, pinned in a test), with a comment that the list is the single place an engine's presence is decided. A `bool(self.radiance_url)`-style `configured` test lives at the call sites (`url` non-empty), matching how `nut_enabled`/`gateway_enabled` work — no new property for nothing.
|
||||
2. `src/dashboards/collectors.py` — `llm_jobs` rewrites its two `if` blocks over `settings.llm_engines()`: one `Job(f"llm:{key}", "llm", display, settings.llm_interval, …)` per **configured** engine; `with_slots=True` when `key == "gufo"`, `with_stats=True` when `key == "radiance"` (the flag plumbing lands in task 04; this task introduces the flag on `fetch_engine` as a no-op parameter so `llm_jobs` reads its final shape now).
|
||||
3. `src/dashboards/api.py`
|
||||
- The `llm_router` 404 gate (today `if not (ctx.settings.gufo_url or ctx.settings.vllm_url)`) becomes "no configured engine" over the registry — same `404 {"configured": false, "detail": …}` shape, and the `detail` sentence lists all three variable names (`GUFO_BASE_URL` / `VLLM_BASE_URL` / `RADIANCE_BASE_URL`) so the not-configured arm names the fix.
|
||||
- The `/api/llm/history` defaults (`(("gufo", ctx.settings.gufo_url), ("vllm", ctx.settings.vllm_url))`) become the registry's configured `(display, url)` pairs — the `engines` query-param filtering behaves exactly as today.
|
||||
- The summary's `llm.enabled` (today `bool(ctx.settings.gufo_url or ctx.settings.vllm_url)`) becomes "any engine configured" over the registry.
|
||||
- `/api/config`'s llm block gains a third key, **additively**: `"radiance": {"enabled": bool(settings.radiance_url), "base_url": strip_userinfo(settings.radiance_url), "auth": bool(settings.radiance_api_key)}` — the existing `gufo`/`vllm` keys keep their exact names and shape (the API shape is additive-only, LOCKED).
|
||||
4. `.env.dev.example` — the engines block keeps the gufo line, keeps the vllm line (vLLM is still a supported engine for other deployments), and adds `export RADIANCE_BASE_URL=http://driveripper.example.net:8000` with the same documentation idiom. `README.md`'s env-var table row (`GUFO_BASE_URL / VLLM_BASE_URL …`) gains radiance in the same sentence shape (the "unset means *not deployed here*" clause applies unchanged).
|
||||
5. `tests/test_config.py`
|
||||
- `RADIANCE_BASE_URL` unset → `radiance_url == ""`, no job, `any_source_enabled` unchanged by it; set → job present, `any_source_enabled` true.
|
||||
- `llm_engines()` pins: the exact ordered triple, the display names (`gufo`, `vLLM`, `radiance`), the api-key passthrough (a set `RADIANCE_API_KEY` travels into the tuple), and that an unset engine is **still listed** with an empty url (the registry is the full fleet, configured-ness is a url test — the `parse_targets` idiom).
|
||||
6. `tests/test_api.py` — `/api/config.llm.radiance` absent-by-default pin (additive key, `enabled: false`); the 404 gate with only `RADIANCE_BASE_URL` set (no 404) and none set (404 + the three-variable detail sentence); the history defaults listing a configured radiance.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the registry pins above; the 404 gate in all three single-engine configurations; the config block's additivity (a diff-pin: the gufo/vllm keys' bytes are unchanged when radiance is added).
|
||||
- Coverage: **>90%** on `config.py` and the touched `api.py`/`collectors.py` paths; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `RADIANCE_BASE_URL=… uv run python -c "from dashboards.config import load; e=[x for x in load().llm_engines() if x[0]=='radiance']; print(e)"` prints the one configured entry; unset, the entry is present with an empty url.
|
||||
- [ ] `grep -n "gufo_url\|vllm_url" src/dashboards/collectors.py src/dashboards/api.py` shows no engine enumeration outside the registry (the settings fields themselves remain — they are the env surface).
|
||||
- [ ] The 404 gate and the history defaults answer from the registry in all configurations (spy-pinned: an unconfigured radiance is never dialled — no client, no fetch).
|
||||
- [ ] `/api/config` serves `llm.radiance` beside byte-identical `llm.gufo`/`llm.vllm`.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's).
|
||||
- [ ] README.md updated for anything user-visible this task changes (the env-var table row); AGENTS.md touched only if a statement in it became stale (the phase's final pass re-checks both)
|
||||
- [ ] no behavior change in completed work (a gufo-only or vLLM-only deployment's `/api/config`, 404, history and summary bodies are byte-identical to before, modulo the new additive `radiance` key in the config block)
|
||||
@@ -0,0 +1,43 @@
|
||||
# Task 03 — The radiance field map: rule-2 precedence, `/stats` as the session source, no invented number
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`src/dashboards/sources/llm.py` parses a radiance poll the way it parses gufo and vLLM — a field map **built from the task-01 fixture** (rule 7), the native windowed throughput gauges winning over a measured Δ (rule 2's precedence, the vLLM-native-gauge case), `n/a` where the engine is silent (never `0`), the KV cell rated through the **existing** `llm.kv_cache_usage_pct` severity row, and `/stats` as the session source beside gufo's `/slots` and vLLM's counts-only arm.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/llm.py` — the map, **names taken from `tests/fixtures/metrics_radiance.txt`, not from memory** (the module docstring's capture ritual already says how; the vLLM-compat subset is part of the same file, so one `Prom` parse serves both families):
|
||||
- `RADIANCE_SERIES` (UI key → (names, agg)), the radiance gauges plus the vLLM-shaped series this build carries: `decode_tokens_per_second_native` (`radiance:decode_tokens_per_second`, avg — the scheduler's recent-window aggregate, the thing that wins), `prompt_tokens_per_second_native` (`radiance:prefill_tokens_per_second`, avg), `num_requests_running` / `num_requests_waiting` (`vllm:…`, sum), `gpu_cache_usage` (`vllm:kv_cache_usage_perc`, max — the same 0-1 ratio the vLLM map feeds `SEVERITY_CELLS`, so `attach_cell_severity` rates it with **no new row**), `kv_cache_max_tokens` / `kv_cache_max_bytes` (`radiance:…`, max), `queue_depth` (`radiance:http_queue_depth`), `admission_rejected_total` (`radiance:http_admission_rejected_total`), `num_preemptions_total` (`vllm:num_preemptions_total`), `prompt_tokens_total` / `generation_tokens_total` (`vllm:…`, sum — the measured-Δ source), `gpu_prefix_cache_hit_rate` (`vllm:gpu_prefix_cache_hit_rate`, avg — a 0-1 gauge; **this build publishes no prefix counters**, so a hits/queries ratio would be an invented number: the gauge is the answer, source named, not derived — the documented asymmetry, the gufo `*_native` note's sibling), `spec_accept_rate` (`radiance:draft_acceptance_ratio`, avg — a 0-1 gauge, **reusing gufo's UI key** so the history name, the fleet block and the UI row all work without a new vocabulary), `decode_step_seconds_recent` / `decode_step_seconds_best` (`radiance:…`, max).
|
||||
- `RADIANCE_HISTOGRAMS`: `time_to_first_token_seconds` (`vllm:time_to_first_token_seconds`), `time_per_output_token_seconds` (`vllm:time_per_output_token_seconds`, `vllm:inter_token_latency_seconds`), `e2e_request_latency_seconds` (`vllm:e2e_request_latency_seconds`) — the vLLM-shaped histograms the build carries. This build has **no** `request_queue_time_seconds`, so `queue_seconds` is `n/a` for radiance by absence, never back-filled.
|
||||
- `RADIANCE_OPTIONAL` (task 01's set of map names absent from the fixture — e.g. the `vllm:num_requests_swapped`-class names) so the hermetic test distinguishes "not on this build" from "not anywhere".
|
||||
2. `parse_radiance_metrics(text, rates, now, stats=None)` — the gufo/vLLM parse functions' idiom, with the **vLLM-side** throughput precedence (native gauge is the aggregate, so it wins):
|
||||
- `decode_tokens_per_second` / `prompt_tokens_per_second`: the native gauge if present (source = the gauge's name, `derived=False`); else the measured Δ of the token counter **per label set** via the existing `_series_delta` (keyed by resolved name + labels — phase 13's carried-counter rule; a refused window is still an answer: `n/a` beside its honest `Δ<window>`); else `field(None, None)`. **No third estimate** (no `running × 1/tpot`): a gauge + a counter + a guess stacked on one cell is exactly the aggregation rule 2 exists to catch, and the engine's own page shows only the first two.
|
||||
- `tokens_total` from the two `vllm:` token counters (prompt + generated, as the vLLM path does); `prefix_cache_hit_ratio` from the gauge (0-1, source named); `spec_positions` from the labelled series `radiance:draft_position_acceptance_ratio` via `parsed.by_label(…, "position")` — an **ordered** list of 7 `{position, ratio}` entries (0-1, in the order 0..6 the engine publishes; absent positions stay absent, not zeroed).
|
||||
- `attach_cell_severity(out)` at the end, unchanged (the `gpu_cache_usage` key carries the verdict; **no** new severity row for spec acceptance — an efficiency number, decision 4).
|
||||
3. `_sessions()` — the third arm, keyed on the `stats` dict task 04's fetch supplies:
|
||||
- `stats` present: `running` / `waiting` / `queue_cap` / `requests` come from it (the engine's own summary — the numbers its own page prints). `requests` empty → `detail` = counts + cap, one honest sentence (the vLLM arm's idiom: "radiance reports counts, not sessions: N running of cap C. Per-request detail appears here while requests are in flight."). Non-empty → `list` = the rows verbatim (the fixture's shape is the contract; a row's fields render as-is, task 05), `source` = "radiance /stats".
|
||||
- `stats` missing (the fetch failed — invariant 4: the poll itself is healthy) → fall back to the `/metrics` gauges `num_requests_running`/`num_requests_waiting` for the counts (they are real numbers, different provenance — the detail says `/stats` did not answer this poll), `queue_cap` `null` (never `0` — rule 2).
|
||||
- `active` stays `metrics["num_requests_running"]["value"]` in every arm (one definition, the `/metrics` gauge).
|
||||
4. `engine_result(engine, base_url, text, rates, *, slots=None, stats=None, now=None)` — the parser dispatch becomes the three-way map (gufo / vllm / radiance); a radiance poll attaches `payload["stats"]` = the small additive subset `{model, version, uptime_s, startup_s, total_requests, cards[]}` (each card `+ vram_used_pct` computed `used/total×100`, `derived: true` — the only derived number in the block; the raw bytes travel as the engine published them). The subset is **whitelisted from the fixture's shape**, so a radiance upgrade that renames a `/stats` key degrades to absent, not a crash (invariant 4's poll-body rule).
|
||||
5. `HISTORY_METRICS` — **additive candidates only** (the existing keys and their candidate order are untouched): `queue_depth` → `("queue_depth",)`, `queue_cap` → `("queue_cap",)` (the `queue_cap` metrics key is the `/stats` passthrough), `decode_step_seconds_recent` → `("decode_step_seconds_recent",)`, `vram_used_pct` → `("vram_used_pct",)` (card 0, the engine's own view of the same card `/system` reads through the agent — different source, different host key, both honest). The existing keys already cover radiance: `decode/prompt_tokens_per_second` (the UI keys the precedence logic fills), `kv_cache_usage` (the `gpu_cache_usage` candidate), `requests_running`/`requests_waiting` (the `num_requests_*` candidates), `tokens_total`, `spec_accept_rate` (the gauge reuses gufo's key). The seven per-position ratios are **snapshot-only** (decision 2: a bounded store, rule 9).
|
||||
6. `series_names(engine)` — the three-way branch; `fleet_totals(engines)` — the code needs **no change** (the aliases `num_requests_running`/`num_requests_waiting`, the `decode/prompt_tokens_per_second` keys, `tokens_total` and the `gpu_cache_usage` max all already cover a radiance payload); **only** the `seams` sentence is rewritten to state, briefly and honestly, the known asymmetries: gufo's prompt counter excludes cache-hit tokens; the summed throughput is per-engine provenance-labelled (native scheduler window vs measured Δ). It does not claim what radiance's `vllm:prompt_tokens_total` includes or excludes about cached tokens — the engine does not say, and neither does the dashboard (rule 2).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration in `tests/test_llm_source.py` (fixture-driven, the conftest skip pattern from task 01):
|
||||
- the fixture parse: every mapped key resolves (shape + source strings), the name set equals the fixture's (task 01's pin), `spec_positions` is the ordered 7, `stats`-arm sessions as specced (empty / non-empty / missing, three fakes), the `payload["stats"]` whitelist (a fixture with a renamed `version` key degrades to absent, the poll still returns), the radiance history-sample names (`llm.queue_depth`, `llm.queue_cap`, `llm.decode_step_seconds_recent`, `llm.vram_used_pct`, …) appear and the gufo/vLLM sample names are untouched.
|
||||
- rule 2, three arms: gauge present → source is the gauge name, `derived` false; gauge line removed from the fixture copy → measured Δ, `derived` true, the window (or the refused-window `n/a` + `Δ` label, a fresh-estimator fake); both absent → `n/a`, no `0` anywhere; `queue_cap` with `stats` missing → `null`, not `0`.
|
||||
- the severity pass-through: the `gpu_cache_usage` cell carries a `severity` verdict from the existing table at a fixture value above the warn bound; `spec_accept_rate` carries **no** severity key (decision 4 pinned both ways — an absence-only assertion, so it is asserted on the payload).
|
||||
- `fleet_totals` with a gufo + radiance payload: the running/waiting sums, the true max KV, `engines_live` 2, the seams copy names the provenance labelling.
|
||||
- the gufo and vLLM parse functions are byte-behaviour-unchanged (their existing pins untouched).
|
||||
- Coverage: **>90%** on `sources/llm.py`; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `parse_radiance_metrics` against the fixture returns every mapped key with its pinned source string; a series removed from a fixture copy answers `{"value": null}` — never `0`.
|
||||
- [ ] the three throughput arms are proven in both directions (gauge wins / measured fills / `n/a` with the honest `Δ` label on a refused window).
|
||||
- [ ] `spec_positions` is the ordered 7 from the fixture; `queue_cap` is `null` (not `0`) when `/stats` is absent; the KV cell carries the severity verdict and the spec cell carries none.
|
||||
- [ ] `engine_result("radiance", …, stats=…)` carries the whitelisted `stats` block with the one computed `vram_used_pct`; a renamed stats key degrades to absent without raising.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's).
|
||||
- [ ] README.md updated for anything user-visible this task changes (none expected — the payload surface is task 05's); AGENTS.md touched only if a statement in it became stale (the phase's final pass re-checks both)
|
||||
- [ ] no behavior change in completed work (gufo and vLLM payloads, samples, alerts and pages behave exactly as before — the suite's existing pins untouched)
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 04 — The `llm:radiance` job: a `/stats` sidecar that can fail without failing the poll
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`collectors.fetch_engine` fetches radiance's `/stats` **best-effort** — a dead or partial `/stats` degrades the session arm (task 03's missing-stats fallback) and never takes the `/metrics` poll down (invariant 4: a failed sidecar keeps the last good payload and marks nothing stale, while a dead `/metrics` is the real `UpstreamError`) — and `llm_jobs` (in the task-02 registry shape) builds the `llm:radiance` job with its own `Job` key, display name and the shared LLM cadence/estimator.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/collectors.py` — `fetch_engine(…, with_slots=False, with_stats=False)`:
|
||||
- The `with_stats` path, beside the existing `/slots` loop: one `GET {base_url}/stats` (no alias — radiance's own page uses exactly that path); on any `httpx.HTTPError` **or** a status ≥ 400 **or** a JSON that is not a dict, `stats` stays `None` and the function returns the metrics result with the missing-stats session arm. No `UpstreamError` is raised from this sidecar, no retry, no second dial — one best-effort read, the `/slots` idiom.
|
||||
- The `UpstreamError` messages for the **metrics** dial keep the existing `strip_userinfo` redaction (invariant 6) — a `RADIANCE_API_KEY`-free URL never leaks, a tokenised one never leaks its token.
|
||||
- `llm_jobs` (task 02's registry shape) passes `with_stats=True` for the radiance entry. No new `RateEstimator`: radiance rides the LLM cadence's existing estimator (task: `app.build_hub` is untouched — one estimator per measuring cadence stays the rule).
|
||||
2. `src/dashboards/app.py` — nothing to change is the check: the `llm` jobs already flow from `llm_jobs` into the hub; confirm `llm:radiance` gets the LLM interval and the LLM-span estimator by the existing wiring (assert in a test, not by comment).
|
||||
3. `tests/test_collectors.py`:
|
||||
- a configured `RADIANCE_BASE_URL` (and no gufo/vllm) builds exactly one `llm:radiance` `Job` with `dashboard == "llm"`, display `radiance`, the LLM interval (the registry's shape from task 02, extended);
|
||||
- a fake server with `/metrics` 200 and `/stats` 500 → the result is a healthy `Result` (payload present, `stats` absent, the session detail names the missing sidecar); `/stats` 200 with a non-JSON body → the same healthy degradation; `/stats` 200 with a valid dict → `stats` travels into the result;
|
||||
- `/metrics` 500 with `/stats` 200 → the `UpstreamError` is the metrics one (the sidecar never masks the real failure);
|
||||
- a tokenised `RADIANCE_BASE_URL` (`http://secret@host:8000`) whose metrics dial fails → the `UpstreamError` message contains no `secret` (the redaction pin, the existing engine-fetch precedent extended);
|
||||
- the job list never dials an unconfigured engine (the existing spy pattern).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the five pins above (healthy / degraded / masked / redacted / undialled).
|
||||
- Coverage: **>90%** on the touched `collectors.py` paths; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `set -a; . ./.env.dev; set +a` + `./scripts/dev-server.sh restart` → `./scripts/dev-server.sh status` shows `llm:radiance` ok (or its real failure) beside the other sources, and `/api/llm/snapshot` (authenticated) carries the radiance payload with the `stats` block.
|
||||
- [ ] a faked dead `/stats` leaves the poll healthy (the test above, and the page shows the metrics panel with the counts-otherwise session arm).
|
||||
- [ ] `grep -n "with_stats" src/dashboards/collectors.py` shows the flag feeding `fetch_engine` from `llm_jobs` only.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's).
|
||||
- [ ] README.md updated for anything user-visible this task changes (none expected — no new env surface beyond task 02's); AGENTS.md touched only if a statement in it became stale (the phase's final pass re-checks both)
|
||||
- [ ] no behavior change in completed work (the gufo `/slots` pass and the vLLM job behave exactly as before — the suite's existing pins untouched)
|
||||
@@ -0,0 +1,39 @@
|
||||
# Task 05 — `/llm` learns radiance: the strip, the 7-bar curve, and the honest empty arms
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The radiance card on `/llm` shows the cells its payload carries — the native/measured throughputs with their provenance tags, the KV bar with its severity colour, the radiance strip (queue cap, build, uptime, per-card VRAM, the 7-bar spec-acceptance curve), and the `requests[]` rows when a request is live, the counts-otherwise sentence at rest — with every chart named for assistive tech and **no verdict colour** on any radiance cell except the KV bar (decision 4).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-llm.js`
|
||||
- The `only:` scopes extend, not new: `ttft`, `tpot`, `queue_mean`, `running` gain `'radiance'` (the build carries the vLLM-shaped histograms and the `num_requests_*` gauges — task 03's map fills the same cell keys); `spec_accept` becomes `['gufo', 'radiance']` (the gauge reuses gufo's key and is the same 0-1 scale, so the existing `%` row renders unchanged).
|
||||
- New **radiance-only** strip cells (the `STRIP_CELLS` idiom, `only: ['radiance']`, each with a `tip` that tells the operator what the number is for):
|
||||
- `cap` — "queue cap", the engine's `queue_cap` (the admission ceiling; `null` when `/stats` did not answer → the cell prints `n/a`, never `0`);
|
||||
- `build` — `stats.version` (e.g. `1.15.1`), tip "The engine's own version string — a radiance upgrade changes it, the panel does not";
|
||||
- `uptime` — `stats.uptime_s` through the existing `app.js` duration formatter;
|
||||
- `vram` — card 0's `vram_used`/`vram_total` as `32.8 / 34.2 GiB` (GiB = 2³⁰, the only conversion on the page, labelled as such in the tip: "the engine's own view of the card; `/system`'s GPU row is the agent's view"), `n/a` when no card answered.
|
||||
- The **7-bar spec curve** (decision 2): a small hand-rolled SVG on the radiance strip (the `charts.js` idiom, no library, no CDN — rule 5), one bar per position in `spec_positions` order (0..6), height ∝ the 0-1 ratio, drawn in a **neutral** token (the page's existing track/cool colours — never a severity colour, the value is an efficiency number, decision 4), with `role="img"` and an `aria-label` that names the chart and its range (e.g. `Spec-decode draft acceptance by position: 82% at position 0 falling to 65% at position 6`) — the label is **computed from the payload** (the "every chart names itself" rule, pinned below).
|
||||
- The `requests[]` arm: non-empty `sessions.list` (source "radiance /stats") renders the rows as a compact table beside the counts (the gufo slot-grid idiom, shrunk — radiance rows are the engine's own fields, rendered **as-is**: the fixture's shape is the contract; a field that is a number prints `n/a`-free only when it is a number, anything else prints its string verbatim, no conversion, no invented key). Empty list → the `sessions.detail` sentence (task 03's counts + cap) in the existing detail slot.
|
||||
- `SLOT_ENGINES` stays `['gufo']` (the slot grid is gufo's); the radiance rows live in the strip, not the slot grid.
|
||||
2. `src/dashboards/web/pages/llm.html`
|
||||
- The empty-state paragraph names all three variables: "Set `GUFO_BASE_URL` (llama.cpp / gufo), `VLLM_BASE_URL` and/or `RADIANCE_BASE_URL`."
|
||||
- The fleet block's `fleet-note`/seams copy is payload-driven (`fleet.seams` from `fleet_totals`, task 03's rewritten sentence) — no new hardcoded copy.
|
||||
3. `src/dashboards/web/static/dash.css` — only the tokens the 7-bar curve and the `requests` table need (reuse the existing neutral/cool/edge tokens; **no** new colour vocabulary — colour comes only from the payload's severity verdict, which for radiance is the KV bar alone).
|
||||
4. `tests/test_api.py` — the served-byte pins extend: the radiance strip markup (the four new cells' `data-key`s), the curve's `role="img"` + `aria-label` joined to the existing chart-aria-label pin list (an absent label fails the test), and the requests-table container present-when-the-payload-carries-rows (a fixture-driven payload), absent-at-rest with the detail sentence.
|
||||
5. Live verification (AGENTS.md rule 10): `./scripts/dev-server.sh restart` with `.env.dev` (gufo + radiance now both configured — the fleet block finally renders: two answering engines), sign in, `/llm`, screenshot the radiance card **at rest** (counts arm, curve, strip) into `data/logs/34-llm-radiance-rest.png`; if a request is in flight during the pass, a second screenshot with the rows visible.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the `test_api.py` pins above (markup, aria-label, both session arms) against fixture-driven payloads.
|
||||
- Coverage: the UI is proven by the payload-contract pins + the screenshots (the JS-tests-none decision, PLAN.md); Python-side **>90%** on anything touched (none expected); the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The radiance card renders all task-03 cells; a payload with `stats` missing shows `n/a` on `cap`/`build`/`uptime`/`vram` (never `0`) and the counts-otherwise session sentence.
|
||||
- [ ] The 7-bar curve is present with its computed `aria-label` (the pin fails on an absent label); no severity-colour token in its markup.
|
||||
- [ ] The `only:` scopes: the vLLM-shaped rows (TTFT/TPOT/queue-time/running) and `spec accept` render on the radiance card from the fixture payload; no radiance row on the gufo card and no gufo row on the radiance card.
|
||||
- [ ] `data/logs/34-llm-radiance-rest.png` exists, showing the card against live data with the fleet block rendered (two engines).
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's).
|
||||
- [ ] README.md updated for anything user-visible this task changes (the `/llm` page section, if it names the engines it draws); AGENTS.md touched only if a statement in it became stale (the phase's final pass re-checks both)
|
||||
- [ ] no behavior change in completed work (the gufo and vLLM cards are byte-identical where this task does not touch them — the served-byte pins)
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 06 — Final pass: live read-only proof, fixture re-capture, README/AGENTS.md, the definition of done
|
||||
|
||||
**Phase:** `34_radiance_llm_engine` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The phase's verification pass: a read-only `live` test that drives the production radiance path against the running engine, the fixture re-capture with a diff read, the documentation pass (README in full; AGENTS.md stale-statements-only), and the definition-of-done checklist run against the dev server.
|
||||
|
||||
## Work
|
||||
|
||||
1. `tests/test_live_fleet.py` — the radiance test (the file's idiom: skipped unless `--live`, each test skips when its source is unset, **live tests must not write** — radiance exposes no write endpoint, so "read-only" is a list of GETs):
|
||||
- skip when `RADIANCE_BASE_URL` is unset (the source gate, rule 3's live twin);
|
||||
- dial the engine through the **production** path — `collectors.fetch_engine` with the LLM estimator (never a hand-rolled `client.get`), one poll;
|
||||
- assert shape, not numbers: the payload's `engine == "radiance"`, the `stats` block's keys (task 03's whitelist), `spec_positions` length 7 when the fixture/running build publishes it, the session arm is one of the two specced arms, every cell that is present is either a number or `None` (never a fake `0` for an absent gauge — the live twin of rule 2), and `unparsed` is empty (a parse error on the live wire is drift, not a tolerance).
|
||||
2. `./scripts/capture-fixtures.sh --with-llm` (`.env.dev` sourced) — re-record the radiance sidecars (and everything else), **read the diff**: a volatile value moving is expected (the headers say so); a renamed or new `radiance:*` series is a contract change — extend task 01's pin (and `RADIANCE_OPTIONAL` where a name is build-optional), never a silent ignore. If the earlier capture's `requests[]` was empty and a request is in flight now, re-capture so the fixture carries the real row shape, and re-run the session-arm tests against it.
|
||||
3. `README.md` — the full final-pass review (the phase's documentation mandate): the env-var table (`RADIANCE_BASE_URL` / `RADIANCE_API_KEY` beside the engine rows, the "unset means *not deployed here*" clause), the LLM-engines section (a radiance paragraph in the gufo/vLLM idiom: what it is, the endpoints it serves, the native-gauge/measured-Δ provenance, the `/stats` sidecar), the capture-script section (three radiance endpoints under `--with-llm`), and every other place the engine count or the driveripper fact appears — fixing **every** inconsistency found, including pre-existing ones.
|
||||
4. `AGENTS.md` — **stale statements only, no expansion** (its own update policy wins): the fleet table's driveripper line `vLLM :8000` → `radiance :8000` (the operator's decision of 2026-10-24); the `/llm` route row's source column (`gufo /metrics + /slots, vLLM /metrics`) gains `radiance /metrics + /stats`; `config.py`'s paragraph names the radiance settings beside the others; `sources/llm.py`'s bullet says the three field maps (gufo, vLLM, radiance) with the rule-2 precedence note already there. Nothing else — no phase narrative, no new rules.
|
||||
5. The definition-of-done run, in order (AGENTS.md):
|
||||
- `uv run pytest` green; `uv run pytest --cov` — coverage floor 94 % held, new/modified Python > 90 %; `uv run ruff check src tests agent && uv run pyright` clean; `uv pip list --outdated` empty (no dependency moved — confirm, do not assume).
|
||||
- `set -a; . ./.env.dev; set +a && ./scripts/dev-server.sh restart` — `/api/health` sane (per-source ok/DOWN; `llm:radiance` answering), `./scripts/dev-server.sh status` read.
|
||||
- `uv run pytest -m live --live` — the new radiance test passes read-only; the existing live tests (gufo, vLLM-where-configured, upsd, agents) still pass.
|
||||
- Screenshots in `data/logs/`: the task-05 rest shot (and the in-flight shot if one was possible), and the overview page (`/`) showing the radiance engine's alert row when its KV bar is rated — the cell-driven `_engine_alerts` needs no code, only proof.
|
||||
- `bash .agents/validate.sh` green.
|
||||
|
||||
## Testing & Quality
|
||||
- The live test is the integration proof; the re-captured fixtures re-pin the hermetic suite; the documentation pass is checked by reading the two files against the project as it stands.
|
||||
- Coverage: no new Python beyond the live test; the whole suite 100 %; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest -m live --live` includes the passing read-only radiance test (skipped, correctly, when unset — proven by one run without the env var).
|
||||
- [ ] the fixture re-capture diff is read and either clean or folded into the pins (`RADIANCE_SERIES`/`RADIANCE_OPTIONAL`); the suite re-run is green.
|
||||
- [ ] README.md: the engine section, env-var table, and capture-script section all name radiance; a full read found no remaining engine-count or driveripper stale statement.
|
||||
- [ ] AGENTS.md: the three stale statements fixed (fleet line, route row, the two module bullets), nothing added or expanded beyond them.
|
||||
- [ ] `./scripts/dev-server.sh status` shows `llm:radiance` answering; the `data/logs/` screenshots exist; `bash .agents/validate.sh` is green; the coverage floor held.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this task's) — the phase's "Complete" gate.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Phase 35 — `/llm` engine windows equalized: the strip and panel rows stretch to a common row height
|
||||
|
||||
**Source:** operator request, 2026-10-25 — "The LLM engines page looks kinda awkward, there's a big gap between gufo's two windows and radiance + the throughput charts. I want the windows to be equalized in height." (With the `data/logs/34-llm-radiance-rest.png` state: the gufo strip card ~320 px shorter than the radiance strip card, the radiance panel shorter than the gufo panel — dead space below the shorter card in each row, before the next row.)
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/web/pages/llm.html` (`#strip`, the `#engines` grid at `grid cols-2 items-start`), `src/dashboards/web/static/dash.css` (`.strip { … align-items: start; }`, the `.grid.items-start` rule with its "cards sit at their own height" comment, `.stack { align-content: start }` which keeps stretched-card content top-aligned), `tests/test_api.py` (`test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling` — the served-byte pin of the **old** decision; the `/system` hosts-grid pin at the `.grid auto items-start` assertion is a *different* context and stays), `README.md` (the one `items-start` mention, L1104–1105, is the system disclosure — stays).
|
||||
|
||||
## Objective
|
||||
|
||||
The two "window" rows on `/llm` — the stats strip row and the engine panel row — render each pair of engine cards at the row's common height instead of their natural heights, so the dead gap below the shorter card (gufo strip under the taller radiance strip; radiance panel under the taller gufo panel, before the four throughput/trend charts) is gone and every row ends on a flat edge.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Equalize = stretch within the row.** The operator wants the window *heights* equalized: each row's cards share the tallest card's height (CSS grid default `stretch`). The extra space on the shorter card is bottom card padding — content stays top-aligned via `.stack { align-content: start }` — never a `0` or an invented number (rule 2 untouched: missing cells still print `n/a`). | `align-items: start` leaves the `.strip` rule; `items-start` leaves the `#engines` div. At ≤ 640 px the `auto-fit` grids are single-column, where stretch is a visual no-op — mobile is unaffected. |
|
||||
| 2 | **This reverses the pinned anti-stretch decision.** The pin `test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling` recorded the `llm_monitor`-era finding (a 172 px blank bottom read as "no data"). The operator now judges the inter-row gap worse than an equalized card and asks for the heights equalized. The pin is **rewritten in place** to pin the new decision both directions (markup carries no `items-start` on `#engines`; the `.strip` block sets no `align-items` and keeps `display: grid` + the `auto-fit` single-engine columns), and the `.grid.items-start` comment in `dash.css` is corrected to the new fact. | `tests/test_api.py` is the only test file touched. The `/system` hosts grid (and the `certs`/`settings`/`gateway`/`disks` `items-start` grids) keep `items-start` — the system case (an expanded disclosure must not stretch collapsed siblings, `README.md` L1104–1105) is a different context and is pinned separately. |
|
||||
| 3 | **No API, no data, no new technology.** | No `/api` route, metric, payload field, dependency, or JS change. The change is two layout declarations and their pin. No `charts.js`/`page-llm.js` edit is needed: `.stack` already top-aligns the content of a stretched card. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `34_radiance_llm_engine` (complete) — the current two-engine strip/panel shape whose height mismatch (the radiance strip's extra rows, spec curve, and `reqs` block vs the gufo strip; the gufo panel's slot grid vs the radiance panel's latency rows) this phase equalizes.
|
||||
- `07_llm_monitor_style_pages` (complete) — the strip/panel split and the served-byte pin this phase rewrites.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_stretch_the_llm_rows.md` — the two layout declarations and the pin that records the operator's decision.
|
||||
2. `02_live_verify_and_docs.md` — the dev-server live browser verification with screenshot, the `/system` no-regress check, the README/AGENTS.md pass.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the rewritten served-byte pin in `tests/test_api.py` — the served `/llm` markup's `#engines` div carries no `items-start`, the served `dash.css` `.strip` block sets no `align-items` while keeping `display: grid` and the `auto-fit` columns, and the served `/system` markup still carries `grid auto items-start` (the unchanged context, re-pinned alongside). The pre-existing `tests/test_api.py` pins for the system hosts grid (`grid auto items-start`) must pass untouched.
|
||||
- Coverage: **>90%** on new/modified code (this phase's code surface is markup + one rewritten pin; the suite's overall floor — 94% via `validate.sh` — must not drop).
|
||||
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's — pre-existing failures included) and achieves **>90%** code coverage on new/modified code.
|
||||
- Every UI change is verified in a real browser against live data, screenshot in `data/logs/` (AGENTS.md rule 10) — with gufo and radiance both answering, the strip row and the panel row each end on a flat edge with no dead gap before the trend charts.
|
||||
|
||||
## Documentation & Consistency
|
||||
- The final pass reviews `README.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For `AGENTS.md` it fixes only *stale statements* (a dead command, changed rule, or outdated fact) and **never adds or expands its content** (a file that states its own update policy, e.g. "updated rarely, if ever", wins).
|
||||
- Tasks update `README.md` immediately when their own changes are user-visible (setup, commands, configuration, features, conventions) — expected here: none beyond confirming the L1104–1105 `items-start` sentence still describes the system disclosure accurately — and `AGENTS.md` only when a statement in it became stale (expected: none — a layout preference is not a rule, command, or fleet fact).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] served `GET /llm` markup: the `#engines` div's class list is `grid cols-2` (no `items-start`); served `dash.css`: the `.strip` block has no `align-items` declaration and still carries `display: grid`, its `gap`, and the `auto-fit` `minmax(min(340px, 100%), 1fr)` columns; the `.grid.items-start` rule still exists (used by `/system` and the other pages).
|
||||
- [ ] live browser, both engines answering: the strip row's two cards and the panel row's two cards each share a common height (flat row bottoms); the gap between a row and the next is the uniform `.85rem` grid gap only — screenshot in `data/logs/`; at a 360 px viewport the page is single-column and unbroken.
|
||||
- [ ] served `GET /system` markup still carries `grid auto items-start` on `#hosts` and its pin passes — the disclosure behaviour is byte-for-byte unchanged.
|
||||
- [ ] full test suite **100% passing** (every test in the project, not just this phase's), coverage not down (94% floor via `bash .agents/validate.sh`); `ruff` + `pyright` clean.
|
||||
- [ ] README.md is accurate and consistent with the whole project; AGENTS.md has no stale statements (verified in the final pass).
|
||||
- [ ] no behaviour change in completed work: payloads, alerts, served bytes of every page this phase does not touch (the suite's existing pins), and the `n/a`/`never 0` rendering of every `/llm` cell are untouched.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 01 — Stretch the LLM strip and panel rows; pin the operator's decision
|
||||
|
||||
**Phase:** `35_llm_equal_window_heights` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The two engine-window rows on `/llm` (the stats strip row and the panel row) share a common row height — CSS grid default `stretch` — so the dead gap below the shorter card is gone, and the served-byte pin records that decision in both directions instead of forbidding it.
|
||||
|
||||
## Work
|
||||
1. `src/dashboards/web/pages/llm.html` — the `#engines` div: `class="grid cols-2 items-start"` → `class="grid cols-2"` (drop `items-start`; the grid's default `align-items: stretch` takes over). The section comment above it stays — it does not name the alignment.
|
||||
2. `src/dashboards/web/static/dash.css` — the `.strip` rule: remove `align-items: start;` from the declaration list (keep `display: grid; gap: .85rem;` and the `auto-fit` `minmax(min(340px, 100%), 1fr)` columns — the single-engine rule is untouched). Also correct the comment block above `.grid.items-start` (the "cards sit at their own height instead of being stretched" prose, which cites the `/llm` 172 px finding): the rule still exists and still serves `/system` (an expanded disclosure must not stretch collapsed siblings — `README.md` L1104–1105) and the `certs`/`settings`/`disks`/`gateway` grids; the `/llm` strip and panel rows now use the default stretch, with the shorter card's extra space reading as bottom card padding (content stays top-aligned through `.stack { align-content: start }`). The comment must name only classes the served CSS still defines.
|
||||
3. `tests/test_api.py` — replace `test_llm_engine_panels_are_not_stretched_to_their_tallest_sibling` with the pin of the **new** decision, e.g. `test_llm_engine_windows_stretch_to_a_common_row_height`:
|
||||
- the served `/llm` markup (comments stripped, as the old pin did via `_code()`): the `#engines` div's class list contains no `items-start`;
|
||||
- the served `dash.css`: the `.strip {` block contains no `align-items` declaration and still contains `display: grid`, its `gap`, and the `auto-fit` `minmax(min(340px, 100%), 1fr)` columns (the rule stays alive, not just de-aligned);
|
||||
- the served `/system` markup still carries `grid auto items-start` on `#hosts` (the unchanged context re-pinned in the same test, so the two contexts cannot silently converge);
|
||||
- the docstring records the operator's 2026-10-25 decision — the inter-row gap read as worse than an equalized card, the heights are equalized by stretch, and the extra space is bottom card padding, never a number (rule 2: missing cells still print `n/a`, never `0`).
|
||||
4. Nothing else: `page-llm.js`, `charts.js`, every other page, and the `README.md` L1104–1105 sentence (it describes the system disclosure and stays accurate) are untouched by this task.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: the rewritten pin above runs in the hermetic suite (`create_app()`, served bytes); the pre-existing `tests/test_api.py` pin for the system hosts grid (`grid auto items-start` at the L2138/L2140 assertions) passes untouched.
|
||||
- Coverage: **>90%** on this task's new/modified code (markup + one rewritten pin; the 94% overall floor via `validate.sh` must not drop).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green, including the new pin and the untouched system-grid pin.
|
||||
- [ ] served `GET /llm` (any authenticated or fixture config): the `#engines` div is `class="grid cols-2"`; the served `dash.css` `.strip` block has no `align-items` line.
|
||||
- [ ] `uv run ruff check src tests agent && uv run pyright` clean; `uv run pytest --cov` floor not down.
|
||||
- [ ] no change to any served page, payload, or asset this task does not name.
|
||||
- [ ] commit with `--no-gpg-sign` and the `LLM-Generated: true` trailer (per task, per PLAN.md's validation workflow).
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 02 — Live verification, the no-regress check, and the documentation pass
|
||||
|
||||
**Phase:** `35_llm_equal_window_heights` · **Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the equalized rows in a real browser against the live fleet (AGENTS.md rule 10), confirm the `/system` disclosure behaviour did not move, and close the phase with the README/AGENTS.md consistency pass.
|
||||
|
||||
## Work
|
||||
1. Dev server and ground truth: source `.env.dev` (`set -a; . ./.env.dev; set +a`), `./scripts/dev-server.sh restart`, and confirm `./scripts/dev-server.sh status` shows the `llm` source ok (gufo and radiance both answering — the strip row only renders two cards when two engines are configured, which is the state the fix is about).
|
||||
2. Live browser verification of `http://127.0.0.1:8090/llm` (sign in with `testpass`), with both engines answering:
|
||||
- the strip row: the gufo and radiance strip cards share one height — a flat bottom edge; the dead space below the shorter (gufo) card is card padding inside the border, not a gap between the row and the panel row;
|
||||
- the panel row: the gufo and radiance panels share one height — the shorter (radiance) panel's extra space is bottom card padding, content top-aligned;
|
||||
- the only vertical space between the strip row, the panel row, and the four trend-chart cards is the uniform `.85rem` grid gap;
|
||||
- every missing value still prints `n/a` (never `0`), and the severity colours are unchanged — the layout change must not have touched a single number;
|
||||
- at a 360 px viewport the page is single-column (the `auto-fit` grids collapse), rows stack, nothing is clipped.
|
||||
Save the screenshot to `data/logs/35-llm-equal-windows.png` (a mobile-360 shot beside it if taken).
|
||||
3. No-regress check: `GET /system` (browser) — expanding one host's disclosure still does not stretch its collapsed siblings (the `items-start` context from `README.md` L1104–1105, byte-for-byte the served markup `grid auto items-start`).
|
||||
4. Documentation pass (the phase's final pass):
|
||||
- `README.md` reviewed in full against the project as it stands after this phase — fix every inconsistency found, including pre-existing ones. Expected outcome: the L1104–1105 `items-start` sentence still describes the system disclosure and needs no change; confirm nothing in the README describes the `/llm` rows as own-height.
|
||||
- `AGENTS.md` — fix only *stale statements* (a dead command, changed rule, or outdated fact); never add or expand. Expected outcome: no change — a layout preference is not a rule, command, or fleet fact, and no statement in the file became stale. Record the check either way.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: `uv run pytest` (full suite, 100% green — pre-existing failures included), `uv run pytest --cov` (floor not down; 94% via `bash .agents/validate.sh`), `uv run ruff check src tests agent && uv run pyright` clean, `bash .agents/validate.sh` green.
|
||||
- Coverage: **>90%** on this task's new/modified code (none expected — verification and documentation only).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `data/logs/35-llm-equal-windows.png` exists and shows both `/llm` rows ending on flat, equal-height card edges with only the uniform grid gap between rows, `n/a` cells still `n/a`, and no colour change.
|
||||
- [ ] the `/system` disclosure no-regress check passes (served markup still `grid auto items-start`; browser behaviour unchanged).
|
||||
- [ ] `./scripts/dev-server.sh status` sane after the restart; `/api/health` answers.
|
||||
- [ ] README.md is accurate and consistent with the whole project; AGENTS.md has no stale statements (both verified in this pass, whatever the expected outcome).
|
||||
- [ ] full test suite **100% passing**, coverage not down, lint/type checks clean, `bash .agents/validate.sh` green.
|
||||
- [ ] commit with `--no-gpg-sign` and the `LLM-Generated: true` trailer (per task, per PLAN.md's validation workflow).
|
||||
@@ -0,0 +1,11 @@
|
||||
**Phase 21 final verification pass** (all 3 task files already in `complete/`, working tree carries the implementation)
|
||||
|
||||
- **Verified** task 01: `#pause`/`Space`/`setPaused`/`_paused` gone — `grep -ri pause src/dashboards/web/` empty, 0 pause hits in all 8 served pages + 10 static assets; re-ran `data/logs/21-task01-nopause.cjs` → 30/30 PASS (hidden tab: 0 data fetches in 22 s, one poll 2 ms after return).
|
||||
- **Verified** task 02: re-ran `21-task02-gpu-values.cjs` on the live fleet → all PASS (deskwork/driveripper show Model/VRAM/Pool/Temp at 16.3 px; gateway = 5 heroes, slot `hidden`, no `n/a`; hero order unchanged; GPU history chart still builds per host).
|
||||
- **Verified** task 03: re-ran `21-task03-scroll.cjs` → 14/14 PASS (`class="data scroll"`, scrollWidth 1021 vs clientWidth 292 at 360 px, Rotate/Revoke/bundle reachable and hit-testable; unchanged at 1280 px). Screenshots refreshed in `data/logs/`.
|
||||
- **Gate**: `uv run pytest` → **1471 passed, 15 skipped**; `--cov --cov-fail-under=94` → **98.94%**; `ruff check src tests agent` clean; `pyright` **0 errors**; `bash .agents/validate.sh` → **EXIT 0** (incl. `node --check`); `uv run pytest -m live --live` → **11 passed**; `/api/health` all 9 sources `ok`.
|
||||
- **Docs fixed** (pre-existing + this phase): "all **eight** signed-in pages" was a miscount of `MARKUP_PAGES` (7 = 5 dashboards + `/settings` + `/certs`; `/login` is the 8th, unsigned) → corrected in `README.md`, `AGENTS.md` (×2) and the `tests/test_api.py` docstring/comment; `AGENTS.md` Commands hermetic tally 1470 → **1471**.
|
||||
- **Docs reviewed, consistent**: Quickstart/commands/flags (`install-agent.sh`, `capture-fixtures.sh`, `dev-server.sh`), the env table vs `config.py`, every documented route vs the served route table, masthead/`/system` GPU/`/certs` scroll descriptions vs code, and all `data/logs/*` + `tests/fixtures/*` references.
|
||||
- **Deviation**: `validate.sh`'s advisory `fastapi 0.153.0 → 0.156.4` left alone — no dependency was touched this phase, so AGENTS rule 8 ("check after touching dependencies") is not triggered; upgrading here would be an unrelated behaviour change.
|
||||
- No live-infra changes made (repo-side + browser verification only).
|
||||
- **Next pending phase**: `22_disks_hide_unconfigured`.
|
||||
@@ -0,0 +1 @@
|
||||
ok
|
||||
@@ -0,0 +1,65 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 24%]
|
||||
...............................................s........................ [ 29%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 67%]
|
||||
............sssssssssss................................................. [ 72%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 92%]
|
||||
............................................................sss......... [ 96%]
|
||||
.............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 32
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 436 0 100%
|
||||
src/dashboards/app.py 185 0 100%
|
||||
src/dashboards/auth.py 265 0 100%
|
||||
src/dashboards/certs.py 292 0 100%
|
||||
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
|
||||
src/dashboards/config.py 120 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
|
||||
src/dashboards/rates.py 73 0 100%
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 134 0 100%
|
||||
src/dashboards/sources/host.py 209 0 100%
|
||||
src/dashboards/sources/llm.py 178 1 99% 285
|
||||
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 313 2 99% 553-554
|
||||
------------------------------------------------------------------
|
||||
TOTAL 3490 37 99%
|
||||
Required test coverage of 94% reached. Total coverage: 98.94%
|
||||
1471 passed, 15 skipped, 1 warning in 156.37s (0:02:36)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
|
||||
fastapi 0.153.0 0.156.4 wheel
|
||||
@@ -0,0 +1,14 @@
|
||||
**Implemented**
|
||||
- `src/dashboards/web/pages/{system,disks,llm,ups}.html` — deleted the `#pause` button line (only served occurrences; `grep -ri pause src/dashboards/web/` now exits 1).
|
||||
- `static/app.js` — dropped `_paused`/`paused`/`setPaused` from `Dash.poll()` and the `#pause` binding + `Space` handler; `document.hidden` skip and the `visibilitychange` refire kept. The `T` theme shortcut that shared that listener survives as `Dash.bindShortcuts()` (same four pages call it; deliberately not moved into `initChrome()`, so its reach is unchanged).
|
||||
- `static/dash.css` — the three comments naming the pause button as a row-3/wrap tenant cleaned.
|
||||
- `tests/test_api.py` — new section: `PAUSE_ARTIFACTS` absent from all 8 served pages (`/login` via its own anonymous client) + 11 assets; `document.hidden`/`visibilitychange` guards still present and no `event.code`/`'Space'` in the shared script; `T` + the four `Dash.bindShortcuts()` call sites pinned.
|
||||
- Browser proof `data/logs/21-task01-nopause.cjs` (+ `.out`, `21-task01-masthead-{,system-}{1280,360}.png`, `21-task01-system-live.png`): 30/30 PASS — no control on 7 pages × 2 widths + `/login`; `Space` presses nothing and polls keep running (0→5 fetches/14 s); hidden tab: **0 data fetches in 22 s** (`document.hidden=true`); return fires one poll in 1 ms.
|
||||
|
||||
**Gate** — `bash .agents/validate.sh` → EXIT=0: `1466 passed, 15 skipped`, coverage 98.94 % (floor 94, unchanged); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean. `uv run pytest -m live --live` → 11 passed (live fleet untouched).
|
||||
|
||||
**Docs** — `README.md`: masthead row-3 list no longer names `⏸ Pause`, plus the "no control stops the polling" sentence and a hidden-tab note on the `POLL_MS_*` row. `AGENTS.md`: same masthead sentence rewritten with the phase-21 rationale.
|
||||
|
||||
**Notes/deviations** — the masthead's `/api/health` status-dot interval in `initChrome()` was never gated on hidden/paused, so the browser check filters it out of "data fetches" (documented in the script); pre-existing behaviour, left alone. `validate.sh` prints a pre-existing advisory (`fastapi 0.153.0` vs `0.156.4`); no dependency was touched by this task, so no upgrade. Concurrent working-tree changes not mine: emptied `NOTES.md`, new `TODO.md` items and `todo/25_…`, `todo/26_…`.
|
||||
|
||||
**Next pending task:** `21_page_cleanups/02_gpu_own_values.md`.
|
||||
@@ -0,0 +1 @@
|
||||
ok
|
||||
@@ -0,0 +1,65 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 24%]
|
||||
..........................................s............................. [ 29%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 68%]
|
||||
.......sssssssssss...................................................... [ 72%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 92%]
|
||||
.......................................................sss.............. [ 97%]
|
||||
......................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 32
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 436 0 100%
|
||||
src/dashboards/app.py 185 0 100%
|
||||
src/dashboards/auth.py 265 0 100%
|
||||
src/dashboards/certs.py 292 0 100%
|
||||
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
|
||||
src/dashboards/config.py 120 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
|
||||
src/dashboards/rates.py 73 0 100%
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 134 0 100%
|
||||
src/dashboards/sources/host.py 209 0 100%
|
||||
src/dashboards/sources/llm.py 178 1 99% 285
|
||||
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 313 2 99% 553-554
|
||||
------------------------------------------------------------------
|
||||
TOTAL 3490 37 99%
|
||||
Required test coverage of 94% reached. Total coverage: 98.94%
|
||||
1466 passed, 15 skipped, 1 warning in 158.10s (0:02:38)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
|
||||
fastapi 0.153.0 0.156.4 wheel
|
||||
@@ -0,0 +1,13 @@
|
||||
**Implemented**
|
||||
- `page-system.js`: `gpuSub()` → `GPU_VALUE_FIELDS` + `gpuValues()` — Model / VRAM / Pool / Temp as four labelled `.metric` cells under a busy-only hero (no `severity`, no bar; model escaped; missing field = `n/a`, never `0 °C`). `renderGpuSlot()` now renders a `.gpu-card` per card and **omits** the slot for a host with no GPU (`data-k="gpus"` ships `hidden`; `[hidden]{display:none!important}` reflows the row to five).
|
||||
- `dash.css`: `.gpu-card` + `.gpu-values` (`auto-fit minmax(min(150px,100%),1fr)`, `overflow-wrap: anywhere`, no colour tokens); `.hero-slot` comment updated.
|
||||
- `tests/test_api.py`: new `GPU_VALUE_CELLS` table read out of the served script and resolved against all three captures **and** over HTTP; pins for the four cells, the vanished `gpuSub`/`join(' · ')`, the no-verdict/no-bar grid + CSS wrap, and the omission (hidden slot, no `n/a`, `[hidden]` rule); six-slot order pin kept + `hidden` pin. `card_contract.py`/`test_system_contract.py`: stale "hero footnote" prose only, assertions untouched.
|
||||
- Browser proof `data/logs/21-task02-gpu-values.cjs` (+ `.out.txt`, `21-task02-gpu-values-{1280,360}.png`, `21-task02-gpu-charts.png`): all checks pass — four values at 16.3px (old sub was 11px), gateway = 5 heroes / no slot / no `n/a`, hero order unchanged, GPU chart still builds per GPU host, no sideways scroll.
|
||||
|
||||
**Gate** — `bash .agents/validate.sh` EXIT=0: `1470 passed, 15 skipped`, coverage 98.94 % (floor 94, unchanged); `ruff check src tests agent` clean; `pyright` 0 errors; `node --check` clean. `uv run pytest -m live --live` → **11 passed**.
|
||||
|
||||
**Docs** — `AGENTS.md`: the `/system` paragraph split (missing *number* → `n/a` uncoloured; missing *device* → slot omitted), four-labelled-values layout, `GPU_VALUE_FIELDS` device, GPU **chart** slot disambiguation, hermetic tally 1464→1470. `README.md`: same rules in the `/system` page description.
|
||||
|
||||
**Decisions/deviations** — Temp's label is the word `Temp` with `°C` in the value (pinned `count("unit: ' °C'") == 1`). `.gpu-values` is `auto-fit` rather than a fixed 2×2: measured hero track is ~180px on the live 3-host fleet at 1280px, so 2 columns re-wrap the TODO's own strings onto 3 lines; wide slots still get 2×2. A missing field on a *present* card now prints `n/a` (the task's rule), replacing phase 06's "print no temperature at all" — the never-`0 °C` half is kept and pinned. Pre-existing `validate.sh` advisory (`fastapi 0.153.0` vs `0.156.4`) left alone: no dependency touched.
|
||||
|
||||
**Next pending task:** `21_page_cleanups/03_certs_table_scroll.md`.
|
||||
@@ -0,0 +1 @@
|
||||
ok
|
||||
@@ -0,0 +1,65 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 24%]
|
||||
..............................................s......................... [ 29%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 67%]
|
||||
...........sssssssssss.................................................. [ 72%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 92%]
|
||||
...........................................................sss.......... [ 96%]
|
||||
............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 32
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 436 0 100%
|
||||
src/dashboards/app.py 185 0 100%
|
||||
src/dashboards/auth.py 265 0 100%
|
||||
src/dashboards/certs.py 292 0 100%
|
||||
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
|
||||
src/dashboards/config.py 120 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
|
||||
src/dashboards/rates.py 73 0 100%
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 134 0 100%
|
||||
src/dashboards/sources/host.py 209 0 100%
|
||||
src/dashboards/sources/llm.py 178 1 99% 285
|
||||
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 313 2 99% 553-554
|
||||
------------------------------------------------------------------
|
||||
TOTAL 3490 37 99%
|
||||
Required test coverage of 94% reached. Total coverage: 98.94%
|
||||
1470 passed, 15 skipped, 1 warning in 153.88s (0:02:33)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
|
||||
fastapi 0.153.0 0.156.4 wheel
|
||||
@@ -0,0 +1,8 @@
|
||||
**Task 03 complete — `/certs` agents table scrolls horizontally.**
|
||||
|
||||
- **Implemented:** `#agents-table` in `src/dashboards/web/pages/certs.html` gains the repo's own `scroll` modifier (`class="data scroll"`) — the existing `table.data.scroll { display:block; overflow:auto; max-height:320px }` rule in `dash.css` is the mechanism, so no CSS added, no wrapper invented (verified the selector form first). `page-certs.js` needed no change (id/`tbody` selectors intact). New served-bytes pin `tests/test_api.py::test_the_certs_agents_table_scrolls_to_its_last_column` pins the class on the table **and** the `table.data.scroll` rule together (either side renaming silently un-scrolls the table).
|
||||
- **Browser verification (invariant 10):** `data/logs/21-task03-scroll.cjs` against the live dev server — 14/14 checks PASS: at 360 px the container overflows (scrollWidth 1021 vs clientWidth 292), scrolling exposes the full Actions column (Rotate/Revoke/bundle all inside, `elementFromPoint` hit-tests to the control — no write sent); at 1280 px nothing changed (no h- or v-scrollbar, all columns inside). Screenshots: `data/logs/certs-scroll-360-{start,mid,end}.png`, `certs-scroll-1280.png` (mid-scroll shows the table scrolled to the serial/expiry columns).
|
||||
- **Tests/lint:** `uv run pytest` → **1471 passed, 15 skipped** (was 1470; +1 new pin); coverage **98.94%** (gate 94%); `ruff` + `pyright` clean; `bash .agents/validate.sh` → **exit 0** (incl. `node --check`); `uv run pytest -m live --live` → **11 passed**. Pre-existing fastapi 0.153→0.156 advisory is non-blocking and outside this task (no dependencies touched — invariant 8 fires only when deps move).
|
||||
- **Docs:** README.md *Managing the certificates* — one sentence that the agents table scrolls at phone width; AGENTS.md `/certs` paragraph — the `table.data.scroll` contract, the pinned class+rule pair, and the measuring script. Gap 12's forward reference to this change remains accurate.
|
||||
- **Decisions:** used the class form (the only one the stylesheet styles); kept the `max-height: 320px` cap as shipped (3 rows ≈ 150 px, verified no vertical scrollbar); no CSS/JS behavior change — row contents, chips, controls, and the `COLUMNS` contract test untouched.
|
||||
- **Next pending task:** none in `todo/21_page_cleanups` — this was the last task; phase 21 is ready for the harness's completion/validation pass.
|
||||
@@ -0,0 +1 @@
|
||||
ok
|
||||
@@ -0,0 +1,65 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 24%]
|
||||
...............................................s........................ [ 29%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 67%]
|
||||
............sssssssssss................................................. [ 72%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 92%]
|
||||
............................................................sss......... [ 96%]
|
||||
.............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 32
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 436 0 100%
|
||||
src/dashboards/app.py 185 0 100%
|
||||
src/dashboards/auth.py 265 0 100%
|
||||
src/dashboards/certs.py 292 0 100%
|
||||
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
|
||||
src/dashboards/config.py 120 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
|
||||
src/dashboards/rates.py 73 0 100%
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 134 0 100%
|
||||
src/dashboards/sources/host.py 209 0 100%
|
||||
src/dashboards/sources/llm.py 178 1 99% 285
|
||||
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 313 2 99% 553-554
|
||||
------------------------------------------------------------------
|
||||
TOTAL 3490 37 99%
|
||||
Required test coverage of 94% reached. Total coverage: 98.94%
|
||||
1471 passed, 15 skipped, 1 warning in 155.49s (0:02:35)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
|
||||
fastapi 0.153.0 0.156.4 wheel
|
||||
@@ -0,0 +1,18 @@
|
||||
All green. Final report:
|
||||
|
||||
**Verified (phase 22 final pass — both task files already in `complete/`)**
|
||||
- `api._disk_visible` (`is not False`) is the single gate, applied to the assembled host list in `/api/disks/snapshot` **and** `/api/summary`'s disks block before the fleet sum and the rows; `sources/disks.py`, `page-disks.js`, `poller.py`, `store.py` and `alerts.py` are byte-unchanged (`git diff --stat`: only `api.py`, `test_api.py`, docs).
|
||||
- Re-ran the live browser pass `data/logs/22-task02-disks-live.cjs` against the running dev server: **34/34 PASS** — no gateway card/why-line, `2/2 hosts answering`, fleet header reads the API's `2 of 2` / `17-17-0-1`, deskwork `nvme0n1` OK 47 °C/19 202 h/11 %, driveripper 16 live cards + `13 uncorrectable` chip, both aria-labelled plots, `/` still 3 fleet rows. Screenshots + `22-task02-api.json` refreshed.
|
||||
- `./scripts/dev-server.sh status`: all **nine sources `ok`**, `disks:gateway` included; live `/api/events?source=disks` still carries gateway transitions (10 of 45), proving the README/AGENTS claim the gate hides a row and not the job's history.
|
||||
|
||||
**Completion criteria**
|
||||
- `false` host absent from both endpoints, `None` + `unreachable` stub present, fleet ≡ row count: **PASS** (6 pins in `tests/test_api.py`, incl. the single-hidden-host = 200-with-empty-list leg).
|
||||
- Live `/disks` = deskwork + driveripper, header 2 hosts, cards unchanged, screenshot in `data/logs/`: **PASS** (re-run above).
|
||||
- `uv run pytest` → **1477 passed, 15 skipped**; `--cov --cov-fail-under=94` → **98.94 %** (`api.py` 100 %); `ruff check src tests agent` → all passed; `pyright` → 0 errors; `bash .agents/validate.sh` → **EXIT 0**. `uv run pytest -m live --live` → **11 passed**.
|
||||
- No behaviour change in completed work: **PASS** (unchanged files above; live numbers match the pre-phase fleet record).
|
||||
|
||||
**Docs** — README: `/disks` page-table row + the agent SMART paragraph now distinguish "cannot read drives" (dimmed cards kept) from "no smartctl" (host absent from the page/API lists). AGENTS.md: hermetic tally 1471→**1477**, and the `collectors.py`/`api.py` passages now note the `false` arm never reaches the `/disks` why-line while an overview alert from a hidden host's `disks` job still names it. Everything else (routes vs served table, env table vs `config.py`, commands, scripts/fixtures/tests references, masthead/POLL_MS/threshold/retention claims) **verified consistent**.
|
||||
|
||||
**Deviations** — `NOTES.md` left empty by the task runs (both notes are tracked: mTLS → phase 24 `todo/`, the admin-links bug → fixed in phase 20 task 04); `/disks`' `#hosts-empty` copy left as pinned markup (it already names smartmontools + the disk group, so an all-hidden fleet is still told the fix); validate's advisory `fastapi 0.153.0 → 0.156.4` untouched (no dependency changed this phase). No live fleet writes of any kind.
|
||||
|
||||
**Next pending phase:** `23_llm_rate_responsiveness`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
== uv run pytest --cov (hermetic: no network, live tests skip) ==
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 24%]
|
||||
.....................................................s.................. [ 28%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 67%]
|
||||
..................sssssssssss........................................... [ 72%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 91%]
|
||||
..................................................................sss... [ 96%]
|
||||
.................................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/dashboards/.venv/lib64/python3.14/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.14.4-final-0 ________________
|
||||
|
||||
Name Stmts Miss Cover Missing
|
||||
------------------------------------------------------------------
|
||||
src/dashboards/__init__.py 3 0 100%
|
||||
src/dashboards/__main__.py 14 1 93% 32
|
||||
src/dashboards/alerts.py 176 0 100%
|
||||
src/dashboards/api.py 440 0 100%
|
||||
src/dashboards/app.py 185 0 100%
|
||||
src/dashboards/auth.py 265 0 100%
|
||||
src/dashboards/certs.py 292 0 100%
|
||||
src/dashboards/collectors.py 205 5 98% 284-285, 289-290, 307
|
||||
src/dashboards/config.py 120 0 100%
|
||||
src/dashboards/debug.py 14 0 100%
|
||||
src/dashboards/poller.py 165 3 98% 192-194, 279
|
||||
src/dashboards/prom.py 112 5 96% 126, 163, 177, 180, 183
|
||||
src/dashboards/rates.py 73 0 100%
|
||||
src/dashboards/severity.py 49 0 100%
|
||||
src/dashboards/sources/__init__.py 0 0 100%
|
||||
src/dashboards/sources/disks.py 134 0 100%
|
||||
src/dashboards/sources/host.py 209 0 100%
|
||||
src/dashboards/sources/llm.py 178 1 99% 285
|
||||
src/dashboards/sources/nut.py 413 20 95% 153, 172, 186, 198, 204, 217, 225, 241, 319-320, 322, 579, 591, 626, 657, 661, 718-719, 724, 736
|
||||
src/dashboards/sources/nut_sim.py 134 0 100%
|
||||
src/dashboards/store.py 313 2 99% 553-554
|
||||
------------------------------------------------------------------
|
||||
TOTAL 3494 37 99%
|
||||
Required test coverage of 94% reached. Total coverage: 98.94%
|
||||
1477 passed, 15 skipped, 1 warning in 155.45s (0:02:35)
|
||||
== uv run ruff check src tests agent ==
|
||||
All checks passed!
|
||||
== uv run pyright ==
|
||||
0 errors, 0 warnings, 0 informations
|
||||
== node --check src/dashboards/web/static/*.js ==
|
||||
advisory: dependencies are out of date (AGENTS.md rule 8 — update on purpose):
|
||||
fastapi 0.153.0 0.156.5 wheel
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user