phase: 29_gateway_dashboard_page
Phase 29 completion pass is done. Final state:
**Verification (all re-run this pass, no task files touched)**
- `uv run pytest` → **1746 passed, 16 skipped**; `uv run pytest --cov` → **99.05 %** (gate 94 %); `bash .agents/validate.sh` → **rc=0** (ruff clean, pyright `0 errors, 0 warnings`, `node --check` clean); `uv run pytest -m live --live` → **12 passed**.
- Live, read-only: dev server **10/10 sources `ok`** (incl. `gateway`); `data/logs/29-task06-live.cjs` → **49 PASS / 0 FAIL** with per-port inbound counts reconciled against a direct `/proc/net/tcp{,6}` read (divergence 0); `data/logs/verify-29-task06-predates.py` → 0 failed (1.1.0 arm renders `WHY_PREDATES`, no pipes, every tally `n/a`, job still `ok`); phase-27 mobile audit re-run over all nine served pages → **371 PASS / 0 FAIL / 0 FINDING** (`data/logs/29-final-mobile-audit.out`).
- Criteria: (1) unconfigured `GATEWAY_HOST` → `404 {configured:false}` with dial spies + `#gate-empty` sentence ✅; (2) 1.1.0/`--no-proxy` → `supported:false`, no `0`s, never red ✅; (3) live gateway 1.2.0 → pipes animate, totals reconcile ✅; (4) no `severity`/verdict colour anywhere, validate green ✅; (5) docs updated + gap 13 logged ✅; (6) the five existing dashboards, jobs and `/api/summary` unchanged (live control leg) ✅.
**Docs (consistency pass, incl. pre-existing drift)**
- `README.md`: architecture diagram now lists `sources/gateway.py`, the 30 s gateway job and the third `/snapshot` reader; `GATEWAY_HOST` row says *a name, never a URL*; the `POLL_MS_*` row states `/gateway`'s fixed 15 s browser poll; "nine pollers" → per-source cadences; phase-28 "until phase 29" wording → current state; new Testing paragraph for `tests/test_gateway_source.py` (join pinned from the captures, reconciliation identity, `getaddrinfo` spy).
- `AGENTS.md`: phase-28 stop-point wording replaced with the shipped phase-29 state; suite/coverage figures → 1746 / 16 skipped / 99.08 %→99.05 % band and `~99 %`; window-chips paragraph records that `/gateway` has **no** chips (fixed 24 h); new testing-policy bullet for the join; stale page counts fixed (seven → eight tabbed pages / `MARKUP_PAGES`, clock-order and badge-count transcripts annotated); gap 12 now records the phase-29 nine-page audit re-run.
- `tests/test_api.py`: comment/docstring-only fixes for the "seven pages" wording that phase 29 made eight.
**Deviations / notes**
- Nothing implemented: all six task files were already in `complete/`, so this run was the final verification + docs pass; no git index or phase file was touched.
- Coverage wobbles 99.05–99.08 % between runs on a pre-existing timing-dependent line (`collectors.py` `/slots`-alias `except`/`break`), not a phase-29 path (`sources/gateway.py` is 100 %).
- `uv pip list --outdated` still lists upgrades; dependencies did not move in this phase, so they were left alone rather than upgraded mid-verification.
- Historical browser-transcript counts (phase 18/20/25 script runs) were left as recorded history and annotated instead of rewritten.
**Next phase:** none — `.agents/phases/todo/` is empty after 29; further work needs a new phase authored first.
This commit is contained in:
@@ -0,0 +1,38 @@
|
||||
# Phase 29 — The gateway dashboard page: animated pipes from config + live sockets
|
||||
|
||||
**Source:** `TODO.md` L3 — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs. I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)." (the dashboard half)
|
||||
**Story:** n/a
|
||||
**Context:** phase 28 (the `haproxy`/`nginx`/`sockets` agent sections, `AGENT_VERSION` 1.2.0, the gateway host rolled out), `src/dashboards/config.py` (`Settings`, `_env`, the `nut_enabled`/`actions_allowed` gate properties), `src/dashboards/sources/disks.py` (the "a source of its own reading the same agents" precedent — the `disks:<host>` job dials the same `/snapshot`), `src/dashboards/poller.py` (job loop, health records, stale/last-good), `src/dashboards/app.py` (`build_hub` — the job list wired once for production and the injected-store test path; the `agent_tls_map` the `host:`/`disks:` jobs share per host), `src/dashboards/api.py` (the snapshot/history route pairs, the `404 {"configured": false}` gate that runs before any dial, `_history_batch`, the `/api/summary` additive blocks), `src/dashboards/alerts.py` (the "an unconfigured source never alerts" rule), `src/dashboards/store.py` (the `sample(source, host, metric, res, ts)` + rollup/retention story — invariant 9), `src/dashboards/web/pages/` + `static/` (the five dashboards, the five-tab masthead pinned by `tests/test_api.py`, `app.js`'s `initChrome` badges, `Dash.hero`/`DashCharts` — no framework, no CDN, invariant 5), `tests/test_api.py` (the served-bytes pins), `tests/test_config.py`, `tests/test_live_fleet.py` (the read-only live suite).
|
||||
|
||||
## Objective
|
||||
|
||||
The fifth dashboard page — `/gateway` — renders the gateway host's connection topology as animated pipes: for every configured frontend (haproxy `bind`, nginx `listen`, the stream `443`) an **inbound** pipe (client → gateway: the port, the `server_name` domains, the live connection count and client addresses) and, fanning out from the gateway, one **outbound** pipe per configured target (haproxy `server`, nginx `proxy_pass`, the stream `map` entries: the target host:port, the live count). The page is a **source of its own** (`gateway`, its own poller job on the same `/snapshot` the `host:` job dials, its own interval/timeout/health/events), its counts are stored as gauges (invariant 9's rollup/retention story) and graphed from `/api/gateway/history`, and it keeps every invariant the fleet already keeps: unconfigured is `404` without a dial (invariant 3), a dead source ages instead of zeroing (invariant 4), no count is a rate (rule 2 — no estimator, no `Δ<window>`), no count carries a verdict (like `net.*`), and an old agent (1.1.0, no sections) reads as "predates gateway support" — dimmed, `n/a`, never red.
|
||||
|
||||
## Dependencies
|
||||
- `28_gateway_agent_proxy` (todo) — the three agent sections on the wire and the gateway host running 1.2.0 (the live half of this phase's verification dials what 28 installed)
|
||||
- `27_mobile_tables_and_audit` (todo) — the page's 360 px behaviour is audited by 27's script shape (the table, if any, scrolls inside its card)
|
||||
|
||||
## Tasks
|
||||
1. `01_config_and_source.md` — `GATEWAY_HOST` in `Settings` + the gate properties; `sources/gateway.py`: the config + sockets join into inbound/outbound pipes, gauges only, the three-way agent support
|
||||
2. `02_wiring.md` — the `gateway` poller job in `build_hub` (30 s), health, the alerts rules, the `/api/summary` block, the `/api/config` entry
|
||||
3. `03_api_routes.md` — `/api/gateway/snapshot` + `/api/gateway/history`: the configured gate before any dial (spy-pinned), history through `_history_batch`, unknown names as empty series
|
||||
4. `04_page.md` — `/gateway` + `page-gateway.js`: the animated pipes (hand-rolled SVG + CSS, no CDN, `prefers-reduced-motion`, `aria-label`), the live counts, the three degradation arms
|
||||
5. `05_nav_and_summary.md` — the sixth tab on every signed-in page (the pinned masthead pins updated + browser-measured), the `/api/summary` pin extended
|
||||
6. `06_docs_and_verify.md` — README/AGENTS (the page row, the `GATEWAY_HOST` row, the fleet fact), the browser screenshot pass against live data, the read-only live verification
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_gateway_source.py` (new — the join over fixture snapshots: an inbound join by listen port, the domain-target port-only join labelled "by port", the raw-IP exact join, the `$var`→`map` join, the 1.1.0-predates arm, the never-answered stub arm); `tests/test_config.py` (`GATEWAY_HOST` default, the gate property); `tests/test_api.py` (the route pins, the summary block, the nav pins)
|
||||
- Coverage: **>90%** on `sources/gateway.py` + the new config field
|
||||
- Real-browser check (invariant 10): task 05's measurement + task 06's screenshots against the live dev server
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- Task 06 updates `AGENTS.md`'s page table (`/gateway` row), the `config.py` bullet (`GATEWAY_HOST`), and the metric-naming paragraph (`gateway.*` gauges); `README.md` gains the page's how-to row. The invariant 9 story: the `gateway.*` series roll up raw → 1 m → 1 h like every other gauge.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (the hermetic suite), coverage not down; `uv run pytest --cov` holds the 94% floor
|
||||
- [ ] `GATEWAY_HOST` unset (or a name not in `HOST_TARGETS`) → `/api/gateway/snapshot` answers `404 {"configured": false}` with no dial (spy-pinned) and the page says "not configured"
|
||||
- [ ] a 1.1.0-style snapshot (no sections) → the page's "agent predates gateway support" arm, dimmed, every number `n/a`, the job still `ok` (the invariant-3 shape, pinned)
|
||||
- [ ] the gateway host (1.2.0, post-phase-28) → the page renders the six haproxy frontends, the nginx server blocks and the stream map, with live counts that match a direct `/proc/net/tcp` read (task 06's verification)
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the nine existing sources read `ok` in `./scripts/dev-server.sh status`, the five existing dashboards render as before, the `/api/summary` pre-existing blocks are byte-identical in shape (the `gateway` block is additive)
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 01 — `GATEWAY_HOST` and the gateway source
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — "…visualizes my incoming connections based on the gateway host's nginx and haproxy configs… data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`Settings` names which `HOST_TARGETS` entry is the gateway (`GATEWAY_HOST`, default `gateway`), and `sources/gateway.py` projects one agent snapshot into the page's data: the **inbound** pipes (each configured frontend — a haproxy `bind` port, a nginx `listen` port, the stream `443` — with its `server_name` domains and the live ESTABLISHED count + client addresses keyed on the listen port) and the **outbound** pipes (each configured target — a haproxy `server`, a nginx `proxy_pass`, a stream `map` entry — with the live ESTABLISHED count). The counts are gauges: **no rate** (no `RateEstimator`, no `window_seconds`, no `Δ<window>` — a count is a count, rule 2), **no verdict** (no `severity` dict on any count — `severity.py` rates no connection count, the `net.*` precedent), and the three-way agent support degrades a pre-1.2.0 agent to "predates gateway support" (invariant 3), never red.
|
||||
|
||||
- ASSUMPTION: **inbound join**: an ESTABLISHED socket belongs to frontend `P` when its **local port** is `P`'s listen port; the client is the socket's **remote** address. A socket whose local port matches no configured frontend is `unmatched` (reported, not dropped — the page shows "N other connections", the operator's fact that the gateway talks to something the configs don't name).
|
||||
- ASSUMPTION: **outbound join**: an ESTABLISHED socket belongs to target `T` when its **remote** address matches — a raw-IP target on an exact (ip, port) match; a **domain target on a port-only match, labelled `by_port: true`** in the payload (the agent never does DNS — invariant 7's "never guess" applied to the join: a port match against a domain is a real socket fact plus a config fact, and the label says which half is which). A socket whose remote matches no target is `unmatched` too.
|
||||
- ASSUMPTION: the payload shape is `{configured: true, supported: true, host: str, haproxy: {frontends: [{name, port, backend: str|null}]}, nginx: {servers: [{port, server_names: [str]}], map_targets: [{key, host, port}]}, inbound: [{port, source: "haproxy"|"nginx", domains: [str], connections: int, clients: [str]}, outbound: [{target: {host, port, name: str|null, by_port: bool}, connections: int, source: str|null}], unmatched: {inbound: int, outbound: int, other: int}, sockets: {listens: int, established: int}}` — `supported: false` (a 1.1.0-style snapshot) carries the same keys as empty/nulls plus `supported: false`, and a never-answered host is the stub the `/api/system` stubs use (`unreachable: true`, every count `null`, invariant 4).
|
||||
- ASSUMPTION: the `$var`→`map` join: a `proxy_pass $targetBackend` whose variable matches a `map` block contributes the map's entries as `map_targets` (key = the domain, the target = the entry's host:port) and **no** literal outbound pipe (the per-connection target is the map's job; the page renders the map entries as the fan-out — the 17-entry `stream.conf` map is exactly the "data going out (domains)" the TODO names).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/config.py` — `Settings.gateway_host: str = _env("GATEWAY_HOST", "gateway")`, `gateway_interval_seconds` (default `30.0`, the `DISKS_INTERVAL_SECONDS` idiom) and `gateway_timeout_seconds` (default `10.0`), and the `gateway_enabled` property (`True` iff `gateway_host` is among the `HOST_TARGETS` names — the `nut_enabled` shape); the `any_source_enabled` property includes it.
|
||||
2. `src/dashboards/sources/gateway.py` — new: `parse(snapshot: dict, host: str) -> dict` (the `sources/disks.py` projection idiom — pure over the snapshot the poller fetched, no I/O of its own): the section readers (an absent section → the `supported: false` arm; `available: false`/`null` → a `why` line in the payload, the invariant-3 arms), the inbound/outbound joins per the assumptions above, the `unmatched` tallies, and the `samples` list: one gauge per distinct listen port (`gateway.in_conns@<port>`) and one per distinct target host (`gateway.out_conns@<host>`; a `by_port` domain target's series is keyed on the host, the port is in the payload not the name — the `@stream` scheme, invariant 9's naming).
|
||||
3. `tests/test_config.py` — `GATEWAY_HOST` default `gateway`, an explicit value winning, `gateway_enabled` true/false over the target list (the `nut_enabled` test shape).
|
||||
4. `tests/test_gateway_source.py` — new: the join over fixture-shaped snapshots (reuse the phase-28 fixtures' parsed shape — the test builds the snapshot dict the agent serves): the inbound join (a socket on local `:443` lands in the stream frontend's pipe with its client address; a socket on an unconfigured port is `unmatched.inbound`); the outbound join (a raw-IP target's exact match; a domain target's port-only match with `by_port: true`; no DNS call anywhere — a `socket.getaddrinfo` spy must record nothing); the `$var`→`map` join (the 17-entry map → 17 `map_targets`, no literal pipe); the `supported: false` arm (a 1.1.0-style snapshot without the sections); the never-answered arm is the API's job (task 03), not the source's.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_config.py` + `tests/test_gateway_source.py` as above
|
||||
- Coverage: **>90%** on `sources/gateway.py` + the new config fields
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GATEWAY_HOST` defaults to `gateway` and `gateway_enabled` is true only for a configured name (pinned)
|
||||
- [ ] `parse()` over a 1.2.0 fixture snapshot yields the exact inbound/outbound lists (counts, clients, `by_port`, the map fan-out) — a rename in the agent's section keys fails a test
|
||||
- [ ] the `getaddrinfo` spy records **nothing** across the whole source (pinned — the join never dials DNS)
|
||||
- [ ] the `supported: false` arm renders the payload's `why` and null counts, never a red verdict (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing `Settings` fields and the `host`/`disks`/`llm`/`nut` sources are byte-identical in behaviour
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 02 — The gateway job: poller, health, alerts, summary, config
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the source is a **job of its own** (the `disks:<host>` precedent): its own interval, timeout, health record, and events, so a wedged config read ages the pipes instead of blinking `/system`
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The `gateway` source runs as its own poller job (the `disks:` shape — one job dials the same `/snapshot` the `host:` job of the gateway host reads, with the gateway's own interval/timeout), so the page ages on its own cadence and a failed poll keeps the last good pipes dimmed instead of zeroing them (invariant 4). The alerts rules keep an unconfigured source silent (invariant 3) and a dead source to exactly one line (the `disks` echo rule — the gateway job dials the same agent, so its failure carries no new information while the `host:` job is the voice; a gateway job failing *while its host answers* — the sections wedged — is the only voice that fault has). `/api/summary` gains the additive `gateway` block, `/api/config` gains the gateway row.
|
||||
|
||||
- ASSUMPTION: the job is `gateway` (one job, the `nut` shape — a single source, not per-host: the page is about the one gateway host), `DISKS`-interval defaults from task 01 (30 s / 10 s), events under the dashboard name `gateway`.
|
||||
- ASSUMPTION: the **no-verdict alert rule**: the source's payload carries no `severity`, so `alerts.py` gains no threshold for `gateway` — the only alerts a `gateway` job raises are reachability (stale > 3× its 30 s interval → `warn`, stopped answering → `fail`, the `disks`-echo suppression applied: a dead gateway job while its `host:` job is also dead adds no line).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/app.py` — `build_hub`: the `gateway` job (one, not per host) wired the `disks:` job is wired — the same `AgentTls` entry the gateway host's `host:`/`disks:` jobs use (so a revocation fails all three with one sentence, the task-03 `agent_tls_map` shape), its `parse` the `sources.gateway.parse` + `samples` pass-through, no `RateEstimator` (the source takes none — task 01). The job exists **only** when `settings.gateway_enabled` (the `disks` job's configured gate: no job at all when the source is unconfigured, invariant 3).
|
||||
2. `src/dashboards/alerts.py` — the `gateway` block: the reachability rules per the assumption above; the `HREF` table gains `/gateway`; **no** verdict rule (asserted: a `gateway` payload with a fabricated `severity` dict still produces no verdict alert — the module may not have a threshold of its own, the phase-09 disk rule's shape).
|
||||
3. `src/dashboards/api.py` — `/api/summary`: the additive `gateway` block (`{configured, supported, host, inbound: int (Σ connections), outbound: int, map_targets: int, unmatched: int, states: {gateway: str}}` — a configured job with no payload yet is a stub, the `system.hosts` stub shape); `/api/config`: the gateway row (`GATEWAY_HOST`'s name + the `Target.display_url` of the entry it names, redacted — invariant 6's boundary).
|
||||
4. `tests/test_api.py` — the `build_hub` wiring pins (the job present iff `gateway_enabled`; the shared `AgentTls` entry — a revocation written into `meta` fails the `gateway` job **before any dial**, the `test_a_revoked_agent_fails_both_of_its_sources_without_a_dial` shape extended to three sources); the `/api/summary` `gateway` block pin (with and without the source configured — unconfigured: the block is `configured: false` and **absent no dial happens**, the spy-pinned shape); the alerts pins (unconfigured → no `gateway` alert; dead job + dead host → one line, not two; a wedged-sections failure while the host answers → one line naming `gateway`).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the wiring/alerts/summary pins above
|
||||
- Coverage: **>90%** on the modified `app.py`/`alerts.py`/`api.py` branches
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] a configured build carries exactly one `gateway` job on the gateway host's shared TLS entry; an unconfigured build carries none (pinned)
|
||||
- [ ] a revocation in `meta` fails the `gateway` job before any dial (spy-pinned, the `host:`/`disks:` pair's shape)
|
||||
- [ ] `/api/summary`'s `gateway` block is pinned with and without configuration; the pre-existing blocks are unchanged
|
||||
- [ ] the alerts pins hold: unconfigured silent, dead-source one line, no verdict rule
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing jobs, alerts, and summary blocks behave exactly as before
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — `/api/gateway/snapshot` and `/api/gateway/history`
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the page's data routes, in the app's established shape: the configured gate before any dial (invariant 3), history through the one batched door (invariant 9 / phase-14 task 02)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`/api/gateway/snapshot` answers the source's payload (task 01's shape: the config, the joined inbound/outbound pipes, the unmatched tallies, `supported`) for the `GATEWAY_HOST` entry — with a configured-but-never-answered host as the `unreachable` stub (every count `null`, invariant 4's page-side half) and `GATEWAY_HOST` unset/unknown as `404 {"configured": false}` **before anything that could reach the network** (no client, no fetch, no job — spy-pinned both ways, the `tests/test_api.py` disks-pair shape). `/api/gateway/history` serves the `gateway.in_conns@<port>` / `gateway.out_conns@<host>` series through the same `_history_batch()` → `store.history_many()` door the other four history routes use (phase-14 task 02 — one batched decision, the same `{source, host, metric, range_seconds, points}` document, unknown names as empty series).
|
||||
|
||||
- ASSUMPTION: the route pair is read-only — a fourth data page grows no write (invariant 1's shape: the app's only POST that dials an upstream stays `/api/ups/command`, and the `tests/test_api.py` route-verb enumeration gains the two GETs to the read-only list).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/api.py` — the `gateway` router: `GET /api/gateway/snapshot` (the configured gate first — `settings.gateway_enabled` false → `404 {"configured": false}` with **no** client construction and **no** `fetch_snapshot` call, spy-pinned; configured → the job's last payload through `sources.gateway.parse`'s never-answered-stub arm) and `GET /api/gateway/history` (the `hosts=`/`metrics=`/`range=` query shape the other three history routes take, answered through `_history_batch`).
|
||||
2. `src/dashboards/api.py` — the route-verb enumeration test's read-only list gains the two paths (the task-01 assumption's pin).
|
||||
3. `tests/test_api.py` — the pins, the disks pair's shape: (a) `GATEWAY_HOST` unset → `404 {"configured": false}`, the `httpx.AsyncClient.get` spy and the `collectors.fetch_snapshot` spy record **nothing**; (b) `GATEWAY_HOST` a name not in `HOST_TARGETS` → the same refusal, the same no-dial proof; (c) configured, the job's payload present → the snapshot's exact keys (the task-01 shape — a rename fails a test); (d) configured, no payload yet → the `unreachable` stub (every count `null`); (e) the history route answered through `_history_batch` — points read straight out of the store with `query()` (a third path, in neither the loop nor the batch, fails the test — the phase-14 task-02 pin shape) for `gateway.in_conns@443` and a `gateway.out_conns@<host>` series, with an unknown name answering as an empty series.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the five pins above
|
||||
- Coverage: **>90%** on the new routes
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] unconfigured (unset or unknown name) → `404 {"configured": false}` with the two spies recording nothing (pinned both arms)
|
||||
- [ ] the never-answered stub is pinned (`unreachable: true`, every count `null`)
|
||||
- [ ] the history route's points equal the store's rows for the `gateway.*` series; unknown names → empty series (pinned)
|
||||
- [ ] the two GETs are in the read-only enumeration; no new POST exists (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean
|
||||
- [ ] no behavior change in completed work: the pre-existing snapshot/history routes answer byte-identical shapes
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 04 — The `/gateway` page: animated pipes
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — "I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The `/gateway` page renders the joined payload (task 01) as animated pipes: one **inbound** pipe per configured frontend — `client → gateway` — labelled with the listen port, the `server_name` domains (haproxy's L4 frontends label their port only), the live connection count, and up to a few client addresses; and, fanning out from the gateway node, one **outbound** pipe per configured target — `gateway → target` — labelled with the target host:port (and `· by port` on a `by_port` match), its live count, and its source (the haproxy backend / nginx location / stream-map key it came through). The animation is **decorative at a fixed speed** — moving dashes whose direction is the data flow — because a count is not a rate (rule 2): the page never claims a tok/s-shaped number from it. No count is coloured by a verdict (the `net.*` precedent: the stylesheet section carries no `ok`/`warn`/`fail` token), and the three degradation arms render their own sentence: unconfigured ("not configured — set `GATEWAY_HOST`"), a 1.1.0 agent ("agent predates gateway support — install 1.2.0", dimmed, `n/a`), a never-answered host (dimmed `.stale`, `n/a`, the row kept, invariant 4).
|
||||
|
||||
- ASSUMPTION: the render is **one card per source family** (haproxy / nginx-https / nginx-stream / unmatched), each card a hand-rolled SVG: a left column of inbound pipe stubs, the gateway node mid-card (one rounded rect, its label the host name), a right column of outbound pipe stubs — the pipe is a `<path>` whose `stroke-dasharray`/`stroke-dashoffset` CSS animation runs only while the pipe has ≥ 1 live connection (a zero-connection pipe is a static faint line + the count `0`, because animating a pipe nobody is using would be a shape claiming a flow the count denies — the sparkline's "a shape is not a verdict" rule applied to motion); `prefers-reduced-motion: reduce` renders every pipe static (no animation at all, the repo's standing rule).
|
||||
- ASSUMPTION: the labels are `textContent`-only (no `innerHTML` for anything upstream-provided — a config domain is upstream text; the `DashCharts.escapeHtml` idiom via the page's own escape), and every SVG pipe carries `role="img"` + an `aria-label` built from its inputs (the battery-glyph convention: the pipe carries information the markup must say in words) — e.g. `"inbound pipe, port 4443, nextcloud.reeseapps.com, 3 connections"`.
|
||||
- ASSUMPTION: the page is the app's standing chrome — the masthead's sixth tab (task 05), the four `#nav-status` badges (the `gateway` job is the fifth dot, `app.js`'s one-dot-per-job rule), the theme toggle, the window chips **absent** (the page's history is the count trend — one `LineChart` per card, the `Charts` disclosure's `/system` shape: collapsed, lazy-built, one `/api/gateway/history` request naming every series the card could draw, `ariaLabel` on the plot), and the page's single `faint` footnote: the `by port` join sentence (the only why-line the operator needs to read a count).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/gateway.html` — the page: the shared `<head>`/masthead (task 05's tab slot), the four badge container, the card grid (the four cards' skeletons with their empty-state sentences), the `#nav-status`, and no `id` the script does not reference (the orphan-`id` audit's shape).
|
||||
2. `src/dashboards/web/static/page-gateway.js` — the page script (the `page-disks.js` idiom): `state` keyed per card; `renderPipes(card, pipes, direction)` — the SVG builder (the `DashCharts` escape discipline, the `aria-label` per the assumption, the zero-connection static arm, the `by port` sub-label); the degradation arms (the unconfigured / predates / never-answered sentences, the `.stale` dim, `n/a` for every count); the `Charts` disclosure's one-history-request rule (the `/system` shape: collapsed default, lazy build, a cached response from another window re-fetched); the `unmatched` card (the "N other connections" sentence — the operator's fact that the gateway talks to something the configs do not name).
|
||||
3. `src/dashboards/web/static/dash.css` — the pipe rules: `.pipe` (the `<path>` stroke, the `stroke-dasharray`), `@keyframes pipe-flow` (the dashoffset drift), `.pipe.still` (the zero-connection arm), and the `@media (prefers-reduced-motion: reduce)` block naming **both** the animating and the still states (the phase-18 reduced-motion shape — naming only one leaves the other animating).
|
||||
4. `tests/test_api.py` — the served-bytes pins: the pipe `@keyframes` + the reduced-motion rule (both states, inside the file's one such block); the zero-connection `.still` arm; the page's absence of any `ok|warn|fail` token in the gateway card section (the no-verdict pin, the phase-06 GPU shape); the `aria-label` on every `LineChart` the page builds (the standing chart-audit pin, extended); the orphan-`id` audit over the served page; the CDN-reference absence (invariant 5).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the served-bytes pins above (the repo has no JS test runner — a `.cjs` browser pass is the behavioural proof, task 06)
|
||||
- Coverage: **>90%** on the new Python-side branches (the pins); the JS is pinned on served bytes + the browser pass
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the page renders the four cards with their pipes, labels (port / domains / host:port / `· by port` / counts / clients), and the `unmatched` card — against a fixture-driven dev store
|
||||
- [ ] a zero-connection pipe is static (`.still`), a ≥ 1-connection pipe animates, `prefers-reduced-motion` renders all static (pinned + measured by task 06's browser pass)
|
||||
- [ ] the three degradation arms print their own sentence, dimmed where the invariant says, every count `n/a` (pinned)
|
||||
- [ ] no verdict token in the gateway card section; no CDN reference; every chart labelled (pinned)
|
||||
- [ ] `uv run pytest` green, `uv run ruff check src tests` and `uv run pyright` clean, `node --check` on the new script
|
||||
- [ ] no behavior change in completed work: the pre-existing pages render byte-identical
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 05 — The sixth tab, and the `/api/summary` pin
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the page is one of the dashboards: the masthead's tab row gains `Gateway` on every signed-in page, and the overview's fleet probe names the new source
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The masthead's tab row — five tabs on all seven signed-in pages (`/`, `/system`, `/disks`, `/llm`, `/ups` + `/settings`/`/certs`'s no-`aria-current` five) — gains the sixth, `Gateway`, in one order on every page, and the pinned markup audits move from a five-tab assertion to a six-tab one (the pins are the contract; they move on purpose, with the browser measurement as the proof). `app.js`'s one-dot-per-job rule makes the `gateway` job the fifth `#nav-status` badge with no JS change (the container already exists; the job is what the badge reads). The `/api/summary` pin (task 02's additive block) is completed here with the page-side half: the overview renders the `gateway` block's one line (the "N inbound · M outbound" count, linked to `/gateway` via the block's own href), and the no-behavior-change control is the five pre-existing dashboards rendering as before.
|
||||
|
||||
- ASSUMPTION: the tab order is `Overview · System · Disks · LLM · UPS · Gateway` (the new tab last — a sixth tab inserted mid-row would move the `aria-current` targets the five pinned pages assert, and a moved tab is a broken-button read on every page at once).
|
||||
- ASSUMPTION: the tab is `aria-current="page"` on `/gateway` only (the standing rule: the five dashboards carry it on their own page, `/settings`/`/certs` carry the five tabs with **no** `aria-current` — the sixth tab inherits the same shape on the two admin pages).
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/pages/*.html` — the six tab pages (`index`, `system`, `disks`, `llm`, `ups`, `gateway`) gain the `Gateway` tab (the `aria-current` on `gateway.html` only); `settings.html` and `certs.html` gain the same sixth tab with no `aria-current` (the two admin pages' shape — the phase-20 task-04 "both links on both admin pages" rule extended to the tab row).
|
||||
2. `tests/test_api.py` — the tab pins move: the five-tab assertion becomes the six-tab one (the `MARKUP_PAGES` loop's tuple gains `Gateway`, the order pin per the assumption, the `aria-current` pin per the assumption); the `test_every_page_links_to_both_admin_pages`-shape loop stays one loop over the pages (the phase-25 task-03 pin's shape — one order, one loop).
|
||||
3. `src/dashboards/web/static/page-index.js` — the overview's `gateway` line: the block's one line in the fleet panel's shape (count + the `/gateway` link via the block's `href`), hidden-when-unconfigured (the `disks` block's invariant-3 shape — an unconfigured source never renders a row, never a red one).
|
||||
4. `tests/test_api.py` — the `/api/summary` pin completed: the `gateway` block's keys pinned (task 02) **and** the overview line's rendering pinned on the served `page-index.js` (the line reads the block's keys — a rename fails a test, the `SLOT_FIELDS` device shape).
|
||||
5. `data/logs/29-task05-tabs.cjs` — the browser measurement (the `25-task03-verify.cjs` shape): eight signed-in pages × two widths (1280 / 360) — six tabs present in order, the `aria-current` on `/gateway` only, the five pre-existing tabs' `aria-current` unchanged, the sixth `#nav-status` badge present and populated on `/gateway`, the 360 px dropdown lists all six, no document-level sideways scroll (phase 18's guarantee), screenshots `data/logs/masthead-29-*.png`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the served-markup pins (the moved tab audit, the `aria-current` pin, the overview-line pin)
|
||||
- Real-browser check (invariant 10): the task-05 script is the proof — eight pages × two widths
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] six tabs on all eight signed-in pages in the pinned order, `aria-current` on `/gateway` only (pinned + measured)
|
||||
- [ ] the `gateway` job is the fifth `#nav-status` badge (measured on `/gateway`, populated from `/api/health`)
|
||||
- [ ] the overview renders the `gateway` line when configured and hides it when unconfigured (pinned)
|
||||
- [ ] the 360 px dropdown lists all six tabs; no document-level sideways scroll (measured)
|
||||
- [ ] `uv run pytest` green, `node --check` on the touched scripts, screenshots in `data/logs/`
|
||||
- [ ] no behavior change in completed work: the five pre-existing dashboards' tab row is byte-identical apart from the added tab (the served-bytes pin proves it)
|
||||
@@ -0,0 +1,31 @@
|
||||
# Task 06 — Docs, and the live verification of the rendered pipes
|
||||
|
||||
**Phase:** `29_gateway_dashboard_page` · **Source:** `TODO.md:3` — the definition of done: UI changes verified in a real browser with a screenshot against **live** data (invariant 10), and the docs updated when a page, a config row, or a fleet fact changed
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The docs carry the new page (the `AGENTS.md` table row, the `config.py` bullet's `GATEWAY_HOST`, the metric-naming paragraph's `gateway.*` gauges, the `README.md` how-to row), and the phase's proof is the read-only live pass: the dev server (1.2.0 gateway host, post-phase-28) renders `/gateway` with the real topology — the six haproxy frontends, the nginx server blocks, the stream map — and the page's live counts **match a direct `/proc/net/tcp` read** on the gateway host (the join is proven end-to-end, not just against fixtures). The pass writes its transcript and screenshots to `data/logs/`.
|
||||
|
||||
- ASSUMPTION: the verification is **read-only** (the `tests/test_live_fleet.py` rule): the pass dials the dashboard's public routes over the session, reads `/proc/net/tcp` on the gateway host via the ssh alias, and compares — it starts no workload, sends no command, writes nothing to the fleet.
|
||||
- ASSUMPTION: the count comparison is **port-bucketed, not exact-timestamped**: the pass reads the page's inbound count for port `P` and the host's ESTABLISHED-on-`P` count within one poll window, and asserts they are equal **or** differ by ≤ 1 (a connection can close between the two reads — the one-socket tolerance is the honest bound; a divergence > 1 fails the pass).
|
||||
|
||||
## Work
|
||||
|
||||
1. `AGENTS.md` — the page table row (`/gateway` — "which domains are coming in on which port, and where the traffic goes — the gateway host's nginx + haproxy configs joined to its live sockets" — source: "the same agents, `haproxy`/`nginx`/`sockets` sections of `GET /snapshot` (agent 1.2.0)"); the `config.py` bullet's `GATEWAY_HOST` line (default `gateway`, the gate property); the metric-naming paragraph's `gateway.in_conns@<port>` / `gateway.out_conns@<host>` gauge line (no rate, no verdict — the `net.*` sentence applied).
|
||||
2. `README.md` — the page's how-to row (the `GATEWAY_HOST` setup, the not-configured arm, the 1.2.0 agent requirement) in the standing page-row shape.
|
||||
3. `data/logs/29-task06-live.cjs` — the read-only live pass (the `24-task04-verify.cjs` shape): sign in, `GET /api/gateway/snapshot` (assert `configured: true`, `supported: true`, the six haproxy frontends, the nginx server blocks, the stream map's 17 entries), `GET /gateway` (the six tabs' page, the four cards rendered, the pipes' `aria-label`s present), then `ssh gateway-ops` → `cat /proc/net/tcp /proc/net/tcp6` → the port-bucketed count comparison per the assumption (inbound on each configured listen port; the `unmatched.inbound` tally against the port complement), and the `by_port` spot-check (one domain target's count equals the ESTABLISHED-on-that-port count).
|
||||
4. `data/logs/29-task06-live.out` + screenshots `data/logs/29-task06-{gateway,card-haproxy,card-stream,charts-expanded}.png` — the transcript (the assertion table, the count comparison, **no** config body or token in it — invariant 6) and the screenshots (the phase's proof, invariant 10).
|
||||
5. The 1.1.0 arm's live proof is the **dev-store** one (not a fleet rollback — a fleet arm is the operator's call): the dev store pointed at a 1.1.0-style fixture snapshot renders the "predates" arm, screenshot `data/logs/29-task06-predates.png`.
|
||||
|
||||
## Testing & Quality
|
||||
- Live (read-only, invariant 10): the task-06 pass is the phase's proof; the 1.1.0 arm is dev-store-driven
|
||||
- The hermetic suite is re-run as the gate (`uv run pytest --cov`), not for this task's logic
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `AGENTS.md`'s page table, the `config.py` bullet, and the metric-naming paragraph name `/gateway` + `GATEWAY_HOST` + the `gateway.*` gauges (a grep over both files finds each)
|
||||
- [ ] `README.md` has the page's how-to row
|
||||
- [ ] the live pass's transcript records: the six frontends, the nginx blocks, the 17 map entries, the port-bucketed count comparison (divergence ≤ 1 on every compared port), the `by_port` spot-check
|
||||
- [ ] the four screenshots + the predates-arm screenshot are in `data/logs/`, and the transcript carries no config body and no token (invariant 6, grepped)
|
||||
- [ ] `uv run pytest --cov` holds the 94% floor, `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: the nine pre-existing sources read `ok` in `./scripts/dev-server.sh status` and the five pre-existing dashboards render as before (the pass's control leg)
|
||||
Reference in New Issue
Block a user