phase: 30_gateway_connection_map
**Phase 30 final verification pass — all 6 tasks already in `complete/`, so this was the phase-completion pass.** - **Verified (re-run, not trusted):** `/gateway` serves exactly two panels (nginx→haproxy, `GATEWAY_CARD_KEYS`); haproxy rows `.solo` straight across, nginx `:443` branched with one tick per leaf (18 live, fixture 17); every pipe computes the theme `--ok` in dark+light, animates only at count ≥1, fully static under `prefers-reduced-motion`; switch = `role="switch"`+`aria-checked`+`aria-disabled`, no listener, footer+title name the missing write path; zero non-GET requests anywhere. - **Live read-only re-run** (`data/logs/30-final-live-verify.out`): 17 PASS — `warnings: []`, 6 haproxy frontends→own `server`, `:443` 18 map leaves incl. `default→127.0.0.1:4443`, `:4443` 6 / `:8443` 3, `:80` bare listener, no `unmatched`/`outbound`/`by_port`/`partial`, no `9120`; inbound reconciliation **page 16 = kernel 16**; `gateway` block = the 12 declared keys; no `gateway:config` alert on a clean config. `git diff --stat agent/ deploy/` empty; agents 1.2.0 (gateway) / 1.1.0 ×2, all `tls`, `bundle_rev 2`. - **Tests/lint (exact):** `uv run pytest` → **1791 passed, 16 skipped**, exit 0 · `bash .agents/validate.sh` → **exit 0**, `--cov` **98.94%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` all page scripts, hygiene clean · `uv run pytest -m live --live` → **12 passed** · `uv pip list --outdated` empty · `./scripts/dev-server.sh status` → **10 sources ok**, `gateway` included · browser pass `30-connection-map.out` 63 PASS/0 FAIL (5 screenshots at 1440/360) · `30-final-mobile-audit.out` 371 PASS/0 FAIL/0 FINDING, exit 0, nine pages. - **Docs fixed:** README `alerts.py` bullet omitted phase 30's keyword-only `gateway_payload` and its one `warn` config-integrity row (+suppression) — added; overview ingress line labelled a dated reading. AGENTS gap 13 corrected 391 px → **361 px** (task 05's pill took 30 px), "task 06 measured" → "phase 29 measured", coverage figure un-pinned to 98.9 %, live-suite count noted (12 today), "all five pages" severity sentence clarified for `/gateway`'s existence-only palette. Everything else reviewed in full and **verified consistent**. - **Defect fixed:** duplicated footer-assertion block at the end of `tests/test_api.py::test_the_switch_is_built_and_explicitly_unwired` (one copy removed). - **Notable:** `NOTES.md` is empty in the working tree — both items in it (non-mTLS agents, disappearing admin links) landed in phases 24 and 20; left cleared. Fixture/live `map`-count drift (17 vs 18) is recorded in AGENTS.md, not silently re-captured. - **Next pending phase:** none — `30_gateway_connection_map` was the only entry in `.agents/phases/todo/`.
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
# Phase 30 — The gateway connection map: two config-derived tables, joined lines, green = in the config
|
||||
|
||||
**Source:** operator request, 2026-10-23 (post-phase-29 review of `/gateway`)
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/sources/gateway.py` (the join this phase rewrites), `agent/host_agent.py` **1.2.0** (its `haproxy` / `nginx` / `sockets` sections — the *only* upstream this phase reads, and it needs **no agent change**), `tests/fixtures/gateway_haproxy.cfg` + `gateway_nginx_main.conf` + `gateway_nginx_conf_d_http.conf` + `gateway_nginx_stream_d_stream.conf` + `gateway_proc_net_tcp{,6}.txt` (the verbatim captures the graph is pinned to), `src/dashboards/api.py` (`/api/gateway/*`, the `/api/summary` `gateway` block), `src/dashboards/alerts.py` (the reachability-only rule this phase changes by decision), `src/dashboards/web/pages/gateway.html` + `static/page-gateway.js` + `static/dash.css` (the pipe vocabulary), `tests/test_gateway_source.py`, `tests/test_api.py` (the served-bytes pins this phase deliberately rewrites).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway` becomes the **connection map** it was meant to be: **two tables — nginx and haproxy** — in which every row is a connection **that exists in a config file**, drawn as a line that physically joins its inbound end to its outbound end (haproxy straight across; nginx branching out of `:443` to the services the stream `map` routes to), each line **green because the config says it exists**, each line carrying the **switch** shape from the operator's sketch (built, visible, not yet wired). The socket table is demoted to a number beside the line instead of deciding its colour or its existence; the `stream map` card and the `unmatched` card are gone; anything the two configs do not describe is **not reported at all**, and the warnings that remain are **config-integrity** problems — a backend no frontend routes to, a `proxy_pass $var` with no `map` — surfaced as a warnings panel on the page and one alert row on `/`.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Green means "in the config."** | Every configured connection draws green. Colour is an **existence state**, not a verdict: `severity.py` still has no `gateway.*` row and no payload key carries `severity`. Red is the **off** state (a route switched out of the config), styled now, unreachable until the write path exists. |
|
||||
| 2 | **Motion means "a socket is open this instant."** | The dash animates while that pipe's live count is ≥ 1 and sits still otherwise. No rolling/recent-peak derivation, no speed tied to volume. |
|
||||
| 3 | **The switch is built, not wired.** | Real `<button role="switch">` per connection, `aria-checked="true"` (in config = on), `aria-disabled="true"` (not `disabled`, which would drop it from the a11y tree), no click handler, a `title` naming why. The UPS controls are the precedent for a gated control; the write path is a **later phase**. |
|
||||
| 4 | **Live counts stay, demoted.** | The inbound trunk carries the real ESTABLISHED count + client addresses; the 24 h per-card chart stays; the header keeps tallies. A leaf endpoint written as a **name** shows `—` (never `0`, never a fake `n/a` column). |
|
||||
| 5 | **Only the unambiguous socket join survives.** | Inbound = the socket's local port is a configured listen port. Outbound = the socket's remote `(address, port)` **equals a target the config wrote as an address**. `by_port`, `unmatched.inbound`, `unmatched.outbound`, `unmatched.groups`, `partial`, `unmatched_listeners` are **deleted**. |
|
||||
| 6 | **Anything outside the two configs is ignored.** | `9120` (this app's own agent), `2222`, DHCP/mDNS/whatever else listens is neither drawn, nor counted, nor warned about. `sockets.listens` / `sockets.established` stay in the payload as the kernel's raw totals for programmatic consumers only. |
|
||||
| 7 | **The warnings list is config integrity.** | Seven kinds (`03_config_warnings.md`) plus three explicit refusals where the payload cannot tell broken from merely unreadable; all derived from config text, all actionable. Panel on `/gateway`, and **one** `warn` row in `/api/summary`'s `alerts` — which changes the "a `gateway` alert is reachability only" rule in `alerts.py` and in `AGENTS.md`. |
|
||||
|
||||
**Out of scope:** any change to `agent/host_agent.py` (1.2.0 already publishes everything needed, including `upstreams` and `maps`), any write path or config edit, any new dependency, any new sample family (the existing `gateway.in_conns@<port>` / `gateway.out_conns@<host>:<port>` names stay, so stored history keeps its meaning).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `28_gateway_agent_proxy` (complete) — agent 1.2.0's `haproxy` / `nginx` / `sockets` sections and the captured fixtures.
|
||||
- `29_gateway_dashboard_page` (complete) — the `gateway` poller job, `/api/gateway/*`, the page's chrome, the six-tab masthead. This phase **reshapes** that payload and page; it does not remove the job, the routes, the tab, or the gauges.
|
||||
- `27_mobile_tables_and_audit` (complete) — the 360 px audit script `data/logs/27-task02-mobile-audit.cjs` must still exit 0 with `/gateway`'s new rows in it.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_connection_graph.md` — `sources/gateway.py`: the config-derived graph — `listeners[]` + `connections[]` (haproxy `frontend → backend → server`, nginx vhost `server_name → proxy_pass`, stream `proxy_pass $var → map` entries, `proxy_pass <pool> → upstream` servers), with no counts yet.
|
||||
2. `02_the_join_simplified.md` — the surviving socket join (inbound by local port, outbound by exact address only), the shared-endpoint rule, the deletion of `outbound` / `by_port` / `unmatched` / `partial`, the gauges re-pointed, and `/api/summary`'s `gateway` block plus the overview's one line reshaped in the same commit.
|
||||
3. `03_config_warnings.md` — `warnings()`: the seven config-integrity kinds (and three refusals), empty on this fleet's captured configs (a pinned assertion), plus one `warn` overview row from `alerts.py`.
|
||||
4. `04_page_two_tables.md` — `gateway.html` + `page-gateway.js` + structural CSS: exactly **two** panels (nginx, haproxy), trunk → leaf rows, the warnings panel on the shared `ul.alerts` vocabulary, four header tallies, the degradation arms kept.
|
||||
5. `05_lines_and_switches.md` — `dash.css` + `page-gateway.js`: the connecting line (straight across for haproxy, branched for nginx), the green `--gate-on`/`--gate-off` existence pair, `live` motion, the switch built to the sketch and pinned as unwired. The `ok|warn|fail`-in-the-pipe-section pin is **restated**, not deleted.
|
||||
6. `06_docs_and_verify.md` — `AGENTS.md` (the `sources/gateway.py`, `api.py`, `alerts.py`, metric-naming, UI-conventions and gap-13 passages) + `README.md`, the browser pass at 1440 px and 360 px with screenshots, the mobile-audit re-run, and the read-only live verification against the real ingress.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: `tests/test_gateway_source.py` is **rewritten around the graph** — every connection the captured configs declare is present and no other; the join identity is restated as *"the inbound counts sum to the ESTABLISHED sockets whose local port a config names"* (the old `inbound + outbound + unmatched == the table` identity dies with `unmatched`); `socket.getaddrinfo` stays spied and **never called**; the six warning kinds each get a synthetic broken config and the captured configs pin **zero** warnings. `tests/test_api.py`'s gateway pins are rewritten in the same tasks that change the payload/page — including `GATEWAY_PAGE_FIELDS`, `GATEWAY_FIELDS`, the stub/answer key-parity pin, the no-verdict/no-window walk, and the colour pin (restated per decision 1).
|
||||
- Integration: the page renders from a fixture-driven dev store; the warnings panel appears only when a warning exists; the chart disclosure stays collapsed-lazy-one-request-per-card.
|
||||
- Coverage: **>90%** on new/modified Python (`sources/gateway.py`, `alerts.py`, `api.py`).
|
||||
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's) and `bash .agents/validate.sh` is green, coverage floor included.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For this phase that specifically means: `AGENTS.md`'s `sources/gateway.py` bullet (the payload keys, the join, the deleted `unmatched`/`by_port`/`partial`), its `api.py` `/api/gateway/*` paragraph and the `/api/summary` `gateway` block field list, its `alerts.py` bullet (the gateway alert is no longer reachability-only), the `/gateway` UI-conventions paragraph (two tables, joined lines, green = in the config, the unwired switch), the metric-naming paragraph, and **gap 13** (the clipped-pipe-label gap, which the new row layout either resolves or restates honestly).
|
||||
- Tasks update `README.md`/`AGENTS.md` immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `/gateway` serves and renders **exactly two** cards, `nginx` and `haproxy`; no `stream map` card, no `unmatched` card, no third table.
|
||||
- [ ] Every haproxy row draws its line **straight across** from `:<bind port>` to its `server` target; the nginx `:443` trunk **branches** to each `map` entry (16 on this fleet) and each http vhost trunk to its `proxy_pass` target — measured in the browser pass, not asserted in prose.
|
||||
- [ ] Every configured line is **green**; it animates only while its live count ≥ 1; both states still static under emulated `prefers-reduced-motion: reduce`. No `severity` key at any depth of the payload; no `severityFrom`/`severityFor` in the page script.
|
||||
- [ ] One switch per connection, `role="switch"` + `aria-checked="true"` + `aria-disabled="true"`, no click handler bound, `title` naming the missing write path — and a test that fails if it becomes clickable without a handler or stops saying it is unwired.
|
||||
- [ ] `POST` nothing: the verb-enumeration pin still finds `/api/ups/command` as the app's only upstream write, and `/gateway` dials only `/api/gateway/snapshot` + `/api/gateway/history`.
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress: `warnings == []`, every one of the 6 haproxy frontends connected to its `server`, `:443` connected to the 17 `map` entries, `:4443`/`:8443` connected to their `proxy_pass` targets, and no `unmatched` / `by_port` / `partial` key anywhere.
|
||||
- [ ] A `gateway` `warn` row appears in `/api/summary.alerts` **iff** `warnings` is non-empty, and disappears when they are; a stale/stopped `gateway` job still alerts exactly as before.
|
||||
- [ ] `uv run pytest` (incl. `--cov` ≥ 94 floor), `uv run ruff check src tests agent`, `uv run pyright`, `node --check` on the changed scripts, `bash .agents/validate.sh` — all green.
|
||||
- [ ] `data/logs/30-*.png` screenshots at 1440 px and 360 px; `data/logs/27-task02-mobile-audit.cjs` still exits 0 with `/gateway` in its sweep.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards and `/settings`/`/certs` render as before; `host:*`, `disks:*`, `llm:*`, `nut` still read `ok` in `./scripts/dev-server.sh status` (10 sources, `gateway` included).
|
||||
Reference in New Issue
Block a user