phase: 30_gateway_connection_map

**Phase 30 final verification pass — all 6 tasks already in `complete/`, so this was the phase-completion pass.**

- **Verified (re-run, not trusted):** `/gateway` serves exactly two panels (nginx→haproxy, `GATEWAY_CARD_KEYS`); haproxy rows `.solo` straight across, nginx `:443` branched with one tick per leaf (18 live, fixture 17); every pipe computes the theme `--ok` in dark+light, animates only at count ≥1, fully static under `prefers-reduced-motion`; switch = `role="switch"`+`aria-checked`+`aria-disabled`, no listener, footer+title name the missing write path; zero non-GET requests anywhere.
- **Live read-only re-run** (`data/logs/30-final-live-verify.out`): 17 PASS — `warnings: []`, 6 haproxy frontends→own `server`, `:443` 18 map leaves incl. `default→127.0.0.1:4443`, `:4443` 6 / `:8443` 3, `:80` bare listener, no `unmatched`/`outbound`/`by_port`/`partial`, no `9120`; inbound reconciliation **page 16 = kernel 16**; `gateway` block = the 12 declared keys; no `gateway:config` alert on a clean config. `git diff --stat agent/ deploy/` empty; agents 1.2.0 (gateway) / 1.1.0 ×2, all `tls`, `bundle_rev 2`.
- **Tests/lint (exact):** `uv run pytest` → **1791 passed, 16 skipped**, exit 0 · `bash .agents/validate.sh` → **exit 0**, `--cov` **98.94%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` all page scripts, hygiene clean · `uv run pytest -m live --live` → **12 passed** · `uv pip list --outdated` empty · `./scripts/dev-server.sh status` → **10 sources ok**, `gateway` included · browser pass `30-connection-map.out` 63 PASS/0 FAIL (5 screenshots at 1440/360) · `30-final-mobile-audit.out` 371 PASS/0 FAIL/0 FINDING, exit 0, nine pages.
- **Docs fixed:** README `alerts.py` bullet omitted phase 30's keyword-only `gateway_payload` and its one `warn` config-integrity row (+suppression) — added; overview ingress line labelled a dated reading. AGENTS gap 13 corrected 391 px → **361 px** (task 05's pill took 30 px), "task 06 measured" → "phase 29 measured", coverage figure un-pinned to 98.9 %, live-suite count noted (12 today), "all five pages" severity sentence clarified for `/gateway`'s existence-only palette. Everything else reviewed in full and **verified consistent**.
- **Defect fixed:** duplicated footer-assertion block at the end of `tests/test_api.py::test_the_switch_is_built_and_explicitly_unwired` (one copy removed).
- **Notable:** `NOTES.md` is empty in the working tree — both items in it (non-mTLS agents, disappearing admin links) landed in phases 24 and 20; left cleared. Fixture/live `map`-count drift (17 vs 18) is recorded in AGENTS.md, not silently re-captured.
- **Next pending phase:** none — `30_gateway_connection_map` was the only entry in `.agents/phases/todo/`.
This commit is contained in:
2026-10-23 22:56:21 -04:00
parent fa4f38c694
commit e366f599f3
51 changed files with 5419 additions and 1519 deletions
+48 -14
View File
@@ -21,9 +21,11 @@ a unit test.
from __future__ import annotations
import asyncio
import json
import os
import sqlite3
from collections.abc import Iterator
from typing import Any
import httpx
import pytest
@@ -257,14 +259,17 @@ async def test_disks_source_charts_the_physical_drives_the_agent_reports(
async def test_the_gateway_source_joins_the_live_ingress(settings: Settings,
agent_tls: dict[str, collectors.AgentTls]) -> None:
"""Phase 29: `sources/gateway.py` over the **real** gateway agent, read-only.
"""Phase 29/30: `sources/gateway.py` over the **real** gateway agent, read-only.
The production path — the same `fetch_snapshot` the poller will run, through the stored
mTLS authority — so what fails here is a drift between the agent's sections and the join,
not a typo in a fixture. The assertions are the ones that hold whatever the traffic is:
the agent answers the three sections, every configured frontend is a pipe, and the
inbound/outbound/unmatched tallies **partition** the kernel's ESTABLISHED table, which is
what lets the page's numbers reconcile against a direct `/proc/net/tcp` read.
the agent answers the three sections, every configured frontend and `listen` port is a
trunk, and the inbound count **re-derived here** from the snapshot's own ESTABLISHED rows
equals the sum the payload prints — the identity of phase 30's simplified join, checked
against the kernel rather than against a fixture. The totals are an inequality by design
(`sum(listeners) + sum(endpoints) <= established`): a socket neither config file describes
is counted nowhere, and on this router that is most of the table.
"""
target = next((item for item in settings.host_targets if item.name == settings.gateway_host),
None)
@@ -283,16 +288,45 @@ async def test_the_gateway_source_joins_the_live_ingress(settings: Settings,
"needs 1.2.0's haproxy/nginx/sockets sections (AGENTS.md gap 11's rollout)")
assert payload["host"] == target.name
assert payload["sockets"]["established"] == len(snapshot["sockets"]["established"])
assert payload["inbound"], "no frontend on either config would leave the page empty"
for pipe in payload["inbound"]:
assert isinstance(pipe["connections"], int), "the socket table answered, so a count is known"
for group in payload["unmatched"]["groups"]:
assert group["connections"] > 0 and group["host_count"] > 1, (
"a group is only ever the unresolved fan-out of several names on one port")
attributed = sum(pipe["connections"] or 0 for pipe in payload["inbound"] + payload["outbound"])
assert attributed + payload["unmatched"]["inbound"] + payload["unmatched"]["outbound"] == \
payload["sockets"]["established"], "every socket counted exactly once"
assert all(sample.source == gateway.SOURCE for sample in gateway.samples(payload, 0.0))
assert payload["listeners"], "no frontend on either config would leave the page empty"
assert payload["connections"], "the two configs declare no route at all"
for trunk in payload["listeners"]:
assert isinstance(trunk["connections"], int), "the socket table answered, so a count is known"
# the join, re-derived from the agent's own rows rather than from the payload's arithmetic
configured = {trunk["port"] for trunk in payload["listeners"]}
assert configured, "a gateway with no configured port has nothing to reconcile"
inbound = [row for row in snapshot["sockets"]["established"]
if ((row.get("local") or {}).get("port")) in configured]
assert sum(t["connections"] for t in payload["listeners"]) == len(inbound), (
"the trunk counts are the ESTABLISHED rows whose local port a config names — no more, "
"no less")
endpoints: dict[tuple[Any, Any], Any] = {}
joinable: set[str] = set()
for route in payload["connections"]:
assert route["in"]["port"] in configured, "a route hangs off no trunk the page draws"
out = route["out"]
assert (route["connections"] is None) is (out["address"] is not True), (
f"{route['id']}: only an endpoint the config wrote as an address can be counted")
if out["address"] is True and isinstance(out["port"], int):
joinable.add(str(route["metric"]))
endpoints.setdefault((out["host"], out["port"]), route["connections"])
else:
assert route["metric"] is None, "a route nobody can count publishes no series"
live_out = sum(value for value in endpoints.values() if isinstance(value, int))
assert sum(t["connections"] for t in payload["listeners"]) + live_out \
<= payload["sockets"]["established"], "the page may under-count the kernel, never exceed it"
# and nothing outside the two config files reached the payload at all — this app's own agent
# port is the case phase 30 deleted the reporting of
text = json.dumps(payload)
assert ":9120" not in text and "@9120" not in text, "the agent's own port leaked into the map"
stored = gateway.samples(payload, 0.0)
assert all(sample.source == gateway.SOURCE for sample in stored)
assert {sample.metric for sample in stored if "out_conns" in sample.metric} <= joinable, (
"only an address endpoint publishes an outbound gauge — a domain-keyed series is the one "
"phase 30 stopped writing, because the figure it carried belonged to the port")
@pytest.mark.parametrize("engine", ["gufo", "vllm"])