phase: 32_gateway_resolved_target_liveness
Fixture and live stream.conf are byte-identical (modulo header comment). Let me test the parser directly:
This commit is contained in:
@@ -0,0 +1,71 @@
|
||||
# Phase 32 — Names become addresses where they are actually dialled: every route gets a real live count
|
||||
|
||||
**Source:** operator request, 2026-10-23 — "The only animated line is `:443 -> 127.0.0.1:4443`." + the decision taken in that conversation: **the agent resolves the names it reads**, because the resolution nginx's `resolver 10.1.0.1` and haproxy's `default-server init-addr last,libc,none` perform happens *on the gateway*, and a laptop- or dashboard-side lookup is a different vantage answering a different question.
|
||||
**Story:** n/a
|
||||
**Context:** `agent/host_agent.py` (1.2.0 → **1.3.0**: the `haproxy`/`nginx` parsers that already produce every target name, the `sockets` reader, the `AGENT_VERSION` + `/schema` contract, the `_smart_lock`/`_smart_refreshing`/`_refresh_smart` cache-and-background-pass precedent, `--no-proxy`/`--no-smart`/`--root`), `src/dashboards/sources/gateway.py` (`_connection()`'s `address`/`joinable`/`metric` rule, `join_sockets()`, `endpoint_of()`, `samples()`), `src/dashboards/collectors.py` (`ADDON_ERROR_PREFIXES` — the cross-module contract AGENTS.md says a new add-on section must satisfy **in the same commit**), `scripts/capture-gateway-config.sh` (the six fixtures + its "no privilege, no copy tool, a bad read is never a fixture" rules + the RFC 5737/3849 address scrub), `tests/test_gateway_source.py`, `tests/test_agent.py`, `tests/fixtures/gateway_*`, `data/logs/verify-28-final.py` (the side-by-side additivity control this phase copies).
|
||||
|
||||
## Objective
|
||||
|
||||
`agent/host_agent.py` **1.3.0** resolves every target name the two proxy configs declare — with `socket.getaddrinfo`, i.e. the same libc resolution haproxy itself uses, on the machine that opens the connections — and publishes the address set, the vantage and the timestamp per name. `sources/gateway.py` then joins the kernel's ESTABLISHED sockets through those addresses, so `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443` carries a **number** and its line **moves**, instead of a permanent em dash on 31 of 33 routes.
|
||||
|
||||
This is a measurement with provenance, not a guess with better marketing: the payload says *which* addresses, *how* they were obtained (`via: "libc"`), and *when*; a name that does not resolve right now publishes no addresses, gets no gauge, and prints the dash with the reason. `sources/gateway.py` still never calls a resolver — that spy pin survives this phase on purpose, because a dashboard resolving through its own network would be measuring a different box's DNS than the one routing the traffic.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Resolution happens on the gateway, in the agent.** | `host_agent.py` gains a `resolver` section. `sources/gateway.py` keeps its `socket.getaddrinfo` spy pin (fails if the module ever resolves), and `tests/test_gateway_source.py` gains the matching assertion that the only resolver in the system is the agent's. The dashboard's own network vantage is not the routing vantage; a lookup from here would be a fact about *this* box's `resolv.conf`. |
|
||||
| 2 | **libc, and the vantage is printed.** | `via: "libc"` on every entry (what `getaddrinfo` is), plus the nginx config's own `resolver` directive captured verbatim as **`resolver.config_directive`** (`"10.1.0.1"` on this fleet) so the page can name both and an operator can see a divergence. It lives inside the new section rather than inside `nginx` so that `nginx` stays byte-stable and the phase's additivity control remains a strict top-level key comparison with no exception list. The agent does **not** hand-roll UDP DNS queries to that resolver: stdlib only, no new dependency, and a divergence is a config finding for the operator rather than a second resolver implementation to maintain. |
|
||||
| 3 | **Only names the configs wrote are ever resolved.** | The name set is derived from the parsed `haproxy` `server` targets and the parsed `nginx` `proxy_pass`/`upstream`/`map` targets — nothing else, ever. Not a name from `/proc`, not a client address, not anything a request supplied. Pinned by asserting the spied `getaddrinfo` calls are exactly the config's target set. |
|
||||
| 4 | **Bounded, cached, never on the request path after the first pass.** | `RESOLVE_MAX_NAMES = 64` (the excess is **one** line naming the count — the include/smart budget shape), `RESOLVE_CACHE_SECONDS = 300`, one single-flight background refresher (the `_smart_refreshing` idiom), a per-pass budget so a black-holing resolver cannot hold `/snapshot`. A wedged lookup must never blow `HOST_TIMEOUT_SECONDS` and turn `/system` red over an add-on it never draws — the exact lesson SMART learned. |
|
||||
| 5 | **Three-way `available`, and `--no-resolve` removes the key in silence.** | `true` resolved what it was asked, `false` no proxy sections to read from, `null` the pass could not run — each non-`true` arm exactly one `resolver:` line (the pinned-wording contract shared with `smartctl:`/`haproxy:`/`nginx:`/`sockets:`). `--no-resolve` is `--no-smart`'s precedent: the key is **absent** ("not collected"), which stays distinguishable from "collected and empty". |
|
||||
| 6 | **A resolved name is joinable; an unresolved one is not.** | `_connection()`'s `metric`/`joinable` test becomes `address is True or resolved-address-set is non-empty`. Name-keyed gauges (`gateway.out_conns@<host>:<port>`) come back **because they now have measurements** — the AGENTS.md passage that retired them said a name-keyed series "would be a chart of a number that is always `null`", and this phase is the reason that stopped being true. The port stays in the name (`:25565`/`:25566` on one host are two services). |
|
||||
| 7 | **The shared-endpoint rule now has a real case and stays exactly as written.** | Two names resolving to the same `(address, port)` are one count **repeated** on both lines with `shared: true` — never halved, never attributed to one of them. `samples()` keeps de-duplicating by metric, and `live_out` in `/api/summary` keeps summing **distinct endpoints** (`api._endpoint_values()`). |
|
||||
| 8 | **Nothing moves that the operator did not ask about.** | `warnings()` is unchanged (resolution is not a config-integrity question); `/api/summary`'s `gateway` block gains **no** thirteenth key; `listeners[]` and the inbound join are untouched; the switch stays unwired and unwire-able (phase 33), and `routeInConfig()` keeps returning `true`. |
|
||||
|
||||
**Out of scope:** any config write, any new sample *family* (only the meaning of `gateway.out_conns@…` widens), any change to `/api/health` or `/api/config`, any `severity` (a count is not graded), the enable/disable path (phase 33), and re-bundling or re-issuing certificates (the agent moves in place on its existing bundle, exactly as phase 28 moved the gateway 1.1.0 → 1.2.0).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `31_gateway_route_readability` (todo, **must complete first**) — the row that reads inbound → counterpart and the green end-to-end route are what a live count paints. Its payload-purity guarantee (`sources/gateway.py` untouched) is what makes this phase's rewrite land cleanly.
|
||||
- `30_gateway_connection_map` (complete) — `connections[]`, the join, `endpoint_of()`, `shared`, the gauge names, `warnings()`.
|
||||
- `28_gateway_agent_proxy` (complete) — the parsers whose target set this phase resolves, the fixture capture script and its scrub rule, and `tests/test_agent.py`'s fake-tree `--root` machinery the resolver tests reuse.
|
||||
- `24_dev_mtls_rollout` / the fleet record — the rollout is phase 28 task 05's procedure on one host (`gateway`), on its existing bundle, with `systemctl restart` and a read-only probe first.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_agent_resolves_the_names_it_reads.md` — agent 1.3.0: the `resolver` section (`available`, `via`, `config_directive`, entries), `RESOLVE_MAX_NAMES`/`RESOLVE_CACHE_SECONDS`/pass budget, the single-flight background pass, three-way `available`, `--no-resolve`, `/schema`, `AGENT_VERSION`.
|
||||
2. `02_resolution_ground_truth.md` — `capture-gateway-config.sh` records the resolver document and the two config files' `resolver` line; the RFC-5737/3849 scrub extended to it; the 1.2.0-vs-1.3.0 **additivity control** (`data/logs/verify-32-additive.py`, copied from `verify-28-final.py`).
|
||||
3. `03_join_through_the_address_the_name_became.md` — `sources/gateway.py`: `out.resolved`/`via`/`resolved_ts`, the widened `joinable`/`metric` rule, the address-set match, `shared` unchanged, `null` refusals, and `collectors.ADDON_ERROR_PREFIXES` gaining `resolver:` **in this commit**.
|
||||
4. `04_the_page_says_which_unknown_it_is.md` — `page-gateway.js` + `gateway.html`: the dash splits into "the name did not resolve in this reading" vs "resolved and nothing is open", the `libc`/timestamp provenance on `title`, footer paragraph 2 rewritten, tallies unchanged.
|
||||
5. `05_roll_the_gateway_to_1_3_0.md` — read-only probe first, then the phase-28 procedure on `gateway` (re-stage, install, restart, verify both ways), the live join numbers recorded, and the fleet record updated with the new version spread.
|
||||
6. `06_docs_and_live_verify.md` — `AGENTS.md` (the agent bullet, `sources/gateway.py`, `collectors.py`, the metric-naming passage, the `/gateway` UI paragraph, the fleet record, gap 13) + `README.md`, the browser pass with screenshots, the mobile-audit re-run, the full gate.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **Hermetic by construction.** No test dials real DNS: `tests/test_agent.py` monkeypatches `socket.getaddrinfo` and asserts (a) it is called with **exactly** the parsed target names and ports and nothing else, (b) the cache means a second `/snapshot` costs zero lookups, (c) a pass over `RESOLVE_MAX_NAMES` reports one line and keeps `available: true`, (d) a raising/timeout-simulating resolver yields `status`-bearing entries and never fails `/snapshot` or `/health`.
|
||||
- **Real captures for shape.** `tests/fixtures/gateway_resolver.json` (documentation addresses, verbatim names) plus the existing six fixtures drive `tests/test_gateway_source.py`: every captured name route gets a count when the recorded `/proc/net/tcp{,6}` holds a socket to its recorded address; `minecraft.reeselink.com:25565` and `:25566` stay two distinct gauges; two names → one endpoint is one count twice with `shared: true`; an entry with no addresses publishes `connections: null`, **no** metric, and no sample.
|
||||
- **The spy pins are restated, not relaxed.** `socket.getaddrinfo` stays forbidden in `sources/gateway.py` (spy) and `0` becomes "no sample", never "an empty chart" — a pipe whose name resolved to nothing publishes no point.
|
||||
- **Live (`-m live --live`)**: one new test reads the real gateway and asserts every `map` entry whose `resolver` status is `ok` carries a finite count **or** `0`, and that none of them is `null` — the operator's complaint, pinned against the box. Skips when `GATEWAY_HOST` is not a target.
|
||||
- Coverage **>90%** on `agent/host_agent.py`, `sources/gateway.py`, `collectors.py`; the whole suite passes 100% and `bash .agents/validate.sh` is green, coverage floor included.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase, fixing every inconsistency **including pre-existing ones**. Specifically: AGENTS.md's agent bullet gains the `resolver` section (and its `--no-resolve`, its bounds, its three-way `available`), its `sources/gateway.py` bullet's join paragraph is rewritten (the join now has two address sources: written-as-address and resolved), the **"no name-keyed `gateway.out_conns@…` name is ever inserted"** sentence in the metric-naming passage is replaced with the widened rule and *why it changed*, the `collectors.ADDON_ERROR_PREFIXES` list gains `resolver` (the "prefix in the same commit" rule), the `/gateway` UI paragraph's dash sentence splits in two, and the **fleet record** moves `gateway` to 1.3.0 with the deskwork/driveripper spread and what the spread costs.
|
||||
- `README.md`: the `/gateway` table row and the gateway prose now describe counted name routes, the resolution vantage, and the capture script's new outputs.
|
||||
- Tasks update both files immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `GET /schema` on the gateway agent answers `agent_version` **1.3.0** with a `resolver` block (its `available` flag named) and `--no-resolve` listed; `tests/test_agent.py` pins the constant and the schema text.
|
||||
- [ ] `GET /api/gateway/snapshot` on the live ingress: **0** `connections[]` rows with `connections: null` among those whose target resolved; `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443` carries a finite count; `socket.getaddrinfo` is still never called anywhere under `src/dashboards/` (spied) and every address in `out.resolved` names `via: "libc"` with a `resolved_ts`.
|
||||
- [ ] Two names resolving to one `(address, port)` draw one count repeated with `shared: true` and **one** stored sample; `/api/summary`'s `gateway.live_out` still equals the sum over **distinct** endpoints (pinned before and after).
|
||||
- [ ] A stubbed `nxdomain` for one name publishes `out.resolved: null`, no `metric`, no sample, and the page prints the "did not resolve in this reading" sentence — never `0`, never a silent blank.
|
||||
- [ ] `--no-resolve` on a loopback agent removes the `resolver` key entirely and `/api/gateway/snapshot` behaves exactly as phase 30/31 (names uncounted, `supported: true`); a 1.2.0 agent on any host behaves identically (the spread is harmless — pinned by a fixture without a `resolver` key).
|
||||
- [ ] `/system` is not `degraded` on a host whose agent reports `resolver: nxdomain …` — the `ADDON_ERROR_PREFIXES` pin (invariant 3), asserted both ways.
|
||||
- [ ] `data/logs/verify-32-additive.py` proves the 1.3.0 document equals the 1.2.0 document key-for-key **plus** `resolver`, and 1.3.0 with `--no-resolve` equals it exactly.
|
||||
- [ ] `uv run pytest` (incl. `--cov` ≥ 94), `uv run pytest -m live --live`, `uv run ruff check src tests agent`, `uv run pyright`, `node --check`, `bash .agents/validate.sh` — all green; `uv pip list --outdated` empty (no dependency moved, and this phase must not add one).
|
||||
- [ ] `data/logs/32-*.png` at 1440 px and 360 px (counted routes, an unresolved arm, both themes) + `data/logs/32-*.out`; `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING.
|
||||
- [ ] `./scripts/dev-server.sh status` lists all ten sources `ok`, `gateway` included, with `host:gateway`/`disks:gateway` unaffected by their host's agent upgrade.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs`, `/login`, the warnings panel, the unwired switch and every non-gateway source render and behave as phase 31 left them.
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Task 01 — The agent resolves the names its own configs declare, off the request path
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** operator decision 2026-10-23 — "the agent resolves the names it reads"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`agent/host_agent.py` goes **1.2.0 → 1.3.0** and gains one additive top-level section, `resolver`: for every `(name, port)` the two parsed proxy configs write as a target, the libc address set, its status, the vantage and a timestamp — obtained on the machine that dials, bounded, cached, and refreshed on a background thread so a slow or black-holing resolver can never hold `/snapshot` open. Everything the agent published before is byte-identical; `--no-resolve` removes the key in silence.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py`
|
||||
- Constants beside the proxy section's bounds, each with a comment naming what it protects:
|
||||
```python
|
||||
AGENT_VERSION = "1.3.0"
|
||||
RESOLVE_MAX_NAMES = 64 # a gateway's config-declared targets, with room to grow
|
||||
RESOLVE_CACHE_SECONDS = 300.0 # a LAN target moves when the operator moves it, not every poll
|
||||
RESOLVE_PASS_BUDGET_SECONDS = 3.0 # a black-holing resolver must not hold /snapshot open
|
||||
RESOLVE_VIA = "libc" # getaddrinfo, i.e. the resolution haproxy's libc path performs
|
||||
```
|
||||
- **New pure helper** `target_names(haproxy: Mapping | None, nginx: Mapping | None) -> list[tuple[str, int | None]]`
|
||||
Derived **only** from the parsed sections: haproxy `backends[].servers[].target` (in config order), then nginx `upstreams` members, `server`-block `proxy_pass` targets, and `maps[].entries[].target`, deduplicated keeping first-seen order. An IP literal is kept as a target with `addresses` set without any lookup (it is already an address, and the lookup would be a round trip to say so) and marked `status: "literal"`. A name from `/proc`, from a request, or from anywhere else must never enter this list — that is decision 3 and it is pinned by asserting the spied call set equals this function's output.
|
||||
- **The pass, following the SMART idiom exactly** (`_smart_lock` / `_smart_cache` / `_smart_refreshing` / `_refresh_smart`):
|
||||
`_resolver_lock`, `_resolver_cache: dict[str, dict[str, Any]]` keyed on `f"{name}:{port}"` (a name on two ports is two targets — `minecraft.reeselink.com` on `25565` **and** `25566` is on this fleet today), `_resolver_refreshing: bool`, `_resolver_errors: list[str]`.
|
||||
```python
|
||||
def resolver_pass(self, targets) -> tuple[dict[str, dict[str, Any]], list[str]]:
|
||||
"""One bounded libc lookup per declared target. Never raises out of this function."""
|
||||
```
|
||||
For each target: `socket.getaddrinfo(name, port, proto=socket.SOCK_STREAM)`, collect the unique `sockaddr[0]` values, keep both families, sort address-family-stable (v4 ascending then v6 ascending) so the payload is deterministic across polls. `socket.gaierror` with `EAI_NONAME`/`EAI_NODATA` → `status: "nxdomain"`, any other `gaierror`/`OSError` → `status: "error"`, both with `addresses: []`. After each lookup compare elapsed against `RESOLVE_PASS_BUDGET_SECONDS`; on exhaustion stop and append **one** line — `resolver: pass budget 3.0s exhausted, 41 of 58 targets looked up` — mirroring the smart pass-budget wording the agent already prints (phase 16 pins that family of tests; match its shape so the next reader has one idiom, not two).
|
||||
- Entry shape (and the whole section):
|
||||
```python
|
||||
"resolver": {
|
||||
"available": True | False | None,
|
||||
"via": "libc",
|
||||
"config_directive": "10.1.0.1" | None, # nginx's own `resolver` line, verbatim arguments
|
||||
"ts": <wall clock of the pass>,
|
||||
"entries": [{"name": …, "port": …, "status": "ok|nxdomain|error|literal",
|
||||
"addresses": […], "ts": <per-entry, when it was answered>}],
|
||||
"errors": [ …at most one line per arm… ],
|
||||
}
|
||||
```
|
||||
`config_directive` lives **here** rather than inside the `nginx` section on purpose: it keeps the `nginx` document byte-stable so phase 32's additivity control stays a strict top-level key comparison instead of an exception list. Read it from the nginx config text the agent already has (`resolver` directive, arguments joined by a space, `None` when the file has none) — no second file read.
|
||||
- `available` is the three-way answer the other add-ons use: `true` the pass ran, `false` there is nothing to resolve because the proxy sections were not read, `null` a pass that could not run — and **exactly one `resolver:` line per non-`true` arm**, the missing/no-proxy case saying **not found** (the pinned cross-module wording the dashboard reads).
|
||||
- Cache policy: the **first** pass runs inline (the SMART precedent, bounded by the budget); an expired cache afterwards refreshes on a single-flight daemon `threading.Thread` while the request is served the previous result; a failed refresh keeps the previous entries and retries on the next expiry instead of hammering DNS. `resolver.entries[].ts` is always the time that entry was actually answered, so a served-from-cache entry cannot pretend to be current.
|
||||
- `--no-resolve` (argparse, no env default — `--no-smart`/`--no-proxy` precedent): the key is **absent** from `snapshot()` ("not collected", which stays distinguishable from "collected and empty"), silently, and `--once` prints the same absence. `--no-proxy` also removes it (nothing to resolve on a box that collects no proxy) — say so in one comment.
|
||||
- `AGENT_VERSION` bump, the module docstring's `/schema` block gains the `resolver` section with its `available` flag and `--no-resolve`, and the status page's path list is unchanged (this task adds no path — task 02 of phase 33 is what adds a POST).
|
||||
2. `tests/test_agent.py`
|
||||
- `test_the_agent_version_is_1_3_0_and_the_schema_names_the_resolver_section` (the constant + the `/schema` text names `resolver`, its `available`, and `--no-resolve`).
|
||||
- `test_only_the_targets_the_configs_declare_are_resolved` — monkeypatch `socket.getaddrinfo`, run a snapshot against the fake-tree configs, assert the `(name, port)` pairs it was called with are **exactly** `target_names(...)`' output for the same tree, in config order, with no duplicates, and that no call carries an address from `/proc/net/tcp`.
|
||||
- `test_the_resolver_cache_means_one_lookup_per_target` — two snapshots, one call set; expiry triggers a background refresh and the first of the two responses is served from cache with its original per-entry `ts`.
|
||||
- `test_a_slow_resolver_cannot_hold_the_snapshot_open` — a `getaddrinfo` stub that sleeps: the pass stops at the budget, prints the one budget line naming the counted and total figure, `available` stays `true`, `/snapshot` and `/health` still answer.
|
||||
- `test_nxdomain_is_a_status_and_never_a_failure` — one raising name yields `status: "nxdomain"`, `addresses: []`, other entries intact, snapshot `ok`.
|
||||
- `test_no_resolve_removes_the_key_in_silence` and `test_a_host_with_no_proxy_sections_says_so_once`.
|
||||
- `test_the_resolver_lines_never_fail_the_snapshot` — every arm, plus the existing "no section can fail `/snapshot`" family extended to cover the new one.
|
||||
- An IP-literal target is reported `status: "literal"` with its address and **no** lookup.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; hermetic by monkeypatching `socket.getaddrinfo` — **no test resolves a real name**.
|
||||
- Coverage **>90%** on the new agent code; `uv run ruff check agent` + `uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Run against a fake tree built the way `tests/test_agent.py::nginx_tree()` builds one (the captured `gateway_nginx_main.conf` + `conf.d` + `stream.d` under `<tmp>/etc/nginx/`, with `--haproxy-config tests/fixtures/gateway_haproxy.cfg`): `python3 agent/host_agent.py --once --no-smart --root <tmp> --haproxy-config tests/fixtures/gateway_haproxy.cfg --nginx-config <tmp>/etc/nginx/nginx.conf` prints a `resolver` block with `available`, `via: "libc"`, `config_directive: "10.1.0.1"` and one entry per declared target — and prints **no** `resolver` key with `--no-resolve` added.
|
||||
- [ ] `grep -n 'AGENT_VERSION = ' agent/host_agent.py` reads `1.3.0` and the `/schema` docstring names `resolver`.
|
||||
- [ ] Two consecutive `--once` runs with the stubbed resolver perform one lookup set, not two.
|
||||
- [ ] `uv run pytest -k agent` **100% passing**; the whole suite green; coverage not down.
|
||||
- [ ] README.md/AGENTS.md updated for the new agent section and flag (this task's own user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: every pre-1.3.0 section is byte-identical for the same `--root` (formally proven in task 02's control).
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 02 — Ground truth for the resolution, and a control that proves 1.3.0 is only additive
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md invariant 7 ("verify against the deployed upstream; never guess a metric name") + the phase-28 additivity precedent
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Two things this phase must be able to point at: a **captured** resolution document from the real gateway, recorded through the same no-privilege, never-a-fixture-on-a-bad-read discipline `scripts/capture-gateway-config.sh` already keeps; and a **control** that proves agent 1.3.0's snapshot equals 1.2.0's key-for-key plus `resolver` — so the upgrade of a live ingress is provably additive rather than "the page still looks fine".
|
||||
|
||||
## Work
|
||||
|
||||
1. `scripts/capture-gateway-config.sh`
|
||||
- After the existing config captures, derive the target set **from the repo's own parser**, never by re-implementing one:
|
||||
```python
|
||||
# The name list is what the agent will ask for, from the configs we just captured — one
|
||||
# source of truth. If `target_names()` changes, this capture follows it, and a fixture
|
||||
# that lists a name the agent no longer resolves is a fixture that fails a test.
|
||||
from agent.host_agent import target_names
|
||||
```
|
||||
(`bash` calls `uv run python - <<'PY'`, same interpreter the tests use.)
|
||||
- One **world-readable, unprivileged** read per name, over the same `$GATEWAY_SSH` alias, which is libc resolution *on the box that dials* — the exact vantage `via: "libc"` claims:
|
||||
`ssh "$GATEWAY_SSH" getent ahostsv4 <name>` then `getent ahostsv6 <name>`. No `sudo`, no `dig`/`host` assumption, no copy tool.
|
||||
- `config_directive` is read from the **already captured** nginx text (the `resolver` directive's arguments) — no extra round trip, and the fixture then matches what the agent will read out of the same bytes.
|
||||
- Write `tests/fixtures/gateway_resolver.json` in the agent's own entry shape, sorted in the agent's order, with `ts` set to a fixed capture constant so the fixture is byte-stable, and:
|
||||
- **names verbatim** — the domains are the ingress map, exactly as the config fixtures keep them;
|
||||
- **addresses scrubbed by the socket table's rule**: keep RFC 1918 / ULA / loopback, rewrite any address `ipaddress` calls globally routable into documentation space (RFC 5737 / RFC 3849), first-seen order, same spelling, **idempotent** — invariant 6, the same reason `/proc/net/tcp` is scrubbed (a global address is not this repo to publish);
|
||||
- a `nxdomain` is recorded as a **status**, not dropped.
|
||||
- Failure discipline, unchanged in spirit and pinned: a failed `ssh`, a missing `getent`, or **every** name failing → exit 1 having written nothing (a broken vantage is not a fixture); a half-written file is removed. One name failing among successes is a fact and is kept.
|
||||
- `--help`/usage comment updated in the same edit (the script's header is the documentation `README.md` points at).
|
||||
2. `tests/fixtures/gateway_resolver.json` — the capture itself, committed.
|
||||
3. `data/logs/verify-32-additive.py` (copy the shape of `data/logs/verify-28-final.py`)
|
||||
- `git stash`-safe pattern: load `agent/host_agent.py` from `git show HEAD:agent/host_agent.py` into a temp file and import it side by side with the working tree's, via `importlib.util.spec_from_file_location` under distinct module names.
|
||||
- Build one fake tree (`nginx_tree`-shaped) plus the haproxy capture, monkeypatch `socket.getaddrinfo` in **both** modules to the same deterministic stub, call `snapshot()` on both with identical arguments, and demand:
|
||||
- `new == old | {"resolver": …}` — every pre-1.3.0 key equal, `resolver` the only addition;
|
||||
- `new --no-resolve == old` **exactly**;
|
||||
- no key removed, renamed, or reordered-meaning-changed anywhere, printed as a diff on failure.
|
||||
- `config_directive` lives inside `resolver` (not inside `nginx`) precisely so this comparison stays a strict top-level key test with no exception list — record that reason in the script's header so nobody later "tidies" the field into the nginx section and weakens the control.
|
||||
- Transcript: `data/logs/32-additive.out`, exit non-zero on any difference.
|
||||
4. `tests/test_deploy_scripts.py` (or the capture-script test module that already covers this script)
|
||||
- Extend the existing pins: the script names `getent` and no `sudo`; it derives the name list by importing `target_names` (so the capture cannot out-ask the agent); a forced-`ssh`-failure run exits 1 and leaves the existing fixture byte-identical; a re-run over the same inputs is idempotent; the new fixture carries no globally routable address (`test_the_gateway_fixtures_carry_no_routable_endpoint`-style walk, extended to JSON).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; `tests/test_agent.py` and `tests/test_gateway_source.py` both consume `gateway_resolver.json` (task 03 wires the second).
|
||||
- Coverage: shell + fixtures, so the gate is the existing script pins plus no Python regression; `bash -n scripts/capture-gateway-config.sh` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `./scripts/capture-gateway-config.sh` run on a box with the `gateway-ops` key writes `tests/fixtures/gateway_resolver.json` and exits 0; the file has one entry per name `target_names()` derives from the captured configs, `via: "libc"`, `config_directive: "10.1.0.1"`, and no globally routable address anywhere in it.
|
||||
- [ ] Killing the alias (or `GATEWAY_SSH=nope`) exits non-zero with every existing fixture byte-identical (`sha256sum` before/after, printed).
|
||||
- [ ] `uv run python data/logs/verify-32-additive.py` exits 0 and its transcript shows the two documents equal plus `resolver`, and the `--no-resolve` arm equal exactly.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md's capture section and AGENTS.md's invariant-7/fixture prose name the new capture output (this task's user-visible change) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: the six existing fixtures are unchanged unless the live router's config genuinely moved, in which case that drift is recorded in the transcript and `tests/test_gateway_source.py`'s tallies are re-counted in the same commit (never edited to match a live read).
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Task 03 — The join counts a name route through the address the gateway itself dials
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** operator complaint — "the only animated line is `:443 -> 127.0.0.1:4443`"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`sources/gateway.py` reads the agent's `resolver` section and widens the one rule that decided a route could never be counted: an endpoint is joinable when the config wrote an **address** *or* when the gateway's own libc resolution published one for the name. `connections[]` grows three provenance fields, name-keyed gauges start carrying measurements, and the two rules that made phase 30 honest stay exactly where they were — the dashboard still never resolves, and a pipe nobody measured still publishes **no** sample.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- New pure reader `resolution_index(snapshot) -> dict[tuple[str, int | None], Mapping[str, Any]]`, keyed on `(name, port)` from `snapshot["resolver"]["entries"]`, ignoring entries with no `addresses` (a `nxdomain` contributes an empty set, never a `None` address). Absent `resolver` section (a 1.2.0 agent, or `--no-resolve`) → `{}`, which is exactly today's behaviour and must be pinned with a fixture that has no `resolver` key at all.
|
||||
- New pure pass `apply_resolution(connections, index, via, ts)` — called between `config_connections()` and the socket join — writing on each connection's `out`:
|
||||
- `resolved`: the address list, or `null` when nothing published one (never `[]`, so the page's "did not resolve" arm is one `is null` test rather than a length check that confuses empty with unknown);
|
||||
- `via`: `"libc"` (the agent's string, copied, never invented here) and `resolved_ts`: that entry's own timestamp;
|
||||
- `joinable`-driven `metric`, widened:
|
||||
```python
|
||||
# A line is countable when the kernel can name its far end: either the config wrote an
|
||||
# address, or the gateway's own libc resolution — the one that opens this connection —
|
||||
# published one. Phase 30 retired name-keyed gauges because they could only ever be
|
||||
# empty; this is the fact that made that untrue. The NAME keeps the config's spelling so
|
||||
# the stored series is the same one phase 29/30 would have written.
|
||||
```
|
||||
`metric` stays `gateway.out_conns@<host>:<port>` with the **config's** host spelling (history continuity, and `:25565`/`:25566` stay two gauges), and stays `null` for a line whose address is unknown.
|
||||
- `endpoint_of(connection)` → returns a **list** of `(address, port)` keys: the written address, or every resolved address. `join_sockets()` builds its `endpoints` map by pointing **every** key at the **same** counter row, so a multi-homed name is one endpoint with several spellings in the kernel's table, and two lines reaching one endpoint still share one counter (`shared: true`, unchanged).
|
||||
- **Address comparison is normalised, spelling is not.** Match with `ipaddress.ip_address(...)` on both sides so `/proc`'s `inet_ntop` output, an IPv4-mapped `::ffff:10.1.2.64`, and a resolver's `10.1.2.64` are the same address; the **payload** keeps publishing whatever the config and the agent wrote, verbatim. No string equality anywhere in the match path — that is the bug class where a route resolves and still never counts.
|
||||
- `_skeleton()` gains `"resolver": {"available": None, "via": None, "entries": []}` so `parse` and `stub` keep one key set (the parity pin), and `parse()` fills `available`/`via` from the section while `entries` carries the entries the index was built from — the page reads provenance from the document it was drawn from, not from a second call.
|
||||
- `samples()`: unchanged logic, now wider in effect (a resolved name route emits its gauge). A `null` count still publishes nothing. De-duplication by metric stays, so a shared endpoint costs one insert.
|
||||
- `supported()` — a 1.3.0 snapshot must answer `supported: true` (its three proxy sections are what the check reads); add the assertion rather than assuming it.
|
||||
- `warnings()` untouched. The `/api/summary` block grows **no** field.
|
||||
2. `src/dashboards/collectors.py` — **the same commit**, per AGENTS.md's cross-module rule for a new add-on section:
|
||||
- `ADDON_ERROR_PREFIXES` gains `resolver`, so a `resolver: … nxdomain …` or `resolver: pass budget … exhausted` line is an *unconfigured* report on `/system`, not `degraded`, and never an overview WARN. Invariant 3, both directions pinned: a host with that line stays out of `alerts`, and a `nginx:`/`smart:` line still behaves as before.
|
||||
3. `tests/test_gateway_source.py`
|
||||
- `test_a_name_route_counts_through_the_address_the_gateway_published` — captured configs + `gateway_resolver.json` + the recorded `/proc/net/tcp{,6}`: a socket whose remote is `10.1.2.64:443` lands on `brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443`'s line, with `out.via == "libc"` and a `resolved_ts`.
|
||||
- `test_the_match_is_an_address_not_a_string` — the same address in `::ffff:` / leading-zero / uppercase spellings on either side still matches; the payload keeps verbatim spelling.
|
||||
- `test_two_names_one_endpoint_is_one_count_shared` and `test_two_ports_on_one_host_are_two_gauges` (`minecraft` `25565`/`25566`).
|
||||
- `test_an_unresolved_name_publishes_no_metric_and_no_sample` — `connections: null`, `metric: null`, zero samples, never `0`.
|
||||
- `test_the_dashboard_still_never_resolves` — `socket.getaddrinfo` spied and asserted uncalled through the whole `parse()` path, **and** a new comment naming where the resolver lives and why (vantage). This pin is restated, not relaxed.
|
||||
- `test_a_snapshot_without_a_resolver_section_is_phase_30_verbatim` — the 1.2.0 fixture: identical payload to today's except `resolver`/`out.resolved`/`via`/`resolved_ts` being absent/`null`, and `supported` still `true`.
|
||||
- The counted-once identity and the `sum(listeners) + sum(endpoints) <= established` inequality are restated with the widened match set, including the `127.0.0.1:4443` loopback hop counting once inbound **and** once outbound on two different pipes.
|
||||
- `tests/test_api.py`: the "no name-keyed `gateway.out_conns@…` series is ever inserted" pin is **rewritten** to the new rule (an endpoint whose address is known — written or resolved — gets its gauge; an unresolved name never does), and `test_the_summary_gateway_block_is_the_payloads_sums_not_the_pages_arithmetic` gains the resolved case so `live_out` still sums **distinct endpoints** (`api._endpoint_values()`), never lines.
|
||||
- `tests/test_live_fleet.py` (marked `live`): against the real ingress, every `connections[]` row whose resolver status is `ok` has a finite `connections` (a number, `0` allowed) — never `null`. Skips when `GATEWAY_HOST` is not a configured target.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit + fixture as above; `tests/test_gateway_source.py` keeps running the **real** `host_agent` parsers over the captures so the join is compared with the agent's document, not with a copy of its field names.
|
||||
- Coverage **>90%** on `sources/gateway.py` and `collectors.py`; whole suite 100%; `uv run ruff check src tests agent` and `uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Against the captures: the number of `connections[]` rows with a non-null `connections` rises from 2 (the two loopback targets) to every row whose name resolved in `gateway_resolver.json`, and `tests/test_gateway_source.py` prints that count so the change is visible rather than implied.
|
||||
- [ ] `grep -rn "getaddrinfo" src/dashboards/` finds nothing; `grep -n "resolver" src/dashboards/collectors.py` shows `ADDON_ERROR_PREFIXES` carrying it.
|
||||
- [ ] `GET /api/gateway/snapshot` on a fixture-driven store shows `out: {…, address: false, resolved: ["10.1.2.64"], via: "libc", resolved_ts: …, …}` with a finite `connections` on that row, and `null`/`null` on an injected `nxdomain` row.
|
||||
- [ ] The stored series for one shared endpoint is **one** insert per poll (asserted by counting rows), and `/api/summary`'s `gateway.live_out` equals the sum over distinct endpoints.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md updated for the widened join rule and the metric-naming passage in the same commit (the "no name-keyed series" sentence is replaced, with the reason) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `warnings()`, `/api/summary`'s key set, the inbound join, `/system`'s `degraded` for the other add-on prefixes, and every non-gateway source are untouched.
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Task 04 — The dash now says *which* unknown it is, and the resolution rides on the tooltip
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** phase-31 decision 8 ("the `—` keeps its meaning and loses its mystery") meeting this phase's new fact
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Two facts that used to share one glyph separate: **"the gateway could not resolve this name in this reading"** and **"resolved, and nothing is open on it right now"**. The first keeps the em dash with a sentence that names the cause; the second prints its `0`. Every counted line carries where its address came from (`libc`) and when, on `title` and in the `aria-label` — provenance the operator needs to explain a number, kept off the ink because the page's copy rule is *would you act differently without it?*
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- Replace the single `NO_COUNT_WHY` with the two sentences the payload can now distinguish, chosen from data and never guessed:
|
||||
```js
|
||||
const UNRESOLVED_WHY = 'no number: the gateway could not resolve this name in this reading';
|
||||
const IDLE_TITLE = (n) => `${n} open connection(s) on this endpoint`
|
||||
+ (resolved ? ` · address from ${via}, resolved ${age}` : '');
|
||||
```
|
||||
`age` comes from `Dash.duration(Dash.ageSeconds(conn.out.resolved_ts))`; `null`/absent prints nothing rather than `NaN`, and `via` is the payload's word (`libc`) copied, not paraphrased.
|
||||
- `leafAria(conn, on)`: the unknown arm becomes `'connections not attributed: the gateway could not resolve this name in this reading'`; the counted arm keeps its existing facts and adds nothing else. A measured `0` keeps its own sentence (`0 open connections`) — never the unresolved wording, because it is a measurement.
|
||||
- `countText()` and the `—` glyph are unchanged; a `0` continues to print `0`.
|
||||
- **Filter amendment** (phase 31 task 04's `haystack`): include `conn.out.resolved` addresses, so `10.1.2.64` finds `brain.reeseapps.com`. They are payload strings, the filter is still an eye, and this is the one case where matching an address helps the operator — pinned with the rest of the haystack contract.
|
||||
- `cardStreams()` is untouched: a line with no `metric` still draws no series, and the chart legend still names only pipes that have points.
|
||||
2. `src/dashboards/web/pages/gateway.html`
|
||||
- Footer paragraph 2 is rewritten to the new truth and stays **one** paragraph (the file's three-paragraph shape is pinned in `tests/test_api.py`):
|
||||
> `a dash means the gateway could not resolve that name in this reading — hover a counted line for the address it matched and when that answer was taken.`
|
||||
- The card comment above `.gate-map` gains one line: `out.resolved` is the agent's libc answer and is why a name line can carry a number.
|
||||
- The `#gate-empty` setup block keeps naming agent **1.2.0** as the minimum (a 1.3.0 agent is not required for the page to work, only for name routes to count) — do not bump it here; the version floor is `sources/gateway.supported()`.
|
||||
3. `tests/test_api.py`
|
||||
- `GATEWAY_PAGE_FIELDS` gains `connections[].out.resolved`, `connections[].out.via`, `connections[].out.resolved_ts`, resolved against a live document in both directions (the existing device), so a rename on the Python side fails by name here.
|
||||
- **New** `test_an_unresolved_name_and_an_idle_route_say_two_different_sentences`: served JS picks the sentence from `out.resolved === null`, the two strings differ, neither is `n/a`, and a `0` is described as a measurement.
|
||||
- `test_an_unattributed_endpoint_prints_a_dash_and_never_a_zero` stays green with its claim intact and its docstring amended to name the new cause (restated, not deleted).
|
||||
- **New** `test_a_counted_line_names_where_its_address_came_from`: the `title` builder reads `out.via` and `out.resolved_ts` through the shared age helpers, and a missing `resolved_ts` yields no age fragment rather than `NaN`/`undefined`.
|
||||
- The filter pin from phase 31 (`test_the_filter_matches_only_payload_text_and_costs_no_request`) is extended: the haystack includes `out.resolved` and still contains no `fetchJSON`, no DOM text, and no `"null"` matchable value.
|
||||
- `test_no_page_narrates_its_own_pipeline` must still pass on the new footer sentence (it names an operator-facing cause and a place to look, not the app's design); if it objects, shorten the sentence rather than touching the audit's allow-list.
|
||||
- The degradation-arm pin gains a case: `resolver: {available: null}` — no red, no extra ink, `why` still the page's one sentence.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: served-bytes pins above + a fixture render for each of the three arms (resolved+live, resolved+idle, unresolved) asserting the glyph, the `title`, and the `aria-label` differ exactly as specified.
|
||||
- Integration: with the resolver section stubbed to `available: null`, every name line keeps the unresolved sentence and the page stays uncoloured; no line anywhere prints `NaN`, `undefined` or `null`.
|
||||
- Coverage: no new Python in this task; `node --check` clean; floor does not fall.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] On live data: a counted name line's `title` reads `… · address from libc, resolved <age>`; an unresolved line (stub one) prints the em dash with the resolution sentence; a resolved line with no socket prints `0`.
|
||||
- [ ] Reading five lines' `aria-label`s aloud (or dumping them) yields three distinct claims and no invented number.
|
||||
- [ ] Typing `10.1.2.64` (or any address from the live `resolver` section) into `#gate-filter` reveals exactly the route(s) that resolve to it, with zero network requests.
|
||||
- [ ] `grep -c "<p" ` on the served footer is still 3 and `test_no_page_narrates_its_own_pipeline` is green.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` dash sentence updated (this task's user-visible change) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: tallies, warnings panel, chart disclosure, switch, phase 31's colour and row order untouched.
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Task 05 — Roll the gateway to 1.3.0, read-only first, exactly as phase 28 moved it to 1.2.0
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md's fleet record (the rollout discipline that kept `host:gateway`/`disks:gateway` `ok` through phase 28) + the standing rule that a path grant is only as current as the file it points at
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
One host moves: `gateway` runs agent **1.3.0** on its existing bundle and its existing unit, and the live page gets real counts on its name routes. Nothing else moves — no `HOST_TARGETS` edit, no certificate, no URL flip, no other host (a 1.1.0 `deskwork`/`driveripper` is unaffected, and a 1.2.0 gateway left running after a failed attempt is a working gateway). The transcript is the artifact.
|
||||
|
||||
## Work
|
||||
|
||||
1. **Read-only probe first** — `data/logs/verify-32-task05-probe.py` (copy `data/logs/verify-28-task04.py`), printing and asserting:
|
||||
- the CA-only `--status`-style probe against `https://10.1.1.27:9120` answers `version` **1.2.0** on `/health`, and still refuses `/snapshot` with `client certificate required`;
|
||||
- the dashboard's own mTLS dial (`collectors.fetch_snapshot` with `certs.agent_tls_map` over the stored authority — the `verify-28-task05.py` idiom, never a hand-rolled `client.get`) reads every pre-1.3.0 section with live numbers and **no** `resolver` key: the control that makes the install provably additive before anything is touched;
|
||||
- `/api/health` reads `host:gateway`, `disks:gateway` `ok` before the change, and the `gateway` job's payload counts how many rows carry a number (the before figure).
|
||||
2. **Re-stage, then install** over the `gateway-ops` key, as `ops`:
|
||||
- `scp agent/host_agent.py deploy/install-agent.sh → /home/ops/agent-install/…` (the directory is `ops`-owned; no root), leaving the previous files beside the grant as `host_agent.py.20261023-backup` / `install-agent.sh.20261023-backup`, which sudo will not run;
|
||||
- read the live unit back (`systemctl cat dashboards-agent`) and run the **same** arguments it carries:
|
||||
`sudo /home/ops/agent-install/deploy/install-agent.sh --bind 10.1.1.27 --port 9120`
|
||||
— no `--cert-bundle` (the bundle on disk, not the flag, is what puts the TLS lines into the render — the phase-28 finding, quoted in the transcript), no `--as-root`/`--caps` (that is `driveripper`'s shape, not this host's);
|
||||
- expect the re-run guard to stay silent (identical render). `hashsum.sha256` the unit before and after and record both; a differing unit means the run did something this phase did not plan, so stop and report rather than `--force`ing;
|
||||
- `sudo systemctl restart dashboards-agent` (the file changed; `enable --now` would not restart it), then `systemctl is-active`.
|
||||
3. **Verify both ways, read-only**:
|
||||
- the CA-only probe answers `version 1.3.0` and still refuses `/snapshot`;
|
||||
- the dashboard's mTLS read shows `agent_version: "1.3.0"`, `resolver.available: true` with an entry per declared target, `haproxy`/`nginx`/`sockets` and every pre-1.3.0 section intact with live numbers, and `/api/gateway/snapshot` answering a **finite** `connections` on every row whose resolver status is `ok` (the after figure, printed beside the before figure);
|
||||
- `./scripts/dev-server.sh status` → all ten sources `ok`; `/system` shows no new `degraded` on any host (the `resolver:` prefix, invariant 3);
|
||||
- a failure at any step is a rollback, not a fix-forward: copy `host_agent.py.20261023-backup` back to `/usr/local/lib/dashboards-agent/host_agent.py` and restart — the unit, the bundle and the URL never changed, so that *is* the whole rollback. Record which arm happened.
|
||||
4. `data/logs/NOTES-32-rollout.md` — sha256 of what was staged where, the unit hash before/after, the before/after "rows with a number" counts, the live `resolver` entry table, the `nxdomain` list if any, the screenshots' names, and the rollback command. The AGENTS.md fleet record's version spread is task 06's edit; this note is the evidence it cites.
|
||||
5. **Do not** touch `deskwork` or `driveripper`. Their 1.1.0 agents have no proxy sections and nothing to resolve; the spread costs nothing because 1.3.0 is additive, and task 02's control is the proof of that claim.
|
||||
|
||||
## Testing & Quality
|
||||
- This task changes no repo code; its quality gate is that the suite is still green afterwards and the two probe scripts exit 0 with their transcripts saved (`data/logs/32-task05-probe.out`, `data/logs/32-task05-verify.out`).
|
||||
- `uv run pytest -m live --live` afterwards: the new live pin (every resolved name route carries a number, never `null`) passes rather than skips.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `systemctl is-active dashboards-agent` is `active` on `gateway`, and the CA-only probe answers **1.3.0**.
|
||||
- [ ] The unit's sha256 is identical before and after the install run, and the guard printed no diff.
|
||||
- [ ] `/api/gateway/snapshot` on the live ingress: `resolver.available: true`, and **0** `connections[]` rows with `connections: null` among the rows whose resolver status is `ok`; the before/after counts are in the transcript.
|
||||
- [ ] `host:gateway` and `disks:gateway` read `ok` across the whole exercise, and no other host changed `degraded` state.
|
||||
- [ ] `data/logs/NOTES-32-rollout.md` exists with the hashes, the two probe transcripts, and the rollback command named.
|
||||
- [ ] `uv run pytest` **100% passing**; `uv run pytest -m live --live` green (the gateway pin passing, not skipping).
|
||||
- [ ] No repo file changed except `data/logs/` (gitignored).
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Task 06 — Docs that match the counted page, the browser measurement, and the full gate
|
||||
|
||||
**Phase:** `32_gateway_resolved_target_liveness` · **Source:** AGENTS.md definition of done (browser verification against live data, plus a full `README.md`/`AGENTS.md` consistency pass)
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Prove the counted page in a browser on the upgraded ingress, carry the resolution facts into the two documents that outlive this conversation, and close the phase with every gate green — including the ones that only bite if something drifted (dependency freshness, the mobile audit, the live suite).
|
||||
|
||||
## Work
|
||||
|
||||
1. `data/logs/32-counted-routes.cjs` — a Playwright pass over the live page (house shape: `NODE_PATH=<playwright node_modules>`, `DASH_BASE`/`DASH_PW`, sign in with `DASH_ADMIN_PASSWORD`, exit non-zero on FAIL, print every measured number):
|
||||
- **motion where the complaint said there was none**: count the animating elements on load (`getComputedStyle(path).animationName === 'pipe-flow'`) and assert the set is non-empty **and** that at least one of them is a **name** endpoint (a `connections[]` row whose `out.address` is `false`); record the counts per panel;
|
||||
- **provenance**: for 5 sampled rows, `title` contains `address from libc` and an age fragment, and `aria-label` states a number where the payload has one and the resolution sentence where it does not;
|
||||
- **no invented zero**: with a stubbed `resolver: {available: null}` and with `sockets.available: null`, no line prints `0` that the payload did not measure, and the trunk prints `n/a`;
|
||||
- **the filter still costs nothing**: query by an IP from the `resolver` section and by a domain, asserting zero network requests and unchanged tallies;
|
||||
- **colour is still existence**: every drawn route — spine, tick, run, switch — computes to `--gate-on` in both themes, and a row given `off` in devtools turns all four together (phase 31's contract, re-asserted after this phase's DOM/provenance changes);
|
||||
- **reduced motion**: nothing animates under emulated `prefers-reduced-motion: reduce`;
|
||||
- **degradation arms**: `unreachable`, `supported: false`, `sections {nginx: false}`, `resolver {available: null}`, `404 {configured:false}` — each in the payload's own words, none red;
|
||||
- **no sideways scroll** at 1440 px and 360 px with the widest fan-out expanded and both charts open;
|
||||
- screenshots `data/logs/32-gateway-{1440,360,counted,light,unresolved,filtered,reduced,phone}.png`, transcript `data/logs/32-counted-routes.out`.
|
||||
2. Re-run `data/logs/27-task02-mobile-audit.cjs` (nine pages, both widths) — exit 0, **0 FAIL / 0 FINDING**, tally recorded in the transcript.
|
||||
3. `AGENTS.md` — update every passage this phase made stale, in full:
|
||||
- the agent bullet: the `resolver` section (its `available` three-way and one-line-per-arm rule, `via: "libc"`, `config_directive`, `RESOLVE_MAX_NAMES`/`RESOLVE_CACHE_SECONDS`/pass budget, the single-flight background pass and *why* it is off the request path, `--no-resolve` removing the key in silence, and the never-resolve-anything-but-config-targets rule), `AGENT_VERSION` **1.3.0**, and the `/schema` addition;
|
||||
- `sources/gateway.py`'s join paragraph: two address sources (written-as-address and resolved), `out.resolved`/`via`/`resolved_ts`, normalised-not-string matching, the shared rule's now-real case, and the sentence that the dashboard **still** never resolves with the vantage reason;
|
||||
- the metric-naming passage: replace "no name-keyed `gateway.out_conns@…` name is ever inserted" with the widened rule and the reason it changed, keeping the note that an old name-keyed series stops growing rather than being rewritten;
|
||||
- `collectors.py`'s `ADDON_ERROR_PREFIXES` list gains `resolver`, beside the "a new add-on section adds its prefix here in the same commit" rule;
|
||||
- the `/gateway` UI-conventions paragraph: the dash now has two meanings and one glyph, provenance lives on `title`, the filter matches resolved addresses;
|
||||
- the **fleet record**: `gateway` runs **1.3.0**, `deskwork`/`driveripper` remain **1.1.0**, what the spread costs (nothing, with the additivity control named), the ten sources `ok` re-verified, and a pointer to `data/logs/NOTES-32-rollout.md`;
|
||||
- **gap list**: add the two follow-on lines (name-route switching = phase 33, amends invariant 1 and needs `PLAN.md` sign-off) and re-check gap 13's clip counts against this phase's measurements.
|
||||
4. `README.md` — the `/gateway` table row (name routes are counted through the gateway's own libc resolution), the gateway prose (the resolver section, its bounds, `--no-resolve`, what a dash means now), the capture-script section (its new output and its scrub rule), and the agent-version references wherever the page's minimum and the fleet's actual are named — kept as two different facts, not one number.
|
||||
5. Freshness and the whole gate:
|
||||
`uv sync --python 3.14 && uv pip list --outdated` → empty (this phase adds **no** dependency; if `cryptography` or anything else moved, that is a separate decision);
|
||||
`uv run pytest`, `uv run pytest --cov`, `uv run pytest -m live --live`, `uv run ruff check src tests agent`, `uv run pyright`, `bash .agents/validate.sh`.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: this task **is** the live proof of the phase's headline claim; every number in the Completion Criteria is printed by `32-counted-routes.out` or a probe transcript, never asserted in prose.
|
||||
- Coverage: the floor is met on the phase's modified Python (`sources/gateway.py`, `collectors.py`, `agent/host_agent.py`); the run is recorded.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `data/logs/32-counted-routes.out` exits 0 and prints: animating-element counts per panel with at least one **name** endpoint among them, the five provenance `title`s, the zero-invention arms, the filter's zero-request proof, the both-themes colour equality, the reduced-motion pass, and 0 px sideways scroll at both widths.
|
||||
- [ ] The eight `data/logs/32-gateway-*.png` screenshots exist, including `counted` (name routes with numbers) and `phone`.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING, tally recorded.
|
||||
- [ ] `uv pip list --outdated` is empty; `uv run pytest` and `uv run pytest --cov` green above the floor; `uv run pytest -m live --live` green; `ruff` + `pyright` clean; `bash .agents/validate.sh` green.
|
||||
- [ ] `grep -n "1.3.0" AGENTS.md` shows the agent bullet **and** the fleet record; `grep -n "no name-keyed" AGENTS.md README.md` finds nothing (the retired sentence is gone, not contradicted); `resolver` appears in `ADDON_ERROR_PREFIXES`' documented list.
|
||||
- [ ] A full read of `README.md` and `AGENTS.md` finds no inconsistency anywhere in either file, including pre-existing ones — fixed in this phase's commits.
|
||||
- [ ] No behaviour change in completed work: `/`, `/system`, `/disks`, `/llm`, `/ups`, `/settings`, `/certs`, `/login` render as before and the unwired switch stays unwired (phase 33 wires it).
|
||||
Reference in New Issue
Block a user