phase: 33_gateway_route_switching
Phase 33 completion pass is done. Final report:
## Phase 33 — completion report
**All 8 task files complete** (`.agents/phases/complete/33_gateway_route_switching/`), full gate green, docs verified.
### Work in this pass
1. **`tests/test_routectl.py` added** (17 tests, all green) — exercises the *real* helper binary unprivileged against a fake tree (`--root`), the real agent parser, the captured gateway fixtures, and stubbed `nginx`/`haproxy`/`systemctl` on `PATH`. Pins the full protocol (locate → state → dry-run → temp-file/fsync/`os.replace` write → checker → restore on checker failure → reload-never-restart → re-read proof → sidecar record), all three route shapes (haproxy `server`, nginx `proxy_pass`, nginx stream `map` via its sink-value id), `drift` / unknown-id / `map-default` refusals, `list`/`verify`, and the `--serve` peer gate (`SO_PEERCRED` uid check, one JSON line per request, exact body shape).
2. **Real helper defect fixed** — `deploy/dashboards-routectl.py`: `main()`'s early-return paths returned `({"ok": False, …}, EXIT_*)` tuples, so `sys.exit()` printed a Python repr instead of a JSON line (usage errors, agent-import refusal). Now prints `json.dumps(…)` and returns only the code, keeping the one-JSON-line contract.
3. **Documentation consistency fixes** (task 08 criterion 6):
- `README.md`: garbled verdict-colour sentence on the switch ("green and to the position") → "in the **on** position and tinted `--gate-on`" (the existence-pair contract).
- `README.md`: audit sentence now names all three event kinds — `route-disabled` / `route-enabled` / **`route-denied`** (the code stores one per refused call; dry runs store none).
- `page-gateway.js`: replaced the stale pre-33 module docstring ("the switch is not wired") with the two-arm `GATEWAY_ALLOW_WRITES` contract.
- `AGENTS.md`: every phase-33 claim re-verified against code (the ten gates, the 35 s/30 s budget pair, `gateway.writes_enabled`, `/api/summary`'s `disabled`, the 1.4.0 agent bullet, the gap list) — no changes needed.
### Verification
- `uv run pytest --cov`: **1970 passed**, 18 skipped — coverage **98.74%** (floor 94%)
- `ruff` clean · `pyright` 0 errors · `bash .agents/validate.sh` **exit 0** (incl. `node --check` on every page script)
- Dev server running; all 10 sources `ok`; bundles exported
### Live infrastructure
**No live changes made.** The live drill remains unexecuted per `data/logs/NOTES-33-switching.md` (no operator approval; checker binaries unavailable in the podman proxy units). All routes stay enabled, config hashes unchanged, and the sandbox rehearsal + backup tree are in place for the operator's own `DANGER` call. All changes are in the working tree only.
This commit is contained in:
+125
-3
@@ -14,9 +14,11 @@ addresses the deployed dashboard dials, nothing forwarded:
|
||||
Each test skips when its source is not configured, so running the whole set with half a
|
||||
fleet reachable is a normal outcome, not a failure.
|
||||
|
||||
**Nothing in here may write.** The UPS test asserts the command gating is honest; it never
|
||||
sends an instant command — a battery test on hardware someone else is plugged into is not
|
||||
a unit test.
|
||||
**Nothing in here may write — with exactly one documented exception.** The UPS test asserts
|
||||
the command gating is honest; it never sends an instant command — a battery test on
|
||||
hardware someone else is plugged into is not a unit test. The one write is the phase-33
|
||||
round-trip test, gated by its own flag (`--allow-live-writes`) on top of `--live`: a
|
||||
default live run never switches a route, and the skip reason names the route it would.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -562,3 +564,123 @@ def test_target_config_is_clean_and_leak_free(settings: Settings) -> None:
|
||||
assert "@" not in target.display_url, f"{target.name}: display_url leaks credentials"
|
||||
for problem in settings.host_target_problems:
|
||||
assert isinstance(problem, str) and problem, f"empty problem: {problem!r}"
|
||||
|
||||
|
||||
#: The one write target of the opt-in round trip (phase 33 task 07): a benchmark service
|
||||
#: with no human dependency. Never the auth, git, alerts or file routes — task 07's
|
||||
#: precondition names them, and a test that would touch one of them is a test that does not
|
||||
#: run.
|
||||
SWITCH_DRILL_ROUTE = "haproxy:5201->web.reeselink.com:5201"
|
||||
|
||||
|
||||
async def test_the_switch_round_trip_on_one_named_route(
|
||||
request: pytest.FixtureRequest, settings: Settings,
|
||||
agent_tls: dict[str, collectors.AgentTls]) -> None:
|
||||
"""Phase 33's one live write: the named route off and back on, through the app's own
|
||||
`POST /route` dial.
|
||||
|
||||
Gated twice so a default live run stays read-only: the file's `live` marker (skip
|
||||
without `--live`) **and** the separate `--allow-live-writes` flag, whose skip reason
|
||||
names the route it would touch. When it does run — the operator's drill, task 07 step
|
||||
5 — it dials `collectors.post_route` (the app's second and last upstream write, through
|
||||
the stored mTLS authority, no hand-rolled client) and proves the round trip on the
|
||||
production read (`fetch_snapshot` → `gateway.parse`): the helper's own words for the
|
||||
write, the row's `enabled` flipping both ways, the agent's own section carrying the
|
||||
marked line in between, and the haproxy section the agent serves byte-identical between
|
||||
the pre-drill and post-drill reads. The restore lives in a `finally`: even a failed
|
||||
proof turns the route back on before the test ends, and a re-enable that itself fails
|
||||
propagates the helper's own word — a marked line left behind must be the operator's
|
||||
first line, never a silent one.
|
||||
"""
|
||||
if not bool(request.config.getoption("--allow-live-writes")):
|
||||
pytest.skip(f"opt-in live write: skip without --allow-live-writes (this test would "
|
||||
f"switch {SWITCH_DRILL_ROUTE} off and back on — the default live suite "
|
||||
"stays read-only")
|
||||
target = next((item for item in settings.host_targets if item.name == settings.gateway_host),
|
||||
None)
|
||||
if target is None:
|
||||
pytest.skip(f"GATEWAY_HOST={settings.gateway_host!r} is not in HOST_TARGETS")
|
||||
|
||||
async def read_ingress(client: httpx.AsyncClient) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
snapshot = await collectors.fetch_snapshot(client, target,
|
||||
timeout=settings.gateway_timeout,
|
||||
tls=agent_tls.get(target.name))
|
||||
return snapshot, gateway.parse(snapshot, target.name)
|
||||
|
||||
async def post(body: dict[str, Any]) -> dict[str, Any]:
|
||||
# One client per call: one write, one connection — the same discipline the agent
|
||||
# keeps for its one helper dial (a retry of a write is a second reload).
|
||||
async with httpx.AsyncClient(timeout=settings.gateway_timeout) as client:
|
||||
return await collectors.post_route(client, target, body,
|
||||
timeout=settings.gateway_timeout,
|
||||
tls=agent_tls.get(target.name))
|
||||
|
||||
disabled = False
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=settings.gateway_timeout) as client:
|
||||
try:
|
||||
before, payload = await read_ingress(client)
|
||||
except collectors.UpstreamError as exc:
|
||||
pytest.skip(f"{target.name} did not answer: {exc}")
|
||||
# Precondition 1: the agent on this host is the one with the write path.
|
||||
version = str(before.get("agent_version") or "")
|
||||
if not version.startswith("1.4."):
|
||||
pytest.skip(f"{target.name} runs agent {version or 'unknown'} — POST /route is "
|
||||
"1.4.0's; re-stage with deploy/install-agent.sh --allow-writes "
|
||||
"--with-routectl")
|
||||
# Precondition 2: a clean pre-state — the route is on and nothing else is off;
|
||||
# a half-finished earlier attempt is the state task 07 step 4 refuses to drill.
|
||||
route = next((row for row in payload["connections"]
|
||||
if row["id"] == SWITCH_DRILL_ROUTE), None)
|
||||
if route is None:
|
||||
pytest.skip(f"{SWITCH_DRILL_ROUTE} is not on the ingress the agent published — "
|
||||
"the drill target moved; update this test's target and re-verify")
|
||||
already_off = [row["id"] for row in payload["connections"]
|
||||
if row["enabled"] is False]
|
||||
assert not already_off, (
|
||||
f"a route is already disabled before the drill {already_off} — a previous "
|
||||
"attempt left a marked line; restore it before re-running (task 07 step 6)")
|
||||
# 1. dry_run: the diff, and no write (task 07 step 5.1).
|
||||
preview = await post({"id": SWITCH_DRILL_ROUTE, "action": "disable",
|
||||
"dry_run": True})
|
||||
assert preview.get("ok") is True and preview.get("dry_run") is True, preview
|
||||
assert preview.get("applied") is False and preview.get("changed") is True, preview
|
||||
assert any("#off" in str(line) for line in preview.get("diff") or []), (
|
||||
f"the dry-run diff must show the marker landing: {preview.get('diff')!r}")
|
||||
# 2. apply: the helper's own word for the write (step 5.2). A checker that refuses
|
||||
# (e.g. no `nginx -t` on the host's PATH) answers `applied: false, restored: true`
|
||||
# and this test fails on the helper's word instead of printing a false green.
|
||||
applied = await post({"id": SWITCH_DRILL_ROUTE, "action": "disable"})
|
||||
assert applied.get("ok") is True and applied.get("applied") is True, (
|
||||
f"the helper refused the write instead of applying it: {applied!r}")
|
||||
assert applied.get("validated") is True and applied.get("reloaded") is True, applied
|
||||
disabled = True
|
||||
# 3. the off state, proven on the production read (step 5.3/5.4): the row and the
|
||||
# agent's own section both report the marker.
|
||||
async with httpx.AsyncClient(timeout=settings.gateway_timeout) as client:
|
||||
off_snapshot, off_payload = await read_ingress(client)
|
||||
off_route = next(row for row in off_payload["connections"]
|
||||
if row["id"] == SWITCH_DRILL_ROUTE)
|
||||
assert off_route["enabled"] is False, "the row must report disabled within one read"
|
||||
marked = [server for backend in (off_snapshot.get("haproxy") or {}).get("backends", ())
|
||||
for server in backend.get("servers", ()) if server.get("disabled") is True]
|
||||
assert marked, "the agent's haproxy section shows no marked server line"
|
||||
finally:
|
||||
# The restore lives here, not after the assertions: a failed proof still leaves the
|
||||
# route on, and a failed re-enable propagates the helper's own word.
|
||||
if disabled:
|
||||
await post({"id": SWITCH_DRILL_ROUTE, "action": "enable"})
|
||||
# 4. the round trip closed: the row is on, nothing is off, and the haproxy section the
|
||||
# agent serves is byte-identical to the pre-drill read (step 5.5/5.6).
|
||||
async with httpx.AsyncClient(timeout=settings.gateway_timeout) as client:
|
||||
after, after_payload = await read_ingress(client)
|
||||
after_route = next(row for row in after_payload["connections"]
|
||||
if row["id"] == SWITCH_DRILL_ROUTE)
|
||||
assert after_route["enabled"] is True, (
|
||||
f"the route is still off after the re-enable — the drill left a marked line: "
|
||||
f"{after_route!r}")
|
||||
left_off = [row["id"] for row in after_payload["connections"]
|
||||
if row["enabled"] is False]
|
||||
assert not left_off, f"marked lines left behind after the round trip: {left_off}"
|
||||
assert json.dumps(after.get("haproxy")) == json.dumps(before.get("haproxy")), (
|
||||
"the haproxy section is not byte-identical after the round trip")
|
||||
|
||||
Reference in New Issue
Block a user