phase: 31_gateway_route_readability
**Phase 31 verification complete — all 5 tasks already in `complete/`; this was the final verification pass.**
Verified / changed
- Re-ran every phase-31 artifact live and green: `verify-31-page-only.py` 6 PASS/0 FAIL; browser pass `31-route-readability.cjs` **64 PASS / 0 FAIL** (exit 0); `27-task02-mobile-audit.cjs` **371 PASS / 0 FAIL / 0 FINDING** (exit 0); rebuilt `data/logs/31-route-readability.out` as the 4-part transcript.
- Added the 4 degradation-arm screenshots the criterion asked for but the pass lacked (`31-gateway-{unconfigured,unreachable,no-nginx,sockets-quiet}.png`) — 12 frames now, one per arm.
- Grep criteria: no gateway connector rule uses `border-strong`; no verdict token in the gateway CSS section or `page-gateway.js` beyond the `alerts.py`-owned `.warn`; exactly 4 `addEventListener` calls (none for the switch); `routeInConfig()` returns `true`; **no `.py` under `src/dashboards/` changed**.
Tests / lint (all 100% green)
- `uv run pytest` → **1796 passed, 16 skipped**; `--cov` → **98.94%** (floor 94); `ruff check src tests agent` → All checks passed; `pyright` → 0 errors; `node --check` clean; `bash .agents/validate.sh` → **exit 0**.
- `uv run pytest -m live --live` → **12 passed / 0 failed**; `./scripts/dev-server.sh status` → all **10 sources ok**.
Completion criteria: colour (33/33 rows every segment `--gate-on`, dark `rgb(63,185,80)` + light `rgb(26,127,55)`, zero grey dashes) ✔ · inbound-first rows + filter (33/33 line 1 = `out.selectors`, line 2 = `→ host:port · kind`; `stackexpected` → **6** rows, not the overview's "five" — the capture declares 6 such routes; tallies byte-identical `10/33/16/0`, zero requests) ✔ · trunk motion truth table + `null` arm prints `n/a` never `0` + reduced-motion stills 43 paths and every knob ✔ · screenshots/transcripts ✔ · payload page-only ✔ (literal byte-diff of a live socket table can never pass; answered by the no-`.py`-diff + shape-equality proof, as recorded in the transcript) · no behaviour change elsewhere ✔.
Docs: fixed 6 inconsistencies — two identifiers split across line breaks (`test_the_pipe_animation_stills_in_both_states_under_reduced_motion`, `32_gateway_resolved_target_liveness`), stale "sixteen routes" count, stale "four machines/hosts" in README **and** AGENTS (fleet is three), "three fields"→"three field types", gap-14 headline wording; updated the pass's screenshot list.
Deviations: `uv pip list --outdated` shows one advisory (`starlette 1.9.0 → 1.10.0`) — not upgraded, this phase moved no dependency (rule 8 / DoD apply "if deps moved"). `TODO.md` carries two new operator disk requests, left untouched.
Next pending phase: `32_gateway_resolved_target_liveness`.
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
# Phase 33 — The switch is wired: turning one route off, and back on, without ever weakening the box
|
||||
|
||||
**Source:** operator request, 2026-10-23 — "Also write the phase to disable and enable services. I want to be able to turn off and on connections as needed in the event of security concerns or upgrades." + the two decisions taken with them in that conversation: **enforcement is a config edit, validated and reloaded, by a narrow root helper on the gateway** (so a disabled route stays disabled across a reload *and* a reboot), and **one switch per connection line, as already drawn**.
|
||||
**Story:** n/a
|
||||
**Context:** `deploy/install-agent.sh` (the unit template, whose `NoNewPrivileges=true` / `ProtectSystem=strict` / `SystemCallFilter=@system-service` / empty `CapabilityBoundingSet` this phase **does not touch**, the `--force` re-run guard, `deploy/agent-sudoers` as the precedent for a path-pinned grant), `agent/host_agent.py` (1.3.0 → **1.4.0**: `do_GET`-only `Handler`, `_certificate_allows`, the `haproxy`/`nginx` parsers that must learn the disabled marker), `src/dashboards/sources/gateway.py` (`connections[]`, `config_connections`, `warnings()`' seven kinds), `src/dashboards/api.py` (`/api/ups/command`'s four-gate shape and the `certs_router()`'s audit-event rule), `src/dashboards/collectors.py` (`AgentTls`, `fetch_gateway_snapshot`, the revoke-before-dial rule), `src/dashboards/config.py` (`GATEWAY_*` block), `src/dashboards/web/static/page-gateway.js` (`buildSwitch`, `routeInConfig`, `renderLeaf`), `tests/test_api.py` (the verb-enumeration pins), `tests/test_auth_api.py` (the served-route walk), `tests/test_deploy_scripts.py` (the fake-root sandbox the helper's tests extend).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway`'s switch becomes a control: **disable** `brain.reeseapps.com` and the nginx `stream` map entry stops routing it, persistently, with the config validated and the proxy reloaded and the page showing that exact route **red** until it is re-enabled. One new write route, `POST /api/gateway/route`, gated seven times, with the write itself performed off-box by a **root helper on the gateway that the agent reaches over a systemd-activated unix socket** — because the agent's own unit forbids privilege (`NoNewPrivileges=true`) and this phase is not going to weaken the sandbox of a read-only collector to give it a root shell.
|
||||
|
||||
This amends AGENTS.md invariant 1 from *one* exception to **two**, and that is the phase's real risk: a bug here is an outage on the fleet's only ingress. Everything below exists to make the blast radius one line of one file, and to make the app refuse rather than improvise whenever it cannot see what it is doing.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Invariant 1 becomes "two exceptions".** | `POST /api/ups/command` and `POST /api/gateway/route`. Both: session, CSRF/origin on a cookie write, an env gate (`NUT_ALLOW_ACTIONS` / `GATEWAY_ALLOW_WRITES`, default **off**), a typed `DANGER`, and every gate re-checked server-side on every request. `PLAN.md`'s anchor table records a read-only app, so this phase's overview carries the sign-off request; the phase never edits `PLAN.md`. |
|
||||
| 2 | **`DANGER` is required for both verbs.** | Enabling is not the safe direction: applying either change re-validates the file and **reloads the proxy**, and a reload touches every route on the box. A word that means "I know this affects the ingress" has to mean it both ways. |
|
||||
| 3 | **The agent keeps its sandbox; the privilege lives outside it.** | No sudo from the agent, no setuid, no change to `NoNewPrivileges`/`ProtectSystem`/`SystemCallFilter`. The write peer is `dashboards-routectl.socket` + `dashboards-routectl@.service` (systemd socket activation, `Accept=yes`, `User=root`, `SocketUser=dashboards-agent`, `SocketGroup=dashboards-agent`, `SocketMode=0600`): the agent connects to a socket it is *filemode-permitted* to connect to, and no other uid on the box can even open it, and the helper proves the peer uid with `SO_PEERCRED`. Access control is a file mode plus the mTLS client certificate that already authorises the agent's other paths — not a capability the agent process can be tricked into. |
|
||||
| 4 | **Only an id crosses a wire. Never a path, never a line, never text.** | The body is `{id, action, danger?, dry_run?}` where `id` is the payload's own `connections[].id`. The helper re-parses the configs itself, maps that id to the one line it declared, and refuses anything it cannot map. There is no `file`, no `line`, no `pattern`, no `command` anywhere in the contract, and the argv to the helper is fixed with no shell. |
|
||||
| 5 | **A route is off in the way that actually stops traffic for its proxy.** | haproxy: the `server` line is commented with a `# dashboards:off … was: <verbatim original>` marker. nginx vhost: the `proxy_pass` line is commented the same way. nginx **stream `map` entry**: commenting it would fall through to the map's `default` (`127.0.0.1:4443`) and **mis-route** the domain instead of blocking it, so the helper rewrites that entry's value to a sink (`127.0.0.1:44450`) and keeps the original in the marker; the sink is a `server` block the helper maintains in `/etc/nginx/stream.d/00-dashboards-sink.conf`, which the existing `stream { include /etc/nginx/stream.d/*.conf; }` glob already loads — so **no hand edit of the main config is ever required**. The map's `default` entry is **refused**: it is the fallback for every unmapped name, not a service. |
|
||||
| 6 | **Validate, restore on failure, reload — never restart.** | Write the new bytes, run `nginx -t` / `haproxy -c -f …`; on failure put the previous bytes back, do **not** reload, and answer `applied: false, validated: false, restored: true`. On success `systemctl reload <unit>` — reload, because a restart cuts every Minecraft session and every in-flight upload on an ingress that is doing its job. A `reload` failure is reported with the unit's own status and the file is left in the applied state (the config is correct; the reload is the operator's next action, said out loud). |
|
||||
| 7 | **Self-describing marker + drift refusal, not silent trust.** | The disabled line carries the original verbatim after `was:`, so a human can restore it by eye and the helper can restore it mechanically. A sidecar `/var/lib/dashboards-routectl/state.json` (`0600`, root) records actor, timestamp and the sha256 of what was written. Enabling restores only if the current line still matches what the helper wrote; otherwise it **refuses** and reports `drift` — a file an operator hand-edited is theirs to fix, not the app's to guess at. |
|
||||
| 8 | **Refuse rather than act blind.** | The write is refused when: writes are not enabled; the id is not in the **current** payload; the route is already in that state (`changed: false`, no reload); either config section is not `available: true`; the `gateway` job is `ok: false` or stale beyond 3× its interval; the `host:<GATEWAY_HOST>` system job is not `ok`; the agent is revoked (refused before any socket, the existing rule); the throttle is tripped. That is the UPS "refused while on battery" rule, translated: do not rewrite a config you cannot currently read end to end. |
|
||||
| 9 | **`off` finally means something the payload said.** | The agent's parsers learn the marker and report the marked `server`/`proxy_pass`/`map` entry with `disabled: true`, so the route stays in the graph as a declaration and `connections[].enabled` becomes **`false`**. `routeInConfig()` stops being a literal, the `off` styling phase 30 built becomes reachable, and `warnings()` learns the marker: a backend emptied by a marker is not `haproxy_backend_empty` (the operator chose that; the page shows it red), while a backend with no `server` line at all still is. |
|
||||
| 10 | **A live route's count is never stamped.** | Disabling does not write `0` into the payload. The sockets drain on the reload and the next poll reports what `/proc` says. Invariant: no code path in this phase writes a count. |
|
||||
| 11 | **The audit is one event, fixed in shape, with no config text in it.** | `gateway`-source events `route-disabled` / `route-enabled` / `route-denied`: verb, id, the session's actor name, `applied`/`validated`/`reloaded`/`drift`, and the proxy's own one-line error when there was one. The unified diff is returned to **the caller's response** (it is what the confirm dialog shows) and never stored — events are served to a browser and `/api/events` is a feed, not a config archive. |
|
||||
| 12 | **The overview is told, because "something is switched off" is not a page-only fact.** | `/api/summary`'s `gateway` block gains **one** field, `disabled` (an int count of `enabled is false` rows, `null` in the arms that did not measure) — the thirteenth key, restated in the pinned key set with its reason. A notifier has to learn about a deliberate outage without scraping a page, and it prints nothing at `0` (the `warnings` rule: a permanent `0 switched off` is ink that trains the eye). |
|
||||
| 13 | **One route, one file, one reload per request.** | A request touches exactly one config file and reloads exactly one proxy. There is no batch verb, no "disable all to this host", no `dry_run` that writes. If a security incident needs five routes off, that is five reviewed requests, each with its own event. |
|
||||
|
||||
**Out of scope:** editing anything a marker cannot express (adding/removing vhosts, certificates, `bind` lines, the map's `default`), haproxy runtime `disable server` via a stats socket (a runtime-only state that a reload silently undoes — decision 5 is persistent instead), disabling an entire **host** or **port**, per-user authorisation (one admin, gap 6), a scheduler ("off at 02:00"), and any change to the agent's unit security directives or to another host's agent (`gateway` alone runs 1.4.0; `deskwork`/`driveripper`'s 1.1.0 agents have no routes and are unaffected).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `32_gateway_resolved_target_liveness` (todo, **must complete first**) — agent 1.3.0's resolver section and the counted name routes are what a switched-off line reads against: `off` + a draining count to `0` is the proof the switch worked, and without resolution a disabled route and a never-counted route look identical.
|
||||
- `31_gateway_route_readability` (todo) — the row that names the inbound domain, the green end-to-end route, and the `off` styling this phase finally reaches.
|
||||
- `30_gateway_connection_map` (complete) — `connections[].id` as the vocabulary, the switch markup and its a11y contract, `routeInConfig()` as the named seam.
|
||||
- `15_agent_mtls` / `24_dev_mtls_rollout` (complete) — the verified-client-certificate gate the new POST path inherits, and the revoke-before-dial rule this phase's collector reuses.
|
||||
- `11_admin_login` (complete) — session, CSRF, origin checks, `LoginThrottle` (the `route:<address>` bucket is its third key shape).
|
||||
- **Operator actions, not repo work:** the `dashboards-agent`-to-helper socket grant lands through the units (no sudoers line needed), but installing units on a host and running the first live drill are the operator's — and the drill **stops** unless they are present. `PLAN.md`'s anchor sign-off for invariant 1 is theirs too.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_root_helper_edits_exactly_one_route.md` — `deploy/dashboards-routectl.py`: `show`/`list`/`diff`/`disable`/`enable`/`verify`, the marker with `was:`, the sink file it manages, validate-restore-reload, drift refusal, `--serve` over a systemd-passed socket, `--root` for tests.
|
||||
2. `02_the_privileged_peer_is_a_socket_not_a_sudo_line.md` — the two systemd units + `install-agent.sh --with-routectl`; the proof that the agent's `NoNewPrivileges=true`/`ProtectSystem=strict` unit is unchanged and can still reach the socket.
|
||||
3. `03_the_agent_accepts_one_write.md` — agent **1.4.0**: `POST /route` behind the verified client certificate **and** `--allow-writes`, body validation, one fixed peer call, plus the parsers reporting `disabled` for a marked line; `/schema` and the additivity control.
|
||||
4. `04_the_source_publishes_enabled_and_the_warnings_learn_the_marker.md` — `connections[].enabled`, `off` reachable in the payload, `warnings()` marker-aware, stub/skeleton parity, the one new `/api/summary` field (`gateway.disabled`), and no count ever written.
|
||||
5. `05_the_api_gates_the_write_ten_times.md` — `GATEWAY_ALLOW_WRITES`, `DASH_READ_ONLY`, `collectors.post_route`, `POST /api/gateway/route`, the ten-gate order, the throttle, the events, `/api/config.gateway.writes_enabled`, and the verb-enumeration + served-route pins restated (this is where invariant 1 is amended in `AGENTS.md`).
|
||||
6. `06_the_switch_obeys.md` — `page-gateway.js` + `gateway.html` + `dash.css`: `DANGER` confirm with the diff from a `dry_run` first, patch from the response, `routeInConfig()` reading `enabled`, the writes-disabled arm that keeps the switch honest, footer paragraph 3.
|
||||
7. `07_rehearse_then_drill_on_the_live_ingress.md` — the fake-tree rehearsal, then the live install on `gateway` and a drill on the one named-safe route, restore, verify, `data/logs/NOTES-33-switching.md`. Stops, correctly, without the operator present.
|
||||
8. `08_docs_and_live_verify.md` — `AGENTS.md` (invariant 1, the agent bullet, `sources/gateway.py`, `api.py`, `collectors.py`, `config.py`'s block, the `/gateway` UI paragraph, the fleet record, gap list) + `README.md` (including the operator's sudo/units steps), the browser pass, the mobile audit, the full gate.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **The helper is tested unprivileged, against a fake tree, with the proxy commands stubbed.** `tests/test_routectl.py` runs the real helper in a `tmp_path` tree (`--root`) with `nginx`/`haproxy`/`systemctl` faked on `PATH`, and pins: one line changed and nothing else in the file (byte-diff assertion); `was:` restores verbatim; a failing validator restores the exact previous bytes and reloads nothing; `default` refused; an unknown id refused; a second disable `changed: false`; the sink file created on first use, left alone if an operator's file is there, removed only when it wrote it and nothing is disabled; drift refused rather than restored; `SO_PEERCRED` refused for a non-`dashboards-agent` uid in the fake-peer test; no `shell=True` anywhere (`grep` pin, the installer's precedent).
|
||||
- **The agent's write path is proven against a fake peer, not a real one.** `tests/test_agent.py` runs the real handler over a TLS loopback port (the phase-15 harness) plus a unix-socket stub: `POST /route` without a client certificate is 403 `client certificate required`; with `--allow-writes` absent it is 403 `writes are not enabled on this agent`; a body with an unexpected key, a bad id, or a missing `danger` is refused **before** the peer is touched (asserted by the stub logging nothing); `GET` behaviour is byte-identical to 1.3.0 (the additivity control re-run).
|
||||
- **The API gates are proven twice, as the UPS ones are.** `tests/test_api.py` walks every gate with spies: writes-disabled refuses without dialling; a stale `gateway` job refuses; a revoked agent refuses without a socket; `DANGER` is required for `disable` **and** `enable`; a right request produces one peer call, one event, and the response's `applied`/`validated`/`reloaded` triple. `tests/test_auth_api.py`'s served-route walk demands a session for `/api/gateway/route`, and the CSRF pin is extended to it on the **wire** (a scripted helper records nothing without the header, once with it) — a 403 after the write is an outage with an error message attached.
|
||||
- **Live, opt-in, and reversible:** `-m live --live` gains `test_the_switch_round_trip_on_one_named_route` which is **skipped unless `--allow-live-writes` is passed as well**, names the route in the skip reason, and always restores in a `finally`. The default live suite stays read-only (AGENTS.md's rule survives this phase; the drill belongs to task 07 with the operator watching).
|
||||
- Coverage **>90%** on `deploy/dashboards-routectl.py`, `agent/host_agent.py`, `sources/gateway.py`, `collectors.py`, `api.py`; the whole suite 100%; `bash .agents/validate.sh` green.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it will stand after this phase and fixes every inconsistency found, **including pre-existing ones**. This phase's specific list: **invariant 1** rewritten to two exceptions with the seven gates named (and the sentence "hiding a button is not a safety control" kept, because it is the reason the switches are gated server-side rather than greyed in JS); the agent bullet's `POST /route` and the `disabled` marker; the `sources/gateway.py` bullet's `enabled`/`off` rules and the marker-aware `warnings()`; the `api.py` bullet's write lists (fleet writes are now two); the `collectors.py` bullet (`post_route`, the revocation check it shares); `config.py`'s `GATEWAY_ALLOW_WRITES` row and README's configuration table; the `/gateway` UI-conventions paragraph (the switch is wired, what `off` looks like, what the writes-disabled arm says); the fleet record (gateway **1.4.0**, its unit's security directives unchanged, the two new systemd units, `NOTES-33-switching.md`); and the gap list (name-route switching **closed**, plus the honest limits: one admin, no scheduler, no per-host kill).
|
||||
- `README.md` gains the operator's install steps for the helper units and the exact `--with-routectl` invocation, plus the recovery command for the state file (`systemctl status dashboards-routectl.socket`, `routectl list`, `routectl verify`).
|
||||
- Tasks update both files immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] With `GATEWAY_ALLOW_WRITES=1`, `/gateway` disables `brain.reeseapps.com`: the confirm dialog shows the **diff from a `dry_run`**, asks for `DANGER`, and the row turns **red** (`off`) on the poll after the reload while its live count drains to `0` on its own — no code wrote that zero.
|
||||
- [ ] `ssh gateway-ops 'sudo /usr/local/sbin/dashboards-routectl.py list'` names exactly that route; `verify` reports no drift; the marked line contains the original verbatim after `was:`; the map entry points at the sink and `stream.d/00-dashboards-sink.conf` exists and is owned by the helper's header.
|
||||
- [ ] Enabling restores the original bytes exactly (byte-compare against a pre-drill copy), the row turns green, and `list` is empty afterwards. **The live ingress ends the phase in the state it started in.**
|
||||
- [ ] Every gate refuses with its own sentence and **without** touching a peer: writes disabled, no session, no CSRF, wrong `Origin`, missing `DANGER`, unknown id, already-in-state, section unreadable, `gateway` job stale, `host:gateway` not `ok`, agent revoked. Each is a test, and each test asserts the stub peer logged nothing.
|
||||
- [ ] The agent's unit is unchanged: `systemctl cat dashboards-agent` still shows `NoNewPrivileges=true`, `ProtectSystem=strict`, `SystemCallFilter=@system-service`, and the same sha256 the phase recorded before installing the units — the sandbox was not traded for the feature.
|
||||
- [ ] A failing validator (`nginx -t` stubbed to exit 1) restores the previous bytes, reloads nothing, answers `applied: false, validated: false, restored: true`, and the page's line never turned `off`.
|
||||
- [ ] `POST` inventory: the app now has exactly two upstream writes (`/api/ups/command`, `/api/gateway/route`) and the enumeration pin lists them as such; `AGENTS.md` invariant 1 says **two** and names both gates; `/api/health` gained nothing.
|
||||
- [ ] `uv run pytest` (with `--cov` ≥ 94), `uv run ruff check src tests agent deploy`, `uv run pyright`, `node --check`, `bash .agents/validate.sh` all green; `uv run pytest -m live --live` green **without** `--allow-live-writes` (the round-trip test skipped by default, its reason naming the route it would touch).
|
||||
- [ ] `data/logs/33-*.png` (switch off in both themes, the `DANGER` dialog with its diff, the writes-disabled arm, 360 px) and `data/logs/33-*.out`; `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING; `data/logs/NOTES-33-switching.md` records the drill, the hashes and the restore.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass), including the operator's steps for undoing all of this (`--without-routectl`, remove the units, re-enable every route).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs`, `/login`, the UPS command path and its four gates, and every non-gateway source behave exactly as phase 32 left them.
|
||||
Reference in New Issue
Block a user