phase: 33_gateway_route_switching
Phase 33 completion pass is done. Final report:
## Phase 33 — completion report
**All 8 task files complete** (`.agents/phases/complete/33_gateway_route_switching/`), full gate green, docs verified.
### Work in this pass
1. **`tests/test_routectl.py` added** (17 tests, all green) — exercises the *real* helper binary unprivileged against a fake tree (`--root`), the real agent parser, the captured gateway fixtures, and stubbed `nginx`/`haproxy`/`systemctl` on `PATH`. Pins the full protocol (locate → state → dry-run → temp-file/fsync/`os.replace` write → checker → restore on checker failure → reload-never-restart → re-read proof → sidecar record), all three route shapes (haproxy `server`, nginx `proxy_pass`, nginx stream `map` via its sink-value id), `drift` / unknown-id / `map-default` refusals, `list`/`verify`, and the `--serve` peer gate (`SO_PEERCRED` uid check, one JSON line per request, exact body shape).
2. **Real helper defect fixed** — `deploy/dashboards-routectl.py`: `main()`'s early-return paths returned `({"ok": False, …}, EXIT_*)` tuples, so `sys.exit()` printed a Python repr instead of a JSON line (usage errors, agent-import refusal). Now prints `json.dumps(…)` and returns only the code, keeping the one-JSON-line contract.
3. **Documentation consistency fixes** (task 08 criterion 6):
- `README.md`: garbled verdict-colour sentence on the switch ("green and to the position") → "in the **on** position and tinted `--gate-on`" (the existence-pair contract).
- `README.md`: audit sentence now names all three event kinds — `route-disabled` / `route-enabled` / **`route-denied`** (the code stores one per refused call; dry runs store none).
- `page-gateway.js`: replaced the stale pre-33 module docstring ("the switch is not wired") with the two-arm `GATEWAY_ALLOW_WRITES` contract.
- `AGENTS.md`: every phase-33 claim re-verified against code (the ten gates, the 35 s/30 s budget pair, `gateway.writes_enabled`, `/api/summary`'s `disabled`, the 1.4.0 agent bullet, the gap list) — no changes needed.
### Verification
- `uv run pytest --cov`: **1970 passed**, 18 skipped — coverage **98.74%** (floor 94%)
- `ruff` clean · `pyright` 0 errors · `bash .agents/validate.sh` **exit 0** (incl. `node --check` on every page script)
- Dev server running; all 10 sources `ok`; bundles exported
### Live infrastructure
**No live changes made.** The live drill remains unexecuted per `data/logs/NOTES-33-switching.md` (no operator approval; checker binaries unavailable in the podman proxy units). All routes stay enabled, config hashes unchanged, and the sandbox rehearsal + backup tree are in place for the operator's own `DANGER` call. All changes are in the working tree only.
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
# Phase 33 — The switch is wired: turning one route off, and back on, without ever weakening the box
|
||||
|
||||
**Source:** operator request, 2026-10-23 — "Also write the phase to disable and enable services. I want to be able to turn off and on connections as needed in the event of security concerns or upgrades." + the two decisions taken with them in that conversation: **enforcement is a config edit, validated and reloaded, by a narrow root helper on the gateway** (so a disabled route stays disabled across a reload *and* a reboot), and **one switch per connection line, as already drawn**.
|
||||
**Story:** n/a
|
||||
**Context:** `deploy/install-agent.sh` (the unit template, whose `NoNewPrivileges=true` / `ProtectSystem=strict` / `SystemCallFilter=@system-service` / empty `CapabilityBoundingSet` this phase **does not touch**, the `--force` re-run guard, `deploy/agent-sudoers` as the precedent for a path-pinned grant), `agent/host_agent.py` (1.3.0 → **1.4.0**: `do_GET`-only `Handler`, `_certificate_allows`, the `haproxy`/`nginx` parsers that must learn the disabled marker), `src/dashboards/sources/gateway.py` (`connections[]`, `config_connections`, `warnings()`' seven kinds), `src/dashboards/api.py` (`/api/ups/command`'s four-gate shape and the `certs_router()`'s audit-event rule), `src/dashboards/collectors.py` (`AgentTls`, `fetch_gateway_snapshot`, the revoke-before-dial rule), `src/dashboards/config.py` (`GATEWAY_*` block), `src/dashboards/web/static/page-gateway.js` (`buildSwitch`, `routeInConfig`, `renderLeaf`), `tests/test_api.py` (the verb-enumeration pins), `tests/test_auth_api.py` (the served-route walk), `tests/test_deploy_scripts.py` (the fake-root sandbox the helper's tests extend).
|
||||
|
||||
## Objective
|
||||
|
||||
`/gateway`'s switch becomes a control: **disable** `brain.reeseapps.com` and the nginx `stream` map entry stops routing it, persistently, with the config validated and the proxy reloaded and the page showing that exact route **red** until it is re-enabled. One new write route, `POST /api/gateway/route`, gated seven times, with the write itself performed off-box by a **root helper on the gateway that the agent reaches over a systemd-activated unix socket** — because the agent's own unit forbids privilege (`NoNewPrivileges=true`) and this phase is not going to weaken the sandbox of a read-only collector to give it a root shell.
|
||||
|
||||
This amends AGENTS.md invariant 1 from *one* exception to **two**, and that is the phase's real risk: a bug here is an outage on the fleet's only ingress. Everything below exists to make the blast radius one line of one file, and to make the app refuse rather than improvise whenever it cannot see what it is doing.
|
||||
|
||||
## Scope decisions taken with the operator (binding for this phase)
|
||||
|
||||
| # | Decision | Consequence |
|
||||
|---|----------|-------------|
|
||||
| 1 | **Invariant 1 becomes "two exceptions".** | `POST /api/ups/command` and `POST /api/gateway/route`. Both: session, CSRF/origin on a cookie write, an env gate (`NUT_ALLOW_ACTIONS` / `GATEWAY_ALLOW_WRITES`, default **off**), a typed `DANGER`, and every gate re-checked server-side on every request. `PLAN.md`'s anchor table records a read-only app, so this phase's overview carries the sign-off request; the phase never edits `PLAN.md`. |
|
||||
| 2 | **`DANGER` is required for both verbs.** | Enabling is not the safe direction: applying either change re-validates the file and **reloads the proxy**, and a reload touches every route on the box. A word that means "I know this affects the ingress" has to mean it both ways. |
|
||||
| 3 | **The agent keeps its sandbox; the privilege lives outside it.** | No sudo from the agent, no setuid, no change to `NoNewPrivileges`/`ProtectSystem`/`SystemCallFilter`. The write peer is `dashboards-routectl.socket` + `dashboards-routectl@.service` (systemd socket activation, `Accept=yes`, `User=root`, `SocketUser=dashboards-agent`, `SocketGroup=dashboards-agent`, `SocketMode=0600`): the agent connects to a socket it is *filemode-permitted* to connect to, and no other uid on the box can even open it, and the helper proves the peer uid with `SO_PEERCRED`. Access control is a file mode plus the mTLS client certificate that already authorises the agent's other paths — not a capability the agent process can be tricked into. |
|
||||
| 4 | **Only an id crosses a wire. Never a path, never a line, never text.** | The body is `{id, action, danger?, dry_run?}` where `id` is the payload's own `connections[].id`. The helper re-parses the configs itself, maps that id to the one line it declared, and refuses anything it cannot map. There is no `file`, no `line`, no `pattern`, no `command` anywhere in the contract, and the argv to the helper is fixed with no shell. |
|
||||
| 5 | **A route is off in the way that actually stops traffic for its proxy.** | haproxy: the `server` line is commented with a `# dashboards:off … was: <verbatim original>` marker. nginx vhost: the `proxy_pass` line is commented the same way. nginx **stream `map` entry**: commenting it would fall through to the map's `default` (`127.0.0.1:4443`) and **mis-route** the domain instead of blocking it, so the helper rewrites that entry's value to a sink (`127.0.0.1:44450`) and keeps the original in the marker; the sink is a `server` block the helper maintains in `/etc/nginx/stream.d/00-dashboards-sink.conf`, which the existing `stream { include /etc/nginx/stream.d/*.conf; }` glob already loads — so **no hand edit of the main config is ever required**. The map's `default` entry is **refused**: it is the fallback for every unmapped name, not a service. |
|
||||
| 6 | **Validate, restore on failure, reload — never restart.** | Write the new bytes, run `nginx -t` / `haproxy -c -f …`; on failure put the previous bytes back, do **not** reload, and answer `applied: false, validated: false, restored: true`. On success `systemctl reload <unit>` — reload, because a restart cuts every Minecraft session and every in-flight upload on an ingress that is doing its job. A `reload` failure is reported with the unit's own status and the file is left in the applied state (the config is correct; the reload is the operator's next action, said out loud). |
|
||||
| 7 | **Self-describing marker + drift refusal, not silent trust.** | The disabled line carries the original verbatim after `was:`, so a human can restore it by eye and the helper can restore it mechanically. A sidecar `/var/lib/dashboards-routectl/state.json` (`0600`, root) records actor, timestamp and the sha256 of what was written. Enabling restores only if the current line still matches what the helper wrote; otherwise it **refuses** and reports `drift` — a file an operator hand-edited is theirs to fix, not the app's to guess at. |
|
||||
| 8 | **Refuse rather than act blind.** | The write is refused when: writes are not enabled; the id is not in the **current** payload; the route is already in that state (`changed: false`, no reload); either config section is not `available: true`; the `gateway` job is `ok: false` or stale beyond 3× its interval; the `host:<GATEWAY_HOST>` system job is not `ok`; the agent is revoked (refused before any socket, the existing rule); the throttle is tripped. That is the UPS "refused while on battery" rule, translated: do not rewrite a config you cannot currently read end to end. |
|
||||
| 9 | **`off` finally means something the payload said.** | The agent's parsers learn the marker and report the marked `server`/`proxy_pass`/`map` entry with `disabled: true`, so the route stays in the graph as a declaration and `connections[].enabled` becomes **`false`**. `routeInConfig()` stops being a literal, the `off` styling phase 30 built becomes reachable, and `warnings()` learns the marker: a backend emptied by a marker is not `haproxy_backend_empty` (the operator chose that; the page shows it red), while a backend with no `server` line at all still is. |
|
||||
| 10 | **A live route's count is never stamped.** | Disabling does not write `0` into the payload. The sockets drain on the reload and the next poll reports what `/proc` says. Invariant: no code path in this phase writes a count. |
|
||||
| 11 | **The audit is one event, fixed in shape, with no config text in it.** | `gateway`-source events `route-disabled` / `route-enabled` / `route-denied`: verb, id, the session's actor name, `applied`/`validated`/`reloaded`/`drift`, and the proxy's own one-line error when there was one. The unified diff is returned to **the caller's response** (it is what the confirm dialog shows) and never stored — events are served to a browser and `/api/events` is a feed, not a config archive. |
|
||||
| 12 | **The overview is told, because "something is switched off" is not a page-only fact.** | `/api/summary`'s `gateway` block gains **one** field, `disabled` (an int count of `enabled is false` rows, `null` in the arms that did not measure) — the thirteenth key, restated in the pinned key set with its reason. A notifier has to learn about a deliberate outage without scraping a page, and it prints nothing at `0` (the `warnings` rule: a permanent `0 switched off` is ink that trains the eye). |
|
||||
| 13 | **One route, one file, one reload per request.** | A request touches exactly one config file and reloads exactly one proxy. There is no batch verb, no "disable all to this host", no `dry_run` that writes. If a security incident needs five routes off, that is five reviewed requests, each with its own event. |
|
||||
|
||||
**Out of scope:** editing anything a marker cannot express (adding/removing vhosts, certificates, `bind` lines, the map's `default`), haproxy runtime `disable server` via a stats socket (a runtime-only state that a reload silently undoes — decision 5 is persistent instead), disabling an entire **host** or **port**, per-user authorisation (one admin, gap 6), a scheduler ("off at 02:00"), and any change to the agent's unit security directives or to another host's agent (`gateway` alone runs 1.4.0; `deskwork`/`driveripper`'s 1.1.0 agents have no routes and are unaffected).
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `32_gateway_resolved_target_liveness` (todo, **must complete first**) — agent 1.3.0's resolver section and the counted name routes are what a switched-off line reads against: `off` + a draining count to `0` is the proof the switch worked, and without resolution a disabled route and a never-counted route look identical.
|
||||
- `31_gateway_route_readability` (todo) — the row that names the inbound domain, the green end-to-end route, and the `off` styling this phase finally reaches.
|
||||
- `30_gateway_connection_map` (complete) — `connections[].id` as the vocabulary, the switch markup and its a11y contract, `routeInConfig()` as the named seam.
|
||||
- `15_agent_mtls` / `24_dev_mtls_rollout` (complete) — the verified-client-certificate gate the new POST path inherits, and the revoke-before-dial rule this phase's collector reuses.
|
||||
- `11_admin_login` (complete) — session, CSRF, origin checks, `LoginThrottle` (the `route:<address>` bucket is its third key shape).
|
||||
- **Operator actions, not repo work:** the `dashboards-agent`-to-helper socket grant lands through the units (no sudoers line needed), but installing units on a host and running the first live drill are the operator's — and the drill **stops** unless they are present. `PLAN.md`'s anchor sign-off for invariant 1 is theirs too.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_the_root_helper_edits_exactly_one_route.md` — `deploy/dashboards-routectl.py`: `show`/`list`/`diff`/`disable`/`enable`/`verify`, the marker with `was:`, the sink file it manages, validate-restore-reload, drift refusal, `--serve` over a systemd-passed socket, `--root` for tests.
|
||||
2. `02_the_privileged_peer_is_a_socket_not_a_sudo_line.md` — the two systemd units + `install-agent.sh --with-routectl`; the proof that the agent's `NoNewPrivileges=true`/`ProtectSystem=strict` unit is unchanged and can still reach the socket.
|
||||
3. `03_the_agent_accepts_one_write.md` — agent **1.4.0**: `POST /route` behind the verified client certificate **and** `--allow-writes`, body validation, one fixed peer call, plus the parsers reporting `disabled` for a marked line; `/schema` and the additivity control.
|
||||
4. `04_the_source_publishes_enabled_and_the_warnings_learn_the_marker.md` — `connections[].enabled`, `off` reachable in the payload, `warnings()` marker-aware, stub/skeleton parity, the one new `/api/summary` field (`gateway.disabled`), and no count ever written.
|
||||
5. `05_the_api_gates_the_write_ten_times.md` — `GATEWAY_ALLOW_WRITES`, `DASH_READ_ONLY`, `collectors.post_route`, `POST /api/gateway/route`, the ten-gate order, the throttle, the events, `/api/config.gateway.writes_enabled`, and the verb-enumeration + served-route pins restated (this is where invariant 1 is amended in `AGENTS.md`).
|
||||
6. `06_the_switch_obeys.md` — `page-gateway.js` + `gateway.html` + `dash.css`: `DANGER` confirm with the diff from a `dry_run` first, patch from the response, `routeInConfig()` reading `enabled`, the writes-disabled arm that keeps the switch honest, footer paragraph 3.
|
||||
7. `07_rehearse_then_drill_on_the_live_ingress.md` — the fake-tree rehearsal, then the live install on `gateway` and a drill on the one named-safe route, restore, verify, `data/logs/NOTES-33-switching.md`. Stops, correctly, without the operator present.
|
||||
8. `08_docs_and_live_verify.md` — `AGENTS.md` (invariant 1, the agent bullet, `sources/gateway.py`, `api.py`, `collectors.py`, `config.py`'s block, the `/gateway` UI paragraph, the fleet record, gap list) + `README.md` (including the operator's sudo/units steps), the browser pass, the mobile audit, the full gate.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **The helper is tested unprivileged, against a fake tree, with the proxy commands stubbed.** `tests/test_routectl.py` runs the real helper in a `tmp_path` tree (`--root`) with `nginx`/`haproxy`/`systemctl` faked on `PATH`, and pins: one line changed and nothing else in the file (byte-diff assertion); `was:` restores verbatim; a failing validator restores the exact previous bytes and reloads nothing; `default` refused; an unknown id refused; a second disable `changed: false`; the sink file created on first use, left alone if an operator's file is there, removed only when it wrote it and nothing is disabled; drift refused rather than restored; `SO_PEERCRED` refused for a non-`dashboards-agent` uid in the fake-peer test; no `shell=True` anywhere (`grep` pin, the installer's precedent).
|
||||
- **The agent's write path is proven against a fake peer, not a real one.** `tests/test_agent.py` runs the real handler over a TLS loopback port (the phase-15 harness) plus a unix-socket stub: `POST /route` without a client certificate is 403 `client certificate required`; with `--allow-writes` absent it is 403 `writes are not enabled on this agent`; a body with an unexpected key, a bad id, or a missing `danger` is refused **before** the peer is touched (asserted by the stub logging nothing); `GET` behaviour is byte-identical to 1.3.0 (the additivity control re-run).
|
||||
- **The API gates are proven twice, as the UPS ones are.** `tests/test_api.py` walks every gate with spies: writes-disabled refuses without dialling; a stale `gateway` job refuses; a revoked agent refuses without a socket; `DANGER` is required for `disable` **and** `enable`; a right request produces one peer call, one event, and the response's `applied`/`validated`/`reloaded` triple. `tests/test_auth_api.py`'s served-route walk demands a session for `/api/gateway/route`, and the CSRF pin is extended to it on the **wire** (a scripted helper records nothing without the header, once with it) — a 403 after the write is an outage with an error message attached.
|
||||
- **Live, opt-in, and reversible:** `-m live --live` gains `test_the_switch_round_trip_on_one_named_route` which is **skipped unless `--allow-live-writes` is passed as well**, names the route in the skip reason, and always restores in a `finally`. The default live suite stays read-only (AGENTS.md's rule survives this phase; the drill belongs to task 07 with the operator watching).
|
||||
- Coverage **>90%** on `deploy/dashboards-routectl.py`, `agent/host_agent.py`, `sources/gateway.py`, `collectors.py`, `api.py`; the whole suite 100%; `bash .agents/validate.sh` green.
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it will stand after this phase and fixes every inconsistency found, **including pre-existing ones**. This phase's specific list: **invariant 1** rewritten to two exceptions with the seven gates named (and the sentence "hiding a button is not a safety control" kept, because it is the reason the switches are gated server-side rather than greyed in JS); the agent bullet's `POST /route` and the `disabled` marker; the `sources/gateway.py` bullet's `enabled`/`off` rules and the marker-aware `warnings()`; the `api.py` bullet's write lists (fleet writes are now two); the `collectors.py` bullet (`post_route`, the revocation check it shares); `config.py`'s `GATEWAY_ALLOW_WRITES` row and README's configuration table; the `/gateway` UI-conventions paragraph (the switch is wired, what `off` looks like, what the writes-disabled arm says); the fleet record (gateway **1.4.0**, its unit's security directives unchanged, the two new systemd units, `NOTES-33-switching.md`); and the gap list (name-route switching **closed**, plus the honest limits: one admin, no scheduler, no per-host kill).
|
||||
- `README.md` gains the operator's install steps for the helper units and the exact `--with-routectl` invocation, plus the recovery command for the state file (`systemctl status dashboards-routectl.socket`, `routectl list`, `routectl verify`).
|
||||
- Tasks update both files immediately when their own changes are user-visible.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] With `GATEWAY_ALLOW_WRITES=1`, `/gateway` disables `brain.reeseapps.com`: the confirm dialog shows the **diff from a `dry_run`**, asks for `DANGER`, and the row turns **red** (`off`) on the poll after the reload while its live count drains to `0` on its own — no code wrote that zero.
|
||||
- [ ] `ssh gateway-ops 'sudo /usr/local/sbin/dashboards-routectl.py list'` names exactly that route; `verify` reports no drift; the marked line contains the original verbatim after `was:`; the map entry points at the sink and `stream.d/00-dashboards-sink.conf` exists and is owned by the helper's header.
|
||||
- [ ] Enabling restores the original bytes exactly (byte-compare against a pre-drill copy), the row turns green, and `list` is empty afterwards. **The live ingress ends the phase in the state it started in.**
|
||||
- [ ] Every gate refuses with its own sentence and **without** touching a peer: writes disabled, no session, no CSRF, wrong `Origin`, missing `DANGER`, unknown id, already-in-state, section unreadable, `gateway` job stale, `host:gateway` not `ok`, agent revoked. Each is a test, and each test asserts the stub peer logged nothing.
|
||||
- [ ] The agent's unit is unchanged: `systemctl cat dashboards-agent` still shows `NoNewPrivileges=true`, `ProtectSystem=strict`, `SystemCallFilter=@system-service`, and the same sha256 the phase recorded before installing the units — the sandbox was not traded for the feature.
|
||||
- [ ] A failing validator (`nginx -t` stubbed to exit 1) restores the previous bytes, reloads nothing, answers `applied: false, validated: false, restored: true`, and the page's line never turned `off`.
|
||||
- [ ] `POST` inventory: the app now has exactly two upstream writes (`/api/ups/command`, `/api/gateway/route`) and the enumeration pin lists them as such; `AGENTS.md` invariant 1 says **two** and names both gates; `/api/health` gained nothing.
|
||||
- [ ] `uv run pytest` (with `--cov` ≥ 94), `uv run ruff check src tests agent deploy`, `uv run pyright`, `node --check`, `bash .agents/validate.sh` all green; `uv run pytest -m live --live` green **without** `--allow-live-writes` (the round-trip test skipped by default, its reason naming the route it would touch).
|
||||
- [ ] `data/logs/33-*.png` (switch off in both themes, the `DANGER` dialog with its diff, the writes-disabled arm, 360 px) and `data/logs/33-*.out`; `data/logs/27-task02-mobile-audit.cjs` exits 0 with 0 FAIL / 0 FINDING; `data/logs/NOTES-33-switching.md` records the drill, the hashes and the restore.
|
||||
- [ ] README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass), including the operator's steps for undoing all of this (`--without-routectl`, remove the units, re-enable every route).
|
||||
- [ ] No behaviour change in completed work: the other five dashboards, `/settings`, `/certs`, `/login`, the UPS command path and its four gates, and every non-gateway source behave exactly as phase 32 left them.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Task 01 — The root helper edits exactly one line, proves it validates, and reloads or gives up
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** operator decision 2026-10-23 — "Edit the config + validate + reload, via a narrow root helper on the gateway"
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`deploy/dashboards-routectl.py` — a stdlib-only root program that turns one `connections[].id` off or back on in one of the two proxy configs, validates the file with the proxy's own checker, reloads that one proxy, and refuses everything else. It is the phase's only writer, it has no network listener of its own (task 02 hands it a systemd-activated socket), and it can be run by hand on the gateway by the operator to see exactly what the dashboard would do.
|
||||
|
||||
The helper **imports the agent's config parser by file path** and never re-implements a rule: two parsers reading one config is two opinions about the same file, which is the failure mode this repo has pinned against since phase 28. `agent/host_agent.py` is world-readable at `/usr/local/lib/dashboards-agent/host_agent.py`, so the helper imports it with `importlib` and refuses to run at all if it cannot.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py` — the shared grammar only (no version bump here; `AGENT_VERSION` moves in task 03):
|
||||
```python
|
||||
ROUTE_MARKER = "# dashboards:off" # the only marker this app writes into a proxy config
|
||||
ROUTE_WAS = "was:" # `<marker> <id> was: <original line verbatim>`
|
||||
def marker_for(route_id: str, original: str) -> str: ...
|
||||
def split_marker(line: str) -> tuple[str, str] | None: # (route_id, original) or None
|
||||
def config_line_index(haproxy_text, haproxy_path, nginx_docs) -> dict[str, dict[str, Any]]:
|
||||
"""route id -> {"file", "line", "original", "kind", "disabled"} for every declared target."""
|
||||
```
|
||||
`config_line_index` walks the parsed declarations **with their source lines**, so the id → (file, line, verbatim text) map is derived by the same parser that produced the graph, and a marked line is reported as the same id with `disabled: true`. It is pure over text; the file reading stays in the callers (`Collector` for the agent, the helper for the writes), both resolved under `--root`.
|
||||
2. `deploy/dashboards-routectl.py` (new, `0755`, stdlib, no third-party import, no `shell=True`)
|
||||
- **Arguments** (all default to the deployed paths, all overridable for the fake tree):
|
||||
`--root /` · `--haproxy-config /etc/haproxy/haproxy.cfg` · `--nginx-config /etc/nginx/nginx.conf` · `--state /var/lib/dashboards-routectl/state.json` · `--sink-conf /etc/nginx/stream.d/00-dashboards-sink.conf` · `--sink-address 127.0.0.1:44450` · `--agent-module /usr/local/lib/dashboards-agent/host_agent.py` · `--nginx-unit nginx` · `--haproxy-unit haproxy` · `--apply` (without it every write verb is a dry run).
|
||||
- **Verbs**, one JSON object to stdout, non-zero exit on refusal:
|
||||
`list` · `show --id ID` · `diff --id ID --action disable|enable` · `disable|enable --id ID [--apply]` · `verify` (marker/hash vs file → drift report) · `--serve` (read one JSON request from the systemd-passed socket, answer one JSON line).
|
||||
- **Per-kind edit, exactly as decision 5** (the three arms and the `default` refusal are this file's core; a fourth arm is a new decision, not a fall-through):
|
||||
| id kind | action | why |
|
||||
|---|---|---|
|
||||
| haproxy `server` | comment the line with `ROUTE_MARKER` + `was:` | the backend stops receiving; the marker keeps the original restorable by eye and by machine |
|
||||
| nginx vhost `proxy_pass` | comment the line the same way | the vhost keeps `listen` + `server_name` and gains no target — a shape `warnings()` already treats as quiet (the `:80 return 301` arm), so "off" does not read as "broken" |
|
||||
| nginx stream `map` entry | **rewrite the value** to the sink address, marker + `was:` on the line, ensure the sink file exists | commenting would fall through to the map's `default` (`127.0.0.1:4443`) and **mis-route the domain instead of blocking it** |
|
||||
| nginx `map` `default` | **refuse** — `{"applied": false, "refused": "map-default"}` | it is the fallback for every unmapped SNI, not a service |
|
||||
- **The sink file**: created on first need with a `# managed by dashboards-routectl — do not edit` header and a stream `server` block that listens on `127.0.0.1:44450` and returns a short refusal string; loaded by the existing `stream { include /etc/nginx/stream.d/*.conf; }` glob, so **the main config is never edited**. If a file already exists at that path without the header, the helper refuses (an operator's file is not its own to overwrite). It deletes the file only when it wrote it **and** nothing is disabled, and never removes it while any map entry still points at the sink.
|
||||
- **Write protocol** (order is the safety property, so it is written as numbered steps in the file):
|
||||
1. re-read the file; index it; map `id` → line (unknown id → refuse, nothing written);
|
||||
2. already in the requested state → `{"applied": false, "changed": false}` and **no reload**;
|
||||
3. `--apply` absent → answer with the unified `diff` only;
|
||||
4. write to `mkstemp` in the same directory, `fsync`, `os.replace`;
|
||||
5. run `nginx -t` or `haproxy -c -f <that file>`; on failure `os.replace` the previous bytes back and answer `{"applied": false, "validated": false, "restored": true}` with the checker's own first line — **and reload nothing**;
|
||||
6. `systemctl reload <unit>`; a failure is reported as `reloaded: false` with the unit's status line, leaving the applied file in place (the config is right; the reload is the operator's next action and the answer says so);
|
||||
7. re-read and re-index to prove the marker is live; record state.
|
||||
- **Never `restart`** — the docstring says why in one sentence: a restart cuts every live connection through the fleet's only ingress, and a reload does not.
|
||||
- **State sidecar**: `0600`, root, `{id: {"actor", "at", "file", "sha256", "action"}}`, written with `os.open(..., O_CREAT, 0o600)` so the mode is its birth mode (the `dev-certs-export.py` idiom). `enable` restores **only** when the current line hashes to what the helper wrote; otherwise `{"applied": false, "drift": true, "hint": "…"}` and no byte changes.
|
||||
- **`--serve`**: `socket.fromfd(3, AF_UNIX, SOCK_STREAM)` (or the `LISTEN_FDS`/stdin idioms, chosen once and documented), read one newline-delimited JSON request, `getsockopt(SO_PEERCRED)` and **refuse any uid that is not the agent's** (`--allow-uid`, default `dashboards-agent`, resolved with `pwd.getpwnam`), dispatch one verb, write one JSON line, exit. One request per activation: systemd spawns the instance per connection (`Accept=yes`), so there is no accept loop, no thread, no timeout policy to invent.
|
||||
- **Nothing in this file takes a path, a line number, a pattern or a command from the wire.** `--serve`'s accepted keys are exactly `{id, action, dry_run}` and every other key is a refusal. Assert it structurally in the tests.
|
||||
- Every subprocess call is a fixed argv list (`subprocess.run([...])`, no `shell=True`), and the checker/unit names come from **flags the operator installed**, never from the request.
|
||||
3. `tests/test_routectl.py` (new) — real helper, fake tree (`--root`), fake `nginx`/`haproxy`/`systemctl` scripts on `PATH` that record their argv and answer to order:
|
||||
- `test_disabling_one_route_changes_exactly_one_line` — unified diff of the whole file is one hunk, one line; every other byte identical (assert on the bytes, not on a count).
|
||||
- `test_a_disabled_map_entry_points_at_the_sink_and_keeps_its_original` — the sink file exists with its header, the marked line carries `was: <verbatim>`, and re-enabling restores the file byte-identical to the pre-drill capture.
|
||||
- `test_the_map_default_is_refused_and_nothing_is_written`, `test_an_unknown_id_is_refused`, `test_disabling_a_disabled_route_changes_nothing_and_reloads_nothing` (the fake `systemctl` log stays empty).
|
||||
- `test_a_failing_validator_restores_the_previous_bytes` — byte-identical restore, `validated: false`, `restored: true`, zero reload calls, non-zero exit.
|
||||
- `test_a_failing_reload_says_so_and_leaves_the_file_applied`, and `test_reload_is_never_a_restart` (grep the source for `restart` in a call position, the installer's `--status` style pin).
|
||||
- `test_the_helper_imports_the_agents_parser_and_refuses_without_it` — point `--agent-module` at a non-file: exit non-zero, no write, and the message names the path (a helper that silently skipped the parser would be a second opinion about the config).
|
||||
- `test_state_is_0600_and_drift_is_refused` — mode at birth; hand-edit the marked line and `enable` refuses with `drift: true` and no byte change.
|
||||
- `test_serve_refuses_a_peer_that_is_not_the_agent_uid`, `test_serve_accepts_exactly_id_action_dry_run`, `test_no_shell_and_no_path_from_the_wire` (structural greps over the file, the `certs.py`/`auth.py` precedent).
|
||||
- `test_the_sink_file_is_left_alone_when_someone_elses_wrote_it`, `test_the_sink_is_removed_only_when_it_is_empty_of_our_routes`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the whole matrix above, hermetic and unprivileged, over the **captured** `gateway_haproxy.cfg` / nginx tree so the edits are pinned against real config bytes rather than a toy file.
|
||||
- Coverage **>90%** on `deploy/dashboards-routectl.py` and the new `agent/host_agent.py` functions; `uv run ruff check agent deploy` + `uv run pyright` clean; `python3 -m py_compile` on the helper (it must run on a host's Python 3.10, so no 3.14-only syntax — pin that with a `# requires: python >=3.10` header and a test that greps for `match `/`typing.Self`-style 3.11+ syntax if the repo has such a pin; otherwise assert `ast` parses with `feature_version=(3,10)`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run python deploy/dashboards-routectl.py --root <tmp> … diff --id nginx:443->reeseapps-brain-of-reese.reeselink.com:443` prints a one-hunk diff and writes nothing; with `--apply` it edits, validates, reloads (against the fakes) and answers `{"applied": true, "validated": true, "reloaded": true}`.
|
||||
- [ ] Re-enabling leaves the tree byte-identical to its pre-drill `sha256sum` for **both** config files.
|
||||
- [ ] `grep -n "shell=True\|restart" deploy/dashboards-routectl.py` finds neither in a call position; `grep -n "def \|argparse" ` shows no `--file`/`--line`/`--pattern` option anywhere.
|
||||
- [ ] `uv run pytest tests/test_routectl.py -q` 100% passing; whole suite green; coverage not down.
|
||||
- [ ] README.md/AGENTS.md note the helper's existence and its verbs (this task's user-visible surface — the operator can run it by hand) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `/snapshot`'s document is unchanged by this task (the `AGENT_VERSION` bump and the `disabled` field are task 03's, with the additivity control).
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Task 02 — The privileged peer is a socket systemd activates, not a privilege the agent is given
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the agent unit's `NoNewPrivileges=true` + `ProtectSystem=strict` (read from `deploy/install-agent.sh`), which this phase must not weaken to get a write
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Install the helper on the gateway as a **socket-activated root service** the agent may connect to and nothing else: `dashboards-routectl.socket` owned by the agent user at `0660`, `dashboards-routectl@.service` running as root with `Accept=yes`. The agent's own unit keeps every one of its security directives — the phase's central safety claim is that a read-only collector stayed read-only in its sandbox and talked to a peer outside it.
|
||||
|
||||
## Work
|
||||
|
||||
1. `deploy/dashboards-routectl.socket` (new)
|
||||
```ini
|
||||
[Socket]
|
||||
ListenStream=/run/dashboards-routectl.sock
|
||||
SocketUser=dashboards-agent
|
||||
SocketGroup=dashboards-agent
|
||||
SocketMode=0600
|
||||
NoDelay=yes
|
||||
[Install]
|
||||
WantedBy=sockets.target
|
||||
```
|
||||
`0600` + `SocketUser=dashboards-agent` is the whole access control at the file layer: only that uid can connect, everyone else gets `EACCES` before any code of ours runs. The peer check in task 01's `--serve` is the **second** look, not the first.
|
||||
2. `deploy/dashboards-routectl@.service` (new)
|
||||
```ini
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/sbin/dashboards-routectl.py --serve \
|
||||
--allow-uid dashboards-agent --state /var/lib/dashboards-routectl/state.json
|
||||
User=root
|
||||
StandardInput=socket
|
||||
StandardOutput=socket
|
||||
StateDirectory=dashboards-routectl
|
||||
ProtectSystem=read-only # it writes two config files by absolute path; see ReadWritePaths
|
||||
ReadWritePaths=/etc/haproxy /etc/nginx/stream.d
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
NoNewPrivileges=true
|
||||
SystemCallFilter=@system-service
|
||||
```
|
||||
Three notes the file must carry as comments: the helper needs `/etc/haproxy` and `/etc/nginx/stream.d` writable and **nothing else** (`/etc/nginx/conf.d` is deliberately absent — vhost edits land there, so the *vhost* arm is listed in `ReadWritePaths` too and that addition is a decision recorded in the file, not smuggled in); `NoNewPrivileges=true` here as well, because it gets its privilege from being started by systemd as root, not from escalation; and `Accept=yes` on the template means systemd spawns one instance per connection with the socket on stdin, which is why the helper handles exactly one request and exits.
|
||||
- **`ReadWritePaths` must list every directory whose file the helper replaces** (`os.replace` needs write on the directory): `/etc/haproxy`, `/etc/nginx/conf.d`, `/etc/nginx/stream.d`. Getting this wrong is a `EROFS` in production, so the live drill in task 07 is the test that matters and the units file names it.
|
||||
3. `deploy/install-agent.sh` — one new flag, `--with-routectl`, off by default, and its inverse `--without-routectl`:
|
||||
- installs `dashboards-routectl.py` to `/usr/local/sbin/` (`0755`, root) and the two units to `/etc/systemd/system/`, `daemon-reload`, `systemctl enable --now dashboards-routectl.socket`;
|
||||
- **changes nothing about the agent's own unit** — assert that structurally: the rendered `dashboards-agent.service` bytes are identical with and without the flag (`tests/test_deploy_scripts.py`), because the temptation to "help" the agent reach the socket by relaxing it is exactly what this task exists to close;
|
||||
- verifies reachability from the agent's own uid before reporting success: `sudo -u dashboards-agent python3 -c 'import socket; s=socket.socket(socket.AF_UNIX); s.connect("/run/dashboards-routectl.sock")'` — a socket the agent cannot connect to is a feature that fails at 03:00 during an incident, so this check is part of the install, and its output is printed;
|
||||
- `--without-routectl` disables and removes the units and the helper, leaves `/etc` and the state directory alone, and reminds the operator to re-enable any route still off (`routectl list` output printed first).
|
||||
- The re-run guard applies as it does to the unit: a differing installed helper is a diff + exit 1 unless `--force`.
|
||||
4. `deploy/gateway-routectl-README.md` (new, short) — the operator's page: what the two units are, how to see them (`systemctl status dashboards-routectl.socket`, `journalctl -u 'dashboards-routectl@*'`), how to run the helper by hand on the box, how to undo it all, and the sentence that **no sudoers line is involved**: the grant is a file mode systemd enforces.
|
||||
5. `tests/test_deploy_scripts.py` — extend the fake-root sandbox:
|
||||
- `test_with_routectl_installs_helper_and_units_and_leaves_the_agent_unit_byte_identical`;
|
||||
- `test_the_socket_is_0600_and_owned_by_the_agent_user` (the fake `install`/`chown`/`chmod` log, the precedent that proved the bundle's `0640`);
|
||||
- `test_the_helper_is_installed_0755_root_and_its_state_dir_is_created`;
|
||||
- `test_without_routectl_removes_the_units_and_not_the_config_or_the_state`;
|
||||
- `test_no_new_sudoers_line_exists` (the `deploy/agent-sudoers` file is unchanged, and nothing in the new units grants sudo);
|
||||
- `test_the_install_checks_the_agent_can_reach_the_socket` — a fake where that connect fails makes the install exit non-zero rather than reporting a success nobody can use.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; the units are also parsed (`systemd-analyze verify` is not available in the sandbox, so the assertions are on the directive lines, and task 07 runs the real `systemctl` on the gateway).
|
||||
- Coverage: the installer is shell; the gate is the sandbox tests plus `bash -n`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `bash -n deploy/install-agent.sh` clean; `grep -n "NoNewPrivileges\|ProtectSystem\|SystemCallFilter" deploy/install-agent.sh` shows the agent unit's values unchanged from before this task (the guard against "fixing" reachability by weakening the sandbox), and `git diff` for this task shows no edit inside the agent unit template block.
|
||||
- [ ] `uv run pytest tests/test_deploy_scripts.py -q` 100% passing, including the byte-identical agent-unit assertion; whole suite green; coverage not down.
|
||||
- [ ] The two unit files list exactly the paths the helper writes and no others, with the `os.replace`-needs-directory reason stated in a comment.
|
||||
- [ ] README.md/AGENTS.md gain the two unit names and the "no sudoers line" sentence (this task's user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: an install **without** `--with-routectl` is byte-identical to an install before this phase, on every host.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Task 03 — Agent 1.4.0: one POST path, behind the certificate and a flag, plus the marker read back as `disabled`
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the agent's `do_GET`-only `Handler` and `_certificate_allows` (phase 15), and phase 30's `routeInConfig()` seam
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` goes **1.3.0 → 1.4.0** with two additions and no other change: the two config parsers report a marker-authored line as a declared target with **`disabled: true`**, and one new path, `POST /route`, which is reachable only with a verified client certificate **and** an agent started with `--allow-writes`, forwards exactly `{id, action, dry_run}` to the task-02 socket, and answers with the helper's JSON. Every `GET` behaviour stays byte-identical, proven by the additivity control.
|
||||
|
||||
## Work
|
||||
|
||||
1. `agent/host_agent.py`
|
||||
- Parsers: each haproxy `server` node and each nginx `proxy_pass` / `upstream` member / `map` entry gains `"disabled": <bool>` — `false` for an ordinary line, `true` for one carrying `ROUTE_MARKER` (read with `split_marker`). Additive: no existing key changes meaning, and a route that is off **stays in the document** because "the config declares this and the operator switched it off" is a different fact from "the config never said it" (rule 2).
|
||||
- **New path** in `Handler`:
|
||||
```python
|
||||
def do_POST(self) -> None: # the BaseHTTPRequestHandler API name — keep it
|
||||
if self.path.split("?", 1)[0].rstrip("/") != "/route":
|
||||
self._send(405, {"ok": False, "detail": "no such write path"})
|
||||
return
|
||||
# the certificate check first, exactly as do_GET does it, revoked serial included;
|
||||
# then the flag; then the body; then the peer. Nothing else in this method writes.
|
||||
```
|
||||
Order, and each refusal is its own sentence: no bundle / no verified certificate → `403 client certificate required` (and `403 client certificate revoked` for a withdrawn serial, on this path too); agent started **without** `--allow-writes` → `403 writes are not enabled on this agent`; body not a JSON object, longer than `_ROUTE_MAX_BODY = 4096`, carrying a key outside `{id, action, dry_run}`, an `action` outside `{disable, enable}`, or an `id` longer than 256 chars → `422` with a fixed message and **no peer call**; no `--routectl-socket` configured → `503 no route helper on this host`.
|
||||
- Peer call: one newline-delimited JSON request over `AF_UNIX`, `settimeout(ROUTE_TIMEOUT_SECONDS = 30)` (the helper may run `nginx -t` and a reload), one JSON line back, oversized or unparsable replies are `502 route helper answered something unreadable` and **never** retried — a retry of a write is a second reload. Any `OSError` → `502` with the errno word, no traceback text, no path echoed beyond the flag's own configured value.
|
||||
- **No token path ever gets this**: on the plain-`http` bearer-token arm, `POST /route` is refused with `403 writes require a certificate` — the same reasoning phase 15 applied to reads (the handshake is the identity), and this app's fleet is mTLS anyway.
|
||||
- `--allow-writes` (default off) and `--routectl-socket /run/dashboards-routectl.sock`; both appear in the `/schema` doc block. `--status` and `--uninstall` are unaffected.
|
||||
- `log_message`: the POST line prints the path and the status only — never the body, never the id (an id is a hostname, and the agent's log line convention is that query strings stay out).
|
||||
- `AGENT_VERSION = "1.4.0"`; `/schema` names `POST /route`, `--allow-writes`, `--routectl-socket`, and the `disabled` field on the proxy sections; the status page lists the write path with its refusal sentence so a human with a browser is not surprised by a 405.
|
||||
2. `tests/test_agent.py` — the TLS loopback harness (phase 15) plus a unix-socket stub peer that logs what it received:
|
||||
- `test_the_agent_version_is_1_4_0_and_the_schema_names_the_write_path`;
|
||||
- `test_a_post_without_a_client_certificate_is_refused` and `test_a_revoked_serial_is_refused_on_the_write_path_too` (the denylist is not a read-side rule);
|
||||
- `test_writes_off_refuses_before_the_peer_is_touched` (stub logs nothing);
|
||||
- `test_the_body_vocabulary_is_id_action_dry_run_exactly` — a key added, a wrong `action`, a 5 KB body, a non-object: all refused, all with the peer idle;
|
||||
- `test_a_good_request_reaches_the_peer_once_and_the_answer_is_passed_through` (including `applied:false` staying `false` — the agent invents no success), `test_the_peer_is_called_once_even_when_it_times_out`, `test_an_unreadable_peer_answer_is_502_and_is_not_retried`;
|
||||
- `test_token_mode_cannot_write`; `test_every_other_path_is_405`;
|
||||
- `test_a_marked_server_is_reported_disabled_and_stays_in_the_document` (haproxy + all three nginx arms), `test_an_unmarked_config_has_disabled_false_everywhere`.
|
||||
- **Additivity control re-run**: `data/logs/verify-33-additive.py` (from task 02 of phase 32) extended to demand 1.4.0's `GET /snapshot` equals 1.3.0's with `disabled` added at the target level and nothing else, and equal *exactly* on an unmarked config.
|
||||
3. `src/dashboards/collectors.py` — a `post_route(tls, target, body)` beside `fetch_gateway_snapshot`, reusing the same `AgentTls` material and the **same revoke-before-socket check**, and redacting the URL in every error string it produces (invariant 6: those strings become events).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the matrix above; hermetic (a stub unix socket, no helper, no root, no proxy binaries).
|
||||
- Coverage **>90%** on the new agent code and `collectors.post_route`; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `python3 agent/host_agent.py --help` lists `--allow-writes` and `--routectl-socket`; `/schema` names `POST /route` and `disabled`.
|
||||
- [ ] Against the fake peer: the full refusal matrix answers without a peer call, and the success path calls it **exactly once** and passes the JSON through unaltered.
|
||||
- [ ] `uv run python data/logs/verify-33-additive.py` exits 0: 1.4.0 equals 1.3.0 plus `disabled`, and equals it exactly on unmarked configs.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md note the write path and its two agent-side gates (this task's user-visible surface) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: `GET /snapshot`, `/health`, `/info`, `/schema` and the TLS matrix behave exactly as 1.3.0; `deskwork`/`driveripper` are untouched.
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Task 04 — `enabled` joins the payload, `off` becomes reachable, and the warnings stop calling a chosen outage a fault
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** phase 30 decision 1 ("red is the **off** state, styled now, unreachable until the write path exists")
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`connections[].enabled` carries the config's own state (`true` declared and active, `false` declared and marker-off), so `routeInConfig()` stops being a literal and the red styling phase 30 built has a referent. And `warnings()` learns the marker: an empty backend, a targetless vhost or a sink-pointed map entry **caused by the operator's own switch** is not a config-integrity fault, while the same shape caused by a hand-deleted line still is. No count is ever written by this phase.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/sources/gateway.py`
|
||||
- `_connection()` gains `"enabled": True`; `config_connections()` sets it from the agent's per-target `disabled` flag (`enabled = not disabled`), and the merge rule keeps its meaning: **a line is enabled only if every claim that merged into it is enabled** — a union of selectors with one disabled line is two lines, so if a merge would mix states the source must not silently union them; state which arm wins and test it (recommended: split into two rows keyed by `enabled`, because "one of these selectors is off" is two facts).
|
||||
- `_skeleton()`/`stub()` gain the key so the parity pin keeps holding; `enabled` is `None` in the stub (nobody read the config), `true`/`false` in an answer — never `false` in an arm that did not read it.
|
||||
- `warnings()` marker-awareness, per kind, and each is a decision to write down rather than a suppression to hope nobody notices:
|
||||
| kind | new rule |
|
||||
|---|---|
|
||||
| `haproxy_backend_empty` | not raised when the backend's only servers are `disabled: true` (the operator emptied it from `/gateway`); still raised when it has no `server` line at all |
|
||||
| `haproxy_route_dangling` / `haproxy_backend_orphan` | unchanged — a switch never renames or removes a `backend`/`use_backend`, so a dangling name is still a broken config |
|
||||
| `nginx_server_unlistened` | unchanged (a disabled `proxy_pass` does not remove a `listen`) |
|
||||
| `nginx_pool_undeclared` / `nginx_variable_unmapped` | unchanged |
|
||||
| `nginx_map_route_missing` | a map entry pointing at the **sink address** is not a missing route; it is a switched-off route, and the sink address is recognised from the payload's own `resolver`/config text rather than hard-coded in the source (a literal in two places is how this drifts) |
|
||||
- The **socket join** is unchanged in code and in ethics: a disabled haproxy `server` has no live sockets after the reload because they drain, and a disabled map entry's sockets to the sink are the sink's own loopback pair. Nothing stamps `0`, nothing zeroes a count, and the `shared` rule is untouched.
|
||||
- `samples()` publishes the gauge for an `enabled: false` line only if the join measured a real count for it (usually `0`, and `0` measured is a legitimate measurement — the distinction is *measured*, never *assumed*).
|
||||
- No `severity` anywhere, still: `off` is a state, not a verdict. The recursive no-verdict walk in `tests/test_api.py` must stay green without an exception for `enabled`.
|
||||
2. `src/dashboards/api.py` — `/api/gateway/snapshot` is otherwise unchanged, but the `gateway` block in `/api/summary` is the one place this phase **does** add a field, because the overview's one line is where "something is switched off" belongs and a page-only fact would be invisible to a notifier: `disabled` (int, count of `enabled is false` rows, `null` in the arms that did not measure). That makes thirteen keys — restate the pinned key set in the same commit, with the reason, and keep `warnings` as a count beside it.
|
||||
3. `tests/test_gateway_source.py`
|
||||
- `test_a_marker_off_route_is_declared_and_disabled` — a config tree with one marked haproxy server + one marked vhost `proxy_pass` + one sink-pointed map entry yields three rows, each `enabled: false`, all their selectors intact, every other row `enabled: true`.
|
||||
- `test_the_source_never_calls_a_route_broken_because_the_operator_disabled_it` and `test_the_same_shape_without_a_marker_is_still_a_warning` — both directions for each of the two affected kinds, run over the **captured** configs plus a marker-injected copy of them (never a toy file).
|
||||
- `test_the_sink_address_is_recognised_not_hard_coded` — a config that writes a different sink port still reads as switched-off, because the sink is identified from the marker+config, not from `44450`.
|
||||
- `test_a_disabled_line_still_publishes_only_a_measured_count` — with no sockets, `connections: 0` only when the table answered; `null` when it did not, and no sample in the second case.
|
||||
- `test_the_stub_and_the_answer_share_one_key_set` gains `enabled`; `test_the_gateway_snapshot_carries_no_verdict_and_no_window` unchanged (extended walk).
|
||||
4. `tests/test_api.py` — the `/api/summary` `gateway` block key set grows `disabled` with its reason in the test docstring; `test_the_overview_line_reads_the_gateway_block_it_claims` learns the new field (and that `0` prints nothing — a permanent `0 switched off` is the ink the empty-state rules exist to avoid, the `warnings` rule restated).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: every rule above from real captures plus injected markers; both directions for each suppression.
|
||||
- Coverage **>90%** on `sources/gateway.py` and `api.py`; whole suite 100%.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/gateway/snapshot` on a fixture with three markers shows three rows with `enabled: false`, their selectors and endpoints intact, and `warnings == []` for the marker-caused shapes.
|
||||
- [ ] Removing the markers from the same tree brings the corresponding warnings back (the suppression is marker-caused, not shape-caused) — asserted as one test pair.
|
||||
- [ ] `/api/summary`'s `gateway.disabled` is `3` on that fixture, `0` on the clean capture, `null` on the stub, and the overview line prints nothing for `0`.
|
||||
- [ ] No `severity` key at any depth, still (the recursive walk passes unmodified).
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` paragraph names `enabled`/`off` and the marker-aware warnings rule — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: the socket join, inbound counts, chart gauges and the phase-32 provenance fields are untouched.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 05 — One route, ten gates: `POST /api/gateway/route` and the amendment of invariant 1
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** operator request — "turn off and on connections as needed in the event of security concerns or upgrades" · **Shape borrowed from:** `/api/ups/command`'s four gates and `/api/certs/*`'s fixed-text audit events
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The app's **second and last** upstream write. `POST /api/gateway/route {id, action, danger?, dry_run?}` refuses in nine places before it dials anything, and in the tenth it forwards one reviewed request through `collectors.post_route` and answers with the helper's own triple. `GATEWAY_ALLOW_WRITES` defaults to **off**, `DASH_READ_ONLY` still refuses everything, and `AGENTS.md` invariant 1 is rewritten in this commit to name two exceptions — because a change to a written invariant that leaves the document describing the old one is the bug, not the omission.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/config.py` — two settings beside the existing `GATEWAY_*` block, each with the reason in its comment:
|
||||
- `GATEWAY_ALLOW_WRITES: bool` (default `False`, the same truthy rule as `NUT_ALLOW_ACTIONS`);
|
||||
- `GATEWAY_TIMEOUT_SECONDS` **unchanged** (reads), plus `GATEWAY_WRITE_TIMEOUT_SECONDS: float = 35.0` — deliberately longer than the agent's own 30 s peer timeout so the dashboard never times out first and leaves the operator guessing whether the reload happened.
|
||||
2. `src/dashboards/collectors.py` — `post_route(tls, target, body, timeout)` beside `fetch_snapshot`: the same `AgentTls` client (built per dial, the phase-15 rule), the same **revocation check before any socket opens**, `Authorization` never used on the certificate path, and every error message run through `strip_userinfo` (those strings reach events and `/api/health`-adjacent payloads — invariant 6).
|
||||
3. `src/dashboards/api.py` — the route, and the gate order is the design, so it is written in the function as numbered comments and mirrored in the tests:
|
||||
| # | Gate | Refusal |
|
||||
|---|---|---|
|
||||
| 1 | session (middleware) | `303`/`401`, unchanged |
|
||||
| 2 | `Origin` / `Sec-Fetch-Site` | `403`, unchanged |
|
||||
| 3 | this session's `X-CSRF-Token` on a cookie write | `403`, unchanged |
|
||||
| 4 | `DASH_READ_ONLY` | `403 "this dashboard is read-only (DASH_READ_ONLY)"` |
|
||||
| 5 | `GATEWAY_ALLOW_WRITES` | `403 "route switching is disabled — set GATEWAY_ALLOW_WRITES=1 to enable it"` |
|
||||
| 6 | throttle `route:<address>` on `ctx.login_throttle` | `429` + `Retry-After` (its third key shape; the per-process caveat stays gap 7) |
|
||||
| 7 | body: `id` non-empty string ≤ 256, `action` in `{disable, enable}`, `dry_run` optional bool, **no other key** | `422`, and it is a `422` rather than a `400` for the same reason the login body is |
|
||||
| 8 | `id` exists in the **current** payload (`409` if the job has never answered) and is not already in the requested state (`409 {changed: false}`) | never a guess about a route it cannot see |
|
||||
| 9 | freshness: `gateway` job `ok` and not stale beyond 3× its interval, `host:<GATEWAY_HOST>` `ok`, `sections.haproxy`/`sections.nginx` both `true`, the agent's `version >= 1.4.0` | `409` naming which one — "do not rewrite a config you cannot currently read end to end" |
|
||||
| 10 | `danger == "DANGER"` for **both** verbs | `422` naming the word, the UPS wording's sibling |
|
||||
Then one `collectors.post_route`, and the response is the helper's object plus `host`, `id`, `action`, `dry_run`, and `hint` sentences owned by `api.py` (the `restart_required` precedent: the prose lives in the route, so the page prints what the route said).
|
||||
- **Events**, one per call, `source="gateway"`, `subject=<the route's host>`, `kind` in `{route-disabled, route-enabled, route-denied}`, severity `warn` for a change and `error` for a refusal-that-matters, message fixed-shape: `disable brain.reeseapps.com → reeseapps-brain-of-reese.reeselink.com:443: applied=true validated=true reloaded=true (admin)`. **No diff, no file path, no config text** — `/api/events` is a feed a browser reads, and a config's bytes are not a log line. A `route-denied` from gates 4–10 writes the event with the gate named and **nothing** about which check the caller came closest to passing.
|
||||
- `/api/config`'s `gateway` row gains `writes_enabled` (bool) so the page can render the honest arm instead of assuming (rule 2), and nothing else in that body changes.
|
||||
4. `tests/test_api.py`
|
||||
- `test_the_gateway_surface_is_gets_only_and_the_write_stays_the_ups_one` is **rewritten**, not deleted: the verb enumeration now lists exactly two fleet writes — `/api/ups/command` and `/api/gateway/route` — beside the session writes and the certificate-authority writes, and the test still fails if a third appears anywhere.
|
||||
- `test_each_gate_refuses_without_touching_the_peer`, parameterised over all ten with spies: `collectors.post_route` swapped for a recorder, and each refusal asserts the recorder is empty **and** the event is what the gate named. The CSRF half is proven on the wire in `tests/test_auth_api.py` the way the UPS one is: a scripted helper records nothing without the header and once with it.
|
||||
- `test_danger_is_required_to_enable_as_well_as_disable`, `test_read_only_refuses_and_names_the_variable`, `test_a_route_that_is_already_off_says_so_and_dials_nothing`, `test_a_stale_gateway_or_dead_host_job_refuses_the_write`, `test_a_revoked_agent_refuses_the_write_without_a_socket`, `test_the_write_timeout_is_longer_than_the_agents_peer_timeout` (the ordering, not the numbers).
|
||||
- `test_no_diff_or_config_text_reaches_an_event` — a successful write whose helper answer carries a `diff` field: the event message contains no `diff`, no `--- `/`+++ ` line, no path, and no `was:` marker.
|
||||
- `test_the_writes_enabled_flag_reaches_the_page_config`.
|
||||
5. `tests/test_auth_api.py` — the served-route walk (`walk_routes`) now sees `POST /api/gateway/route` and demands a session; the public-surface list stays exactly four entries; the CSRF-on-the-wire test gains this endpoint.
|
||||
6. `AGENTS.md` — **in the same commit**: invariant 1 rewritten from "exactly one exception" to "exactly two, and both are gated on purpose", naming the two endpoints and the gate list, keeping the sentences that make it work ("Hiding a button is not a safety control", "the UI confirmation … `DANGER` for anything that can cut power" now extended: *or that can cut a route through the fleet's only ingress*). Also: `api.py`'s bullet (its three write lists become four facts and now include the gateway route), `collectors.py`'s bullet (`post_route`), `config.py`'s paragraph (the two new settings), and the `/gateway` UI-conventions sentence about the unwired switch is **removed** — it becomes the wired description in task 06, and the two must not both exist.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit/integration: every gate in both directions; the wire-level CSRF proof; the event-shape proof; the verb enumeration restated.
|
||||
- Coverage **>90%** on `api.py` and `collectors.py`; the whole suite 100%; `ruff` + `pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `curl` against the dev server with writes off answers the `GATEWAY_ALLOW_WRITES` sentence; with it on and no `DANGER` answers the `DANGER` sentence; with both and a live fake agent it answers `applied: true` and the event line is in `/api/events?source=gateway`.
|
||||
- [ ] `grep -n "DASH_READ_ONLY" src/dashboards/api.py` shows it refusing on **both** write paths.
|
||||
- [ ] The verb-enumeration test lists exactly two fleet writes and fails on a third by name; `walk_routes` covers the new POST; `PUBLIC_PATHS` is still four entries.
|
||||
- [ ] `/api/config` serves `gateway.writes_enabled` and the page's own arm is task 06's.
|
||||
- [ ] `AGENTS.md` invariant 1 names two exceptions and their gates in the same commit as the code that made it true.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `bash .agents/validate.sh` green.
|
||||
- [ ] No behaviour change in completed work: `/api/ups/command`'s four gates behave exactly as before (the suite's existing UPS pins are untouched), and no read route changed shape.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Task 06 — The switch works, asks for `DANGER` with the diff on screen, and never claims a change it didn't get
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** the operator's sketch (phase 30) + this phase's decision that a route off is a config edit with a reload behind it · **Shape borrowed from:** `page-ups.js`'s typed-`DANGER` confirm and `dash.css`'s `dialog` + `input[type="text"].confirm`
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Clicking a switch on `/gateway` shows the **exact one-line diff** the gateway would apply, asks for `DANGER`, applies it, and paints the row from the **answer** — not from hope. Where writes are disabled the switch stays visibly unwired and says which variable would change that, because `/gateway` must look the same on a deployment that chose not to have this feature as it did before the feature existed.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/web/static/page-gateway.js`
|
||||
- `routeInConfig(conn)` becomes a reading, not a literal: `return conn.enabled !== false;` — with the comment rewritten: `off` is now reachable, it means "declared in the config, switched off by the operator", and `enabled === undefined` (a pre-1.4.0 agent, or the stub arm) still means "on", so an old agent's page never goes red.
|
||||
- `buildSwitch(conn)` splits into two arms, driven by `/api/config`'s `gateway.writes_enabled` (never inferred):
|
||||
- **writes off** → today's markup exactly: `aria-disabled="true"`, `cursor: not-allowed`, `title` = `Route switching is off — set GATEWAY_ALLOW_WRITES=1 on the dashboard to enable it.` (the sentence from `api.py`, copied into a constant that `tests/test_api.py` compares against the route's refusal so the two cannot drift);
|
||||
- **writes on** → a real control: no `aria-disabled`, `aria-checked` from `enabled`, a full accessible name (`endpoint, reached by <selectors>, in the <source> config, on|off, activate to switch it off`), `title` naming what activating does (edits the file, validates, reloads the proxy).
|
||||
- **The flow**, one async handler, and the order is the safety:
|
||||
1. `POST /api/gateway/route` with `{id, action, dry_run: true}` → `diff` + `hint`;
|
||||
2. open a real `<dialog>` (`showModal()`) whose body is: the row's inbound word and endpoint, the config's own `selectors`, the diff in a labelled `<pre>`, the route's `hint`, and a `input[type="text"].confirm` asking for `DANGER`;
|
||||
3. submit only when the typed word is exactly `DANGER` (client-side check of the *client's* convenience; the server re-checks, always);
|
||||
4. switch `aria-busy="true"`, `POST` with `danger: "DANGER"`, `dry_run: false`;
|
||||
5. paint from the **response**: `applied: true` → the row's `enabled` flips on the next poll (and immediately, optimistically **only** for the switch's own `aria-checked`, with `aria-busy` cleared and a note that the config is applied and the page re-reads); `applied: false` → nothing flips, the refusal prints verbatim through `textContent`;
|
||||
6. `await refresh()` so the numbers come from the agent's next reading rather than from the click.
|
||||
- No optimistic colour, ever: the page does not turn a line green or red because a button was pressed. `pending` is the only state a click may create, and `aria-busy` is how it says so.
|
||||
- A `429`/`409`/`403`/`422` prints the server's `detail` + `hint` in the dialog (`role="alert"`), leaves the dialog open so the operator can read it, and returns focus to the confirm field — the `/login` lesson about a form that eats its own error.
|
||||
2. `src/dashboards/web/pages/gateway.html`
|
||||
- One `<dialog id="gate-switch-dialog" aria-labelledby="gate-switch-title" aria-modal="true">` with the four slots the script fills, shipped `hidden`/closed; no per-row dialogs (the same rows are patched, so one dialog is the whole surface, and a dialog per row is 33 focus targets in the tab order).
|
||||
- Footer paragraph 3 becomes the truth and stays **one** paragraph:
|
||||
> `the switch edits that route in its own config file, validates it with the proxy's own checker and reloads that proxy — nothing is restarted, and no route goes off unless GATEWAY_ALLOW_WRITES is set on the dashboard.`
|
||||
- The `#gate-empty` setup block is untouched.
|
||||
3. `src/dashboards/web/static/dash.css`
|
||||
- `#gate-switch-dialog pre` — mono, `overflow-x: auto`, `max-height` bounded, added/removed lines tinted from the **existing** tokens only (`.diff-add`/`.diff-del` deriving from `--gate-on`/`--gate-off`, the phase-30 existence pair, so no new colour enters the map and the diff reads as the same red/green the rows do);
|
||||
- `button.gate-switch:not([aria-disabled="true"]):hover` and `cursor: pointer` — added **only** under that selector, so a writes-off switch never grows a hover affordance it cannot honour;
|
||||
- any new transition (dialog, knob while `aria-busy`) is named in the file's one `@media (prefers-reduced-motion: reduce)` block — naming one and missing the other is the phase-18 lesson, and there is a test for it;
|
||||
- at ≤640 px the dialog stays inside the viewport (`width: calc(100% - 2rem)` is already the shared rule) and the diff scrolls in its own box, never the document (phase 27's rule).
|
||||
4. `tests/test_api.py`
|
||||
- `test_the_switch_is_built_and_explicitly_unwired` is **replaced** by `test_the_switch_is_wired_only_when_the_deployment_enables_writes`: with `writes_enabled: false` the served script keeps `aria-disabled="true"`, the `GATEWAY_ALLOW_WRITES` sentence, and no listener; with it true the listener exists, `aria-disabled` is gone, and the accessible name states the state and the consequence. The old test's *intent* survives: a dead-looking-live control still fails.
|
||||
- `test_a_toggle_asks_for_a_preview_then_the_word_then_sends_once` — the served script issues `dry_run: true` before any `DANGER` send, requires the exact word, and sends exactly once per activation (no retry loop in the handler).
|
||||
- `test_the_row_is_painted_from_the_answer_not_the_click` — no code path sets `off`/`on` other than the payload/`applied` pass; `applied: false` leaves `aria-checked` untouched.
|
||||
- `test_the_switch_sentence_matches_the_routes_refusal` — the writes-off `title` and `api.py`'s `GATEWAY_ALLOW_WRITES` refusal are the same string, read from both files.
|
||||
- `test_no_gateway_count_is_coloured_by_a_verdict` gains the reachable-`off` fact and keeps every assertion: the diff tints are `--gate-*`, no `.ok/.warn/.fail` or `--ok/--warn/--fail` enters the gateway section, and the warnings panel's `.warn` stays the one exception.
|
||||
- `test_the_pipe_animation_stills_in_both_states_under_reduced_motion` extended to the new transitions (knob busy state, dialog) — and it still asserts there is **one** such block in the file.
|
||||
- `test_the_gateway_page_is_exactly_two_connection_panels_and_a_warnings_list` and the footer-paragraph pin stay green (three paragraphs, one dialog, `<section>` count unchanged).
|
||||
- The orphan-`id` audit must find `#gate-switch-dialog` referenced by the script; every slot inside it is `[data-k=…]`, not an `id`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pins above; a fixture render for each arm (writes on/off, `enabled: false` from the payload, `applied: false` from the response) asserting class, `aria-checked`, `aria-busy`, `title` and the accessible name.
|
||||
- Integration: the dev server against a stub agent (the `tests/conftest.py` fake) answering `applied: true` then a payload with the row `enabled: false` — the row turns red **because of the payload**, and the second read of the switch's name says `off`.
|
||||
- Coverage: `node --check` clean; floor does not fall.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With `GATEWAY_ALLOW_WRITES=0` the page is indistinguishable from phase 32's: `aria-disabled`, the `GATEWAY_ALLOW_WRITES` sentence, no click effect, no hover affordance.
|
||||
- [ ] With it `=1` (and the fake agent): clicking shows the one-line diff and the `DANGER` field; the wrong word sends nothing; the right word sends once; the row's colour changes on the payload that follows and never before.
|
||||
- [ ] A stubbed `applied: false, validated: false, restored: true` leaves every row exactly as it was and prints the route's own sentence.
|
||||
- [ ] Tab order: dialog opens with focus in the confirm field, Escape closes it without a request, focus returns to the switch. At 360 px the dialog and its diff stay inside the viewport and the document scrolls no sideways pixels.
|
||||
- [ ] `uv run pytest` **100% passing**, coverage not down; `node --check` and `bash .agents/validate.sh` green.
|
||||
- [ ] README.md/AGENTS.md's `/gateway` paragraph describes the wired switch and both arms (replacing the "unwired" sentence task 05 deleted — the two must not coexist) — the phase's final pass re-checks both project-wide.
|
||||
- [ ] No behaviour change in completed work: tallies, warnings panel, charts, filters, colour semantics, and the other seven pages.
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
# Task 07 — Install the helper, prove it from both sides, and drill one named route with the operator watching
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** AGENTS.md's rollout discipline (phases 24 and 28) applied to the one change in this repo that can take the fleet's ingress down
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`gateway` runs agent **1.4.0** with the socket-activated helper, and one route is switched off and back on **for real**, with the live ingress ending the task in the state it started in. The drill is not a formality: it is the only proof that `ReadWritePaths`, `SocketMode`, `SO_PEERCRED`, `nginx -t` and the reload all line up on a machine, which no sandbox can tell us. **If the operator is not present in the session, this task stops after step 3 and reports — that is the correct outcome, not a failure**, and ntfy is not a way to ask (it is outbound-only).
|
||||
|
||||
## Work
|
||||
|
||||
1. **Backups first, hashed.** On `gateway-ops` as root:
|
||||
`cp -a /etc/haproxy /root/dashboards-33-backup-<utc>/haproxy && cp -a /etc/nginx /root/dashboards-33-backup-<utc>/nginx`, then `sha256sum` every file that the helper can edit (`haproxy.cfg`, `nginx.conf`, `conf.d/*.conf`, `stream.d/*.conf`) into the transcript. This is the restore point for every later step.
|
||||
2. **Sandbox rehearsal on the box** (writes nothing real): run the helper with `--apply` against a copy tree (`--root /tmp/dashboards-33-rehearsal`) with the *real* `nginx -t` and `haproxy -c` binaries on `PATH`, for each of the three marker arms and the `default` refusal. Record the three diffs. A checker that fails on a copied tree for path reasons is expected and is *not* the rehearsal's point — the arms and the restore byte-equality are.
|
||||
3. **Install, read-only-verified first**:
|
||||
- re-stage `/home/ops/agent-install/` (repo `agent/host_agent.py` **1.4.0**, `deploy/install-agent.sh`, `deploy/dashboards-routectl.py`, the two unit files), leaving `*.20261023-backup` copies sudo will not execute;
|
||||
- `systemctl cat dashboards-agent` → record its sha256 and the four security directives verbatim;
|
||||
- `sudo /home/ops/agent-install/deploy/install-agent.sh --bind 10.1.1.27 --port 9120 --allow-writes --with-routectl` (arguments read back off the unit; **no** `--cert-bundle`, the bundle on disk is what drives the TLS lines — the phase-28 finding);
|
||||
- the guard's diff is inspected before `--force`: the only legitimate change to the agent unit is the added `--allow-writes` in `ExecStart` — if `NoNewPrivileges`, `ProtectSystem`, `PrivateDevices`, `SystemCallFilter` or `CapabilityBoundingSet` moved, **stop** and report: the sandbox was not traded for this feature;
|
||||
- `sudo systemctl restart dashboards-agent`; `systemctl is-active` for it and for `dashboards-routectl.socket`;
|
||||
- reachability both ways: as `dashboards-agent`, connect to `/run/dashboards-routectl.sock` and call `list` (must answer `[]`); as any other non-root uid, connect and get `EACCES` (the mode is the first gate and the peer check is the second);
|
||||
- read-only probes: CA-only `--status` answers **1.4.0**; the dashboard's mTLS read shows `agent_version 1.4.0`, every section intact with live numbers, `disabled: false` everywhere, and `/api/gateway/snapshot` rendering identically to phase 32's.
|
||||
4. **Preconditions for the drill — all checked, all recorded, any failure stops it:**
|
||||
- the operator has said go, in this session, in words;
|
||||
- `nut` is not on battery and no UPS event is in the last hour (the same "don't do the risky thing during a power event" rule the UPS commands keep);
|
||||
- `gateway`, `host:gateway`, `disks:gateway` all `ok` and fresh; `warnings == []`;
|
||||
- `nginx -t` and `haproxy -c -f /etc/haproxy/haproxy.cfg` pass on the box before anything is touched;
|
||||
- `routectl list` is empty (no half-finished state from an earlier attempt);
|
||||
- the drill target is **`iperf3-5201`** (`haproxy:5201->web.reeselink.com:5201`) — a benchmark service with no human dependency. **Never** the auth, git, alerts or file routes: not `keycloak*`, `gitea*`, `forgejo*`, `ntfy*`, `nextcloud*`, `immich*`, `brain*`, `keycloak`, or any `map` entry an operator would notice within a minute. If even `iperf3` is in doubt, the drill does not run and the task reports that instead.
|
||||
5. **The drill, in order, with the page open:**
|
||||
1. `dry_run` through `POST /api/gateway/route` → the diff, and the page's dialog shown;
|
||||
2. apply with `DANGER`; the helper answers `applied/validated/reloaded`;
|
||||
3. prove it four independent ways: `routectl list` names the route; the marked line is in `haproxy.cfg` with `was:`; `haproxy -c` still passes; `curl -sv --resolve <host>:25565:127.0.0.1` (or the drill port) from the box connects and is refused/closed rather than proxied;
|
||||
4. the page: the row is **red** within one poll, its live count drains to `0` on its own, no other row changed, and `/api/summary`'s `gateway.disabled` is `1`;
|
||||
5. re-enable through the page; `routectl list` empty; `sha256sum` of every file equal to step 1's capture, byte-for-byte;
|
||||
6. the page green again, `disabled` back to `0`, and `journalctl -u 'dashboards-routectl@*'` showing exactly the two activations.
|
||||
6. **Abort rules, written where they can be followed:** if any step fails, restore from the step-1 backup (`cp -a` back, `haproxy -c`, reload), re-read the page, and write down what happened. Never leave a marked line behind without saying so in the transcript's first line. Never run two drills at once, never run one while the `gateway` job is stale, and never re-run an apply to "make sure" — the helper is the place idempotence lives, and the answer tells you which state you are in.
|
||||
7. `data/logs/NOTES-33-switching.md`: the hashes before and after, the unit diff that was guarded, the socket's owner/mode, both probe transcripts, the four proofs, the `journalctl` window, the screenshots' names, and — if the drill did not run — **exactly** which precondition stopped it, so the next operator can finish it deliberately.
|
||||
8. `tests/test_live_fleet.py`: `test_the_switch_round_trip_on_one_named_route`, marked `live` **and** gated on a separate `--allow-live-writes` flag, skipped by default with a reason naming the route it would touch, restoring in a `finally`. The default live suite stays read-only.
|
||||
|
||||
## Testing & Quality
|
||||
- This task's code change is the one opt-in live test; everything else is procedure plus transcript.
|
||||
- After it: `uv run pytest`, `uv run pytest -m live --live` (green, round-trip **skipped**), `./scripts/dev-server.sh status` (ten sources `ok`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `systemctl cat dashboards-agent` shows the four security directives **identical** to step 3's capture, with `--allow-writes` as the only added argument, and its pre/post sha256 pair is in the note.
|
||||
- [ ] `dashboards-routectl.socket` is `active`, owned `dashboards-agent:dashboards-agent` mode `0600`; a non-agent uid gets `EACCES`; the agent's own uid gets a JSON answer.
|
||||
- [ ] The drill's four proofs are in the transcript, plus the byte-identical `sha256sum` restore of every editable config file, and `routectl list` is empty at the end.
|
||||
- [ ] `/gateway` showed the row red and its count draining, and returned to green — screenshotted at both states.
|
||||
- [ ] `uv run pytest -m live --live` passes with the round-trip **skipped** by default; `uv run pytest -m live --live --allow-live-writes` passes too, if and only if the drill ran.
|
||||
- [ ] `data/logs/NOTES-33-switching.md` exists and its first line states the final state of the ingress (`all routes enabled`, or exactly what is off and why).
|
||||
- [ ] No repo file changed except the opt-in live test and `data/logs/`.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Task 08 — Docs that describe a dashboard which can change the fleet, the browser pass, the full gate
|
||||
|
||||
**Phase:** `33_gateway_route_switching` · **Source:** AGENTS.md definition of done + the fact that this phase changed an invariant, so the documents are part of the change rather than a summary of it
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
`README.md` and `AGENTS.md` describe the write path honestly — what it touches, what refuses, how to turn it off entirely, and how to recover a route by hand at 03:00 when the dashboard is the thing that is broken. Then the browser pass measures the wired page in both arms, the mobile audit re-runs, and every gate is green.
|
||||
|
||||
## Work
|
||||
|
||||
1. `AGENTS.md` — the full consistency pass, with these named as the passages this phase makes stale if left:
|
||||
- **invariant 1** — two exceptions, both gated, the gate list, and the reason a typed word survives in the design (`Hiding a button is not a safety control`);
|
||||
- `agent/host_agent.py` bullet — **1.4.0**, the marker grammar (`ROUTE_MARKER`, `split_marker`, `config_line_index`), `disabled` on every target, `POST /route` behind certificate + `--allow-writes`, the single peer call and its no-retry rule;
|
||||
- the new deploy surface — `deploy/dashboards-routectl.py` (verbs, marker + `was:`, validate-restore-reload, never restart, the sink file, drift refusal), the two units, `--with-routectl`/`--without-routectl`, and the sentence that the agent's unit lost no security directive and gained no sudoers line;
|
||||
- `sources/gateway.py` bullet — `enabled`, the merge rule with mixed states, the marker-aware warnings table, the `/api/summary` `disabled` field;
|
||||
- `api.py` bullet — the two fleet writes, `/api/gateway/route`'s gate order, the fixed-shape audit events with no config text, `/api/config.gateway.writes_enabled`;
|
||||
- `collectors.py` bullet — `post_route` and the shared revoke-before-socket rule;
|
||||
- `config.py` paragraph + README's table — `GATEWAY_ALLOW_WRITES` (default off) and `GATEWAY_WRITE_TIMEOUT_SECONDS` (and why it exceeds the agent's own timeout);
|
||||
- the `/gateway` UI-conventions paragraph — the switch is wired, the preview-then-`DANGER`-then-paint-from-the-answer flow, `off`'s appearance, the writes-disabled arm's sentence, the diff box scrolling in its own container;
|
||||
- the **fleet record** — `gateway` at **1.4.0** with the helper units installed, the unit's directives re-read, the drill result and `NOTES-33-switching.md`;
|
||||
- **gap list** — name-route switching **closed**, and the honest limits written as gaps: one admin (gap 6 unchanged, so anyone with the session can switch a route), no scheduler, no per-host kill switch, no runtime haproxy state (a hand reload cannot undo a marker), and `GATEWAY_ALLOW_WRITES` off by default means the feature is invisible on a fresh deployment until it is deliberately enabled.
|
||||
2. `README.md` + both tracked env templates — `.env.example` and `.env.dev.example` gain `GATEWAY_ALLOW_WRITES` with its default and a comment naming exactly what enabling it authorises, and `tests/test_auth_docs.py` gains the pin (the same device that keeps the auth settings declared in the templates), because a write gate no template names is a write gate nobody finds — and never a value, only the declaration. Then the operator-facing half: the `/gateway` table row (routes can be switched off **and on**), a short *Switching a route* subsection (what the dialog does, what `DANGER` means here, that the edit is persistent across reload and reboot, and the three recovery commands: `routectl list`, `routectl verify`, `routectl enable --id …` on the box), the *Enabling it* subsection (`GATEWAY_ALLOW_WRITES=1`, `--with-routectl`, the two units, how to undo everything), and the configuration table's two new rows. Also: the sentence that a route off is a **deliberate outage** and appears in the event feed, so an operator does not mistake it for an incident.
|
||||
3. `data/logs/33-switching.cjs` — Playwright pass (house shape, exit non-zero on FAIL, prints what it measured):
|
||||
- writes-**off** arm: `aria-disabled`, the `GATEWAY_ALLOW_WRITES` sentence, a click does nothing (no request on the wire), no hover affordance;
|
||||
- writes-**on** arm against the stubbed agent: the dialog opens with the diff and focus in the confirm field; the wrong word sends nothing; the right word sends exactly one request; the row's colour changes only after the payload says `enabled: false`; `applied: false` leaves the row untouched and prints the route's sentence; Escape closes with no request and returns focus;
|
||||
- `off` colour contract: spine halves, tick, run, switch and diff tint all compute to the same `--gate-off` value in dark **and** light, and an `on` row is fully `--gate-on` — no fifth colour, no verdict token anywhere in the section;
|
||||
- reduced motion: nothing animates, including the dialog and the busy knob;
|
||||
- 360 px: the dialog and its `<pre>` stay inside the viewport, the diff scrolls in its own box, and the document scrolls zero sideways pixels;
|
||||
- screenshots `data/logs/33-gateway-{writes-off,dialog,danger,off,off-light,refused,phone,phone-dialog}.png`, transcript `data/logs/33-switching.out`.
|
||||
4. Re-run `data/logs/27-task02-mobile-audit.cjs` (nine pages, both widths): exit 0, **0 FAIL / 0 FINDING**, tally recorded — the dialog is new markup and the audit is what catches a fact painted past a card edge.
|
||||
5. The whole gate, recorded in the transcript:
|
||||
`uv run pytest`, `uv run pytest --cov`, `uv run pytest -m live --live`, `uv run ruff check src tests agent deploy`, `uv run pyright`, `bash .agents/validate.sh`, `uv pip list --outdated` (empty — no dependency is added by this phase, and `deploy/dashboards-routectl.py` is stdlib on a host's Python 3.10).
|
||||
6. `./scripts/dev-server.sh status` with writes **off** → ten sources `ok` and no change; with `GATEWAY_ALLOW_WRITES=1` and the fake agent → still ten `ok`, and a write through `curl` round-trips.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: this task is the phase's proof that the documents and the shipped page agree; every completion-criteria number comes from `33-switching.out`, `NOTES-33-switching.md` or a gate's own output.
|
||||
- Coverage: floor green with the phase's new modules included; no `# pragma: no cover` added to reach it.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `grep -n "one exception\|exactly one write" AGENTS.md` finds nothing; invariant 1 names **two** endpoints and their gates, and the `/gateway` paragraph no longer says the switch is unwired (the two statements must not coexist).
|
||||
- [ ] `grep -n "GATEWAY_ALLOW_WRITES" README.md AGENTS.md .env.example .env.dev.example` — declared in both templates with its default and a comment saying what enabling it authorises, and documented in both documents.
|
||||
- [ ] `data/logs/33-switching.out` exits 0 with the both-arms results, the `--gate-off`/`--gate-on` equalities in both themes, the reduced-motion pass and 0 px sideways scroll at 360 px; the eight screenshots exist.
|
||||
- [ ] `data/logs/27-task02-mobile-audit.cjs` exits 0, 0 FAIL / 0 FINDING, tally recorded.
|
||||
- [ ] All five gates green, `uv pip list --outdated` empty, and the live suite green with the round-trip skipped by default.
|
||||
- [ ] A full read of `README.md` and `AGENTS.md` finds no inconsistency anywhere, including pre-existing ones, fixed in this phase's commits.
|
||||
- [ ] No behaviour change in completed work: the UPS write path and its four gates, the other five dashboards, `/settings`, `/certs`, `/login`, and every read route's payload are exactly as phases 31–32 left them.
|
||||
Reference in New Issue
Block a user