phase: 28_gateway_agent_proxy
**Phase 28 final pass — complete, 100 % green.** All five tasks were already `complete/`; this pass verified the criteria, re-ran the live half, and did the docs pass. - Gate `bash .agents/validate.sh` **rc=0**: `1650 passed, 15 skipped`, coverage **98.97 %** (floor 94), `ruff` clean, `pyright` **0 errors**, `node --check` clean, hygiene clean (`/tmp/validate-done.log`). - Live: `pytest -m live --live` **11 passed**; `./scripts/dev-server.sh status` nine sources `ok`; mTLS re-read of `gateway` → agent **1.2.0**, all three sections `available: true`, 6/6 frontends routed, 11 nginx servers + `$targetBackend`, **19** listeners, no core errors (`data/logs/28-finalpass-live.out`); installed agent file byte-identical to this tree (`b0e9ebf1b718…`). - Criteria: 1 ✅ pytest/coverage · 2 ✅ three sections over a captured tree, `false`/`null` + one `errors` line each elsewhere with `cpu`/`mem`/`disk_io`/`system` intact · 3 ✅ `AGENT_VERSION` 1.2.0 + SCHEMA names all three · 4 ✅ six fixtures verbatim, capture refuses a bad read, 116 socket address columns hold no globally routable endpoint · 5 ✅ gateway runs 1.2.0 and answers over mTLS (transcript `data/logs/28-task05-rollout.md`, `28-finalpass-live.out`) · 6 ✅ validate · 7 ✅ additive: HEAD's 1.1.0 and this 1.2.0 `collect()`ed over one `--root` are key-for-key equal plus the three new sections, `--no-proxy` equal exactly (`data/logs/verify-28-final.py` → 41 PASS, `28-final-verify.out`). - Docs fixed (README/AGENTS/Containerfile, all re-green): README's opening said "Four dashboards" above a five-row table and said the new sections are "stored in the snapshot" (they are in the **agent's** document — no `/api` payload, no `sample` row, nothing charts them until phase 29); `Containerfile`'s header still said "three dashboards"; AGENTS's coverage figure (98.94 → **98.97 %**), invariant 6 gained the captured-socket-table identifier rule, the agent bullet gained "phase 28 stops at the wire", and the fleet record names the stash-and-compare control plus the volatile listener count. Full record: `data/logs/NOTES-28-final.md`. - Deviations: none functional; no git index/history touched, phase files untouched (`PLAN.md` and `complete/` unchanged). `uv pip list --outdated` still lists 4 packages (fastapi 0.153→0.161 etc.) — advisory only, dependencies untouched by this phase, left for a deliberate pass. - Next pending phase: `.agents/phases/todo/29_gateway_dashboard_page/` (the page that consumes these three sections).
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
# Phase 28 — The agent reads the gateway's proxy configs and its live sockets
|
||||
|
||||
**Source:** `TODO.md` L3 — "Create a 'gateway' page that visualizes my incoming connections based on the gateway host's nginx and haproxy configs. I want it to show animated pipes with data coming in (IPs, ports, domains, etc.) and data going out (IPs, ports, domains, etc.)." (the agent half)
|
||||
**Story:** n/a
|
||||
**Context:** `agent/host_agent.py` (the stdlib-only single-file agent, `Collector._safe` best-effort sections, `snapshot()`, the SCHEMA docstring, `AGENT_VERSION` 1.1.0, the `--no-smart` opt-out and the three-way availability precedent it sets), `tests/test_agent.py` (the in-process harness over a fake `root` tree, the real-`/proc` test, the smartctl-capture fixture pins), `scripts/capture-fixtures.sh` (the ssh-capture convention: plain command first, `sudo -n` fallback, refuse to overwrite good fixtures with unreadable output), `tests/fixtures/` (the verbatim-capture ground-truth directory), the gateway host's live state verified 2026-10-21 (read-only): `nginx` and `haproxy` both `active`, `/etc/haproxy/haproxy.cfg` (6 `frontend` blocks — `bind :::22 v4v6` et cetera — each with one `default_backend`; 6 `backend` blocks, each one `server <name> <domain>:<port> check inter 5s`), `/etc/nginx/nginx.conf` (a catch-all `listen 80` / `server_name _` block plus `include /etc/nginx/conf.d/*.conf;` and `include /etc/nginx/stream.d/*.conf;`), `/etc/nginx/conf.d/http.conf` (server blocks: `listen 4443 ssl proxy_protocol` / `listen 8443 ssl`, `server_name <domain>`, `location / { proxy_pass http://<domain>:<port>; }`), `/etc/nginx/stream.d/stream.conf` (a `map $ssl_preread_server_name $targetBackend { … }` block and `proxy_pass $targetBackend` with `ssl_preread on`), all three files world-readable and readable by the `dashboards-agent` user, `/proc/net/tcp` readable the same way. The dashboard side (source, wiring, page) is phase 29; this phase makes the data exist on the wire.
|
||||
|
||||
## Objective
|
||||
|
||||
`host_agent.py` 1.2.0 serves three new **additive** snapshot sections on the gateway host — `haproxy` (frontends + binds, backends + server targets, the frontend→backend routing), `nginx` (server blocks with `listen` + `server_name`, `upstream`, `proxy_pass` including the `$var`-to-`map` join, `map` blocks) and `sockets` (LISTEN + ESTABLISHED from `/proc/net/tcp{,6}`) — each best-effort (a missing config degrades its own section, never the snapshot), `--no-proxy` opting out silently, the real configs captured verbatim as fixtures, the rollout onto the gateway host done, and the sections proven live over mTLS. Nothing changes for token-mode or pre-1.2.0 consumers: an old agent's snapshot is byte-identical in every old field.
|
||||
|
||||
## Dependencies
|
||||
- `27_mobile_tables_and_audit` (todo) — not a code dependency (different files); ordered after it so the fleet rollout in task 05 does not race the mobile-audit's browser passes on the shared dev box
|
||||
- `15_agent_mtls` / `24_dev_mtls_rollout` (complete) — the installed-bundle install path (`deploy/install-agent.sh --cert-bundle`), the `--status` CA-only probe, and the gap-10 re-stage-first rule the rollout follows
|
||||
|
||||
## Tasks
|
||||
1. `01_haproxy_section.md` — the agent reads + parses `haproxy.cfg` into the additive `haproxy` section, three-way availability, `--no-proxy` opt-out, never fails the snapshot
|
||||
2. `02_nginx_section.md` — the agent reads the main config + one level of `include <glob>` (bounded) into the additive `nginx` section, with the `proxy_pass $var` → `map` join
|
||||
3. `03_socket_section.md` — the agent parses `/proc/net/tcp` + `/proc/net/tcp6` (bounded) into the additive `sockets` section: LISTEN + ESTABLISHED with local/remote ip:port
|
||||
4. `04_version_and_capture.md` — `AGENT_VERSION` 1.2.0, the SCHEMA docstring, `scripts/capture-gateway-config.sh` (real configs → `tests/fixtures/`), the live probe that the sections answer on the gateway host
|
||||
5. `05_rollout_gateway.md` — re-stage `/home/ops/agent-install/`, install 1.2.0 on the gateway host, restart the unit, verify the sections over mTLS — read-only live verification, transcript in `data/logs/`
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/test_agent.py` — every new section over a fake `root` tree (present config → parsed shape; missing file → `available: false` + exactly one errors line; unreadable → `null` + one errors line; `--no-proxy` → sections absent and no errors line; a corrupt config degrades its own section and the snapshot still succeeds with the other sections intact)
|
||||
- Fixture: `tests/fixtures/gateway_haproxy.cfg` + `gateway_nginx_*.conf` + `gateway_proc_net_tcp{,6}.txt` are verbatim captures from the gateway host (invariant 7); the parser tests run against them, so a config drift fails a test before a dashboard does
|
||||
- Coverage: **>90%** on the new/modified code (the three collectors + the parsers)
|
||||
- Live (read-only, invariant 10): task 04's probe over the mTLS dial, and task 05's post-restart verification with its transcript
|
||||
|
||||
## Documentation & Consistency
|
||||
|
||||
- The final pass (task 05) updates `AGENTS.md`'s agent bullet (the new sections, `AGENT_VERSION` 1.2.0, the `--no-proxy` line) and the `SCHEMA` docstring is the self-documenting half on the wire. `README.md` is checked for any agent-version sentence.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (the hermetic suite, no network), coverage not down
|
||||
- [ ] a 1.2.0 agent over a captured-config root tree answers all three sections; over a tree without them it answers `available: false`/`null` with one errors line each and the snapshot's other sections are intact
|
||||
- [ ] `AGENT_VERSION` is 1.2.0 and the SCHEMA docstring names the three new sections
|
||||
- [ ] the fixture files are verbatim captures (the capture script's refusal-to-overwrite-with-unreadable pin holds)
|
||||
- [ ] the gateway host runs 1.2.0 (`systemctl is-active dashboards-agent` → `active`, the file's `AGENT_VERSION` reads 1.2.0) and the sections answer over mTLS — transcript in `data/logs/`
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed work: a 1.2.0 agent's snapshot is identical to 1.1.0's in every pre-existing field for a non-gateway host (the new keys are the only difference, and `--no-proxy` removes even those)
|
||||
Reference in New Issue
Block a user