services/dashboards
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 33_gateway_route_switching

Browse Source 
Phase 33 completion pass is done. Final report:

## Phase 33 — completion report

**All 8 task files complete** (`.agents/phases/complete/33_gateway_route_switching/`), full gate green, docs verified.

### Work in this pass

1. **`tests/test_routectl.py` added** (17 tests, all green) — exercises the *real* helper binary unprivileged against a fake tree (`--root`), the real agent parser, the captured gateway fixtures, and stubbed `nginx`/`haproxy`/`systemctl` on `PATH`. Pins the full protocol (locate → state → dry-run → temp-file/fsync/`os.replace` write → checker → restore on checker failure → reload-never-restart → re-read proof → sidecar record), all three route shapes (haproxy `server`, nginx `proxy_pass`, nginx stream `map` via its sink-value id), `drift` / unknown-id / `map-default` refusals, `list`/`verify`, and the `--serve` peer gate (`SO_PEERCRED` uid check, one JSON line per request, exact body shape).

2. **Real helper defect fixed** — `deploy/dashboards-routectl.py`: `main()`'s early-return paths returned `({"ok": False, …}, EXIT_*)` tuples, so `sys.exit()` printed a Python repr instead of a JSON line (usage errors, agent-import refusal). Now prints `json.dumps(…)` and returns only the code, keeping the one-JSON-line contract.

3. **Documentation consistency fixes** (task 08 criterion 6):
   - `README.md`: garbled verdict-colour sentence on the switch ("green and to the position") → "in the **on** position and tinted `--gate-on`" (the existence-pair contract).
   - `README.md`: audit sentence now names all three event kinds — `route-disabled` / `route-enabled` / **`route-denied`** (the code stores one per refused call; dry runs store none).
   - `page-gateway.js`: replaced the stale pre-33 module docstring ("the switch is not wired") with the two-arm `GATEWAY_ALLOW_WRITES` contract.
   - `AGENTS.md`: every phase-33 claim re-verified against code (the ten gates, the 35 s/30 s budget pair, `gateway.writes_enabled`, `/api/summary`'s `disabled`, the 1.4.0 agent bullet, the gap list) — no changes needed.

### Verification
- `uv run pytest --cov`: **1970 passed**, 18 skipped — coverage **98.74%** (floor 94%)
- `ruff` clean · `pyright` 0 errors · `bash .agents/validate.sh` **exit 0** (incl. `node --check` on every page script)
- Dev server running; all 10 sources `ok`; bundles exported

### Live infrastructure
**No live changes made.** The live drill remains unexecuted per `data/logs/NOTES-33-switching.md` (no operator approval; checker binaries unavailable in the podman proxy units). All routes stay enabled, config hashes unchanged, and the sandbox rehearsal + backup tree are in place for the operator's own `DANGER` call. All changes are in the working tree only.
This commit is contained in:
ducoterra
2026-11-01 11:50:51 -05:00
parent 0271387053
commit 86c8bf4992
91 changed files with 7568 additions and 519 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent/host_agent.py
+297 -31
View File
@@ -2,13 +2,19 @@
"""host_agent.py — read-only host metrics agent for the homelab dashboards.
**Stdlib only, one file, Python 3.10+.** Copy it to a host, run it, and the dashboard can
read that host. It never writes anything anywhere: every number comes from reading
/proc, /sys and statvfs, and the HTTP server only answers GET.
read that host. Every number comes from reading /proc, /sys and statvfs, and the HTTP
server answers GET — with exactly one exception: 1.4.0's `POST /route`, the fleet's one
narrow write (switching a route's `#off` marker). Its gates are the client certificate and
`--allow-writes`, and it never touches a config file itself: it dials the socket-activated
root helper (`--routectl-socket`) once and passes the helper's JSON through verbatim.
GET /health -> {"ok": true, "hostname": ..., "version": ...}
GET /info -> slow-changing identity (os, kernel, cpu model, cores, gpu list)
GET /snapshot -> everything, one JSON document (see SCHEMA below)
GET /schema -> this docstring's SCHEMA section, for self-documenting deploys
POST /route -> the one write: forward exactly {id, action, dry_run} to the
routectl helper and answer its JSON verbatim (1.4.0; certificate
plus --allow-writes, in that order — a token-mode POST is refused)
Design rules
------------
@@ -89,6 +95,20 @@ Design rules
are simply absent — `null` a pass that could not run), and `--no-resolve` removes the key in
silence the way `--no-smart` does; `--no-proxy` removes it too, because a box that collects no
proxy has no declared names to look up.
* **The one write is a forward, not a file edit.** `POST /route` (1.4.0) is the only non-GET
path and the only thing this process ever sends to an upstream. The order is the design:
the client certificate (the handshake is the identity — a bearer token never gets this
path), then `--allow-writes` (the operator's decision, off by default), then the body
(exactly `{id, action, dry_run}`, ≤ 4096 bytes, an `action` of `disable`/`enable`, an
`id` of at most 256 characters — a refused body never reaches the peer), then the
helper's socket. The dial is `AF_UNIX` with a 30 s budget (the helper may run `nginx -t`
and a reload), one newline-delimited request, one line back, and **never a retry — a
retry of a write is a second reload**: every failure is a 502 (an unreadable or
oversized answer, a timeout, or the errno word) and the helper's JSON is passed through
verbatim, refusal included. Without `--routectl-socket` (the flag's value set empty) the
answer is `503 no route helper on this host`. The bytes are written by the socket-
activated root helper (`deploy/dashboards-routectl.py`); this process opens no config
file for writing, so `--allow-writes` on a box without the helper is a 503, not a write.
* **The state** is a small pid->cpu-time cache so `processes.top_cpu` can answer a
question /proc cannot (CPU needs two samples), bounded and cleared when stale, plus
the short-TTL SMART result cache above.
@@ -139,18 +159,27 @@ SCHEMA (GET /snapshot)
top_cpu: [{pid, name, cpu_pct}]}
haproxy {available: true|false|null, config: str, frontends: [{name, binds:
[{addr, port}], default_backend: str|null, backends: [str]}],
backends: [{name, servers: [{name, target, port}]}]}
backends: [{name, servers: [{name, target, port,
disabled: bool}]}]}
# parsed from --haproxy-config; additive (phase 28) and absent entirely
# with --no-proxy. `addr` is the bind's address as written (`:::22` ->
# `::`), `port` null when a target names none (never 0), and the
# frontend->backend routing is the names themselves: the dashboard
# joins them, the agent reports only what the config says.
# `disabled` is the route switch's read side (phase 33, agent 1.4.0):
# true when the line carries the helper's `#off <name>` marker
# (ROUTE_MARKER), false otherwise. The marker is a comment in both
# configs, so a marked line is still a live line to haproxy/nginx until
# the helper rewrites the value — the agent reports the state, the
# dashboard draws it, and the kernel does not know markers.
nginx {available: true|false|null, config: str, files: [str],
servers: [{listen: [{addr, port}], server_names: [str], proxy_passes:
[{location: str|null, scheme: str|null,
target: {host, port}|null, variable: str|null}]}],
upstreams: [{name, servers: [{host, port}]}],
maps: [{variable, entries: [{key, target: {host, port}|null}]}]}
target: {host, port}|null, variable: str|null,
disabled: bool}]}],
upstreams: [{name, servers: [{host, port, disabled: bool}]}],
maps: [{variable, entries: [{key, target: {host, port}|null,
disabled: bool}]}]}
# parsed from --nginx-config plus the files its own `include` lines name
# (one level, at most 8 files / 1 MB of included text; `files` is the
# main config then every include actually merged); additive (phase 28)
@@ -158,6 +187,10 @@ SCHEMA (GET /snapshot)
# written (`[::]:8443` -> `"::"`, a bare `80` -> a null addr, never `*`),
# `server_names` keeps the catch-all `_` verbatim, `location` is null for
# a proxy_pass written in the server block itself (the stream form), and
# `disabled` (phase 33, agent 1.4.0) is the line's `#off <name>` marker
# read by the tokenizer, exactly as the haproxy section's: on a
# `proxy_pass`, an `upstream` member line, or a `map` entry line. See
# the haproxy note for what the flag does and does not mean.
# a `proxy_pass $var` is served with `variable` (the `$` dropped) and
# `target: null`: the map whose variable matches decides per connection,
# so both facts travel unmerged and the dashboard joins them.
@@ -213,7 +246,12 @@ other protection is `--cert-bundle`/AGENT_CERT_BUNDLE: one installed PEM (server
key, the dashboard's root CA, `# BUNDLE-REV <n>` and `# REVOKED <serial>` lines) turns the
server into HTTPS that answers every path but `/health` only to a client certificate signed
by that root, and refuses one on the denylist on every path — the certificate is the auth, so
no token is read on that path.
no token is read on that path. The write path (1.4.0, the gateway) is `--allow-writes`
(default off) plus `--routectl-socket` (default /run/dashboards-routectl.sock, the helper's
socket; set the flag's value empty for the `503 no route helper on this host` arm): a
`POST /route` is then answered certificate-first, then flag, then body, then one dial to
that socket, and the helper's JSON is passed through verbatim — see the design rule above
for the refusal sentences.
"""
from __future__ import annotations
@@ -237,8 +275,65 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any, cast
AGENT_VERSION = "1.3.0"
AGENT_VERSION = "1.4.0"
SCHEMA_VERSION = 1
#: POST /route: one newline-delimited JSON request to the routectl helper, one line back.
#: The budget covers the helper's own checker run (`nginx -t` / `haproxy -c -f`) plus reload.
ROUTE_TIMEOUT_SECONDS = 30
#: The largest body the write path accepts, checked before a byte is read off the wire.
_ROUTE_MAX_BODY = 4096
#: An answer longer than this is "unreadable" — the helper's status JSON never is.
_ROUTE_MAX_ANSWER = 131072
# The route switch's marker, one constant the parser and the write path both read (phase 33).
# A comment in both haproxy and nginx, so `line, marker = split_marker(line)` on a route line
# is the whole of the read side: a marked line is still a live line to the proxy until the
# helper rewrites its value, and this is the state, not a behaviour the agent enforces.
ROUTE_MARKER = "#off"
def split_marker(line: str) -> tuple[str, str | None]:
"""Split a config line into ``(line, marker)`` for the helper's ``#off <name>`` marker.
The marker is the **whole** trailing comment (``#...`` through end of line), so the
grammar is one and the write path (task 01's ``routectl``) and the parser share it: a
line is marked when its comment *is* ``#off <name>`` — a marker with nothing after the
word, and nothing but the word, after it. Anything else in the comment is the operator's
own note, not a switch, and the line is reported ``disabled: false``. A marked line stays
in the document (the helper edits the value, never the line), which is what lets the
dashboard tell "declared, off" from "never declared" (that split is the dashboard's
``enabled`` field, phase 33 task 04).
"""
for i, ch in enumerate(line):
if ch == "#":
marker = line[i:]
if marker == ROUTE_MARKER or marker.startswith(ROUTE_MARKER + " "):
name = marker[len(ROUTE_MARKER):].strip()
if name and " " not in name:
return line[:i].rstrip(), name
return line, None
return line, None
def marker_for(name: str) -> str:
"""The full marker comment for one route name (the write path's spelling, pinned here so
the parser and task 01's helper cannot drift). One name per marker: a marker is a whole
comment and the name has no spaces, so the split above is exact."""
return f"{ROUTE_MARKER} {name}"
def config_line_index(text: str, name: str) -> int:
"""The 1-based line number of the route line carrying ``name``'s marker, or ``-1``.
Task 01's helper needs the line to edit; the marker names the route, so the search is
the marker itself, never the route's text (a target string appears in many lines and
only one of them is the route). `-1` is "not found", which the helper maps to its
`not-found` refusal.
"""
marker = marker_for(name)
for i, line in enumerate(text.splitlines()):
if split_marker(line)[1] == name and marker in line:
return i + 1
return -1
PAGE_SIZE = 0 # resolved from sysconf at runtime
# Filesystems that are not storage. Mirrors the spirit of node_exporter's default
@@ -2037,6 +2132,11 @@ def _scan_haproxy_config(text: str) -> tuple[list[dict[str, Any]], list[dict[str
if backend is not None and backend not in current_fe["backends"]:
current_fe["backends"].append(backend)
if current_be is not None and directive == "server" and len(words) > 2:
# the route switch's read side (phase 33, agent 1.4.0): the marker is a trailing
# comment, `#off <name>`, and a marked line is still a live line to haproxy until
# the helper rewrites the value — the agent reports the state, the dashboard
# draws it (`enabled: false`), and the kernel does not know markers.
marked = split_marker(stripped)[1]
target = _haproxy_addr_port(words[2])
current_be["servers"].append({
"name": words[1],
@@ -2044,6 +2144,7 @@ def _scan_haproxy_config(text: str) -> tuple[list[dict[str, Any]], list[dict[str
# port — "absent is never 0", on the one field haproxy makes optional
"target": words[2] if target is None else target[0],
"port": None if target is None else target[1],
"disabled": marked is not None,
})
return frontends, backends, recognised
@@ -2052,9 +2153,12 @@ def parse_haproxy_config(text: str) -> dict[str, Any]:
"""Parse a haproxy config's **text** into the `haproxy` section's data. Pure, never raises.
`{frontends: [{name, binds: [{addr, port}], default_backend, backends}], backends:
[{name, servers: [{name, target, port}]}], recognised: bool}` — the frontend→backend routing
is the names themselves (`default_backend` plus every `use_backend`), so the dashboard joins
them and this file interprets no rule.
[{name, servers: [{name, target, port, disabled}]}], recognised: bool}` — the frontend→
backend routing is the names themselves (`default_backend` plus every `use_backend`), so the
dashboard joins them and this file interprets no rule. `disabled` is the line's `#off
<name>` marker (phase 33, agent 1.4.0; `ROUTE_MARKER`, `split_marker`), `true` on a marked
line, `false` on every other line — the write path (the helper) is the only writer, and a
marker with anything but one name after it is the operator's comment, not a switch.
`recognised` answers the separate question "is this text a haproxy config at all" (does its
top level name a section keyword?) and is deliberately **not** part of the served section:
@@ -2070,7 +2174,7 @@ def parse_haproxy_config(text: str) -> dict[str, Any]:
def _nginx_tokens(text: str) -> list[tuple[str, str]]:
"""Split nginx config text into `("w", word)`, `("{", "")`, `("}", "")`, `(";", "")`.
"""Split nginx config text into `("w", word)`, `("{", "")`, `("}", "")`, `(";", comment)`.
nginx's grammar is words closed by `;` or opened by `{`, with `#` comments and quoted
strings that may contain any of those three — the captured gateway config writes
@@ -2083,16 +2187,28 @@ def _nginx_tokens(text: str) -> list[tuple[str, str]]:
A `#` opens a comment only where a word would have started, which is where nginx's own
lexer reads one: `proxy_pass http://host/#/app` keeps its fragment, and the
`# include /etc/nginx/old.conf;` line an operator commented out contributes nothing.
The one value a token carries is a `;` (or `}`) token's **end-of-line comment**: a comment
on the same line, after the statement that closed it, is attached to that token, and every
other comment is dropped as before. That is the route switch's read side (phase 33):
`server s1 10.1.1.9:443 #off s1` is a live line with a state, and the statement scanner is
what hands the comment to the directive. A comment at the start of a line (a commented-out
directive) belongs to no statement and is dropped, exactly as today.
"""
tokens: list[tuple[str, str]] = []
word: list[str] = []
started = False # an empty quoted string is still a word: the `''` key of a map
line_terminated = False # a `;` or `}` has been emitted on this line
i, n = 0, len(text)
while i < n:
char = text[i]
if char == "#" and not started and (i == 0 or text[i - 1] in " \t\r\n"):
while i < n and text[i] != "\n":
i += 1
end = i
while end < n and text[end] != "\n":
end += 1
if line_terminated and tokens and tokens[-1][0] in ";}":
tokens[-1] = (tokens[-1][0], text[i:end])
i = end
continue
if char in "\"'":
quote = char
@@ -2116,12 +2232,15 @@ def _nginx_tokens(text: str) -> list[tuple[str, str]]:
tokens.append(("w", "".join(word)))
word, started = [], False
tokens.append((char, ""))
line_terminated = True
i += 1
continue
if char.isspace():
if started:
tokens.append(("w", "".join(word)))
word, started = [], False
if char == "\n":
line_terminated = False
i += 1
continue
word.append(char)
@@ -2132,15 +2251,28 @@ def _nginx_tokens(text: str) -> list[tuple[str, str]]:
return tokens
def _comment_off(comment: str) -> bool:
"""Does this end-of-line comment carry the route switch's marker?
The grammar is `split_marker`'s — the one the write path (task 01's helper) shares —
applied to a line whose only comment is the one the tokenizer attached: `#off <name>`
with exactly one name after the word. The delegation is the point: the parser and the
helper cannot read the same marker differently.
"""
return split_marker(f"line {comment}")[1] is not None
def _nginx_statements(text: str) -> list[tuple[Any, ...]]:
"""The brace-block scanner: tokens in, nested statements out. Raises on an unbalanced file.
A statement is `("d", name, args)` for a directive or `("b", name, args, children)` for a
block. That is the whole difference between the two config languages this agent reads:
haproxy's structure is its indentation, nginx's is its braces, so nginx gets a scanner
instead of a line loop — and with it the two shapes that make the language what it is, a
`server { … location { … } … }` nested as deep as it goes and an `upstream` body whose
`server` lines are *targets* while an `http` body's `server {` is a *block*.
A statement is `("d", name, args, comment)` for a directive — `comment` is the end-of-line
comment the tokenizer attached (the route switch's marker lives there, phase 33) or `""` —
and `("b", name, args, children)` for a block. That is the whole difference between the two
config languages this agent reads: haproxy's structure is its indentation, nginx's is its
braces, so nginx gets a scanner instead of a line loop — and with it the two shapes that
make the language what it is, a `server { … location { … } … }` nested as deep as it goes
and an `upstream` body whose `server` lines are *targets* while an `http` body's `server {`
is a *block*.
`ValueError` here means "nginx itself would refuse to start on this file": a stray brace or
a directive that runs into the end of the file without its `;`. The caller turns that into
@@ -2156,7 +2288,7 @@ def _nginx_statements(text: str) -> list[tuple[Any, ...]]:
continue
if kind == ";":
if words: # a bare `;` is whitespace to nginx
containers[-1].append(("d", words[0], words[1:]))
containers[-1].append(("d", words[0], words[1:], value))
words = []
continue
if kind == "{":
@@ -2271,6 +2403,10 @@ def _collect_nginx(statements: _NginxStatements, out: dict[str, Any], *, root: b
"""
for statement in statements:
kind, name, args = statement[0], statement[1], statement[2]
# the end-of-line comment, if the tokenizer attached one: the route switch's marker
# (phase 33) lives on the route line itself, and `_comment_off` is its read
comment = statement[3] if len(statement) > 3 and isinstance(statement[3], str) else ""
marked = _comment_off(comment)
if kind == "d":
if name == "include" and args:
if root:
@@ -2284,6 +2420,7 @@ def _collect_nginx(statements: _NginxStatements, out: dict[str, Any], *, root: b
# keeps its name and a null port rather than losing the line entirely
"host": args[0] if found is None else found[0],
"port": None if found is None else found[1],
"disabled": marked,
})
continue
if map_ctx is not None:
@@ -2297,6 +2434,7 @@ def _collect_nginx(statements: _NginxStatements, out: dict[str, Any], *, root: b
# `upgrade` / `close` values are header words, and calling one a host
# would be this section inventing a fact to fill a field
"target": None if found is None else {"host": found[0], "port": found[1]},
"disabled": marked,
})
continue
if server is not None:
@@ -2309,7 +2447,9 @@ def _collect_nginx(statements: _NginxStatements, out: dict[str, Any], *, root: b
# "any domain" is what the page says about it, not what the agent decides
server["server_names"].extend(args)
elif name == "proxy_pass" and args:
server["proxy_passes"].append(_nginx_proxy_pass(args[0], location))
entry = _nginx_proxy_pass(args[0], location)
entry["disabled"] = marked
server["proxy_passes"].append(entry)
continue
children = cast("_NginxStatements", statement[3])
if name == "upstream" and args:
@@ -2367,9 +2507,14 @@ def parse_nginx_config(main_text: str,
"""Parse an nginx main config plus the documents its includes name. Pure, never raises.
`{servers: [{listen: [{addr, port}], server_names: [str], proxy_passes: [{location, scheme,
target, variable}]}], upstreams: [{name, servers: [{host, port}]}],
maps: [{variable, entries: [{key, target}]}], files: [str], includes: [str], unparsed:
[str], recognised: bool, error: str|null, resolver: str|null}`.
target, variable, disabled}]}], upstreams: [{name, servers: [{host, port, disabled}]}],
maps: [{variable, entries: [{key, target, disabled}]}], files: [str], includes: [str],
unparsed: [str], recognised: bool, error: str|null, resolver: str|null}`.
`disabled` on a `proxy_pass`, an `upstream` member line, or a `map` entry line is the
route switch's read side (phase 33, agent 1.4.0): `true` when that line carries the
helper's `#off <name>` marker, `false` otherwise — one grammar (`split_marker`), the same
one the haproxy section uses, and the same one the write path (task 01's helper) shares.
Four of those keys are the ones `Collector.nginx` does not serve, the way
`parse_haproxy_config`'s `recognised` is not: `error` is why the main text could not be
@@ -2979,6 +3124,12 @@ STATUS_PAGE = """<!doctype html>
</table>
<p><a href="/snapshot">/snapshot</a> · <a href="/info">/info</a> ·
<a href="/health">/health</a> · <a href="/schema">/schema</a></p>
<p style=color:#8b93a7>POST /route — the one write (switch a route's #off marker; 1.4.0).
Refusals, in order: 405 no such write path · 403 writes require a certificate (token
mode) · 403 client certificate required · 403 client certificate revoked · 403 writes
are not enabled on this agent · 422 body refusals · 503 no route helper on this
host · 502 helper unreachable / timed out / unreadable answer. The bytes are written
by the socket-activated root helper, never by this process.</p>
"""
@@ -2987,6 +3138,8 @@ class Handler(BaseHTTPRequestHandler):
collector: Collector
token: str | None = None
bundle: TlsBundle | None = None
allow_writes: bool = False
routectl_socket: str | None = None
def do_GET(self) -> None: # the BaseHTTPRequestHandler API name — keep it
path = self.path.split("?", 1)[0].rstrip("/") or "/"
@@ -3032,6 +3185,102 @@ class Handler(BaseHTTPRequestHandler):
else:
self._send(404, {"ok": False, "detail": f"no such path: {path}"})
def do_POST(self) -> None: # the BaseHTTPRequestHandler API name — keep it
# The one write (1.4.0). The order is the design, and each refusal is its own
# sentence: the certificate (the handshake is the identity — the token arm is
# refused outright, the denylist is not a read-side rule), then the operator's
# flag, then the body (a refused body never reaches the peer), then the helper's
# socket. Nothing in this method writes a file; the bytes are the helper's.
path = self.path.split("?", 1)[0].rstrip("/") or "/"
if path != "/route":
self._send(405, {"ok": False, "detail": "no such write path"})
return
if self.bundle is None:
self._send(403, {"ok": False, "detail": "writes require a certificate"})
return
if not self._certificate_allows(self.bundle, path):
return
if not self.allow_writes:
self._send(403, {"ok": False, "detail": "writes are not enabled on this agent"})
return
try:
length = int(self.headers.get("content-length", "0"))
except ValueError:
length = -1
if length < 0 or length > _ROUTE_MAX_BODY:
self._send(422, {"ok": False,
"detail": "body longer than the write path accepts"})
return
raw = self.rfile.read(length)
try:
body = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, ValueError):
self._send(422, {"ok": False, "detail": "body is not a JSON object"})
return
if not isinstance(body, dict):
self._send(422, {"ok": False, "detail": "body is not a JSON object"})
return
if not set(body) <= {"id", "action", "dry_run"}:
self._send(422, {"ok": False, "detail": "body carries a key the write path does not take"})
return
action = body.get("action")
if action not in ("disable", "enable"):
self._send(422, {"ok": False, "detail": "action must be disable or enable"})
return
rid = body.get("id")
if not isinstance(rid, str) or not rid or len(rid) > 256:
self._send(422, {"ok": False,
"detail": "id must be a string of at most 256 characters"})
return
if self.routectl_socket is None:
self._send(503, {"ok": False, "detail": "no route helper on this host"})
return
self._route_peer({"id": rid, "action": action,
"dry_run": bool(body.get("dry_run", False))})
def _route_peer(self, payload: dict[str, Any]) -> None:
"""Dial the routectl helper once, forward `payload`, pass the one answer through.
One connection, one request, one answer: a retry of a write is a second reload,
so every failure is a 502 and never a second dial. The helper's JSON travels
verbatim — a refusal it answers stays a refusal, and the agent invents no
success of its own. The log line prints the path and the status only: the
`id` is a hostname, and this process's log lines stay out of its body.
"""
buf = b""
sock = None
try:
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(ROUTE_TIMEOUT_SECONDS)
sock.connect(self.routectl_socket)
sock.sendall(json.dumps(payload).encode("utf-8") + b"\n")
while not buf.endswith(b"\n"):
chunk = sock.recv(65536)
if not chunk:
break
buf += chunk
if len(buf) > _ROUTE_MAX_ANSWER:
self._send(502, {"ok": False,
"detail": "route helper answered something unreadable"})
return
except TimeoutError:
self._send(502, {"ok": False, "detail": "route helper timed out"})
return
except OSError as exc:
word = socket.errorname.get(exc.errno, str(exc.errno))
self._send(502, {"ok": False, "detail": f"route helper: {word}"})
return
finally:
if sock is not None:
sock.close()
try:
answer = json.loads(buf.decode("utf-8").strip())
except (UnicodeDecodeError, ValueError):
self._send(502, {"ok": False,
"detail": "route helper answered something unreadable"})
return
self._send(200, answer)
def _certificate_allows(self, bundle: TlsBundle, path: str) -> bool:
"""Does this connection's client certificate authorise `path`? Sends the refusal if not.
@@ -3094,17 +3343,21 @@ class Handler(BaseHTTPRequestHandler):
def serve(host: str, port: int, collector: Collector, token: str | None,
bundle: TlsBundle | None = None) -> None:
bundle: TlsBundle | None = None, *, allow_writes: bool = False,
routectl_socket: str | None = None) -> None:
"""Serve forever on host:port — plain HTTP with an optional token, or HTTPS with a bundle.
All three class attributes are assigned on every call: `Handler` is process-wide state, so a
server started without a bundle must not inherit one from a server that had it (that would
answer a plain-HTTP socket with 403 on every path, or worse, accept a client certificate the
operator revoked).
All five class attributes are assigned on every call: `Handler` is process-wide state, so a
server started without a bundle (or without the write flag) must not inherit one from a
server that had it (that would answer a plain-HTTP socket with 403 on every path, or
worse, accept a client certificate the operator revoked — or answer a write to a helper
this process was never told about).
"""
Handler.collector = collector
Handler.token = token
Handler.bundle = bundle
Handler.allow_writes = allow_writes
Handler.routectl_socket = routectl_socket
httpd = ThreadingHTTPServer((host, port), Handler)
if bundle is not None:
httpd.socket = bundle.context.wrap_socket(httpd.socket, server_side=True)
@@ -3178,6 +3431,17 @@ def main(argv: list[str] | None = None) -> int:
f"documented degradation (available=false, one error entry), "
f"not a failure")
parser.add_argument("--once", action="store_true", help="print one snapshot as JSON and exit")
parser.add_argument("--allow-writes", action="store_true",
help="answer POST /route on this agent (the fleet's one narrow write: "
"switch a route's #off marker). The gates are the client "
"certificate and this flag, in that order; the bytes are written "
"by the socket-activated root helper, never by this process")
parser.add_argument("--routectl-socket",
default=os.environ.get("AGENT_ROUTECTL_SOCKET",
"/run/dashboards-routectl.sock"),
help="the routectl helper's socket that POST /route dials once per "
"request; set the value empty for the "
"`503 no route helper on this host` arm")
args = parser.parse_args(argv)
collector = Collector(args.root, top_processes=args.top, net_all=args.net_all,
@@ -3203,7 +3467,9 @@ def main(argv: list[str] | None = None) -> int:
"metrics are sensitive); pass --token, --token-file, --cert-bundle or set "
"AGENT_TOKEN / AGENT_CERT_BUNDLE", flush=True)
return 2
serve(args.host, args.port, collector, token, bundle)
serve(args.host, args.port, collector, token, bundle,
allow_writes=args.allow_writes,
routectl_socket=args.routectl_socket or None)
return 0