services/dashboards
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 33_gateway_route_switching

Browse Source 
Phase 33 completion pass is done. Final report:

## Phase 33 — completion report

**All 8 task files complete** (`.agents/phases/complete/33_gateway_route_switching/`), full gate green, docs verified.

### Work in this pass

1. **`tests/test_routectl.py` added** (17 tests, all green) — exercises the *real* helper binary unprivileged against a fake tree (`--root`), the real agent parser, the captured gateway fixtures, and stubbed `nginx`/`haproxy`/`systemctl` on `PATH`. Pins the full protocol (locate → state → dry-run → temp-file/fsync/`os.replace` write → checker → restore on checker failure → reload-never-restart → re-read proof → sidecar record), all three route shapes (haproxy `server`, nginx `proxy_pass`, nginx stream `map` via its sink-value id), `drift` / unknown-id / `map-default` refusals, `list`/`verify`, and the `--serve` peer gate (`SO_PEERCRED` uid check, one JSON line per request, exact body shape).

2. **Real helper defect fixed** — `deploy/dashboards-routectl.py`: `main()`'s early-return paths returned `({"ok": False, …}, EXIT_*)` tuples, so `sys.exit()` printed a Python repr instead of a JSON line (usage errors, agent-import refusal). Now prints `json.dumps(…)` and returns only the code, keeping the one-JSON-line contract.

3. **Documentation consistency fixes** (task 08 criterion 6):
   - `README.md`: garbled verdict-colour sentence on the switch ("green and to the position") → "in the **on** position and tinted `--gate-on`" (the existence-pair contract).
   - `README.md`: audit sentence now names all three event kinds — `route-disabled` / `route-enabled` / **`route-denied`** (the code stores one per refused call; dry runs store none).
   - `page-gateway.js`: replaced the stale pre-33 module docstring ("the switch is not wired") with the two-arm `GATEWAY_ALLOW_WRITES` contract.
   - `AGENTS.md`: every phase-33 claim re-verified against code (the ten gates, the 35 s/30 s budget pair, `gateway.writes_enabled`, `/api/summary`'s `disabled`, the 1.4.0 agent bullet, the gap list) — no changes needed.

### Verification
- `uv run pytest --cov`: **1970 passed**, 18 skipped — coverage **98.74%** (floor 94%)
- `ruff` clean · `pyright` 0 errors · `bash .agents/validate.sh` **exit 0** (incl. `node --check` on every page script)
- Dev server running; all 10 sources `ok`; bundles exported

### Live infrastructure
**No live changes made.** The live drill remains unexecuted per `data/logs/NOTES-33-switching.md` (no operator approval; checker binaries unavailable in the podman proxy units). All routes stay enabled, config hashes unchanged, and the sandbox rehearsal + backup tree are in place for the operator's own `DANGER` call. All changes are in the working tree only.
This commit is contained in:
ducoterra
2026-11-01 11:50:51 -05:00
parent 0271387053
commit 86c8bf4992
91 changed files with 7568 additions and 519 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/34_radiance_llm_engine/00_phase.md
+54
View File
@@ -0,0 +1,54 @@
# Phase 34 — Radiance is a first-class engine: a third `/llm` panel that says what its /metrics and /stats publish, nothing invented
**Source:** operator request, 2026-10-24 — "add support for a new llm engine 'radiance' which is running on driveripper now. Check it out at http://driveripper.reeselink.com:8000/ and come up with a plan." + the two decisions taken with them in that conversation: **radiance replaced vLLM at driveripper:8000** (the dev config's `VLLM_BASE_URL` points at the engine that is now radiance, so the dev config moves to `RADIANCE_BASE_URL`; vLLM *code* stays — the API shape is additive-only and other deployments may run vLLM), and **the per-position spec-acceptance curve is in** (seven positions, each its own acceptance ratio).
**Story:** n/a
**Context:** `src/dashboards/sources/llm.py` (the gufo/vLLM field maps, the rule-2 throughput precedence, `HISTORY_METRICS`, `_sessions()`, `fleet_totals()`, `SEVERITY_CELLS`), `src/dashboards/config.py` (the `*_BASE_URL`/`*_API_KEY` engine settings, `any_source_enabled`), `src/dashboards/collectors.py` (`fetch_engine`, the best-effort `/slots` fetch, `llm_jobs`), `src/dashboards/api.py` (the `llm_router` 404 gate, `/api/config`'s llm block, the summary's llm block, the history defaults), `src/dashboards/alerts.py` (the cell-driven `_engine_alerts`), `src/dashboards/severity.py` (the only threshold table), `src/dashboards/web/static/page-llm.js` + `pages/llm.html` (the `only:` scoping, strip cells, slot grid), `tests/test_llm_source.py` / `tests/test_collectors.py` / `tests/test_api.py` / `tests/test_config.py` / `tests/test_live_fleet.py`, `scripts/capture-fixtures.sh`, `.env.dev` / `.env.dev.example`, `README.md`.
## Objective
The dashboard treats radiance (a vLLM-derived engine, v1.15.1, model `juggernaut`, on `driveripper:8000`) as a **first-class third engine** — `llm:radiance` — with its own field map pinned by a verbatim capture, its native windowed throughput gauges winning per rule 2, its `/stats` endpoint as the session source (radiance's answer to gufo's `/slots`), and a radiance strip on `/llm` (queue cap, version/uptime, per-card VRAM, the 7-bar spec-acceptance curve) — while gufo and vLLM keep their exact payloads, history names, and behaviour.
## Scope decisions taken with the operator (binding for this phase)
| # | Decision | Consequence |
|---|----------|-------------|
| 1 | **Radiance replaced vLLM at driveripper:8000.** | `.env.dev`'s `VLLM_BASE_URL` line becomes `RADIANCE_BASE_URL` (same address). The vLLM engine **code and env var stay** — `VLLM_BASE_URL` still configures a vLLM engine for other deployments; only the dev fleet's config moves. `AGENTS.md`'s fleet table says `vLLM :8000` for driveripper; that is the one stale fact the final pass fixes. |
| 2 | **The per-position spec-acceptance curve is in.** | `radiance:draft_position_acceptance_ratio{position="0".."6"}` renders as a 7-bar hand-rolled SVG on the radiance strip, `role="img"` + `aria-label`, pinned in `tests/test_api.py` beside every other chart. Snapshot-only: the seven per-position ratios do **not** get history series (a bounded store, rule 9); the overall `draft_acceptance_ratio` does. |
| 3 | **Engine internals the operator cannot act on are out.** | `link`/`passes`/`steps*`/`ckpt*`/`hit_cuts`/`released`/`kv_stream*`/`lookup*`/`linear*`/`attn_cached`/`client_disconnect`/`stream_overflow` stay out of the payload (flavour text is not helpful information). What the payload carries: native + measured throughputs, KV usage + max tokens/bytes, queue depth/cap/admission-rejected, preemptions, the prefix hit-rate gauge, overall + per-position spec acceptance, the decode-step gauges, `requests[]` sessions, per-card VRAM, `model`/`version`/`uptime_s`/`startup_s`, lifetime `tokens_total`. |
| 4 | **No new thresholds, no new write path.** | The only rated radiance cell is KV pressure, via the **existing** `llm.kv_cache_usage_pct` row (radiance publishes the same 0-1 ratio the vLLM map already feeds it). Spec acceptance is an efficiency number, not a verdict — uncoloured. No radiance endpoint accepts writes; the app's only upstream write stays `POST /api/ups/command` (invariant 1 unchanged). No new dependency, no new `/api` route — the served-route walk and the public-surface pin are untouched. |
## Dependencies
- `23_llm_rate_responsiveness` (complete) — the per-cadence `RateEstimator` and the refused-window → `n/a` + `Δ<window>` semantics this phase reuses for the measured Δ of radiance's vLLM-shaped counters.
- `13_llm_page_and_decode_rate` (complete) — the engine-panel/strip shape, the `only:` scoping, the served-byte pins this phase extends.
- `24_dev_mtls_rollout` (complete) — the direct-dial fleet; radiance is dialled the same plain way the other engines are (no token, no certificate — the operator's address answers unauthenticated).
- Independent of `33_gateway_route_switching` (todo, in progress) — orthogonal module; no shared file besides `api.py`'s already-present llm routes, which this phase does not touch.
## Tasks
1. `01_capture_radiance_ground_truth.md` — `capture-fixtures.sh` records radiance's `/metrics`, `/stats` and `/v1/models` verbatim into `tests/fixtures/`, and the fixture name-set is pinned hermetically.
2. `02_config_and_the_engine_registry.md` — `RADIANCE_BASE_URL` / `RADIANCE_API_KEY`, one `llm_engines()` registry replacing the five hardcoded gufo/vllm enumeration sites, the dev config moves to radiance.
3. `03_the_radiance_field_map.md` — `RADIANCE_SERIES`/`RADIANCE_HISTOGRAMS`, `parse_radiance_metrics()`, the radiance `_sessions()` arm, the additive history series, `fleet_totals`' seams copy.
4. `04_the_radiance_job.md` — `fetch_engine` learns the best-effort `/stats` fetch, the `llm:radiance` job.
5. `05_the_llm_page_learns_radiance.md` — the radiance strip (queue cap, version/uptime, per-card VRAM, the 7-bar spec curve), the `only:` scopes, `requests[]` rows, the empty-state text.
6. `06_docs_and_live_verify.md` — the live read-only test, the fixture re-capture, README/AGENTS.md pass, the live-data screenshots.
## Testing & Quality
- Unit/integration: the radiance parse against the verbatim fixtures (name-set drift pin, rule-2 precedence in both directions, absent → `n/a`, the refused-window arm, severity pass-through, the per-position shape, both session arms, `fleet_totals` with a radiance payload), the collector's best-effort `/stats` (a dead `/stats` keeps the `/metrics` poll healthy — invariant 4), the registry (one place lists the engines), the API surface (`/api/config.llm.radiance`, the 404 gate, the history defaults, the summary), and the live read-only test.
- Coverage: **>90%** on new/modified code; the whole suite 100%; `ruff` + `pyright` clean.
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's — pre-existing failures included) and achieves **>90%** code coverage on new/modified code.
- Every UI change is verified in a real browser against live data, screenshot in `data/logs/` (AGENTS.md rule 10) — with gufo and radiance both answering, the cross-engine fleet block finally renders on the dev fleet, which is the phase's own proof that the sum is honest.
## Documentation & Consistency
- The final pass reviews `README.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by this phase**. For `AGENTS.md` it fixes only *stale statements* (a dead command, a changed rule, an outdated fact) and **never adds or expands its content** (a file that states its own update policy, e.g. "updated rarely, if ever", wins).
- Tasks update `README.md` immediately when their own changes are user-visible (setup, commands, configuration, features, conventions), and `AGENTS.md` only when a statement in it became stale.
## Completion Criteria
- [ ] `RADIANCE_BASE_URL` set and `GUFO_BASE_URL` set: `curl -s http://127.0.0.1:8090/api/llm/snapshot | jq .engines | jq 'map(.engine)'` lists `gufo`, `radiance` (and `vllm` only if configured), and `fleet.engines_live` is 2; `/api/config.llm.radiance.enabled` is `true`.
- [ ] `RADIANCE_BASE_URL` unset: `/api/llm/snapshot` answers `404 {"configured": false}` with no dial (spy-pinned), and no `llm:radiance` job exists in the hub.
- [ ] `tests/fixtures/metrics_radiance.txt` and `tests/fixtures/stats_radiance.json` exist, are verbatim captures, and a hermetic test fails if the fixture's `radiance:*`/`vllm:*` name set drifts from `RADIANCE_SERIES`/`RADIANCE_HISTOGRAMS`.
- [ ] The radiance panel on `/llm` shows the native `Δ<window>`-labelled throughputs when the gauge is absent, the KV bar with its severity colour, the 7-bar spec curve with its `aria-label`, the `requests[]` rows when a request is live and the counts-otherwise arm at rest — screenshot in `data/logs/`.
- [ ] `uv run pytest -m live --live` includes a radiance test that passes read-only against the running engine.
- [ ] full test suite **100% passing** (every test in the project, not just this phase's), coverage >90%; `ruff` + `pyright` clean; `bash .agents/validate.sh` green.
- [ ] README.md is accurate and consistent with the whole project; AGENTS.md has no stale statements (verified in the final pass — the `vLLM :8000` fleet fact included).
- [ ] no behaviour change in completed work: the gufo and vLLM payloads, history names, alerts, and served pages are byte-identical where this phase does not touch them (the suite's existing pins untouched).