phase: 34_radiance_llm_engine

Everything is verified and green. Final report:

**Phase 34 final verification pass — complete**

- Verified all 6 completed tasks' work against the live fleet and hermetic suite: `llm:radiance` job registered, `llm_engines()` registry drives 404 gate / history defaults / `/api/config` (additive, byte-identical gufo/vllm keys), radiance field map + `/stats` sidecar + 7-position spec curve, `/llm` strip/curve/rows.
- **Defect found & fixed (pre-existing, task 05):** the vLLM-only sentence "vLLM exposes no per-slot JSON…" rendered on the **radiance** panel (`slotSection`'s non-gufo branch). Fix: vLLM line gated to `engine === 'vllm'`; radiance's panel slot section renders nothing (its session home is the strip) — `page-llm.js` + the served-byte pin in `tests/test_api.py` (now asserts the radiance branch is empty and the sentence can't leak). Browser-verified; re-shot `data/logs/34-llm-radiance-rest.png` (the 4.5-min wait found no in-flight request, so the busy shot remains the earlier pass's — its strip rows still stand; only the panel text predates the fix).
- Tests: `uv run pytest` → **2031 passed, 19 skipped** (100%); `uv run pytest --cov` → **98.73%** (floor 94%; config 100%, llm 99%, collectors 98%, api 99%); `uv run ruff check src tests agent && uv run pyright` → clean; `node --check` OK; `uv pip list --outdated` → empty.
- `uv run pytest -m live --live` → **13 passed, 2 skipped** (vLLM unset — correct; live-write opt-in — correct); the read-only radiance test passed.
- Completion criteria: (1) engines `gufo`+`radiance`, `fleet.engines_live` 2, `config.llm.radiance.enabled` true — **pass (live curl)**; (2) unset → 404 no-dial + no job — **pass (spy-pinned, green)**; (3) fixtures present, both families (79 `radiance:` / 198 `vllm:` lines), drift pin — **pass**; (4) panel/curve/rows — **pass (re-shot rest screenshot)**; (5) live radiance test — **pass**; (6) suite/coverage/lint/`bash .agents/validate.sh` (exit 0) — **pass**; (7) docs — below; (8) gufo/vLLM behaviour unchanged — **pass (existing pins untouched)**.
- Documentation: **README.md** full review — env table, LLM-engines section, architecture diagram, capture-script and `/llm` section all name radiance correctly; no inconsistencies found, no changes needed. **AGENTS.md** — the three task-06 fixes (fleet line, route row, module bullets) are in place and all other statements match reality; no changes needed.
- Dev server healthy (`./scripts/dev-server.sh status`: all sources ok, `llm:radiance` answering); all changes left in the working tree for the harness.
- Next pending phase: none — 34 was the last phase in `todo/`.
This commit is contained in:
2026-11-01 18:53:46 -05:00
parent 86c8bf4992
commit 219d7a7c3b
77 changed files with 3517 additions and 149 deletions
+81
View File
@@ -25,6 +25,7 @@ from __future__ import annotations
import asyncio
import ipaddress
import json
import math
import os
import sqlite3
from collections.abc import Iterator
@@ -479,6 +480,86 @@ async def test_gufo_slots_answer_the_pinned_shape(settings: Settings) -> None:
assert_slot_shape(rows)
async def test_radiance_answers_the_pinned_shape(settings: Settings) -> None:
"""Phase 34: the radiance `/metrics` + `/stats` poll over the **running** engine.
The production path — `collectors.fetch_engine` with the LLM cadence's estimator,
`with_stats` the way `llm_jobs` runs it — so what fails here is a rename between the
engine and the map, not a typo in a fixture. Read-only: radiance exposes no write
endpoint, so "read-only" is a list of GETs. Skips when `RADIANCE_BASE_URL` is unset
(invariant 3's live twin: unset is not deployed here, not broken). The assertions
are shape, never numbers: the `stats` block's whitelist, the ordered 7-position spec
curve (when the build publishes it), one of the two specced session arms, and every
cell the engine answered is a finite number or `None` — an absent gauge is `None`,
never a faked `0` (rule 2's live twin). `unparsed` must be empty: a parse error on
the live wire is drift, not a tolerance.
"""
if not settings.radiance_url:
pytest.skip("RADIANCE_BASE_URL is unset — that engine is not deployed here")
# The span the LLM jobs really get: phase 23 spans the LLM estimator by the poll
# interval, not by the host's `RATE_WINDOW_SECONDS` (the other tests' idiom).
rates = RateEstimator(span=settings.llm_interval)
async with httpx.AsyncClient(timeout=settings.llm_timeout) as client:
result = await collectors.fetch_engine(client, "radiance", settings.radiance_url,
settings.radiance_api_key, rates,
with_stats=True)
payload = result.payload
assert payload["engine"] == "radiance"
assert payload["unparsed"] == 0, (
f"a parse error on the live wire is drift, not a tolerance: {payload['unparsed']}")
metrics = payload["metrics"]
# The /stats block, task 03's whitelist: present only when the sidecar answered, and
# then only those keys (the engine internals stay out — decision 3).
stats = payload.get("stats")
if stats is not None:
allowed = {"model", "version", "uptime_s", "startup_s", "total_requests", "cards"}
extra = set(stats) - allowed
assert not extra, f"the /stats subset carried keys the whitelist does not name: {sorted(extra)}"
for card in stats.get("cards", []):
assert set(card) <= {"index", "name", "vram_used", "vram_total", "vram_used_pct"}, card
# The spec-acceptance curve: the ordered seven, when the build publishes it.
positions = metrics["spec_positions"]
assert isinstance(positions, list)
if positions:
assert len(positions) == 7, (
f"the spec curve has {len(positions)} positions — the pin is 0..6, "
f"not {[p['position'] for p in positions]}")
assert [p["position"] for p in positions] == list(range(7)), ("not 0..6 in order")
for pos in positions:
ratio = pos["ratio"]
assert ratio is None or (isinstance(ratio, int | float)
and not isinstance(ratio, bool)
and math.isfinite(ratio) and 0.0 <= ratio <= 1.0), (
f"position {pos['position']}'s ratio {ratio!r} is not a 0-1 gauge")
# Every cell the engine answered is a finite number or None — never a faked 0.
for key, cell in metrics.items():
if key == "spec_positions":
continue # the ordered list, checked above
if not isinstance(cell, dict) or "value" not in cell:
continue
value = cell["value"]
assert value is None or (isinstance(value, int | float)
and not isinstance(value, bool)
and math.isfinite(value)), (
f"{key}: {value!r} is neither a number nor None")
# The session arm is one of the two specced arms: /stats answered (the engine's own
# summary is the source) or it did not (the /metrics gauges, and it says so).
sessions = payload["sessions"]
if stats is not None:
assert sessions["source"] == "radiance /stats"
assert "running" in sessions and "waiting" in sessions and "queue_cap" in sessions
else:
assert sessions["source"] is None
assert sessions.get("queue_cap") is None, ("an absent /stats leaves the cap n/a, never 0")
assert "/stats" in (sessions.get("detail") or ""), (
f"the fallback arm must name the missing sidecar: {sessions.get('detail')!r}")
assert sessions["active"] is None or isinstance(sessions["active"], int | float)
def require_upsd(settings: Settings) -> None:
"""Skip, not fail, when there is no real upsd to talk to."""
if not settings.nut_enabled: