phase: 34_radiance_llm_engine
Everything is verified and green. Final report: **Phase 34 final verification pass — complete** - Verified all 6 completed tasks' work against the live fleet and hermetic suite: `llm:radiance` job registered, `llm_engines()` registry drives 404 gate / history defaults / `/api/config` (additive, byte-identical gufo/vllm keys), radiance field map + `/stats` sidecar + 7-position spec curve, `/llm` strip/curve/rows. - **Defect found & fixed (pre-existing, task 05):** the vLLM-only sentence "vLLM exposes no per-slot JSON…" rendered on the **radiance** panel (`slotSection`'s non-gufo branch). Fix: vLLM line gated to `engine === 'vllm'`; radiance's panel slot section renders nothing (its session home is the strip) — `page-llm.js` + the served-byte pin in `tests/test_api.py` (now asserts the radiance branch is empty and the sentence can't leak). Browser-verified; re-shot `data/logs/34-llm-radiance-rest.png` (the 4.5-min wait found no in-flight request, so the busy shot remains the earlier pass's — its strip rows still stand; only the panel text predates the fix). - Tests: `uv run pytest` → **2031 passed, 19 skipped** (100%); `uv run pytest --cov` → **98.73%** (floor 94%; config 100%, llm 99%, collectors 98%, api 99%); `uv run ruff check src tests agent && uv run pyright` → clean; `node --check` OK; `uv pip list --outdated` → empty. - `uv run pytest -m live --live` → **13 passed, 2 skipped** (vLLM unset — correct; live-write opt-in — correct); the read-only radiance test passed. - Completion criteria: (1) engines `gufo`+`radiance`, `fleet.engines_live` 2, `config.llm.radiance.enabled` true — **pass (live curl)**; (2) unset → 404 no-dial + no job — **pass (spy-pinned, green)**; (3) fixtures present, both families (79 `radiance:` / 198 `vllm:` lines), drift pin — **pass**; (4) panel/curve/rows — **pass (re-shot rest screenshot)**; (5) live radiance test — **pass**; (6) suite/coverage/lint/`bash .agents/validate.sh` (exit 0) — **pass**; (7) docs — below; (8) gufo/vLLM behaviour unchanged — **pass (existing pins untouched)**. - Documentation: **README.md** full review — env table, LLM-engines section, architecture diagram, capture-script and `/llm` section all name radiance correctly; no inconsistencies found, no changes needed. **AGENTS.md** — the three task-06 fixes (fleet line, route row, module bullets) are in place and all other statements match reality; no changes needed. - Dev server healthy (`./scripts/dev-server.sh status`: all sources ok, `llm:radiance` answering); all changes left in the working tree for the harness. - Next pending phase: none — 34 was the last phase in `todo/`.
This commit is contained in:
@@ -33,9 +33,13 @@ for entry in ${targets//,/ }; do # space or comma separated, like config.parse
|
||||
done
|
||||
# Engine URLs come from the same env too, so this script carries no deployment
|
||||
# detail of its own — and never records a tokenised URL into a fixture by accident.
|
||||
# Engine entries are `<name> <url>` pairs; the name is the fixture suffix and the env
|
||||
# var that dials the engine (gufo / vllm / radiance — a vLLM derivative that also
|
||||
# answers /stats and /v1/models, captured in the same pass).
|
||||
ENGINES=()
|
||||
[ -n "${GUFO_BASE_URL:-}" ] && ENGINES+=("gufo:${GUFO_BASE_URL%/}/metrics")
|
||||
[ -n "${VLLM_BASE_URL:-}" ] && ENGINES+=("vllm:${VLLM_BASE_URL%/}/metrics")
|
||||
[ -n "${GUFO_BASE_URL:-}" ] && ENGINES+=("gufo ${GUFO_BASE_URL%/}/metrics")
|
||||
[ -n "${VLLM_BASE_URL:-}" ] && ENGINES+=("vllm ${VLLM_BASE_URL%/}/metrics")
|
||||
[ -n "${RADIANCE_BASE_URL:-}" ] && ENGINES+=("radiance ${RADIANCE_BASE_URL%/}/metrics")
|
||||
|
||||
failed=0
|
||||
|
||||
@@ -211,17 +215,78 @@ fi
|
||||
|
||||
if [ "${1:-}" = "--with-llm" ]; then
|
||||
if [ ${#ENGINES[@]} -eq 0 ]; then
|
||||
echo " ! no GUFO_BASE_URL / VLLM_BASE_URL set (see .env.dev) — nothing to capture"
|
||||
echo " ! no GUFO_BASE_URL / VLLM_BASE_URL / RADIANCE_BASE_URL set (see .env.dev) — nothing to capture"
|
||||
failed=1
|
||||
fi
|
||||
for entry in "${ENGINES[@]}"; do
|
||||
name=${entry%%:*}; url=${entry#*:}
|
||||
name=${entry%% *}; url=${entry#* }
|
||||
if curl -fsS --max-time 20 "$url" -o "$FIX/metrics_${name}.txt"; then
|
||||
# Radiance's capture carries the script's verbatim header, like the other fixtures —
|
||||
# gufo's and vLLM's metrics files stay raw, as before.
|
||||
if [ "$name" = "radiance" ]; then
|
||||
{ echo "# Verbatim /metrics reply from the radiance engine, recorded by"
|
||||
echo "# ./scripts/capture-fixtures.sh — the engine's own Prometheus text"
|
||||
echo "# format: the radiance: native series + the vllm: compatibility family."
|
||||
echo "# Volatile values (counters, gauges) move every poll, so a diff here"
|
||||
echo "# on re-capture is expected; the contract asserts the series-name set"
|
||||
echo "# against RADIANCE_SERIES/RADIANCE_HISTOGRAMS/RADIANCE_OPTIONAL in"
|
||||
echo "# sources/llm.py — never the numbers (the nut_*.vars precedent)."
|
||||
cat "$FIX/metrics_${name}.txt"; } > "$FIX/.metrics_radiance.txt.tmp" &&
|
||||
mv "$FIX/.metrics_radiance.txt.tmp" "$FIX/metrics_${name}.txt"
|
||||
fi
|
||||
echo " ✓ metrics_${name}.txt ($(wc -l <"$FIX/metrics_${name}.txt") lines)"
|
||||
else
|
||||
echo " ✗ metrics_${name}.txt — $url unreachable (VPN? engine down?)"
|
||||
failed=1
|
||||
fi
|
||||
# radiance is a vLLM derivative and answers two more read-only endpoints: /stats
|
||||
# (the engine's own JSON snapshot: cards[], requests[], draft_positions[], ...)
|
||||
# and /v1/models (the model's capabilities — the only place the engine states what
|
||||
# it can do). Both are recorded verbatim in the same pass the gufo /slots JSON
|
||||
# document is, with the same verbatim header; the hermetic shape tests in
|
||||
# tests/test_llm_source.py need them on disk.
|
||||
if [ "$name" = "radiance" ]; then
|
||||
base=${url%/metrics}
|
||||
stats_tmp="$FIX/.stats_radiance.json.tmp"
|
||||
models_tmp="$FIX/.models_radiance.json.tmp"
|
||||
if curl -fsS --max-time 10 "${base}/stats" -o "$stats_tmp"; then
|
||||
info=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('in_flight', 0), len(d.get('requests', [])))" "$stats_tmp" 2>/dev/null || echo '? 0')
|
||||
inflight=${info%% *}; rows=${info##* }
|
||||
{ echo "# Verbatim /stats reply from the radiance engine, recorded by"
|
||||
echo "# ./scripts/capture-fixtures.sh — one JSON document, the engine's own"
|
||||
echo "# shape. Volatile values (uptime_s, now_ms, the counters, kv_*) move"
|
||||
echo "# every poll, so a diff here on re-capture is expected; the contract"
|
||||
echo "# asserts the top-level key set and the cards[]/requests[]/"
|
||||
echo "# draft_positions[] shape — never those numbers (the nut_*.vars"
|
||||
echo "# precedent)."
|
||||
echo "# in_flight was ${inflight} at capture (requests[] has ${rows} row(s))."
|
||||
cat "$stats_tmp"; } > "$FIX/.stats_radiance.json.tmp2" &&
|
||||
mv "$FIX/.stats_radiance.json.tmp2" "$FIX/stats_radiance.json"
|
||||
rm -f "$stats_tmp"
|
||||
if [ "$inflight" = "0" ]; then
|
||||
echo " (requests[] was empty — re-run with a request in flight to record its shape)"
|
||||
fi
|
||||
echo " ✓ stats_radiance.json ($(wc -c <"$FIX/stats_radiance.json") bytes)"
|
||||
else
|
||||
rm -f "$stats_tmp" # never overwrite a good fixture with a dead engine's empty file
|
||||
echo " ✗ stats_radiance.json — ${base}/stats unreachable"
|
||||
failed=1
|
||||
fi
|
||||
if curl -fsS --max-time 10 "${base}/v1/models" -o "$models_tmp"; then
|
||||
{ echo "# Verbatim /v1/models reply from the radiance engine, recorded by"
|
||||
echo "# ./scripts/capture-fixtures.sh — one JSON document. The model id and"
|
||||
echo "# max_model_len are stable; created is an epoch timestamp that moves."
|
||||
echo "# The contract asserts the object/data shape — never the numbers."
|
||||
cat "$models_tmp"; } > "$FIX/.models_radiance.json.tmp2" &&
|
||||
mv "$FIX/.models_radiance.json.tmp2" "$FIX/models_radiance.json"
|
||||
rm -f "$models_tmp"
|
||||
echo " ✓ models_radiance.json ($(wc -c <"$FIX/models_radiance.json") bytes)"
|
||||
else
|
||||
rm -f "$models_tmp" # never overwrite a good fixture with a dead engine's empty file
|
||||
echo " ✗ models_radiance.json — ${base}/v1/models unreachable"
|
||||
failed=1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo " (skipped engine /metrics captures — pass --with-llm when the engines are reachable)"
|
||||
|
||||
Reference in New Issue
Block a user