phase: 23_llm_rate_responsiveness

Phase 23 final verification complete — all green.

```
VERIFIED (criteria)
1  /api/llm/snapshot windows ≈2 s live (1.98–2.00), /api/system window_seconds ≈10 s (9.996–10.005);
   /api/config llm.rate_window_seconds=2=LLM_INTERVAL_SECONDS; browser re-check ALL CHECKS PASSED
   (data/logs/23-window-{llm,system}.png — /llm prints Δ2.0s, /system footnote Δ10.0s)
2  Phase-13 pins unchanged: floor, 2×span ceiling, 8× jump gate + independent anchor — all pass
   (only import/span-line diffs in tests/; git diff confirms no assertion rewritten)
3  Live proof re-derived from artifacts: 780 rows, 0 unlabelled, windows 0.61–3.38 s, 5 starts +
   5 ends settled by poll 2 (0 unsettled), 1 m rollups max 42.3 vs raw 201.6, 0 over 300 tok/s;
   counterfactual 10 s window: median settle poll 3, |ln| 0.924 vs 0.037
4  uv run pytest: 1494 passed / 15 skipped · coverage 98.94% (floor 94) · ruff clean · pyright 0
   errors · node --check clean · bash .agents/validate.sh exit 0 · live 11 passed · store_perf 4 passed
   Dev server up, all nine sources ok. No git add/commit run; everything left in the working tree.

DOCS PASS — 4 inconsistencies fixed in AGENTS.md (last two runs above include them)
- UI conventions hero example still said the `.sub` footnote reads `Δ10.0s`; the only hero that
  prints a Δ is /llm's → now `Δ2.0s` on /llm, "at the current load" on /ups
- gap 9: "found 10 busy-edges" contradicted its own artifact (17 edges, 10 judged, 7 reported-not-
  judged) → 17; and the poll-2 ratios were a 3-of-5 cherry-pick → full five values both windows
- gap 12: the responsive-table fix is now scheduled — names 27_mobile_tables_and_audit
README, config table, .env.example, rates/prom docstrings, app/api bullets and the testing-policy
pins all re-checked against code and found consistent (store pragmas, budgets, throttle, fleet).

NOTES
- Advisory only, not phase 23: fastapi 0.153.0 → 0.156.8 available; dependencies untouched here,
  so rule 8's "update on purpose" is left to the operator.
- NOTES.md is empty in the working tree (its items became phases 24–27); left as-is, untracked-file
  rules forbid me removing it.
- Next phase: 24_dev_mtls_rollout (25–27 follow).
```
This commit is contained in:
2026-10-21 08:04:21 -04:00
parent 895f309f8e
commit 5f3ed9bcd4
37 changed files with 1694 additions and 31 deletions
+60
View File
@@ -12,6 +12,7 @@ import math
import pytest
from dashboards.rates import RateEstimator
from rate_window_contract import HOST_WINDOW, LLM_WINDOW
def test_first_sample_has_no_rate() -> None:
@@ -210,6 +211,65 @@ def test_a_held_jump_is_not_confirmed_by_a_window_on_the_same_anchor() -> None:
assert published[-1] == pytest.approx(45.0), "the stall ends in the truth, not a tail"
# --- phase 23: the gates are the estimator's, not the old 10 s span's --------- #
# `app.build_hub` now spans the `llm` jobs by `LLM_INTERVAL_SECONDS` (2 s), a fifth of the host's
# `RATE_WINDOW_SECONDS`. Everything phase 13 built has to survive the shrink or the smaller window
# is a licence for the same 3284.59 tok/s row, so each gate runs here at both spans. The numbers
# scale with the span: what these three pin is the *relation*, which is what "unchanged" means.
# The fourth protection — a held jump needing a **different anchor** — is deliberately not
# parametrized: it compares two sample timestamps and no span arithmetic, and a 2 s window holds
# one sample per poll interval anyway, so a stale anchor ages out one poll later instead of five.
# Its span-10 regression pair (`test_a_held_jump_is_not_confirmed_by_a_window_on_the_same_anchor`
# above, `test_gufo_a_stale_scrape_body_never_becomes_a_rate_above_the_engine` in
# `tests/test_llm_source.py`) runs unchanged, and the floor row of this section is the one that
# could actually have been weakened by the smaller window: `MIN_WINDOW_SECONDS` is absolute.
@pytest.mark.parametrize("span", [LLM_WINDOW, HOST_WINDOW])
def test_the_floor_refuses_the_catch_up_gap_at_either_window(span: float) -> None:
"""The poll loop's ~0.2 s catch-up is a gap whatever the panel's span is.
0.22 s is a tenth of a 10 s window and an eighth of a 2 s one, and neither is a measurement:
the floor is absolute on purpose (phase 13 pinned that), so the LLM's narrower claim is not a
narrower guard.
"""
r = RateEstimator(span=span)
r.rate("x", 0.0, now=0.0)
r.rate("x", 50.0 * span, now=span) # the honest cadence: 50 tok/s
rate, window = r.rate("x", 50.0 * span + 11.0, now=span + 0.22)
assert rate is None, f"a 0.22 s catch-up became a rate at span={span}"
assert window == pytest.approx(0.22), "the n/a still names the window it refused"
@pytest.mark.parametrize("span", [LLM_WINDOW, HOST_WINDOW])
def test_the_ceiling_is_two_spans_at_either_window(span: float) -> None:
"""`2 x span` is relative, so a stall stays a stall at either window size."""
r = RateEstimator(span=span)
r.rate("x", 0.0, now=0.0)
assert r.rate("x", 100.0, now=2.5 * span) == (None, None), "past the ceiling: no rate"
rate, window = r.rate("x", 200.0, now=3.5 * span)
assert rate == pytest.approx(100.0 / span)
assert window == pytest.approx(span), "the next poll re-bases on the fresh sample"
@pytest.mark.parametrize("span", [LLM_WINDOW, HOST_WINDOW])
def test_a_jump_is_held_and_confirmed_at_either_window(span: float) -> None:
"""An 8x step holds one poll at a 2 s window exactly as at a 10 s one.
The ratio is dimensionless, so narrowing the window does not loosen the gate: a real burst
outlives one poll interval of either size and prints one poll late, while the held value keeps
the `Δ<window>` label it was published with.
"""
r = RateEstimator(span=span)
r.rate("x", 0.0, now=0.0)
r.rate("x", 10.0 * span, now=span) # 10 tok/s published
rate, window = r.rate("x", 90.0 * span, now=2.0 * span) # 80 tok/s = 8x: held
assert rate == pytest.approx(10.0)
assert window == pytest.approx(span), "the held value keeps the Δ label it was published with"
rate, window = r.rate("x", 172.0 * span, now=3.0 * span) # 82: a new anchor confirms it
assert rate == pytest.approx(82.0)
assert window == pytest.approx(span)
def test_a_rise_from_zero_is_never_held() -> None:
"""The first work an idle engine does must print on its own poll, not one late."""
r = RateEstimator(span=10)