dashboards/.agents/phases/todo/23_llm_rate_responsiveness/00_phase.md

Phase 23 — LLM rate responsiveness

Source: TODO.md L8 — "The LLM monitor has a strange delay and a 'ramp up/ramp down' effect that makes it look innacurate. Data reporting should be accurate to as small a delta as possible." Story: n/a Context: src/dashboards/app.py line 122 (build_hub builds one RateEstimator(span=settings.rate_window) and hands it to both host_jobs (line 124) and llm_jobs (line 130)), src/dashboards/config.py lines 197–199 (llm_interval 2 s, llm_timeout 3 s, rate_window 10 s), src/dashboards/collectors.py line 477 (llm_jobs(settings, client, rates) — the estimator is a parameter, so a second one is a wiring change, not a signature redesign), src/dashboards/rates.py (line 55 MIN_WINDOW_SECONDS = 0.5, line 61 MAX_WINDOW_RATIO = 2.0, the phase-13 jump gate — all untouched by this phase), src/dashboards/web/static/page-llm.js (line 39 pollMs: 2000, the Δ<window> labels printed from the payload's window), src/dashboards/sources/llm.py (the rate cells carry computed:Δ<series>/Δt + the honest Δ<window>; the window the estimator used travels with the value — rule 2), the mechanism the TODO describes (gap 9 / phase 13 in AGENTS.md: a 10 s window over a 2 s poll ramped the hero up over ~5 polls when a slot started and decayed over ~5 polls when it ended — the "ramp up/ramp down" the operator sees; the sub-8× skew family and the stall are already closed by phases 13/14 and stay closed: this phase changes the window size, nothing else)

Objective

The LLM throughput rates are measured over a window as small as the app's own cadence allows — the LLM source gets its own RateEstimator whose span is the LLM poll interval (2 s), so a slot start reads steady within two polls instead of ramping over ten, while the host source keeps the 10 s RATE_WINDOW_SECONDS window its 10 s cadence needs (a 2 s window over a 10 s poll would refuse every window — the estimator's floor and ceiling would answer n/a forever). The phase-13 protections (the 0.5 s floor, the 2 × span ceiling, the 8× jump gate with its independent-anchor confirmation) apply to the smaller window unchanged; the Δ<window> label prints the smaller window it actually used, so the number always says what it is.

Dependencies

• — (none) — the phase touches app.py/collectors.py wiring and the LLM page; it is scheduled after 22_disks_hide_unconfigured only to keep the fleet-changing phase 24 last.

Tasks

1. 01_llm_window_per_poll.md — the second estimator: build_hub hands llm_jobs a RateEstimator(span=settings.llm_interval), the host keeps the 10 s one
2. 02_window_pins.md — the pins: LLM rate cells label the 2 s window, host cells keep 10 s, the phase-13 gates still hold over the smaller window
3. 03_live_verification.md — the live proof: a slot start reads steady within two polls (no 10 s ramp), the Δ2.0s label travels with the value, screenshots

Testing & Quality

• Unit/integration: tests/test_llm.py (or wherever the parse_gufo_metrics/parse_vllm_metrics tests live — grep rate_window/RateEstimator in tests/ first) — the window-value pins for both sources, the phase-13 tests (floor refusal, ceiling, jump gate, the stale-anchor tests) run unchanged and green; tests/test_api.py's /api/llm/snapshot shape pins stay green (the shape is unchanged — only the window values move)
• Coverage: >90% on new/modified code
• The phase is "Complete" only when the entire project test suite passes 100% (every test in the project, not just this phase's)

Documentation & Consistency

• The final pass reviews README.md and AGENTS.md in full against the project as it stands after this phase — fixing every inconsistency found, including pre-existing ones not caused by this phase. AGENTS.md's app.py/poller.py bullets (the "one RateEstimator" sentence) and rates.py's docstring context gain the two-estimator fact; gap 9's "the estimator no longer turns a slow fetch into 0.2 s arithmetic" history stays — this phase is the window-size decision on top of it, and the RATE_WINDOW_SECONDS=10 line in .env.example stays (it is the host's knob now, and gains a comment saying so if it has none).

Completion Criteria

• /api/llm/snapshot rate cells carry a window ≈ 2 s (the label prints Δ2.0s on the page) and /api/system/snapshot rate cells keep ≈ 10 s (pinned both ways)
• the phase-13 tests (floor, ceiling, jump gate, stale anchor) pass unchanged
• live: a slot start reads steady within two polls and a slot end decays within two — the 10 s ramp is gone (screenshots + the stored 1 m rollup in data/logs/)
• uv run pytest green, coverage not down, ruff + pyright clean
• bash .agents/validate.sh green
• no behavior change in completed work: the host rates, the estimator's gates, the rollups and the Δ<window> labelling are all unchanged in kind