phase: 108_history_wire_check
Final verification pass complete — everything independently re-verified green. Report: **Phase 108 — History wire check (final verification pass; verdict: NO BUG, tests-only per D13)** - Verified all 3 tasks complete in `.agents/phases/complete/108_history_wire_check/`; re-ran every pin and gate myself - Layer 1 (unit): 2 owner-shape pins in `tests/unit/test_history.py` (2-turn keep-whole + A4 thinking mapping) — pass - Layer 2 (integration): `test_endpoint_two_turn_history_reaches_the_llm` — captured LLM request = `[system, user Q1, assistant R1, user Q2]`, `history_msgs=2` — pass - Layer 3 (E2E): `tests/e2e/test_history_wire_check.py` — owner's verbatim 3-message scenario, echo byte-exact (T2: `2 prior messages`+R1 tail; T3: `4 prior messages`+R2 tail) — pass - `VERDICT.md` present in phase dir: layer outcomes + "no bug — model behavior/user error" + pin names **Gates (exact results)** - `uv run pytest --cov=app --cov-report=term-missing` → 2305 passed, TOTAL **99%** (>90%) - `uv run pytest tests/e2e/test_history_wire_check.py -v --no-cov` → 2 passed (isolation, DB up) - `uv run pytest tests/e2e/test_llm_history.py -v --no-cov` (phase-74 regression) → 3 passed - `uv run ruff check .` → clean; `uv run pyright` → 0 errors **Completion criteria:** all six met — no defects found; diff scoped to `tests/**`, `VERDICT.md`, `.agents/phases/**`, `.agents/reports/**` (no `app/`/`frontend/` changes, consistent with no-bug verdict). Per executor rules, no git commit made — left for the harness. **Next pending phase:** `109_turn_progress_loader`
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
# Phase 108 Verdict — History wire check (TODO.md L4)
|
||||
|
||||
**Verdict: NO BUG.** The follow-up history wire is proven complete at all
|
||||
three layers, for the owner's exact scenario, byte-exact via the phase-74
|
||||
`echo my history` oracle (D14 — no new marker). The reported instance
|
||||
("What did I just ask you?" → "This is the first question you've asked")
|
||||
was **model behavior / user error**, exactly the owner's own hypothesis
|
||||
("Just check if there's a bug, there may not be"). No `app/` or
|
||||
`frontend/` changes were needed — the phase ships tests-only (D13).
|
||||
|
||||
## Layer outcomes
|
||||
|
||||
| Layer | What it isolates | Outcome | Pins (evidence) |
|
||||
|-------|------------------|---------|-----------------|
|
||||
| 1 — unit: the trimmer | `history_to_messages` (`app/rag/prompts.py`) | **PASS** — the owner's 2-turn history (Q1 "What is my name?" / R1 "Your name is Reese.") under the DEFAULT budgets (40 turns / 24 000 chars, no overrides) survives whole: chronological, roles mapped, thinking → `reasoning_content` only when non-empty (A4) | `tests/unit/test_history.py::test_short_two_turn_history_kept_whole_and_chronological`, `::test_two_turn_history_thinking_mapping` |
|
||||
| 2 — integration: the endpoint | `POST /api/chat` splice, both branches (`app/api/chat.py`) | **PASS** — the SAME 2-turn history through the real endpoint (follow-up "What did I just ask you?") makes the LLM request exactly `[system, user Q1, assistant R1, user Q2]`; SSE `done`; log line carries `history_msgs=2`. Branch-agnostic by design (phase 74 pinned the splice on both branches) | `tests/integration/test_chat_api.py::test_endpoint_two_turn_history_reaches_the_llm` |
|
||||
| 3 — E2E: the full browser wire | client record → `conversation.slice(0, -1)` → body → LLM (`frontend/assets/app.js`) | **PASS** — the owner's verbatim 3-message scenario in a real browser: T2's echo = `history: 2 prior messages` + R1's exact 24-char tail; T3's echo = `history: 4 prior messages` + R2's exact tail (tails read from the persisted `bor.chat.v1` record — what the client saved is what it sends). The missing-first-turn bug would render T2 as `0 prior messages` / `last answer tail: none`; it does not | `tests/e2e/test_history_wire_check.py::test_cold_start_echo_shows_no_phantom_history`, `::test_owner_scenario_three_turns_carry_the_full_prior_history` |
|
||||
|
||||
## Why "no bug" is sound
|
||||
|
||||
- Layers 1 and 2 prove the server drops nothing: a short history is
|
||||
orders of magnitude under both budgets, and the captured LLM request on
|
||||
the real endpoint carries the complete prior exchange, chronological.
|
||||
- Layer 3 proves the client sends what it saved: the echo's `N prior
|
||||
messages` count and `last answer tail` are computed by the mock from
|
||||
the request itself, and both matches hold byte-exactly for turns 2 and 3.
|
||||
- The deterministic mock leaves no room for "the model forgot" at the
|
||||
wire level — the model (turbo) simply receives the full history and
|
||||
occasionally mis-reports it. The owner's third message working correctly
|
||||
is consistent with model non-determinism over a complete wire.
|
||||
|
||||
## The pins stay as the permanent guard
|
||||
|
||||
A future regression that drops the first turn — in the trimmer, the
|
||||
endpoint splice, or the client's record→history mapping (including the
|
||||
phase-49 retry / phase-53 stale-regen pop paths) — fails layer 1, 2, or 3
|
||||
immediately.
|
||||
Reference in New Issue
Block a user