3.3 KiB
3.3 KiB
Phase 108 Verdict — History wire check (TODO.md L4)
Verdict: NO BUG. The follow-up history wire is proven complete at all
three layers, for the owner's exact scenario, byte-exact via the phase-74
echo my history oracle (D14 — no new marker). The reported instance
("What did I just ask you?" → "This is the first question you've asked")
was model behavior / user error, exactly the owner's own hypothesis
("Just check if there's a bug, there may not be"). No app/ or
frontend/ changes were needed — the phase ships tests-only (D13).
Layer outcomes
| Layer | What it isolates | Outcome | Pins (evidence) |
|---|---|---|---|
| 1 — unit: the trimmer | history_to_messages (app/rag/prompts.py) |
PASS — the owner's 2-turn history (Q1 "What is my name?" / R1 "Your name is Reese.") under the DEFAULT budgets (40 turns / 24 000 chars, no overrides) survives whole: chronological, roles mapped, thinking → reasoning_content only when non-empty (A4) |
tests/unit/test_history.py::test_short_two_turn_history_kept_whole_and_chronological, ::test_two_turn_history_thinking_mapping |
| 2 — integration: the endpoint | POST /api/chat splice, both branches (app/api/chat.py) |
PASS — the SAME 2-turn history through the real endpoint (follow-up "What did I just ask you?") makes the LLM request exactly [system, user Q1, assistant R1, user Q2]; SSE done; log line carries history_msgs=2. Branch-agnostic by design (phase 74 pinned the splice on both branches) |
tests/integration/test_chat_api.py::test_endpoint_two_turn_history_reaches_the_llm |
| 3 — E2E: the full browser wire | client record → conversation.slice(0, -1) → body → LLM (frontend/assets/app.js) |
PASS — the owner's verbatim 3-message scenario in a real browser: T2's echo = history: 2 prior messages + R1's exact 24-char tail; T3's echo = history: 4 prior messages + R2's exact tail (tails read from the persisted bor.chat.v1 record — what the client saved is what it sends). The missing-first-turn bug would render T2 as 0 prior messages / last answer tail: none; it does not |
tests/e2e/test_history_wire_check.py::test_cold_start_echo_shows_no_phantom_history, ::test_owner_scenario_three_turns_carry_the_full_prior_history |
Why "no bug" is sound
- Layers 1 and 2 prove the server drops nothing: a short history is orders of magnitude under both budgets, and the captured LLM request on the real endpoint carries the complete prior exchange, chronological.
- Layer 3 proves the client sends what it saved: the echo's
N prior messagescount andlast answer tailare computed by the mock from the request itself, and both matches hold byte-exactly for turns 2 and 3. - The deterministic mock leaves no room for "the model forgot" at the wire level — the model (turbo) simply receives the full history and occasionally mis-reports it. The owner's third message working correctly is consistent with model non-determinism over a complete wire.
The pins stay as the permanent guard
A future regression that drops the first turn — in the trimmer, the endpoint splice, or the client's record→history mapping (including the phase-49 retry / phase-53 stale-regen pop paths) — fails layer 1, 2, or 3 immediately.