feat(rag): pass chat history with prior thinking to the LLM
Build and Push Containers / build-and-push-app (push) Successful in 1m39s
Build and Push Containers / build-and-push-db (push) Successful in 11s

Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH
the conversation so far — every prior user/brain turn and the prior
thinking blocks on brain turns (preserve-thinking) — while
POST /api/chat stays stateless (A10): the client provides the history
in the request body and the server stores nothing new.

Server (task 01):
- ChatRequest.history: optional list[HistoryTurn] (who: user|brain,
  text, optional thinking) — absent/empty keeps the request
  byte-identical to pre-phase-74 (the two-message [system, user]
  request; the kill-switch semantics are pinned in the integration
  suite).
- app.rag.prompts.history_to_messages: pure mapper — walks the turns
  newest-first against the settings budgets (history_max_turns=40 /
  history_max_chars=24000, BOR_HISTORY_MAX_TURNS /
  BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut
  mid-answer); the kept window is returned oldest-first; brain turns
  carry their thinking as reasoning_content (A4) only when
  non-empty.
- Both branches feed it: the deflected path splices it between the
  system prompt and the current user message (the phase-71 recovery
  still rebuilds from messages[1:]), the grounded agent receives
  run_agent(..., history=hist); llm.py's message params widen to
  list[dict[str, Any]] (string-only messages stay byte-identical on
  the wire — the SDK passes message dicts through verbatim).
- The per-turn log line (PLAN §9) gains history_msgs=N after
  kb_chars=N.
- Pins: tests/unit/test_history.py (mapper: mapping, reasoning
  gating, both budgets, drop-whole, ordering, empty default),
  tests/unit/test_config.py (the two settings + env overrides),
  tests/unit/test_agent.py (the history splice + the default),
  tests/integration/test_chat_api.py (deflected AND grounded forward
  the history incl. reasoning_content, no-history byte-identity, 422
  pins, the log field).

Client (task 02):
- runTurn — the single funnel for fresh send / phase-49 retry /
  phase-53 stale-regen — sends history = the conversation record
  minus the current question, with thinking only on brain records
  that streamed one (undefined drops the key from the JSON, the
  record's convention); the question is never duplicated into the
  history.

Wire proof (task 03):
- The mock's echo my history marker (HISTORY_TRIGGER) answers with
  the deterministic history echo — history: N prior messages; last
  answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE
  the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both
  turn branches whatever the gate says; the module docstring records
  the user/assistant-only history invariant that keeps every
  existing (tool-result-classified) marker flow unaffected.
- tests/e2e/test_llm_history.py (isolated): a grounded follow-up and
  a deflected follow-up both receive history: 2 prior messages +
  thinking: yes + the byte-exact tail of turn 1's answer (derived
  from the persisted bor.chat.v1 record — the same array the client
  maps into the body); a cold start receives history: 0 prior
  messages / last answer tail: none / thinking: no.
- Regressions green in isolation: chat_rag, chat_history (phase 50),
  agent_document_tools, harness_aligned_tools, stop_generation,
  retry_answer, response_to_docs.
This commit is contained in:
2026-09-05 16:04:40 -04:00
parent a16130c71d
commit 055c0b5d85
29 changed files with 1418 additions and 18 deletions
+170 -2
View File
@@ -89,7 +89,10 @@ class FakeRagLLM:
#: recovered answer endpoint for the pre-first-piece retry rule.
self.stream_fail_count = stream_fail_count
self.question_embeds: list[str] = []
self.seen_messages: list[list[dict[str, str]]] = []
#: Phase 74: assistant history messages may carry
#: ``reasoning_content`` — the dict values stay strings, but the
#: key set is wider than the pre-phase ``{role, content}`` shape.
self.seen_messages: list[list[dict[str, Any]]] = []
#: Every request's ``tools`` value (phase 37) — ``None`` is the
#: pre-phase request shape (the key is absent from the payload).
self.seen_tools: list[list[dict[str, Any]] | None] = []
@@ -138,7 +141,7 @@ class FakeRagLLM:
async def chat_stream(
self,
messages: list[dict[str, str]],
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
scaffolding: ScaffoldingFilter | None = None,
):
@@ -1226,3 +1229,168 @@ def test_deflected_mixed_scaffolding_and_content_needs_no_recovery(
assert len(flaky.seen_messages) == 1 # the clean content stands — no recovery
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and f"scaffold_stripped={len(span)}" in lines[-1]
# ---------- phase 74: client-provided history (with prior thinking) ----------
#: The client's prior turns (oldest first — the ``bor.chat.v1`` record
#: minus the current question): two user turns, two brain turns, the
#: FIRST brain turn carrying a prior thinking block (A4) and the second
#: not (the ``reasoning_content`` gate has both shapes on one request).
HISTORY: list[dict[str, Any]] = [
{"who": "user", "text": "What port does Tailscale run on?"},
{
"who": "brain",
"text": "Tailscale runs on 41641/udp.",
"thinking": "The Tailscale wire protocol uses 41641/udp.",
},
{"who": "user", "text": "And the subnet router?"},
{"who": "brain", "text": "The subnet router shares the same port."},
]
#: What :func:`app.rag.prompts.history_to_messages` must produce for
#: :data:`HISTORY` — chronological, ``reasoning_content`` ONLY on the
#: turn that had thinking.
HISTORY_MESSAGES: list[dict[str, Any]] = [
{"role": "user", "content": "What port does Tailscale run on?"},
{
"role": "assistant",
"content": "Tailscale runs on 41641/udp.",
"reasoning_content": "The Tailscale wire protocol uses 41641/udp.",
},
{"role": "user", "content": "And the subnet router?"},
{"role": "assistant", "content": "The subnet router shares the same port."},
]
def _stream_chat_with_history(
client: TestClient, message: str, history: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Phase 74 variant of :func:`_stream_chat`: sends ``history`` (the
client's prior turns, oldest first) in the request body."""
with client.stream(
"POST", "/api/chat", json={"message": message, "history": history}
) as r:
assert r.status_code == 200
assert r.headers["content-type"].startswith("text/event-stream")
buf = ""
frames: list[dict[str, Any]] = []
for part in r.iter_text():
buf += part
while "\n\n" in buf:
frame, buf = buf.split("\n\n", 1)
frame = frame.strip()
if frame.startswith("data:"):
frames.append(json.loads(frame.removeprefix("data:").strip()))
assert buf.strip() == "", "stream must end on a frame boundary"
return frames
def test_deflected_turn_forwards_history_with_prior_thinking(
client,
db,
seeded_kb: FakeRagLLM,
caplog: pytest.LogCaptureFixture,
) -> None:
"""A DEFLECTED turn sends the prior turns — chronological, with the
prior brain turn's thinking as ``reasoning_content`` — between the
LOW system prompt and the current question (A2/A3/A4); the per-turn
log line carries ``history_msgs=4``."""
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
caplog.set_level(logging.INFO, logger="app.chat")
frames = _stream_chat_with_history(client, OFF_TOPIC, HISTORY)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["type"] == "done"
assert frames[-1]["deflected"] is True
assert len(seeded_kb.seen_messages) == 1
(messages,) = seeded_kb.seen_messages
assert messages[0]["role"] == "system"
assert "DEFLECT_MODE" in messages[0]["content"] # the LOW prompt
assert messages[1:-1] == HISTORY_MESSAGES # the prior turns, chronological
assert messages[-1] == {"role": "user", "content": OFF_TOPIC}
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and "history_msgs=4" in lines[-1]
def test_grounded_turn_forwards_history_through_the_agent(
client,
db,
seeded_kb: FakeRagLLM,
caplog: pytest.LogCaptureFixture,
) -> None:
"""The GROUNDED agent branch receives the same block: its first
request is ``[HIGH system, *history, current question]`` (the tool
rounds then append to that same list); ``history_msgs=4``."""
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
caplog.set_level(logging.INFO, logger="app.chat")
frames = _stream_chat_with_history(client, QUESTION, HISTORY)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["type"] == "done"
assert frames[-1]["deflected"] is False
assert len(seeded_kb.seen_messages) == 1 # the canned answer ends the loop
(messages,) = seeded_kb.seen_messages
assert messages[0]["role"] == "system"
assert "<tools>" in messages[0]["content"] # the HIGH prompt
assert messages[1:-1] == HISTORY_MESSAGES
assert messages[-1] == {"role": "user", "content": QUESTION}
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and "history_msgs=4" in lines[-1]
def test_request_without_history_sends_exactly_system_and_user(
client,
db,
seeded_kb: FakeRagLLM,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Byte-identical pin (A2): a request WITHOUT ``history`` sends
exactly the two-message ``[system, user]`` request on BOTH branches
(deflected + grounded), and the per-turn log line carries
``history_msgs=0``."""
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
caplog.set_level(logging.INFO, logger="app.chat")
_stream_chat(client, OFF_TOPIC) # deflected branch
_stream_chat(client, QUESTION) # grounded branch
finally:
fastapi_app.dependency_overrides.clear()
assert len(seeded_kb.seen_messages) == 2
for messages in seeded_kb.seen_messages:
assert [m["role"] for m in messages] == ["system", "user"]
assert seeded_kb.seen_messages[0][1] == {"role": "user", "content": OFF_TOPIC}
assert seeded_kb.seen_messages[1][1] == {"role": "user", "content": QUESTION}
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert len(lines) == 2
assert all("history_msgs=0" in line for line in lines)
def test_history_rejects_unknown_who(client, db) -> None:
"""Schema pin: ``who`` is a ``Literal["user", "brain"]`` — anything
else is a 422 at the boundary (the same trust model as the saved-
chat ``ChatMessage``)."""
r = client.post(
"/api/chat",
json={"message": "hi", "history": [{"who": "alien", "text": "x"}]},
)
assert r.status_code == 422
def test_history_rejects_more_than_100_entries(client, db) -> None:
"""Schema pin: the DoS sanity ceiling is 100 turns — 101 is a 422
(the config budgets do the real trimming; this only keeps a
pathological body from wasting the mapper's work)."""
r = client.post(
"/api/chat",
json={
"message": "hi",
"history": [{"who": "user", "text": f"q{i}"} for i in range(101)],
},
)
assert r.status_code == 422