feat(rag): pass chat history with prior thinking to the LLM
Build and Push Containers / build-and-push-app (push) Successful in 1m39s
Build and Push Containers / build-and-push-db (push) Successful in 11s

Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH
the conversation so far — every prior user/brain turn and the prior
thinking blocks on brain turns (preserve-thinking) — while
POST /api/chat stays stateless (A10): the client provides the history
in the request body and the server stores nothing new.

Server (task 01):
- ChatRequest.history: optional list[HistoryTurn] (who: user|brain,
  text, optional thinking) — absent/empty keeps the request
  byte-identical to pre-phase-74 (the two-message [system, user]
  request; the kill-switch semantics are pinned in the integration
  suite).
- app.rag.prompts.history_to_messages: pure mapper — walks the turns
  newest-first against the settings budgets (history_max_turns=40 /
  history_max_chars=24000, BOR_HISTORY_MAX_TURNS /
  BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut
  mid-answer); the kept window is returned oldest-first; brain turns
  carry their thinking as reasoning_content (A4) only when
  non-empty.
- Both branches feed it: the deflected path splices it between the
  system prompt and the current user message (the phase-71 recovery
  still rebuilds from messages[1:]), the grounded agent receives
  run_agent(..., history=hist); llm.py's message params widen to
  list[dict[str, Any]] (string-only messages stay byte-identical on
  the wire — the SDK passes message dicts through verbatim).
- The per-turn log line (PLAN §9) gains history_msgs=N after
  kb_chars=N.
- Pins: tests/unit/test_history.py (mapper: mapping, reasoning
  gating, both budgets, drop-whole, ordering, empty default),
  tests/unit/test_config.py (the two settings + env overrides),
  tests/unit/test_agent.py (the history splice + the default),
  tests/integration/test_chat_api.py (deflected AND grounded forward
  the history incl. reasoning_content, no-history byte-identity, 422
  pins, the log field).

Client (task 02):
- runTurn — the single funnel for fresh send / phase-49 retry /
  phase-53 stale-regen — sends history = the conversation record
  minus the current question, with thinking only on brain records
  that streamed one (undefined drops the key from the JSON, the
  record's convention); the question is never duplicated into the
  history.

Wire proof (task 03):
- The mock's echo my history marker (HISTORY_TRIGGER) answers with
  the deterministic history echo — history: N prior messages; last
  answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE
  the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both
  turn branches whatever the gate says; the module docstring records
  the user/assistant-only history invariant that keeps every
  existing (tool-result-classified) marker flow unaffected.
- tests/e2e/test_llm_history.py (isolated): a grounded follow-up and
  a deflected follow-up both receive history: 2 prior messages +
  thinking: yes + the byte-exact tail of turn 1's answer (derived
  from the persisted bor.chat.v1 record — the same array the client
  maps into the body); a cold start receives history: 0 prior
  messages / last answer tail: none / thinking: no.
- Regressions green in isolation: chat_rag, chat_history (phase 50),
  agent_document_tools, harness_aligned_tools, stop_generation,
  retry_answer, response_to_docs.
This commit is contained in:
2026-09-05 16:04:40 -04:00
parent a16130c71d
commit 055c0b5d85
29 changed files with 1418 additions and 18 deletions
+18 -4
View File
@@ -176,7 +176,7 @@ import logging
import re
from collections.abc import AsyncIterator, Sequence
from dataclasses import dataclass, field
from typing import Any, cast
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
@@ -821,6 +821,7 @@ async def run_agent(
seed_docs: Sequence[Document],
settings: Settings,
holder: AgentHolder,
history: Sequence[dict[str, Any]] = (),
) -> AsyncIterator[StreamPiece | ToolCallPiece | RetryPiece]:
"""Run the grounded-turn tool loop, yielding every stream piece.
@@ -830,6 +831,18 @@ async def run_agent(
SSE ``retry`` events. After the loop finishes, *holder* carries the
read documents and the executed tool-call count (re-lists included).
History (phase 74, TODO L4): *history* is the client's prior turns
already mapped to model messages by
:func:`app.rag.prompts.history_to_messages` (trimmed newest-first
against the settings budgets; assistant turns carry their prior
thinking as ``reasoning_content``). It is spliced between the system
prompt and the current user message —
``[system, *history, user]`` — and everything downstream (the tool
rounds, the phase-71 recovery rebuilding from ``messages[1:]``, the
retry restarts) already operates on that one ``messages`` list,
unchanged. ``()`` (the default) keeps the pre-phase-74 two-message
request byte-identical.
Retries (phase 67, owner-locked A2): every model request goes through
:func:`chat_stream_retried` — a failed round is retried **before** its
first piece (same messages, ``settings.llm_retries`` restarts, a flat
@@ -862,6 +875,7 @@ async def run_agent(
"""
messages: list[dict[str, Any]] = [
{"role": "system", "content": system_prompt},
*history, # phase 74: the client's prior turns (empty by default)
{"role": "user", "content": user_message},
]
# Phase 45: no per-tool budgets — the tools stay offered for the
@@ -892,7 +906,7 @@ async def run_agent(
# a quiet no-op.
stream = chat_stream_retried(
llm,
cast("list[dict[str, str]]", messages),
messages,
tools=tools,
retries=settings.llm_retries,
delay=settings.llm_retry_delay,
@@ -948,7 +962,7 @@ async def run_agent(
recovery_filter = ScaffoldingFilter()
recovered = chat_stream_retried(
llm,
cast("list[dict[str, str]]", messages_recovered),
messages_recovered,
tools=None,
retries=settings.llm_retries,
delay=settings.llm_retry_delay,
@@ -1030,7 +1044,7 @@ async def run_agent(
final_filter = ScaffoldingFilter()
final = chat_stream_retried(
llm,
cast("list[dict[str, str]]", messages),
messages,
tools=None,
retries=settings.llm_retries,
delay=settings.llm_retry_delay,