feat(rag): pass chat history with prior thinking to the LLM
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH the conversation so far — every prior user/brain turn and the prior thinking blocks on brain turns (preserve-thinking) — while POST /api/chat stays stateless (A10): the client provides the history in the request body and the server stores nothing new. Server (task 01): - ChatRequest.history: optional list[HistoryTurn] (who: user|brain, text, optional thinking) — absent/empty keeps the request byte-identical to pre-phase-74 (the two-message [system, user] request; the kill-switch semantics are pinned in the integration suite). - app.rag.prompts.history_to_messages: pure mapper — walks the turns newest-first against the settings budgets (history_max_turns=40 / history_max_chars=24000, BOR_HISTORY_MAX_TURNS / BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut mid-answer); the kept window is returned oldest-first; brain turns carry their thinking as reasoning_content (A4) only when non-empty. - Both branches feed it: the deflected path splices it between the system prompt and the current user message (the phase-71 recovery still rebuilds from messages[1:]), the grounded agent receives run_agent(..., history=hist); llm.py's message params widen to list[dict[str, Any]] (string-only messages stay byte-identical on the wire — the SDK passes message dicts through verbatim). - The per-turn log line (PLAN §9) gains history_msgs=N after kb_chars=N. - Pins: tests/unit/test_history.py (mapper: mapping, reasoning gating, both budgets, drop-whole, ordering, empty default), tests/unit/test_config.py (the two settings + env overrides), tests/unit/test_agent.py (the history splice + the default), tests/integration/test_chat_api.py (deflected AND grounded forward the history incl. reasoning_content, no-history byte-identity, 422 pins, the log field). Client (task 02): - runTurn — the single funnel for fresh send / phase-49 retry / phase-53 stale-regen — sends history = the conversation record minus the current question, with thinking only on brain records that streamed one (undefined drops the key from the JSON, the record's convention); the question is never duplicated into the history. Wire proof (task 03): - The mock's echo my history marker (HISTORY_TRIGGER) answers with the deterministic history echo — history: N prior messages; last answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both turn branches whatever the gate says; the module docstring records the user/assistant-only history invariant that keeps every existing (tool-result-classified) marker flow unaffected. - tests/e2e/test_llm_history.py (isolated): a grounded follow-up and a deflected follow-up both receive history: 2 prior messages + thinking: yes + the byte-exact tail of turn 1's answer (derived from the persisted bor.chat.v1 record — the same array the client maps into the body); a cold start receives history: 0 prior messages / last answer tail: none / thinking: no. - Regressions green in isolation: chat_rag, chat_history (phase 50), agent_document_tools, harness_aligned_tools, stop_generation, retry_answer, response_to_docs.
This commit is contained in:
+32
-4
@@ -115,6 +115,23 @@ recovery, via the holder; deflected: this turn's filters), 0 on clean
|
||||
turns (the field is uniform, the phase-67 ``retries=N`` pattern); the
|
||||
recovery does not bump ``retries=N`` (it is not a phase-67
|
||||
endpoint-retry).
|
||||
|
||||
Chat history (phase 74, TODO L4, owner-locked A2/A3/A4 2026-09-08):
|
||||
``POST /api/chat`` accepts an optional ``history`` — the client's prior
|
||||
turns, oldest first (the ``bor.chat.v1`` record minus the current
|
||||
question; the endpoint stays stateless per A10 — nothing is stored). It
|
||||
is mapped ONCE per turn by :func:`app.rag.prompts.history_to_messages`
|
||||
— trimmed newest-first against the settings budgets
|
||||
(``history_max_turns`` / ``history_max_chars``; a capped-out turn is
|
||||
dropped whole, never truncated) — and fed to the model on BOTH turn
|
||||
branches: the deflected path splices it between the system prompt and
|
||||
the current user message (the phase-71 recovery still rebuilds from
|
||||
``messages[1:]`` — unchanged), and the grounded agent receives it as
|
||||
``run_agent(..., history=hist)``. Prior brain turns' thinking travels
|
||||
as ``reasoning_content`` on the assistant message (the preserve-
|
||||
thinking wire convention, A4). The per-turn log line records
|
||||
``history_msgs=N`` after ``kb_chars=N`` (0 when the request carries no
|
||||
history — the two-message request stays byte-identical).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -150,7 +167,7 @@ from app.rag.llm import (
|
||||
chat_stream_retried, # phase 67: the retry-before-first-piece primitive
|
||||
)
|
||||
from app.rag.overview import load_kb_overview
|
||||
from app.rag.prompts import build_deflect_prompt, build_high_prompt
|
||||
from app.rag.prompts import build_deflect_prompt, build_high_prompt, history_to_messages
|
||||
from app.rag.retriever import RetrievedChunk, retrieve, select_documents, weak_hit_titles
|
||||
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
|
||||
from app.rag.suggestions import derive_suggestions
|
||||
@@ -306,6 +323,14 @@ async def chat(
|
||||
retries_used = 0 # phase 67: LLM requests restarted this turn (log line)
|
||||
try:
|
||||
settings = get_settings()
|
||||
# Phase 74 (TODO L4): the client's prior turns, mapped ONCE
|
||||
# per turn — trimmed newest-first against the settings
|
||||
# budgets, assistant turns carrying their prior thinking as
|
||||
# ``reasoning_content`` (A4). BOTH branches below (deflected
|
||||
# + grounded agent) reuse the same block; an absent/empty
|
||||
# history yields ``[]`` (the byte-identical two-message
|
||||
# request, A2).
|
||||
hist = history_to_messages(request.history, settings)
|
||||
|
||||
# 1. Embed the question.
|
||||
# Phase 67: a dead embeddings endpoint is retried before any
|
||||
@@ -384,8 +409,9 @@ async def chat(
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
messages = [
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": plan.system_prompt},
|
||||
*hist, # phase 74: the trimmed prior turns (empty by default)
|
||||
{"role": "user", "content": request.message},
|
||||
]
|
||||
|
||||
@@ -434,6 +460,7 @@ async def chat(
|
||||
seed_docs=plan.docs,
|
||||
settings=settings,
|
||||
holder=holder,
|
||||
history=hist, # phase 74: the same trimmed prior turns
|
||||
)
|
||||
thinking_chars = 0
|
||||
content_chars = 0 # phase 71: the turn's visible (clean) content
|
||||
@@ -636,8 +663,8 @@ async def chat(
|
||||
|
||||
logger.info(
|
||||
"question=%r embed_ms=%d top_score=%.3f fts_hits=%d summary_hits=%d tuning=%d "
|
||||
"kb_chars=%d threshold=%.2f deflected=%s sources=%r thinking_chars=%d "
|
||||
"tool_calls=%d total_ms=%d retries=%d scaffold_stripped=%d",
|
||||
"kb_chars=%d history_msgs=%d threshold=%.2f deflected=%s sources=%r "
|
||||
"thinking_chars=%d tool_calls=%d total_ms=%d retries=%d scaffold_stripped=%d",
|
||||
request.message,
|
||||
embed_ms,
|
||||
plan.top_score,
|
||||
@@ -645,6 +672,7 @@ async def chat(
|
||||
plan.summary_hits,
|
||||
plan.tuning_count,
|
||||
plan.kb_chars,
|
||||
len(hist),
|
||||
settings.relevance_threshold,
|
||||
plan.deflected,
|
||||
source_paths,
|
||||
|
||||
Reference in New Issue
Block a user