feat(rag): pass chat history with prior thinking to the LLM
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH the conversation so far — every prior user/brain turn and the prior thinking blocks on brain turns (preserve-thinking) — while POST /api/chat stays stateless (A10): the client provides the history in the request body and the server stores nothing new. Server (task 01): - ChatRequest.history: optional list[HistoryTurn] (who: user|brain, text, optional thinking) — absent/empty keeps the request byte-identical to pre-phase-74 (the two-message [system, user] request; the kill-switch semantics are pinned in the integration suite). - app.rag.prompts.history_to_messages: pure mapper — walks the turns newest-first against the settings budgets (history_max_turns=40 / history_max_chars=24000, BOR_HISTORY_MAX_TURNS / BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut mid-answer); the kept window is returned oldest-first; brain turns carry their thinking as reasoning_content (A4) only when non-empty. - Both branches feed it: the deflected path splices it between the system prompt and the current user message (the phase-71 recovery still rebuilds from messages[1:]), the grounded agent receives run_agent(..., history=hist); llm.py's message params widen to list[dict[str, Any]] (string-only messages stay byte-identical on the wire — the SDK passes message dicts through verbatim). - The per-turn log line (PLAN §9) gains history_msgs=N after kb_chars=N. - Pins: tests/unit/test_history.py (mapper: mapping, reasoning gating, both budgets, drop-whole, ordering, empty default), tests/unit/test_config.py (the two settings + env overrides), tests/unit/test_agent.py (the history splice + the default), tests/integration/test_chat_api.py (deflected AND grounded forward the history incl. reasoning_content, no-history byte-identity, 422 pins, the log field). Client (task 02): - runTurn — the single funnel for fresh send / phase-49 retry / phase-53 stale-regen — sends history = the conversation record minus the current question, with thinking only on brain records that streamed one (undefined drops the key from the JSON, the record's convention); the question is never duplicated into the history. Wire proof (task 03): - The mock's echo my history marker (HISTORY_TRIGGER) answers with the deterministic history echo — history: N prior messages; last answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both turn branches whatever the gate says; the module docstring records the user/assistant-only history invariant that keeps every existing (tool-result-classified) marker flow unaffected. - tests/e2e/test_llm_history.py (isolated): a grounded follow-up and a deflected follow-up both receive history: 2 prior messages + thinking: yes + the byte-exact tail of turn 1's answer (derived from the persisted bor.chat.v1 record — the same array the client maps into the body); a cold start receives history: 0 prior messages / last answer tail: none / thinking: no. - Regressions green in isolation: chat_rag, chat_history (phase 50), agent_document_tools, harness_aligned_tools, stop_generation, retry_answer, response_to_docs.
This commit is contained in:
+18
-4
@@ -176,7 +176,7 @@ import logging
|
||||
import re
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, cast
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
@@ -821,6 +821,7 @@ async def run_agent(
|
||||
seed_docs: Sequence[Document],
|
||||
settings: Settings,
|
||||
holder: AgentHolder,
|
||||
history: Sequence[dict[str, Any]] = (),
|
||||
) -> AsyncIterator[StreamPiece | ToolCallPiece | RetryPiece]:
|
||||
"""Run the grounded-turn tool loop, yielding every stream piece.
|
||||
|
||||
@@ -830,6 +831,18 @@ async def run_agent(
|
||||
SSE ``retry`` events. After the loop finishes, *holder* carries the
|
||||
read documents and the executed tool-call count (re-lists included).
|
||||
|
||||
History (phase 74, TODO L4): *history* is the client's prior turns
|
||||
already mapped to model messages by
|
||||
:func:`app.rag.prompts.history_to_messages` (trimmed newest-first
|
||||
against the settings budgets; assistant turns carry their prior
|
||||
thinking as ``reasoning_content``). It is spliced between the system
|
||||
prompt and the current user message —
|
||||
``[system, *history, user]`` — and everything downstream (the tool
|
||||
rounds, the phase-71 recovery rebuilding from ``messages[1:]``, the
|
||||
retry restarts) already operates on that one ``messages`` list,
|
||||
unchanged. ``()`` (the default) keeps the pre-phase-74 two-message
|
||||
request byte-identical.
|
||||
|
||||
Retries (phase 67, owner-locked A2): every model request goes through
|
||||
:func:`chat_stream_retried` — a failed round is retried **before** its
|
||||
first piece (same messages, ``settings.llm_retries`` restarts, a flat
|
||||
@@ -862,6 +875,7 @@ async def run_agent(
|
||||
"""
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": system_prompt},
|
||||
*history, # phase 74: the client's prior turns (empty by default)
|
||||
{"role": "user", "content": user_message},
|
||||
]
|
||||
# Phase 45: no per-tool budgets — the tools stay offered for the
|
||||
@@ -892,7 +906,7 @@ async def run_agent(
|
||||
# a quiet no-op.
|
||||
stream = chat_stream_retried(
|
||||
llm,
|
||||
cast("list[dict[str, str]]", messages),
|
||||
messages,
|
||||
tools=tools,
|
||||
retries=settings.llm_retries,
|
||||
delay=settings.llm_retry_delay,
|
||||
@@ -948,7 +962,7 @@ async def run_agent(
|
||||
recovery_filter = ScaffoldingFilter()
|
||||
recovered = chat_stream_retried(
|
||||
llm,
|
||||
cast("list[dict[str, str]]", messages_recovered),
|
||||
messages_recovered,
|
||||
tools=None,
|
||||
retries=settings.llm_retries,
|
||||
delay=settings.llm_retry_delay,
|
||||
@@ -1030,7 +1044,7 @@ async def run_agent(
|
||||
final_filter = ScaffoldingFilter()
|
||||
final = chat_stream_retried(
|
||||
llm,
|
||||
cast("list[dict[str, str]]", messages),
|
||||
messages,
|
||||
tools=None,
|
||||
retries=settings.llm_retries,
|
||||
delay=settings.llm_retry_delay,
|
||||
|
||||
Reference in New Issue
Block a user