feat(rag): pass chat history with prior thinking to the LLM
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH the conversation so far — every prior user/brain turn and the prior thinking blocks on brain turns (preserve-thinking) — while POST /api/chat stays stateless (A10): the client provides the history in the request body and the server stores nothing new. Server (task 01): - ChatRequest.history: optional list[HistoryTurn] (who: user|brain, text, optional thinking) — absent/empty keeps the request byte-identical to pre-phase-74 (the two-message [system, user] request; the kill-switch semantics are pinned in the integration suite). - app.rag.prompts.history_to_messages: pure mapper — walks the turns newest-first against the settings budgets (history_max_turns=40 / history_max_chars=24000, BOR_HISTORY_MAX_TURNS / BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut mid-answer); the kept window is returned oldest-first; brain turns carry their thinking as reasoning_content (A4) only when non-empty. - Both branches feed it: the deflected path splices it between the system prompt and the current user message (the phase-71 recovery still rebuilds from messages[1:]), the grounded agent receives run_agent(..., history=hist); llm.py's message params widen to list[dict[str, Any]] (string-only messages stay byte-identical on the wire — the SDK passes message dicts through verbatim). - The per-turn log line (PLAN §9) gains history_msgs=N after kb_chars=N. - Pins: tests/unit/test_history.py (mapper: mapping, reasoning gating, both budgets, drop-whole, ordering, empty default), tests/unit/test_config.py (the two settings + env overrides), tests/unit/test_agent.py (the history splice + the default), tests/integration/test_chat_api.py (deflected AND grounded forward the history incl. reasoning_content, no-history byte-identity, 422 pins, the log field). Client (task 02): - runTurn — the single funnel for fresh send / phase-49 retry / phase-53 stale-regen — sends history = the conversation record minus the current question, with thinking only on brain records that streamed one (undefined drops the key from the JSON, the record's convention); the question is never duplicated into the history. Wire proof (task 03): - The mock's echo my history marker (HISTORY_TRIGGER) answers with the deterministic history echo — history: N prior messages; last answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both turn branches whatever the gate says; the module docstring records the user/assistant-only history invariant that keeps every existing (tool-result-classified) marker flow unaffected. - tests/e2e/test_llm_history.py (isolated): a grounded follow-up and a deflected follow-up both receive history: 2 prior messages + thinking: yes + the byte-exact tail of turn 1's answer (derived from the persisted bor.chat.v1 record — the same array the client maps into the body); a cold start receives history: 0 prior messages / last answer tail: none / thinking: no. - Regressions green in isolation: chat_rag, chat_history (phase 50), agent_document_tools, harness_aligned_tools, stop_generation, retry_answer, response_to_docs.
This commit is contained in:
@@ -89,7 +89,10 @@ class FakeRagLLM:
|
||||
#: recovered answer endpoint for the pre-first-piece retry rule.
|
||||
self.stream_fail_count = stream_fail_count
|
||||
self.question_embeds: list[str] = []
|
||||
self.seen_messages: list[list[dict[str, str]]] = []
|
||||
#: Phase 74: assistant history messages may carry
|
||||
#: ``reasoning_content`` — the dict values stay strings, but the
|
||||
#: key set is wider than the pre-phase ``{role, content}`` shape.
|
||||
self.seen_messages: list[list[dict[str, Any]]] = []
|
||||
#: Every request's ``tools`` value (phase 37) — ``None`` is the
|
||||
#: pre-phase request shape (the key is absent from the payload).
|
||||
self.seen_tools: list[list[dict[str, Any]] | None] = []
|
||||
@@ -138,7 +141,7 @@ class FakeRagLLM:
|
||||
|
||||
async def chat_stream(
|
||||
self,
|
||||
messages: list[dict[str, str]],
|
||||
messages: list[dict[str, Any]],
|
||||
tools: list[dict[str, Any]] | None = None,
|
||||
scaffolding: ScaffoldingFilter | None = None,
|
||||
):
|
||||
@@ -1226,3 +1229,168 @@ def test_deflected_mixed_scaffolding_and_content_needs_no_recovery(
|
||||
assert len(flaky.seen_messages) == 1 # the clean content stands — no recovery
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and f"scaffold_stripped={len(span)}" in lines[-1]
|
||||
|
||||
|
||||
# ---------- phase 74: client-provided history (with prior thinking) ----------
|
||||
|
||||
#: The client's prior turns (oldest first — the ``bor.chat.v1`` record
|
||||
#: minus the current question): two user turns, two brain turns, the
|
||||
#: FIRST brain turn carrying a prior thinking block (A4) and the second
|
||||
#: not (the ``reasoning_content`` gate has both shapes on one request).
|
||||
HISTORY: list[dict[str, Any]] = [
|
||||
{"who": "user", "text": "What port does Tailscale run on?"},
|
||||
{
|
||||
"who": "brain",
|
||||
"text": "Tailscale runs on 41641/udp.",
|
||||
"thinking": "The Tailscale wire protocol uses 41641/udp.",
|
||||
},
|
||||
{"who": "user", "text": "And the subnet router?"},
|
||||
{"who": "brain", "text": "The subnet router shares the same port."},
|
||||
]
|
||||
|
||||
#: What :func:`app.rag.prompts.history_to_messages` must produce for
|
||||
#: :data:`HISTORY` — chronological, ``reasoning_content`` ONLY on the
|
||||
#: turn that had thinking.
|
||||
HISTORY_MESSAGES: list[dict[str, Any]] = [
|
||||
{"role": "user", "content": "What port does Tailscale run on?"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Tailscale runs on 41641/udp.",
|
||||
"reasoning_content": "The Tailscale wire protocol uses 41641/udp.",
|
||||
},
|
||||
{"role": "user", "content": "And the subnet router?"},
|
||||
{"role": "assistant", "content": "The subnet router shares the same port."},
|
||||
]
|
||||
|
||||
|
||||
def _stream_chat_with_history(
|
||||
client: TestClient, message: str, history: list[dict[str, Any]]
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Phase 74 variant of :func:`_stream_chat`: sends ``history`` (the
|
||||
client's prior turns, oldest first) in the request body."""
|
||||
with client.stream(
|
||||
"POST", "/api/chat", json={"message": message, "history": history}
|
||||
) as r:
|
||||
assert r.status_code == 200
|
||||
assert r.headers["content-type"].startswith("text/event-stream")
|
||||
buf = ""
|
||||
frames: list[dict[str, Any]] = []
|
||||
for part in r.iter_text():
|
||||
buf += part
|
||||
while "\n\n" in buf:
|
||||
frame, buf = buf.split("\n\n", 1)
|
||||
frame = frame.strip()
|
||||
if frame.startswith("data:"):
|
||||
frames.append(json.loads(frame.removeprefix("data:").strip()))
|
||||
assert buf.strip() == "", "stream must end on a frame boundary"
|
||||
return frames
|
||||
|
||||
|
||||
def test_deflected_turn_forwards_history_with_prior_thinking(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""A DEFLECTED turn sends the prior turns — chronological, with the
|
||||
prior brain turn's thinking as ``reasoning_content`` — between the
|
||||
LOW system prompt and the current question (A2/A3/A4); the per-turn
|
||||
log line carries ``history_msgs=4``."""
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
frames = _stream_chat_with_history(client, OFF_TOPIC, HISTORY)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["type"] == "done"
|
||||
assert frames[-1]["deflected"] is True
|
||||
assert len(seeded_kb.seen_messages) == 1
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
assert messages[0]["role"] == "system"
|
||||
assert "DEFLECT_MODE" in messages[0]["content"] # the LOW prompt
|
||||
assert messages[1:-1] == HISTORY_MESSAGES # the prior turns, chronological
|
||||
assert messages[-1] == {"role": "user", "content": OFF_TOPIC}
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "history_msgs=4" in lines[-1]
|
||||
|
||||
|
||||
def test_grounded_turn_forwards_history_through_the_agent(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""The GROUNDED agent branch receives the same block: its first
|
||||
request is ``[HIGH system, *history, current question]`` (the tool
|
||||
rounds then append to that same list); ``history_msgs=4``."""
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
frames = _stream_chat_with_history(client, QUESTION, HISTORY)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["type"] == "done"
|
||||
assert frames[-1]["deflected"] is False
|
||||
assert len(seeded_kb.seen_messages) == 1 # the canned answer ends the loop
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
assert messages[0]["role"] == "system"
|
||||
assert "<tools>" in messages[0]["content"] # the HIGH prompt
|
||||
assert messages[1:-1] == HISTORY_MESSAGES
|
||||
assert messages[-1] == {"role": "user", "content": QUESTION}
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "history_msgs=4" in lines[-1]
|
||||
|
||||
|
||||
def test_request_without_history_sends_exactly_system_and_user(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""Byte-identical pin (A2): a request WITHOUT ``history`` sends
|
||||
exactly the two-message ``[system, user]`` request on BOTH branches
|
||||
(deflected + grounded), and the per-turn log line carries
|
||||
``history_msgs=0``."""
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
_stream_chat(client, OFF_TOPIC) # deflected branch
|
||||
_stream_chat(client, QUESTION) # grounded branch
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert len(seeded_kb.seen_messages) == 2
|
||||
for messages in seeded_kb.seen_messages:
|
||||
assert [m["role"] for m in messages] == ["system", "user"]
|
||||
assert seeded_kb.seen_messages[0][1] == {"role": "user", "content": OFF_TOPIC}
|
||||
assert seeded_kb.seen_messages[1][1] == {"role": "user", "content": QUESTION}
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert len(lines) == 2
|
||||
assert all("history_msgs=0" in line for line in lines)
|
||||
|
||||
|
||||
def test_history_rejects_unknown_who(client, db) -> None:
|
||||
"""Schema pin: ``who`` is a ``Literal["user", "brain"]`` — anything
|
||||
else is a 422 at the boundary (the same trust model as the saved-
|
||||
chat ``ChatMessage``)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={"message": "hi", "history": [{"who": "alien", "text": "x"}]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
|
||||
|
||||
def test_history_rejects_more_than_100_entries(client, db) -> None:
|
||||
"""Schema pin: the DoS sanity ceiling is 100 turns — 101 is a 422
|
||||
(the config budgets do the real trimming; this only keeps a
|
||||
pathological body from wasting the mapper's work)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={
|
||||
"message": "hi",
|
||||
"history": [{"who": "user", "text": f"q{i}"} for i in range(101)],
|
||||
},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
|
||||
Reference in New Issue
Block a user