"""Deterministic OpenAI-compatible mock for E2E tests (aipi stand-in). Implements just enough of the aipi surface: * ``GET /v1/models`` * ``POST /v1/embeddings`` — real bag-of-words vectors (768-dim, L2-normed). Because similarity is *genuine token overlap*, the relevance threshold behaves the same way it will in production: related questions score high, unrelated ones score low and trigger honest deflection. * ``POST /v1/chat/completions`` — streaming (SSE) or not. The content keys off markers in the system prompt: - user message containing ``write a long answer`` -> a ~900-word deterministic numbered answer (long-answers story, phase 11) - ``SUMMARY_MODE`` -> the deterministic summary digest: the first 24 tokens of the user message (the summarizer puts the capped document content there) — byte-stable for a given fixture (document summaries, phase 30) - ``KB_OVERVIEW_MODE`` -> the deterministic outline: the first 8 tokens of the user message (the generator puts the document list there) — byte-stable for a given KB (KB overview, phase 31) - ``FOLDER_SUMMARY_MODE`` -> the deterministic folder one-liner ``Documentation for .`` (phase 155 task 01: the new voice — a natural opener naming the folder, never the forbidden "This folder contains…" openers) — is the user message's ``Folder: …`` header line (the generator names the source/folder there, phase 94), so the stored row always names its folder and an E2E can assert on it. Checked BEFORE the ``SUMMARY_MODE`` branch: the folder marker CONTAINS the summary marker as a substring, so the summary branch would otherwise shadow every folder-summary call - ``IMAGE_DESCRIPTION_MODE`` (phase 122, image documents) -> the fixed ``IMAGE_DESCRIPTION_ANSWER`` description: the CHAT model's (vision) reply to ``summarizer.describe_image``'s multimodal user message (the marker is its first text part; the call is non-streaming, so only this answer path serves it). The reply is the image document's whole content + summary, so it is byte- stable and token-dense (its cosine against the story suite's question clears the mock-calibrated gate) - ``PICTORIAL_PAGE_MODE`` (phase 140, pictorial PDF pages) -> the fixed ``PICTORIAL_PAGE_ANSWER`` description: the CHAT model's (vision) reply to ``summarizer.describe_pdf_page``'s multimodal user message (one rendered sparse PDF page — the marker is its first text part; the call is non-streaming, so only this answer path serves it). The reply is appended to the page's indexed content block as ``[Page N illustration: …]`` (LOCKED A4), so it is byte-stable: a re-sync with the same reply re-digests to the same content and lands as ``unchanged`` - ``DEFLECT_MODE`` -> honest "I haven't done anything like that" answer - otherwise -> upbeat answer quoting the provided document context - user message containing ``pretend to think slowly`` -> 3s warm-up delay (used by the loading-feedback story). - user message containing ``think out loud`` -> the answer is preceded by ~2 700 chars of deterministic ``reasoning_content`` chunks (the thinking-display story, phase 17; lengthened in phase 21 so the rendered scratchpad overflows the 320px ``.thinking-text`` window) - user message containing ``think out loud then hesitate`` -> the ``think out loud`` stream, then a 4s pause before the first content frame (the sources-midstream story, phase 20 — a deterministic "leave during pure thinking" navigation window). - user message containing ``think in paragraphs`` (``THINK_PARAS_TRIGGER``) -> the ``think out loud`` scratchpad WITH REAL paragraph breaks ("\n\n"), streamed at 60-char frames (vs the mock's 12-char default). One frame renders several lines — a real-model-sized delta, the condition under which a POST-render pin-state reading (the old app.js) measured the chunk's height instead of the user's position and the think-window follow died at the first 2-newline gap. The regression pin for the pre-render capture in app.js (2026-08-29, owner report). Checked BEFORE ``think out loud`` (it is the more specific phrase); existing E2E questions carry neither, so every other suite is unaffected. - system prompt containing ```` (phase 15, steering notes) -> the composed answer ends with `` (tuning: )`` — makes prompt injection observable in the UI deterministically. - system prompt containing ```` (phase 31, KB overview) -> the composed answer ends with `` (kb: )`` — the same echo convention for the overview's prompt injection. - user message containing ``show the end of your notes`` (phase 24; phase 118 re-targeted — the story's dedicated suite ``tests/e2e/test_summary_seed_context.py``) -> the answer quotes the **last 160 chars of the ```` block** — a tail echo, byte-stable across runs. (Phase 37: the HIGH prompt now ends with a ```` section after ````, so the echo targets the block itself; its tail still includes the closing tag.) Phase 118 (A6): the block carries the suggested documents' SUMMARIES (never full texts), so the echoed tail is the LAST suggested document's SUMMARY tail (its digest + the ``Source: /`` pointer line) — a sentinel on a document's *last line* appears in the rendered answer iff the FULL content (not the summary) was in the prompt, which under the summary-seed contract is only through a ``read`` tool result. - user message containing ``use your tools`` (phase 37, agent document tools; phase 70: the flow emits the harness-aligned names — ``ls`` / ``read`` with the combined ``source/path`` identity; phase 94: the flow DRILLS through the tree ``ls`` — the top-level listing carries sources only, so the flow takes one drill step, ``ls`` scoped to the first source, before the first file line exists) **and** the system prompt carries the ```` section -> the deterministic SINGLE-READ tool flow, discriminated statelessly from the messages (the ``tools`` parameter gates the list/read steps — a no-tools request with no tool results is not the flow): * request 1 (``tools`` offered, no tool results yet): stream ONLY ``tool_calls`` deltas — ``ls`` (synthetic id ``call_0``, no arguments), ``finish_reason: "tool_calls"``, no content; * request 2 (the top-level source listing in the messages — the agent's ``ls`` tree format, phase 94: no file lines yet): stream a ``tool_calls`` delta — ``ls`` scoped to the FIRST source of the listing (id ``call_1``) — the drill step (one drill per flow — ``_drill_target`` skips already-drilled sources, so the second listing is the folder level); * request 3 (a ``tool``-role folder listing with file lines in the messages): parse the FIRST file line (``source: X | path: Y | title: Z`` — the labeled ``source:`` / ``path:`` fields, phase 63) and stream a ``tool_calls`` delta calling ``read`` on the JOINED combined ``source/path`` (the mock joins the two labeled fields — the file-line format is unchanged, so this join is the only parse change, phase 70) (id ``call_2``); * request 4 (a ``tool``-role read result in the messages — content starting with the agent's ``"Document :"`` header): a content answer, deterministic: ``Read . `` — so a suite can assert the read document reached the model and landed in the answer. Reached regardless of the ``tools`` parameter (phase 45 keeps the tools offered until the round cap). The single-read flow stops at ONE read result; the MULTI-READ variant below reads two. - user message containing BOTH ``use your tools`` AND ``read two documents`` (``MULTI_READ_TRIGGER``, phase 45 task 02) **and** the system prompt carries the ```` section -> the deterministic MULTI-READ flow (list → drill → read #1 → read #2 → answer; phase 94: the drill step between the top-level listing and the first read), classified by the COUNT of ``tool``-role read results (content starting with the agent's ``"Document :"`` prefix); phase 70: the same flow on the harness-aligned names — ``ls``, the drill ``ls`` scoped to the first source, then ``read`` on the JOINED combined ``source/path`` of each file line: * 0 read results, no listing yet: ``ls`` (id ``call_0``); * 0 read results, top-level listing only (no file lines yet): the drill — ``ls`` scoped to the first source (id ``call_1``); * 0 read results, file lines present: ``read`` on the JOINED combined ``source/path`` of the FIRST file line (id ``call_2``); * 1 read result: ``read`` on the JOINED combined ``source/path`` of the SECOND file line — the first listing line whose ``source/path`` differs from the one already read (id ``call_3``); a one-file listing degenerates to the single-read answer (nothing second to read); * 2 read results: the forced answer, byte-stable: the single-read shape quoting the FIRST read result, plus the line ``I read and .`` naming both read paths in read order — so a suite can assert the model used BOTH documents. All other requests (including the marker without a ```` section, or with the tool conversation not yet started and no tools offered — e.g. ``agent_max_rounds=0``) behave exactly as today. ``E2E_REAL_LLM=1`` ignores the mock entirely (the real model does what it does). - user message containing ``search your documents`` (``SEARCH_TRIGGER``, phase 68 search tool — renamed to the harness-aligned ``grep`` in phase 70, same match/output contract) **and** the system prompt carries the ```` section -> the deterministic SEARCH tool flow, discriminated statelessly from the messages (streaming only): * request 1 (``tools`` offered, no search result yet): stream ONLY ``tool_calls`` deltas — ``grep`` with ``{"pattern": SEARCH_PATTERN}`` (id ``call_0``); * request 2 (a ``tool``-role search result in the messages — recognizable by its ``source/path:line: text`` match lines or the sentinel in its content): the content answer, deterministic: ``Found `` — so a suite can assert the search result reached the model and landed in the answer. Checked BEFORE the plain ``use your tools`` flow (it is the more specific phrase — same convention as ``think in paragraphs``); no existing E2E question or fixture file contains the trigger, so every other suite is unaffected. - user message containing ``search within one document`` (``SCOPED_SEARCH_TRIGGER``, phase 146 task 02 — the search tool's SCOPE: a re-search of ONE document, so the UI can show where the search runs and repeated patterns read as distinct searches, TODO.md L3) **and** the system prompt carries the ```` section -> the deterministic SCOPED-SEARCH flow, discriminated statelessly from the messages (streaming only; the bare-grep ``SEARCH_TRIGGER`` flow above is the byte-identity pin and stays untouched): * request 1 (``tools`` offered, no search result yet): stream ONLY ``tool_calls`` deltas — ``grep`` with ``{"pattern": SEARCH_PATTERN, "path": SCOPED_SEARCH_PATH}`` (id ``call_0``) — the SCOPED form (pattern + the re-searched document's combined ``source/path``); * request 2 (a ``tool``-role search result in the messages — the agent's REAL scoped grep matched the fixture's sentinel line): the content answer, deterministic: ``Found `` (the same echo shape as the bare flow). Checked BEFORE the bare ``SEARCH_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72 ordering convention; the trigger needs the ```` section, so deflected turns never hit it); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``emit raw tool markup`` (``SCAFFOLD_TRIGGER``, phase 71, tool-scaffolding guardrails — the 2026-09-03 incident where a deflected round streamed the model's raw ``<|tool_call_start|>…<|tool_call_end|>`` markup into the UI) **or** ``always emit raw tool markup`` (``SCAFFOLD_ALWAYS_TRIGGER``, checked FIRST — it contains the former phrase) -> the deterministic SCAFFOLDING flow, independent of the ```` marker (both grounded and deflected turns hit it): * ``SCAFFOLD_ALWAYS_TRIGGER``: EVERY request (the one bounded recovery included) streams ONLY ``delta.content`` chunks carrying the incident span ``SCAFFOLD_SPAN`` — ``<|tool_call_start|>[read(path='search_docs/reese-notes.md')] <|tool_call_end|>`` — split across the mock's 12-char chunks (the filter's boundary path), ``finish_reason: "stop"``, no structured ``tool_calls``, no reasoning — the terminal malformed-reply path. * ``SCAFFOLD_TRIGGER``: request 1 (no ``CORRECTION_INSTRUCTION`` in the system prompt) streams the same scaffolding-only span; request 2 (the system prompt carries the harness constant — a stable substring of ``app.rag.agent.CORRECTION_INSTRUCTION``, IMPORTED into this module so the mock can never drift from it: the one bounded recovery, ``tools=None`` with the correction folded into the single system prompt) streams the clean ``SCAFFOLD_RECOVERY_ANSWER`` — the recovery path. Checked BEFORE the ``SEARCH_TRIGGER`` / ``TOOLS_TRIGGER`` flows (the trigger needs no ```` section); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``list the files in this directory`` (``LS_TEACH_TRIGGER``, phase 72, teaching refusals — the 2026-09-03 incident where the harness-prior ``ls(path='.')`` misuse met the terse refusal and the model re-reasoned the same paragraphs over and over) **and** the system prompt carries the ```` section -> the deterministic LS-TEACHING flow, discriminated statelessly from the messages (streaming only; phase 94: the correction's top-level listing carries sources only, so the flow drills one level before the first file line exists): * request 1 (``tools`` offered, no ``tool``-role result in the messages yet): stream ONLY ``tool_calls`` deltas — ``ls`` with ``{"path": "."}`` (synthetic id ``call_0``), ``finish_reason: "tool_calls"``, no content — the incident's misuse, deterministic; * request 2 (a ``tool``-role result present that is NOT a listing with file lines — i.e. the teaching refusal): a ``tool_calls`` delta — ``ls`` with no arguments (id ``call_1``) — the correction; * request 3 (the top-level source listing in the messages — file lines still absent): the drill — a ``tool_calls`` delta — ``ls`` scoped to the FIRST source of the listing (id ``call_2``); * request 4 (a ``tool``-result with a ``source: X | path: Y | title: Z`` file line in the messages — the folder listing): a deterministic content answer — ``These are the indexed documents: `` (the line, parsed with the ``_CATALOG_LINE_RE`` machinery), ``finish_reason: "stop"`` — the loop ended in ONE correction + ONE drill, not at the round cap. Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (the trigger phrases are disjoint substrings — the phase-71 ordering convention); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``drill down the tree`` (``DRILL_TRIGGER``, phase 94 task 04 — the drill-down ``ls``'s dedicated story suite ``tests/e2e/test_ls_tree_drilldown.py``) **and** the system prompt carries the ```` section -> the deterministic SCRIPTED DRILL-DOWN flow: the question carries its own tool call after the colon — ``drill down the tree: ls [target]`` (no target = the top level; ``target`` = a source name or ``source/folder`` path) or ``drill down the tree: read source/path`` — parsed by ``_DRILL_CALL_RE`` from the RAW user message (the target keeps its case), then discriminated statelessly from the tool results (streaming only): * request 1 (``tools`` offered, no ``tool``-role result in the messages yet): the SCRIPTED call — ``ls`` with no arguments for the top level, ``ls``/``read`` with the parsed target otherwise (synthetic id ``call_0``); * the last tool result is the NOT-A-FOLDER teaching refusal (it carries ``"is not a folder"`` — the phase-94 task-03 teaching line, the refusal being VISIBLE to the model is what fires this branch): the scripted one-round recovery — ``ls`` the target's SOURCE segment (id ``call_1``); * the last tool result is a READ result (``"Document :…``): the deterministic echo answer ``Read . `` (the phase-37 single-read shape — the grounded- turn citation contract); * the last tool result is the agent's ALREADY_IN_CONTEXT dedupe refusal (phase 118: the read target is a document ALREADY READ into full-text context earlier in the same turn — the seeds are summaries, so a first read of any document succeeds and only a re-read is refused): the model answers FROM THE PROMPT — the deterministic answer ``Already in context: Read . `` block>`` (same citation shape as the read-result branch — the document text reached the model either way, and the answer proves it; the block it quotes now carries the document's SUMMARY). No suite exercises this branch today (the drill questions read once per turn) — it is kept for the still-real already-read refusal; * any other last result (a top-level or folder LISTING landed): the deterministic ECHO answer ``Here's the level I listed:\n `` — the mock echoes what it received (the house scripted-turn way of asserting on tool results): the suite asserts on the exact tree level — the ``— N documents`` lines, the stored folder summaries, the ``source: X | path: Y | title: Z`` file lines, and the 50-line cap + ``…and N more documents…`` note — through the rendered answer, the only E2E lens on the LLM's context. Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72 ordering convention; the trigger needs the ```` section, so deflected turns never hit it); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``read the capped document`` (``READ_CAP_TRIGGER``, phase 95 task 03 — the read cap's dedicated story suite ``tests/e2e/test_read_truncation_cap.py``) **and** the system prompt carries the ```` section -> the deterministic SCRIPTED CAPPED-READ flow: the question carries its own tool call after the colon — ``read the capped document: read source/path`` — parsed by ``_READ_CAP_CALL_RE`` from the RAW user message (the target keeps its case), then discriminated statelessly from the tool results (streaming only): * request 1 (``tools`` offered, no ``tool``-role result in the messages yet): the scripted call — ``read`` with the parsed target (synthetic id ``call_0``); * a ``tool``-role result is in the messages: the deterministic ECHO — the answer carries the LAST tool result VERBATIM (``Here's what the read returned:\n``): a read result (``"Document :…``) lands in the answer with its FULL content — the first cap chars + ``[…truncated…]`` + the pinned grep-pointer notice when the cap fired, the plain body byte-identical to the pre-phase-95 shape when it did not — and a refusal (the premise broke) lands just as visibly, so the suite fails loudly on it. The mock is the only E2E lens on the LLM's context, so the echo is the assertion surface for both the marker's presence AND its absence. Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72/94 ordering convention; the trigger needs the ```` section, so deflected turns never hit it); verified 2026-09-10: no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``read the suggested document`` (``SUMMARY_SEED_READ_TRIGGER``, phase 118 task 06 — the summary-seed context's dedicated story suite ``tests/e2e/test_summary_seed_context.py``) **and** the system prompt carries the ```` section -> the deterministic SCRIPTED SUMMARY-READ flow: the question carries its own tool call after the colon — ``read the suggested document: read source/path`` — parsed by ``_SUMMARY_SEED_READ_CALL_RE`` from the RAW user message (the target keeps its case), then discriminated statelessly from the tool results (streaming only): * request 1 (``tools`` offered, no ``tool``-role result in the messages yet): the scripted call — ``read`` with the parsed target (synthetic id ``call_0``); * a ``tool``-role result is in the messages: the deterministic ECHO — the answer carries the LAST tool result VERBATIM (``Here's what the read returned:\n``): under the phase-118 summary-seed contract a first ``read`` of ANY document succeeds (the seeds are summaries, not full text), so a read result (``"Document :…`` — header + the phase-106 D5 ``date:`` line + the FULL content) lands in the answer with its tail intact — the story suite's lens on the full text the ``read`` tool delivered (a tail sentinel on the document's last line appears in the answer iff the full content reached the model through the read, not the seed); a refusal (the premise broke) lands just as visibly, so the suite fails loudly on it. The mock is the only E2E lens on the LLM's context, so the echo is the assertion surface. Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72/94 ordering convention; the trigger needs the ```` section, so deflected turns never hit it); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``what are the correct llama.cpp arguments`` (``GREP_TEACH_TRIGGER``, the 2026-09-05 incident — the harness prior is that grep takes a REGEX; this app's grep is a case-insensitive fixed substring, owner-locked A5) **and** the system prompt carries the ```` section -> the deterministic GREP-REGEX-TEACHING flow, discriminated statelessly from the messages (streaming only): * request 1 (``tools`` offered, no ``tool``-role result yet): stream ONLY ``tool_calls`` deltas — ``grep`` with ``{"pattern": GREP_TEACH_PATTERN}`` (``qwen.*3\\.8``, id ``call_0``) — the incident's regex-shaped first grep, which a fixed-substring grep can NEVER match; * request 2 (the last tool result is the server's TEACHING no-match line — it carries ``GREP_TEACH_MARKER``): ``grep`` with the plain form ``GREP_TEACH_PLAIN`` (``qwen3.8``, id ``call_1``) — the one-round correction; * request 3 (the last tool result carries ``source/path:line: text`` match lines): ``read`` the FIRST match line's document by its combined ``source/path`` (id ``call_2``); * request 4 (the last tool result is a read result, the ``"Document :\n"`` shape): the deterministic echo answer ``Read . ``, ``finish_reason: "stop"`` — the loop ended in ONE correction, not at the round cap. * A PLAIN no-match as the last result (no match line, no teaching marker — e.g. the plain pattern genuinely absent) is the deterministic terminal answer ``No matches — the knowledge base has no such text.`` (the flow cannot loop on a well-formed pattern). Checked BEFORE the SEARCH / TOOLS_TRIGGER flows (disjoint trigger phrases — the phase-72 ordering convention); no existing E2E question or fixture file contains the phrase, so every other suite is unaffected. - user message containing ``show me a table`` (phase 44, markdown tables, TODO.md L6) -> the fixed table answer (``TABLE_ANSWER``): a 3-column service table, an ```` XSS probe line, and a deliberately wide 5-column table — byte-stable, so the story E2E can assert the rendered ```` shape, the escaped XSS line, and the wrapper's horizontal scroll inside the 72rem container column (phase 100). Checked BEFORE the ``DEFLECT_MODE`` branch (a deflection prompt never carries the marker, same reasoning as ``SUMMARY_MODE``), so a marker question always gets the table answer; the E2E asks it against an on-topic fixture (HIGH gate) and asserts non-deflection. - user message containing ``show me the blockquote`` (phase 138, markdown blockquotes, TODO.md L3) -> the fixed quote answer (``QUOTE_ANSWER``): a single-line quote, a 3-line quote group, a nested ``>>`` collapse line, an ``\n" "\n" "```\n" "> inside a fence stays fence content\n" "```\n" "\n" "| Cell |\n" "|---|\n" "| > x stays a cell |\n" "\n" "And the quote ends here — a plain closing line." ) # --------------------------------------------------------------------------- # Phase 67 (LLM retry, TODO.md L3): deterministic failure injection # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) gets #: ``RETRY_DEAD_ATTEMPTS`` dead streaming attempts (500, JSON body) before #: the normal composed answer streams — 1 original attempt + 1 retry under #: the default ``BOR_LLM_RETRIES=3`` (see the module docstring). RETRY_TRIGGER = "fail then answer" #: App-level attempts the endpoint stays dead for before the answer. RETRY_DEAD_ATTEMPTS = 2 #: A user message containing this substring (case-insensitive) makes #: EVERY streaming chat/completions request respond 500 — the #: retry-budget exhaustion path (the terminal error banner in the UI). ALWAYS_FAIL_TRIGGER = "always fail" #: An embeddings request whose input contains this substring #: (case-insensitive) 500s on its FIRST POST; the next returns the normal #: bag-of-words vector — the endpoint's pre-stream embedding retry loop. EMBED_FAIL_TRIGGER = "embed fail once" # --------------------------------------------------------------------------- # Phase 120 (failed-turn retry, TODO.md L3–4): deterministic failure # injections for the failed-turn E2E suite (tests/e2e/ # test_failed_turn_retry.py) — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) dies for #: the ENTIRE first turn — every app-level attempt 500s (the #: retry-budget exhaustion, the zero-frame terminal ``error``) — and #: streams the normal composed answer from the SECOND turn on (the #: banner/in-bubble Retry's re-ask, phase 111/120). The dead window is #: exactly ONE turn: ``FAIL_FIRST_TURN_DEAD_ATTEMPTS`` = the e2e #: forced-default budget (conftest pins ``BOR_LLM_RETRIES`` to the code #: default 3 → 4 app-level attempts = 12 POSTs at #: ``_HTTPS_PER_DEAD_ATTEMPT``). Each sequence resets after the success #: it guards, so a second question carrying the trigger re-drives the #: failure from zero (the phase-67 ``_fail_posts`` convention). FAIL_FIRST_TURN_TRIGGER = "fail first turn" #: One full turn under the forced-default budget (3 retries → 4 #: attempts): the whole first turn dies, the second turn's first #: attempt streams (the Retry's re-ask). FAIL_FIRST_TURN_DEAD_ATTEMPTS = 4 #: A user message containing this substring (case-insensitive) streams #: ``PARTIAL_FAIL_TEXT`` as ordinary ``delta.content`` chunks on its #: FIRST matching request and then the mock's generator RAISES (a #: mid-stream connection reset — the SSE body ends without #: ``finish_reason``/``[DONE]``; a mid-stream failure never re-POSTs). #: The app's ``chat_stream_retried`` re-raises the wrapped ``LLMError`` #: (a piece already emitted — the locked retry-before-first-frame rule) #: and the chat endpoint settles with the terminal SSE ``error`` frame #: AFTER the partial deltas. The SECOND request streams the normal #: composed answer (the retry's re-ask) and resets the sequence. PARTIAL_FAIL_TRIGGER = "partial then fail" #: Matching requests that partial-then-die (one POST — see above). PARTIAL_FAIL_DEAD_REQUESTS = 1 #: The deterministic partial answer (byte-stable — the E2E asserts the #: bubble KEEPS exactly this text, with the error note appended). PARTIAL_FAIL_TEXT = "Here is the start of the answer that never finished." # --------------------------------------------------------------------------- # Phase 71 (tool-scaffolding guardrails, 2026-09-03 incident): # deterministic raw-markup flows — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) drives #: the scaffolding flow: request 1 streams ONLY the incident's raw tool #: markup as ``delta.content``; the follow-up request carrying the #: harness correction in the system prompt (the one bounded recovery) #: streams the clean answer. Independent of the ```` marker — #: both grounded and deflected turns hit it. Existing E2E questions do #: not contain the phrase, so every other suite is unaffected. SCAFFOLD_TRIGGER = "emit raw tool markup" #: A user message containing this substring (checked BEFORE #: ``SCAFFOLD_TRIGGER`` — it contains that phrase) streams the #: scaffolding-only span on EVERY request, recovery included — the #: terminal malformed-reply path (the dedicated error frame, no done). SCAFFOLD_ALWAYS_TRIGGER = "always emit raw tool markup" #: The incident span (2026-09-03): the model's chat-template tool #: syntax, emitted as plain ``delta.content`` although no tools were #: offered. Streamed through the mock's 12-char chunking, so it always #: spans ≥2 wire chunks (the filter's boundary path). SCAFFOLD_SPAN = ( "<|tool_call_start|>[read(path='search_docs/reese-notes.md')]" "<|tool_call_end|>" ) #: The clean answer the one bounded recovery produces (byte-stable — #: the dedicated E2E suite asserts the recovered bubble and the wire's #: delta text against it). SCAFFOLD_RECOVERY_ANSWER = "Here is the plain-text answer the recovery produced." #: The stable substring of the harness-owned correction constant the #: recovery request carries in its system prompt. Keyed on a substring #: (not the whole constant) so a re-wrap of the constant cannot silently #: re-route the mock; the module-level assert below fails loudly if the #: substring ever leaves the constant (the mock must never drift from #: ``app.rag.agent.CORRECTION_INSTRUCTION``). _CORRECTION_MARKER = "no tool syntax" assert _CORRECTION_MARKER in CORRECTION_INSTRUCTION, ( "mock drift: the correction marker left CORRECTION_INSTRUCTION" ) # --------------------------------------------------------------------------- # Phase 72 (teaching refusals — the 2026-09-03 incident's ls misuse): # the deterministic LS-TEACH self-correction flow — see the module # docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the deterministic LS-TEACHING flow (the incident's #: ``ls(path='.')`` misuse → the teaching refusal → the corrected #: no-arg ``ls()`` → the catalog answer). Checked BEFORE the plain #: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71 #: ordering convention); verified: no existing E2E question or fixture #: file contains the phrase, so every other suite is unaffected. LS_TEACH_TRIGGER = "list the files in this directory" #: The deterministic GREP-REGEX-TEACH flow (the 2026-09-05 "Qwen 3.8" #: incident — the harness prior is that grep takes a REGEX; this app's #: grep is a case-insensitive fixed substring, owner-locked A5, so a #: regex-shaped pattern can NEVER match, and the bare no-match line #: made the turbo model trust the miss and end the turn with a wrong #: "I searched the entire knowledge base" refusal). The flow pins the #: self-correction on the SSE wire: the regex-shaped first grep → the #: server's TEACHING no-match line (``GREP_TEACH_MARKER``) → the #: plain-form retry grep → the match → the read → the deterministic #: echo answer. Checked BEFORE the SEARCH / TOOLS_TRIGGER flows #: (disjoint trigger phrases — the phase-71/72 ordering convention); #: verified: no existing E2E question or fixture file contains the #: phrase, so every other suite is unaffected. GREP_TEACH_TRIGGER = "what are the correct llama.cpp arguments" #: The incident's regex-shaped first grep (it can never match a #: fixed-substring grep — that is the point of the flow). GREP_TEACH_PATTERN = "qwen.*3\\.8" #: The plain-form retry — the server's teaching line hands over exactly #: this hint (``app.rag.agent.plain_form(GREP_TEACH_PATTERN)``). GREP_TEACH_PLAIN = "qwen3.8" #: The marker of the agent's teaching no-match line (app.rag.agent #: ``NO_MATCHES_REGEX`` / ``NO_MATCHES_REGEX_SCOPED``) — the mock's #: plain step keys on it (a plain no-match line carries it not). GREP_TEACH_MARKER = "grep matches a plain substring" # --------------------------------------------------------------------------- # Phase 94 (task 04, the drill-down ls's dedicated story suite): # the deterministic SCRIPTED drill-down turns — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the scripted DRILL-DOWN flow: the question carries its own tool call #: after the colon (``drill down the tree: ls [target]`` / ``drill #: down the tree: read source/path`` — the suite's scripted turns, #: ``tests/e2e/test_ls_tree_drilldown.py``). Checked BEFORE the plain #: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72 #: ordering convention); verified: no existing E2E question or fixture #: file contains the phrase, so every other suite is unaffected. DRILL_TRIGGER = "drill down the tree" #: The scripted call in the drill-down question (case-insensitive — the #: suite's questions capitalize the trigger's first letter): the verb #: (``ls`` / ``read``) plus the optional target — a source name or #: ``source/folder`` path (``ls``) or a combined ``source/path`` #: (``read``), parsed from the RAW user message so the target keeps its #: case. The target is a ``[a-z0-9_./-]`` run (case-insensitively), so #: the suite's `` — `` flavor separator (em dash) can never bleed into #: it; no run = the top-level ``ls`` (no ``path`` argument). _DRILL_CALL_RE = re.compile( r"drill down the tree:\s*(?Pls|read)(?:\s+(?P[a-z0-9_./-]+))?", re.I, ) #: The agent's NOT-A-FOLDER teaching refusal marker (app.rag.agent #: ``NOT_A_FOLDER`` — ``'{arg}' is not a folder — {parent} has: #: {subfolders}``): the drill-down flow's scripted recovery keys on it #: in the LAST tool result (the refusal being visible to the model is #: exactly what triggers the one-round recovery — the phase-72 #: self-correction contract carrying the tree's teaching line). _NOT_A_FOLDER_MARKER = "is not a folder" #: The stable substring of the harness-owned dedupe refusal the #: drill-down flow's ``ctx_answer`` branch keys on (phase 118: a read #: target ALREADY READ into full-text context in the same turn — the #: seeds are summaries, so a first read of any document succeeds and #: only a re-read is refused; the mock then answers from the document's #: summary in the ```` prompt block, exactly what the #: refusal instructs). Keyed on a substring (not the whole #: constant) so a re-wrap of the constant cannot silently re-route the #: mock; the module-level assert below fails loudly if the substring #: ever leaves the constant (the mock must never drift from #: ``app.rag.agent.ALREADY_IN_CONTEXT``). _ALREADY_IN_CONTEXT_MARKER = "Already in your context" assert _ALREADY_IN_CONTEXT_MARKER in ALREADY_IN_CONTEXT, ( "mock drift: the dedupe marker left ALREADY_IN_CONTEXT" ) #: One ```` block of the HIGH prompt's ```` #: section (``app.rag.prompts.build_high_prompt``): the block is the #: document identity (``source``/``path``/``title`` attributes — plus, #: since phase 106 D5, the ``date`` attribute, the row's ``created_at`` #: UTC date part, APPENDED after ``title``) plus the document's #: SUMMARY (phase 118, A6 — the summary-seed contract: the seeded #: blocks are summaries, never full text; full text enters the context #: only through the capped ``read`` tool) between the tags. The #: ``date`` group is OPTIONAL so the mock tolerates the pre- and #: post-phase block shapes (house rule: the marker/regex lands with #: the prompt change). _DOCUMENT_BLOCK_RE = re.compile( r'\n(?P.*?)\n', re.S, ) def _document_block(system: str, source: str, path: str) -> str | None: """The stored text of one ```` block (or ``None``). The drill-down flow's ``ctx_answer`` branch: when the agent's read of a document ALREADY READ in the same turn gets the ALREADY_IN_CONTEXT dedupe (phase 118 — the seeds are summaries, so a first read of any document succeeds and only a re-read is refused), the mock (the model) extracts the block's text by the identity attributes and quotes it, answering from the prompt as the refusal instructs. The block carries the document's SUMMARY (phase 118 A6) — no suite exercises the branch today; it is kept for the still-real already-read refusal. """ for block in _DOCUMENT_BLOCK_RE.finditer(system): if block.group("source") == source and block.group("path") == path: return block.group("content") return None #: The agent's drill-down ``ls`` result shapes (app.rag.agent #: ``render_ls_top`` / ``render_folder_listing``, phase 94): the #: top-level header ``"N sources:"`` and the per-source block line #: ``" — N documents"`` (summary lines are indented — they #: never match); the folder-level header ``" — N documents, #: M folders:"`` (identity = the source name or ``source/folder``). _SOURCE_HEADER_RE = re.compile(r"^\d+ sources:$") _SOURCE_LINE_RE = re.compile(r"^(?P.+?) — (?P\d+) documents$") _FOLDER_HEADER_RE = re.compile(r"^(?P.+?) — \d+ documents, \d+ folders:$") #: One DEAD app-level chat attempt costs exactly this many HTTP POSTs #: while the endpoint stays down: the openai SDK's default policy #: (max_retries=2 — the app's ``LLMClient`` keeps it) re-POSTs a 500'd #: streaming request twice before surfacing the error to #: ``chat_stream_retried``. The failure counters below therefore count #: app-level attempts (groups of this size), not raw POSTs — the visible #: sequence (one SSE ``retry`` frame after each dead attempt, the answer #: on the third) stays deterministic regardless of the SDK's internal #: backoff pacing. _HTTPS_PER_DEAD_ATTEMPT = 3 #: Module-level failure counters — the mock is single-conversation per #: e2e server. Keyed by trigger phrase (reset per trigger): the number #: of matching POSTs served so far. Each sequence resets after the #: success it guards, so a second question carrying the same trigger #: re-drives the failure sequence from zero. _fail_posts: dict[str, int] = {} #: Phase 123 (task 04 — question images): a small ring buffer of the #: recent ``/v1/chat/completions`` request bodies, exposed on the #: ``/v1/e2e/captured`` pair below. Purely OBSERVATIONAL (the buffer #: never influences an answer — determinism is untouched): it lets a #: story suite assert on the EXACT request the app built — e.g. that a #: question-image turn delivered the multimodal user content list #: (text part + ``image_url`` data URL) to the model. _CAPTURED: list[dict[str, Any]] = [] _CAPTURE_MAX = 100 def _llm_500(why: str) -> JSONResponse: """A dead-proxy 500 with a JSON error body (phase 67 injection).""" return JSONResponse( status_code=500, content={ "error": { "message": f"upstream connection reset ({why})", "type": "proxy_error", } }, ) def _bump_fail(key: str) -> int: n = _fail_posts.get(key, 0) + 1 _fail_posts[key] = n return n def _chat_dead(key: str, dead_attempts: int) -> bool: """Bump *key*'s counter; True while the endpoint stays dead. Counted in app-level attempts (see ``_HTTPS_PER_DEAD_ATTEMPT``): the first ``dead_attempts * _HTTPS_PER_DEAD_ATTEMPT`` POSTs 500 and the next attempt's first POST streams (the caller resets the counter on the success). """ return _bump_fail(key) <= dead_attempts * _HTTPS_PER_DEAD_ATTEMPT def _partial_fail_stream() -> Any: """The phase-120 partial-then-fail stream (``PARTIAL_FAIL_TRIGGER``): ``PARTIAL_FAIL_TEXT`` in 12-char ``delta.content`` chunks (the mock's default cadence — NO ``finish_reason``, NO ``[DONE]``), then a RAISE that ends the SSE body abruptly (a connection reset mid-stream — the client SDK sees a truncated body, the app wraps it in ``LLMError`` after the pieces already emitted, and the chat endpoint's terminal ``error`` frame lands AFTER the partial deltas on the client's stream).""" model = "turbo" chunk_id = f"chatcmpl-{uuid.uuid4()}" for piece in re.findall(r".{1,12}", PARTIAL_FAIL_TEXT, re.S): payload = { "id": chunk_id, "object": "chat.completion.chunk", "created": int(time.time()), "model": model, "choices": [ {"index": 0, "delta": {"content": piece}, "finish_reason": None} ], } yield f"data: {json_dumps(payload)}\n\n" time.sleep(0.02) raise RuntimeError("e2e mid-stream failure (phase 120 partial-then-fail)") #: The agent's ``read`` tool-result prefix (app.rag.agent #: ``_execute_tool``): ``"Document :\n"``. _READ_RESULT_PREFIX = "Document " #: One line of the agent's ``ls`` output (app.rag.agent #: ``_execute_tool``, phase 63): labeled, pipe-delimited fields — #: ``source: X | path: Y | title: Z`` — unambiguous for LLM parsing even #: when the path contains ``/`` characters. _CATALOG_LINE_RE = re.compile( r"^source: (?P.+?) \| path: (?P.+?) \| title: .+$" ) def _read_results(body: dict[str, Any]) -> list[tuple[str, str]]: """The read results in the messages, in order: ``(source/path, content)``. A read result is a ``tool``-role message whose content starts with the agent's read-result prefix (``app.rag.agent`` ``_execute_tool``): ``"Document :\n"``. The header is stripped of the prefix AND the trailing colon so the path stays clean. """ out: list[tuple[str, str]] = [] for m in _messages(body): if m.get("role") != "tool": continue content = str(m.get("content") or "") if content.startswith(_READ_RESULT_PREFIX): header, _, doc_content = content.partition("\n") sp = header[len(_READ_RESULT_PREFIX):].strip().removesuffix(":") out.append((sp, doc_content)) return out def _catalog_docs(body: dict[str, Any]) -> list[tuple[str, str]]: """Every ``(source, path)`` in the catalog tool result, in listing order. Catalog lines are ``source: X | path: Y | title: Z`` (the agent's ``ls`` output — phase 63: labeled, pipe-delimited fields, unambiguous even for paths full of ``/``): the line-level regex recovers the ``source`` and ``path`` fields directly. The ``"N documents:"`` header line matches no line and is skipped; read-result messages are full documents, not listings, and are skipped too. """ docs: list[tuple[str, str]] = [] for m in _messages(body): if m.get("role") != "tool": continue content = str(m.get("content") or "") if content.startswith(_READ_RESULT_PREFIX): continue for line in content.splitlines(): match = _CATALOG_LINE_RE.match(line) if match: docs.append((match.group("source"), match.group("path"))) return docs def _tool_results(body: dict[str, Any]) -> list[str]: """Every ``tool``-role result content in the messages, in order. (Phase 72, LS-TEACH flow: the flow is discriminated statelessly from the tool results — a catalog listing vs the teaching refusal vs none yet.) """ return [ str(m.get("content") or "") for m in _messages(body) if m.get("role") == "tool" ] def _first_file_line(body: dict[str, Any]) -> str | None: """The first file line (``source: X | path: Y | title: Z``) across the ``tool``-role results in the messages, in message order (read results — full documents, not listings — skipped; the ``_CATALOG_- LINE_RE`` machinery). Phase 94: the drill-down ``ls`` carries file lines only at folder levels — the top-level source listing and the teaching refusals have none, so ``None`` here means "no folder level reached yet". """ for content in _tool_results(body): if content.startswith(_READ_RESULT_PREFIX): continue for line in content.splitlines(): if _CATALOG_LINE_RE.match(line): return line return None def _drill_target(body: dict[str, Any]) -> str | None: """The next source to drill into (phase 94, the drill step). The drill-down ``ls`` top level lists SOURCES only (no file lines), so a deterministic flow that needs a file line must drill one level: ``ls`` scoped to a source. The target is the FIRST source of the top-level listing (registry order — the listing's own order) that does not yet have a folder-level listing in the messages (a folder header whose identity is the source or ``source/…``); ``None`` when no top-level listing is in the messages yet (nothing to drill from) or every listed source has been drilled (an empty KB — the flow degenerates to the old re-list loop, settling at the round cap exactly like the phase-70 empty-catalog case). """ sources: list[str] = [] drilled: set[str] = set() for content in _tool_results(body): lines = content.splitlines() if not lines: continue if _SOURCE_HEADER_RE.match(lines[0]): for line in lines[1:]: match = _SOURCE_LINE_RE.match(line) if match: sources.append(match.group("source")) else: folder = _FOLDER_HEADER_RE.match(lines[0]) if folder: drilled.add(folder.group("identity")) for source in sources: if not any( identity == source or identity.startswith(source + "/") for identity in drilled ): return source return None #: One line of the agent's ``grep`` output (app.rag.agent #: ``_execute_tool``, phase 68 — phase 70 renamed the tool, the line #: format is unchanged): ``source/path:LINE: text``. The #: non-greedy prefix keeps nested paths (``/`` in the path) intact. _SEARCH_LINE_RE = re.compile(r"^(?P.+?):(?P\d+): (?P.*)$") def _search_result_line(body: dict[str, Any]) -> str | None: """The first matched line's text of a search result in the messages. A search result is a ``tool``-role message — never a read result (those start with the agent's ``"Document "`` prefix) — that either carries ``source/path:LINE: text`` match lines (the agent's ``grep`` output, phase 68) or the sentinel pattern itself (its no-match line quotes the pattern). Returns the first match line's ``text`` part (already 200-char-capped server-side), or the message's first line in the sentinel-only shape, or ``None`` when no search result is in the messages yet. """ sentinel = SEARCH_PATTERN.lower() for m in _messages(body): if m.get("role") != "tool": continue content = str(m.get("content") or "") if content.startswith(_READ_RESULT_PREFIX): continue for line in content.splitlines(): match = _SEARCH_LINE_RE.match(line) if match: return match.group("text") if sentinel in content.lower(): lines = content.splitlines() return lines[0] if lines else "" return None def _search_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a SEARCH_TRIGGER request into a step of the search flow. * ``("search",)`` — ``tools`` are offered and no search result is in the messages yet: the model greps the whole KB for ``SEARCH_PATTERN`` (id ``call_0``). * ``("found", first_line)`` — a ``tool``-role search result is in the messages: the model answers, quoting the first matched line (``Found ``). Reached regardless of the ``tools`` parameter (phase 45 keeps the tools offered until the round cap). * ``None`` — not the search flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no search result is in the messages yet (e.g. ``agent_max_rounds=0``). """ if SEARCH_TRIGGER not in _user(body).lower(): return None if "" not in _system(body): return None first_line = _search_result_line(body) if first_line is not None: return ("found", first_line) if not body.get("tools"): return None return ("search",) def _scoped_search_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a SCOPED_SEARCH_TRIGGER request into a step. Phase 146 (task 02): the SCOPED search flow — the owner's "looks like the LLM is repeating itself" scenario (TODO.md L3): a re-search of ONE document (``grep(pattern, path)``) so the UI can show WHERE the search runs. The same scripted-turn idiom as ``_search_flow`` (the phase-70/72 grep branches): stateless classification from the messages (streaming only). * ``("search",)`` — ``tools`` are offered and no search result is in the messages yet: the model greps the ONE fixture document (``SCOPED_SEARCH_PATH``) for ``SEARCH_PATTERN`` (id ``call_0``) — the scoped form (``pattern`` + ``path``). * ``("found", first_line)`` — a ``tool``-role search result is in the messages (the agent's REAL scoped grep matched the fixture's sentinel line): the model answers, quoting the first matched line (``Found ``). Reached regardless of the ``tools`` parameter (phase 45 keeps the tools offered until the round cap). * ``None`` — not the scoped flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no search result is in the messages yet (e.g. ``agent_max_rounds=0``). """ if SCOPED_SEARCH_TRIGGER not in _user(body).lower(): return None if "" not in _system(body): return None first_line = _search_result_line(body) if first_line is not None: return ("found", first_line) if not body.get("tools"): return None return ("search",) def _scaffold_flow(body: dict[str, Any]) -> str | None: """Classify a phase-71 scaffolding request (see the module docstring). * ``"scaffold"`` — stream ONLY the incident span (``SCAFFOLD_SPAN``) as ``delta.content`` chunks: ``finish_reason: "stop"``, no structured ``tool_calls``, no reasoning. EVERY request for ``SCAFFOLD_ALWAYS_TRIGGER`` (the recovery included), and the FIRST request of ``SCAFFOLD_TRIGGER`` (no correction in the system prompt yet). * ``"recovery"`` — ``SCAFFOLD_TRIGGER`` whose system prompt carries the harness correction (the one bounded recovery: ``tools=None``, the constant folded into the single system prompt by ``app.api.chat`` / ``app.rag.agent``): stream the clean ``SCAFFOLD_RECOVERY_ANSWER``. * ``None`` — not the scaffolding flow. The discrimination is stateless, like the other marker flows: the trigger phrase in the user message plus the correction's presence in the system prompt. """ user = _user(body).lower() if SCAFFOLD_ALWAYS_TRIGGER in user: # checked FIRST — it contains SCAFFOLD_TRIGGER return "scaffold" if SCAFFOLD_TRIGGER in user: if _CORRECTION_MARKER in _system(body): return "recovery" return "scaffold" return None def _tool_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a marker request into one step of the tool flow. Single-read (phase 37 — the user message carries ``TOOLS_TRIGGER`` only): * ``("list", "", "")`` — ``tools`` are offered and no tool results are in the messages yet: the model lists the catalog. * ``("read", source, path, "call_1")`` — a ``tool``-role catalog result is in the messages: the model reads its FIRST ``source: X | path: Y | title: Z`` line (the labeled ``source:`` / ``path:`` fields, phase 63), emitted as ``read`` on the JOINED combined ``source/path`` (phase 70: the mock joins the two fields — the canonical document identity). * ``("answer", "source/path", content)`` — a ``tool``-role read result (``"Document :\n"``) is in the messages: the model answers, quoting the read document. Reached regardless of the ``tools`` parameter (phase 45 keeps the tools offered until the round cap). Multi-read (phase 45 task 02 — the user message carries BOTH ``TOOLS_TRIGGER`` and ``MULTI_READ_TRIGGER``), classified by the count of ``tool``-role read results: * 0 read results: ``("list", "", "")`` (no listing yet), ``("drill", source, "call_1")`` when the top-level source listing is in the messages but no file line yet (phase 94: the drill — the top level carries sources only), or ``("read", source, path, "call_2")`` on the FIRST file-line doc. * 1 read result: ``("read", source, path, "call_3")`` on the SECOND file-line doc — the first listing line whose ``source/path`` differs from the one already read. A one-file listing degenerates to the single-read ``("answer", ...)`` shape (nothing second to read). * 2 read results: ``("multi_answer", "", text)`` — the forced answer, byte-stable: the single-read shape quoting the FIRST read result, plus ``I read and .`` (both read paths, read order). The second element is unused. * ``None`` — not the marker flow: the request behaves exactly as before (marker absent, no ```` section, or a no-tools request with no tool results — e.g. ``agent_max_rounds=0``). """ user = _user(body).lower() if TOOLS_TRIGGER not in user: return None if "" not in _system(body): return None reads = _read_results(body) if MULTI_READ_TRIGGER in user: if not reads: if not body.get("tools"): return None docs = _catalog_docs(body) if docs: return ("read", docs[0][0], docs[0][1], "call_2") drill = _drill_target(body) if drill is not None: return ("drill", drill, "call_1") return ("list", "", "") if len(reads) == 1: skip = reads[0][0] second = next( (d for d in _catalog_docs(body) if f"{d[0]}/{d[1]}" != skip), None ) if second is None: # One-file listing: nothing second to read — the # single-read answer shape (deterministic degenerate). return ("answer", reads[0][0], reads[0][1]) return ("read", second[0], second[1], "call_3") (sp1, c1), (sp2, _c2) = reads[0], reads[1] answer = f"Read {sp1}. {c1[:80]} I read {sp1} and {sp2}." return ("multi_answer", "", answer) # Phase-37 single-read flow (phase 94: the drill step between the # top-level listing and the first file line). if reads: return ("answer", reads[0][0], reads[0][1]) if not body.get("tools"): return None docs = _catalog_docs(body) if docs: return ("read", docs[0][0], docs[0][1], "call_2") drill = _drill_target(body) if drill is not None: return ("drill", drill, "call_1") return ("list", "", "") def _ls_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-72 LS-TEACH request (see the module docstring). * ``("misuse",)`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the incident's misuse — ``ls`` with ``{"path": "."}`` (id ``call_0``), ``finish_reason: "tool_calls"``, no content. * ``("correct",)`` — a ``tool``-role result is in the messages and no file line exists yet (the teaching refusal): the correction — ``ls`` with no arguments (id ``call_1``). * ``("drill", source, "call_2")`` — the top-level source listing is in the messages (the correction ran) but no file line yet (phase 94: the top level carries sources only): the drill — ``ls`` scoped to the first source of the listing. * ``("answer", line)`` — a ``source: X | path: Y | title: Z`` file line is in the messages (the folder listing): the deterministic content answer ``These are the indexed documents: `` (the first file line), ``finish_reason: "stop"`` — the loop settled in ONE correction + ONE drill, not at the round cap. * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ if LS_TEACH_TRIGGER not in _user(body).lower(): return None if "" not in _system(body): return None line = _first_file_line(body) if line is not None: return ("answer", line) drill = _drill_target(body) if drill is not None: return ("drill", drill, "call_2") if _tool_results(body): return ("correct",) if not body.get("tools"): return None return ("misuse",) def _grep_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a GREP-TEACH request (the 2026-09-05 incident — see the ``GREP_TEACH_*`` constants). Stateless over the messages, like the other marker flows: * ``("regex",)`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the incident's regex-shaped first grep — ``grep`` with ``{"pattern": GREP_TEACH_PATTERN}`` (id ``call_0``). * ``("plain",)`` — the LAST tool result is the server's TEACHING no-match line (it carries ``GREP_TEACH_MARKER``): the one-round correction — ``grep`` with the plain form (id ``call_1``). * ``("read", combined, "call_2")`` — the last tool result carries ``source/path:line: text`` match lines: ``read`` the FIRST match line's document by its combined ``source/path`` identity. * ``("answer", combined, content)`` — the last tool result is a read result (``"Document :\n"``): the deterministic echo answer ``Read . ``. * ``("nomatch",)`` — the last tool result is a PLAIN no-match (no match line, no teaching marker): the deterministic terminal ``No matches — the knowledge base has no such text.`` answer. * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body).lower() if GREP_TEACH_TRIGGER not in user: return None if "" not in _system(body): return None results = [ str(m.get("content") or "") for m in _messages(body) if m.get("role") == "tool" ] if not results: if not body.get("tools"): return None return ("regex",) last = results[-1] if last.startswith(_READ_RESULT_PREFIX): head, _, content = last.partition("\n") # The read result is ``"Document :\n"`` — the # head carries the server's appended ``:`` (removed here; a # document path never legitimately ends with one). return ("answer", head[len(_READ_RESULT_PREFIX):].removesuffix(":"), content) if GREP_TEACH_MARKER in last: return ("plain",) for line in last.splitlines(): m = _SEARCH_LINE_RE.match(line) if m: return ("read", m.group("sp"), "call_2") return ("nomatch",) def _drill_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-94 scripted drill-down request (see the module docstring). The question carries the scripted call (``drill down the tree: ls [target]`` / ``drill down the tree: read source/path``); the step is then discriminated statelessly from the tool results, like the other marker flows: * ``("call", verb, target, "call_0")`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the scripted call (``ls`` with NO path for the top level — empty target —, ``ls`` with the target for source/folder levels, ``read`` with the combined ``source/path``). * ``("recover", source)`` — the LAST tool result is the NOT-A-FOLDER teaching refusal (it carries :data:`_NOT_A_FOLDER_MARKER`): the scripted one-round recovery — ``ls`` the target's SOURCE segment (the refusal listing the parent's subfolders is what lets the model — and this mock — self-correct in the next round, the phase-72 contract). * ``("read_answer", combined, quote)`` — the LAST tool result is a read result (``"Document :\n"``): the deterministic echo answer ``Read . `` (the phase-37 single-read shape — the grounded-turn citation contract the suite asserts). * ``("ctx_answer", target, quote)`` — the scripted call was a ``read`` and the LAST tool result is the ALREADY_IN_CONTEXT dedupe refusal (the target is already a top-2 retrieval document — deterministic for the drill fixture: the read question names the file's path, so the file self-matches the hybrid gate): the model answers FROM THE PROMPT — ``Already in context: Read . `` block>`` (the same citation shape as the read-result branch — the document text reached the model either way). Falls through to the echo when the target is not a ```` block (the premise broke — the suite fails loudly on the answer). * ``("echo", listing)`` — a listing landed (top-level or folder — any other last result): the deterministic ECHO — the answer carries the listing VERBATIM (``Here's the level I listed:\n ``), so the suite asserts on the exact tree level the model saw (source ``— N documents`` lines + stored summaries, subfolder lines, the ``source: X | path: Y | title: Z`` file lines, the 50-line cap + note) through the rendered answer — the house scripted-turn way of asserting on tool results (the mock is the only E2E lens on the LLM's context). * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the scripted call is unparseable, or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if DRILL_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _DRILL_CALL_RE.search(user) if match is None: return None verb = match.group("verb") target = match.group("arg") or "" results = _tool_results(body) if not results: if not body.get("tools"): return None return ("call", verb, target, "call_0") last = results[-1] if last.startswith(_READ_RESULT_PREFIX): head, _, content = last.partition("\n") # The read result is ``"Document :\ndate: …\n"`` # — the head carries the server's appended ``:`` (removed here; # a document path never legitimately ends with one); the # phase-106 D5 ``date:`` line sits between the header and the # document text — skipped so the quote stays pure document # content (the suite's byte-identical pin). combined = head[len(_READ_RESULT_PREFIX):].strip().removesuffix(":") if content.startswith("date: "): content = content.partition("\n")[2] return ("read_answer", combined, content[:80]) if _ALREADY_IN_CONTEXT_MARKER in last and verb == "read" and "/" in target: # Phase 118: the dedupe fires only for a document ALREADY READ # in the same turn (the seeds are summaries — a first read of # a suggested document succeeds). Answer from the prompt (the # refusal's instruction), quoting the block's text (now the # document's summary). src, _, p = target.partition("/") content = _document_block(_system(body), src, p) if content is not None: return ("ctx_answer", target, content[:80]) if _NOT_A_FOLDER_MARKER in last: return ("recover", target.split("/")[0]) return ("echo", last) # --------------------------------------------------------------------------- # Phase 95 (task 03, the read cap's dedicated story suite): # the deterministic SCRIPTED capped read — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the scripted CAPPED-READ flow (the read cap's story suite, #: ``tests/e2e/test_read_truncation_cap.py``): the question carries its #: own tool call after the colon — ``read the capped document: read #: source/path`` — the mock emits the scripted ``read``, then ECHOES #: the ENTIRE tool result into its answer (the house scripted-turn lens #: on the LLM's context: the truncated shape — first cap chars + #: ``[…truncated…]`` + the pinned grep-pointer notice — or the plain #: shape, byte-identical to the pre-phase-95 result, lands in the #: rendered answer, and the suite asserts both directions through it). #: Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger #: phrases — the phase-71/72/94 ordering convention); verified #: 2026-09-10: no existing E2E question or fixture file contains the #: phrase, so every other suite is unaffected. READ_CAP_TRIGGER = "read the capped document" #: The scripted call in the read-cap question (case-insensitive — the #: suite's questions capitalize the trigger's first letter): the verb #: (``read``) plus the target — a combined ``source/path``, parsed from #: the RAW user message so the target keeps its case. The target is a #: ``[a-z0-9_./-]`` run (case-insensitively), so the suite's `` — `` #: flavor separator (em dash) can never bleed into it (the drill-down #: convention, ``_DRILL_CALL_RE``). _READ_CAP_CALL_RE = re.compile( r"read the capped document:\s*read\s+(?P[a-z0-9_./-]+)", re.I, ) def _read_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-95 scripted read-cap request (see the module docstring). The question carries the scripted call (``read the capped document: read source/path``); the step is then discriminated statelessly from the tool results, like the other marker flows: * ``("call", target, "call_0")`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the scripted ``read`` on the parsed target (synthetic id ``call_0``). * ``("echo", result)`` — a ``tool``-role result is in the messages: the deterministic ECHO — the answer carries the LAST tool result VERBATIM (``Here's what the read returned:\n``): a read result (``"Document :…``) lands in the answer with its full content — the truncation marker + the pinned notice when the cap fired, the plain body when it did not — and a refusal (the premise broke) lands just as visibly, so the suite fails loudly on it. * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the scripted call is unparseable, or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if READ_CAP_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _READ_CAP_CALL_RE.search(user) if match is None: return None results = _tool_results(body) if not results: if not body.get("tools"): return None return ("call", match.group("arg"), "call_0") return ("echo", results[-1]) # --------------------------------------------------------------------------- # Phase 118 (task 06, the summary-seed context's dedicated story suite): # the deterministic SCRIPTED summary read — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the scripted SUMMARY-READ flow (the summary-seed context's story #: suite, ``tests/e2e/test_summary_seed_context.py``): the question #: carries its own tool call after the colon — ``read the suggested #: document: read source/path`` — the mock emits the scripted ``read`` #: (phase 118: the seeds are summaries, so a first read of a suggested #: document SUCCEEDS — the full text arrives through the read), then #: ECHOES the ENTIRE tool result into its answer (the house #: scripted-turn lens on the LLM's context — the full content's tail #: reaches the rendered answer iff the read delivered it). Checked #: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — #: the phase-71/72/94 ordering convention); no existing E2E question #: or fixture file contains the phrase, so every other suite is #: unaffected. SUMMARY_SEED_READ_TRIGGER = "read the suggested document" #: The scripted call in the summary-read question (case-insensitive — #: the suite's questions capitalize the trigger's first letter): the #: verb (``read``) plus the target — a combined ``source/path``, parsed #: from the RAW user message so the target keeps its case. The target #: is a ``[a-z0-9_./-]`` run (case-insensitively), so the suite's #: `` — `` flavor separator (em dash) can never bleed into it (the #: read-cap/drill convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``). _SUMMARY_SEED_READ_CALL_RE = re.compile( r"read the suggested document:\s*read\s+(?P[a-z0-9_./-]+)", re.I, ) # --------------------------------------------------------------------------- # Phase 124 (task 07, image chips and answers, TODO L3): the scripted # SHOW_IMAGE flow + the zero-read bug-fix trigger — see the module # docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ``show_image`` prompt clause in the system #: prompt (the ``BOR_ANSWER_IMAGES`` toggle's server-side marker — #: the clause rides inside the HIGH prompt's ```` section ONLY #: when the toggle is on) — drives the deterministic SHOW_IMAGE flow #: (the mock shows the seeded image document and answers straight #: from the seed — ZERO ``read`` calls). On a default-OFF app the #: clause is absent, so the trigger NEVER fires there and the mock #: behaves exactly as pre-phase for every other suite. Disjoint from #: every other trigger phrase — the phase-71/72/94/118 ordering #: convention; verified 2026-09-26: no existing E2E question or #: fixture file contains the phrase, so every other suite is #: unaffected. SHOW_IMAGE_TRIGGER = "show me the picture of" #: The phase-124 BUG-FIX-ONLY trigger (case-insensitive): the mock #: answers straight from the seed with ZERO tool calls and QUOTES THE #: TAIL of the seeded image doc's ```` block — proving the #: description was in the prompt (for an image doc the seed block IS #: its entire content) and that its chip appears with NO tool calls #: at all (the phase-124 auto-cite). Unlike ``SHOW_IMAGE_TRIGGER`` #: it needs NO clause — the chips fix is not gated by #: ``BOR_ANSWER_IMAGES`` (it belongs to the ``BOR_IMAGES``-indexed #: docs), so it also fires on a default-OFF app. Checked BEFORE the #: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker #: lives in the user message — the TABLE/HISTORY convention), so a #: marker question always gets the echo whatever the honesty gate #: says. DIAGRAM_TRIGGER = "which diagram is in your notes" #: The phase-125 (task 06, image-anchor context, LOCKED A4) trigger #: (case-insensitive): the mock QUOTES THE CONTEXT LINE of the seeded #: image doc's ```` block (the same image-doc pick as #: :data:`DIAGRAM_TRIGGER`). For an ENRICHED image doc the block #: leads with the one-line ``Referenced in /: ""`` #: line (the task-05 rendering of the stored ``image_context``, #: BEFORE the description) — quoting the block's first line proves #: the line reached the prompt; for a non-enriched (standalone) doc #: the block is the bare description (byte-identical pre-phase) and #: its first line never starts with ``Referenced in `` (the mock's #: fixed description cannot begin that way) — the mock then answers #: that the image carries no reference line. Checked BEFORE the #: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker #: lives in the user message — the TABLE/HISTORY convention), so a #: marker question always gets the echo whatever the honesty gate #: says. Disjoint from every other trigger phrase; verified #: 2026-09-27: no existing E2E question or fixture file contains the #: phrase, so every other suite is unaffected. CONTEXT_LINE_TRIGGER = "quote the image reference" #: The agent's ``show_image`` SUCCESS result's confirming first line #: (``app.rag.agent``'s ``show_image`` branch — the line ``read``'s #: result never carries): the SHOW_IMAGE flow's follow-up request #: keys on it in a ``tool``-role result (stateless discrimination — #: the house marker-flow convention). _SHOW_CONFIRM_MARKER = "Image shown to the user" #: The image-doc extension family (``app.rag.summarizer.IMAGE_MIMES`` #: — phase 122's ext→mime map, imported so the mock can never drift #: from the app's image family): a seeded ```` block is #: the IMAGE document when its ``path`` attribute ends in one of #: these (the suite's single-PNG fixture guarantees exactly one such #: block — the pick is deterministic). _IMAGE_PATH_SUFFIXES = tuple(IMAGE_MIMES) def _seeded_image_doc(system: str) -> tuple[str, str, str] | None: """The seeded image doc's ``(source, path, content)`` — or ``None``. Phase 124 (task 07): the ```` block (the :data:`_DOCUMENT_BLOCK_RE` machinery) whose ``path`` attribute ends in an image extension (:data:`_IMAGE_PATH_SUFFIXES`) is the seeded image document — for an image doc the block's content IS the document's entire text (the vision description — content == summary, phase 122 task 03). Both of this phase's triggers pick the doc this way (the suite's single-PNG fixture guarantees exactly one such block; ``None`` = a text-only KB, the premise broke — the caller degrades loudly). """ for block in _DOCUMENT_BLOCK_RE.finditer(system): if block.group("path").lower().endswith(_IMAGE_PATH_SUFFIXES): return (block.group("source"), block.group("path"), block.group("content")) return None def _seed_context_line(block_content: str) -> str | None: """The context line of a seeded image doc's block (phase 125 task 06, LOCKED A4; phase 137 task 04/05) — or ``None``. An ENRICHED image doc's ```` block leads with the one-line context rendering (the task-05/task-04 :func:`app.rag.prompts._extract_context_line` output, BEFORE the description): the one-line ``Referenced in …`` reference shape (phase 125) or the one-line ``From / (page N)`` provenance shape (phase 137 — a PDF-embedded image, whose block leads with the document-context clause, a combined context joining both with ``; ``). Its first line, verbatim, is the line the mock quotes (the prompt-injection echo convention). A non-enriched block is the bare description — its first line never starts with ``Referenced in `` or ``From `` (the mock's fixed :data:`IMAGE_DESCRIPTION_ANSWER` begins "A network diagram" — the context-aware :data:`IMAGE_DESCRIPTION_CONTEXT_ANSWER` begins "From " but only ever rides a doc whose stored context makes the ``From …`` CONTEXT LINE the block's first line, so the bare description is never the first line for it), so ``None`` pins the byte-identical pre-phase shape. """ first = block_content.split("\n", 1)[0].strip() if first.startswith("Referenced in ") or first.startswith("From "): return first return None def _show_image_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-124 SHOW_IMAGE request (see the module docstring). Stateless over the messages, like the other marker flows: * ``("call", combined)`` — the system prompt carries the ``show_image`` clause (the toggle-ON gate) and no ``tool``-role result has confirmed the show yet: stream ONLY the ``show_image`` tool-call delta on the seeded image doc's combined ``source/path`` (synthetic id ``call_show``, ``finish_reason: "tool_calls"``, no content) — NO ``read`` (the zero-read turn). * ``("answer",)`` — a ``tool``-role result carrying the agent's confirming line (:data:`_SHOW_CONFIRM_MARKER`) is in the messages: the deterministic FINAL answer — a short sentence ending with the ```` block's tail echo (the phase-24 END_OF_NOTES echo convention — it proves the seed was there), NO further tool calls. * ``None`` — not the flow: the trigger phrase is absent, the clause is absent (a default-OFF app — the mock behaves exactly as pre-phase), the turn is deflected (no ```` seed — a deflection prompt never carries one), or no seeded image doc exists (a text-only KB — the premise broke; the suite fails loudly on the missing figure). """ if SHOW_IMAGE_TRIGGER not in _user(body).lower(): return None system = _system(body) if "show_image" not in system: return None doc = _seeded_image_doc(system) if doc is None: return None if any(_SHOW_CONFIRM_MARKER in r for r in _tool_results(body)): return ("answer",) if not body.get("tools"): return None return ("call", f"{doc[0]}/{doc[1]}") def _summary_seed_read_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-118 scripted summary-read request (see the module docstring). The question carries the scripted call (``read the suggested document: read source/path``); the step is then discriminated statelessly from the tool results, like the other marker flows: * ``("call", target, "call_0")`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the scripted ``read`` on the parsed target (synthetic id ``call_0``). * ``("echo", result)`` — a ``tool``-role result is in the messages: the deterministic ECHO — the answer carries the LAST tool result VERBATIM (``Here's what the read returned:\n``): a read result (``"Document :…`` — header + date line + FULL content) lands in the answer with its tail intact (the full text the ``read`` tool delivered — the phase-118 contract: a first read of a suggested document succeeds, the seed was a summary); a refusal (the premise broke) lands just as visibly, so the suite fails loudly on it. * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the scripted call is unparseable, or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if SUMMARY_SEED_READ_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _SUMMARY_SEED_READ_CALL_RE.search(user) if match is None: return None results = _tool_results(body) if not results: if not body.get("tools"): return None return ("call", match.group("arg"), "call_0") return ("echo", results[-1]) # --------------------------------------------------------------------------- # Phase 131 (task 04, the read line-ranges' dedicated story suite): # the deterministic grep -> read(offset/limit) dance — see the module # docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the deterministic RANGE-READ flow (the read line-ranges' story #: suite, ``tests/e2e/test_read_line_ranges.py``): the owner's stuck #: scenario (TODO.md L7) — the huge document: ``grep`` finds the #: line, the ``read`` lands exactly that part with ``offset`` / #: ``limit``, and the answer echoes the read slice. Checked BEFORE #: the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the #: phase-71/72/94/118 ordering convention); verified 2026-09-28: no #: existing E2E question or fixture file contains the phrase, so #: every other suite is unaffected. READ_LINES_TRIGGER = "read the lines" #: The canned pattern the flow greps for (the suite's fixture document #: carries exactly one line containing it — the flow's read target #: line; the sentinel convention of ``SEARCH_PATTERN``). READ_LINES_PATTERN = "LINE_MARKER" #: The line count the flow's range read requests (a PART of the #: document, not the whole — the owner's dance, TODO.md L7). READ_LINES_LIMIT = 30 def _read_call_offset(body: dict[str, Any]) -> int: """The offset the model's ``read`` call passed (phase 131, task 04). The agent loop echoes every executed call back into the messages (the assistant ``tool_calls`` message, the arguments as a JSON string — ``app/rag/agent.py``), so the range-read request (request 3 of the flow) carries the call it was made from: this parses the FIRST assistant ``read`` call's ``offset``. ``0`` when absent or unparseable (the premise broke — the suite fails loudly on ``line 0``). """ for m in _messages(body): if m.get("role") != "assistant": continue # The agent loop's echoed call (the ``tool_calls`` LIST on the # assistant message — ``app/rag/agent.py``; :func:`_messages` # is typed for string content, so the list comes back as # ``Any`` here — the house mock convention over the wire # shape). raw_calls: Any = m.get("tool_calls") for call in (raw_calls if isinstance(raw_calls, list) else []): if not isinstance(call, dict): continue fn = call.get("function") or {} if not isinstance(fn, dict) or fn.get("name") != "read": continue try: args = json.loads(fn.get("arguments") or "{}") except json.JSONDecodeError: continue offset = args.get("offset") if isinstance(args, dict) else None if isinstance(offset, int) and not isinstance(offset, bool): return offset return 0 def _read_lines_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-131 range-read request (see the module docstring). Classified statelessly by the tool results in the messages (the house count pattern), like the other marker flows: * ``("answer", offset, quote)`` — a ``tool``-role read result (content starting with the agent's ``"Document "`` prefix) is in the messages: the forced answer, byte-stable — ``Found it at line N. `` with N the offset the model passed (parsed from the read call's arguments in the assistant tool-call message — a string on this return for the homogeneous ``tuple[str, ...]`` shape; the dispatch formats it into the answer, the display surface) and quote the read result's content with the header lines (the phase-106 D5 ``date:`` line + the phase-131 ``lines: a-b of N`` extent) skipped — the pure read slice, first 80 chars (the phase-94 read-echo convention: the RANGE reached the model's context). * ``("read", combined, line, "call_1")`` — a ``tool``-role grep result carrying ``source/path:line: text`` match lines is in the messages: the RANGE READ on the FIRST match line's combined ``source/path`` — ``offset`` = the parsed line number (the grep's own string here — the dispatch parses the int for the wire) and ``limit`` = :data:`READ_LINES_LIMIT`. * ``("nomatch",)`` — a tool result is in the messages but it is neither a read result nor a grep match (the grep found nothing — the premise broke): the deterministic terminal answer (the grep-teach ``nomatch`` line — the suite fails loudly on the missing marker). * ``("grep",)`` — ``tools`` are offered and no tool results are in the messages yet: the first step — ``grep`` with ``{"pattern": READ_LINES_PATTERN}`` (id ``call_0``). * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ if READ_LINES_TRIGGER not in _user(body).lower(): return None if "" not in _system(body): return None reads = _read_results(body) if reads: _sp, content = reads[-1] # The read result's header lines sit between the ``Document # …:`` header and the document text — the phase-106 D5 # ``date:`` line always, the phase-131 ``lines: a-b of N`` # extent on a range read (a plain read carries no extent # line) — skipped (the drill-flow convention) so the quote is # the pure read content (the suite's marker-line assertion). if content.startswith("date: "): content = content.partition("\n")[2] if content.startswith("lines: "): content = content.partition("\n")[2] return ("answer", str(_read_call_offset(body)), content[:80]) for result in _tool_results(body): if result.startswith(_READ_RESULT_PREFIX): continue for line in result.splitlines(): match = _SEARCH_LINE_RE.match(line) if match: return ( "read", match.group("sp"), match.group("line"), "call_1", ) if _tool_results(body): return ("nomatch",) if not body.get("tools"): return None return ("grep",) # --------------------------------------------------------------------------- # Phase 139 (task 03): the owner's "same file over and over" scenario — # the model range-reads the SAME document TWICE with different # offset/limit, so the two Reading lines are distinguishable. See the # module docstring. # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the deterministic SAME-FILE-TWICE flow (the read range-tool-lines' #: story suite, ``tests/e2e/test_read_range_tool_lines.py`` — the #: owner's "same file over and over" confusion, TODO.md L4). Checked #: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — #: the phase-71/72/94/118/131 ordering convention); verified #: 2026-09-30: no existing E2E question or fixture file contains the #: phrase, so every other suite is unaffected. SAME_FILE_TRIGGER = "read the same file twice" #: The scripted target in the same-file-twice question (case- #: insensitive — the suite's question capitalizes the trigger's first #: letter): the verb (``read``) plus the target — a combined #: ``source/path``, parsed from the RAW user message so the target #: keeps its case. The target is a ``[a-z0-9_./-]`` run (case- #: insensitively), so a `` — `` flavor separator can never bleed into #: it (the read-cap / drill-down convention, ``_READ_CAP_CALL_RE`` / #: ``_DRILL_CALL_RE``). _SAME_FILE_CALL_RE = re.compile( r"read the same file twice:\s*read\s+(?P[a-z0-9_./-]+)", re.I, ) #: The FIRST range read's requested extent — the model's REQUEST #: (ASSUMPTION A1: the suffix shows the request, not the actual slice #: extent). ``offset`` 10 + ``limit`` 20 renders `` (lines 10–29)``. SAME_FILE_FIRST_OFFSET = 10 SAME_FILE_FIRST_LIMIT = 20 #: The SECOND range read's requested extent — the SAME file again, a #: different range (the owner's confusion made visible). ``offset`` 40 #: + ``limit`` 15 renders `` (lines 40–54)``. SAME_FILE_SECOND_OFFSET = 40 SAME_FILE_SECOND_LIMIT = 15 def _same_file_quote(doc_content: str) -> str: """The phase-94/131 read-echo quote of ONE read result's content. The read result's header lines sit between the ``Document …:`` header and the document text — the phase-106 D5 ``date:`` line always, the phase-131 ``lines: a-b of N`` extent on a range read (a plain read carries no extent line) — skipped (the drill-flow convention) so the quote is the pure read content, first 80 chars (the RANGE reached the model's context). Mirrors the phase-131 ``_read_lines_flow`` quote logic so the two flows' echo shapes stay byte-consistent. """ content = doc_content if content.startswith("date: "): content = content.partition("\n")[2] if content.startswith("lines: "): content = content.partition("\n")[2] return content[:80] def _same_file_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-139 same-file-twice request (see the module docstring). The question carries its own target (``read the same file twice: read source/path``); the step is then discriminated statelessly by the READ results in the messages (the house count pattern — the flow only ever issues ``read`` calls, so counting read results is reliable): * ``("answer", quote1, quote2)`` — TWO or more ``tool``-role read results are in the messages (both range reads landed): the forced answer, byte-stable, in the phase-131 read-echo shape — ``Read the same file twice — lines 10–29: ; lines 40–54: `` where each ```` is that read result's pure content (header lines skipped, first 80 chars — the :func: ``_same_file_quote`` convention). * ``("read2", target)`` — EXACTLY ONE read result is in the messages (the first read landed): the SECOND range read on the SAME target — ``offset`` = :data:`SAME_FILE_SECOND_OFFSET`, ``limit`` = :data:`SAME_FILE_SECOND_LIMIT` (id ``call_1``). * ``("read1", target)`` — ``tools`` are offered and no read result is in the messages yet: the FIRST range read on the parsed target — ``offset`` = :data:`SAME_FILE_FIRST_OFFSET`, ``limit`` = :data:`SAME_FILE_FIRST_LIMIT` (id ``call_0``). * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the target is unparseable, or ``tools`` are not offered and no read result is in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if SAME_FILE_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _SAME_FILE_CALL_RE.search(user) if match is None: return None target = match.group("arg") reads = _read_results(body) if len(reads) >= 2: return ( "answer", _same_file_quote(reads[0][1]), _same_file_quote(reads[1][1]), ) if len(reads) == 1: return ("read2", target) if not body.get("tools"): return None return ("read1", target) # --------------------------------------------------------------------------- # Phase 141 (task 04): the owner's stuck scenario — the model # range-reads ONE document in TWO different ranges; the # coverage-tracked server proceeds with the second (the pre-phase # pair-set dedupe refused it ``ALREADY_IN_CONTEXT`` while only the # first slice was in the prompt). See the module docstring. # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the deterministic TWO-RANGES flow (the read range-dedupe fix's #: story suite, ``tests/e2e/test_read_range_dedupe.py`` — the #: owner's stuck scenario, TODO.md L3). Checked BEFORE the plain #: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the #: phase-71/72/94/118/131/139 ordering convention); verified #: 2026-10-01: no existing E2E question or fixture file contains #: the phrase, so every other suite is unaffected. TWO_RANGES_TRIGGER = "read two ranges" #: The scripted target in the two-ranges question (case- #: insensitive — the suite's question capitalizes the trigger's #: first letter): the verb (``read``) plus the target — a combined #: ``source/path``, parsed from the RAW user message so the target #: keeps its case (the ``_SAME_FILE_CALL_RE`` convention; the target #: is a ``[a-z0-9_./-]`` run, so a `` — `` flavor separator can #: never bleed into it). _TWO_RANGES_CALL_RE = re.compile( r"read two ranges:\s*read\s+(?P[a-z0-9_./-]+)", re.I, ) #: The FIRST range read's requested extent — range A (lines 1–100 #: of the suite's 200-line fixture document). TWO_RANGES_FIRST_OFFSET = 1 TWO_RANGES_FIRST_LIMIT = 100 #: The SECOND range read's requested extent — range B (lines 101– #: …): the SAME file again, a different range. Pre-phase the server #: refused this call ``ALREADY_IN_CONTEXT`` (the false "full text is #: in your prompt" claim — the owner's stuck turn); the coverage- #: tracked server proceeds with it (its lines are NOT covered by #: the first read's span). TWO_RANGES_SECOND_OFFSET = 101 TWO_RANGES_SECOND_LIMIT = 100 def _two_ranges_extent(content: str) -> str | None: """The ``lines: a-b of N`` extent line of one read result's content (the ``Document …:`` header already stripped by :func:`_read_results`, the phase-106 D5 ``date:`` line skipped) — ``None`` when the result carries no extent line (a plain read). The flow keys its step on this line (the mock's own keying, asserted — a refusal or a differently-shaped result is the broken premise, never a step input). """ body = content if body.startswith("date: "): body = body.partition("\n")[2] if body.startswith("lines: "): return body.partition("\n")[0] return None def _two_ranges_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-141 two-ranges request (see the module docstring). The question carries its own target (``read two ranges: read source/path``); the step is then discriminated statelessly by the tool results in the messages (the house count pattern — the flow only ever issues ``read`` calls, so counting read results is reliable): * ``("answer", quote)`` — TWO or more ``tool``-role read results are in the messages and the SECOND carries the ``lines: 101- …`` extent header (asserted — the second range read landed as advertised): the forced answer, byte-stable, in the existing read-echo shape — ``Both parts read. `` with quote the second result's pure content (header lines skipped, first 80 chars — the :func:`_same_file_quote` convention; the marker line lives in range B, so an echo of it proves the slice reached the model). * ``("read2", target)`` — EXACTLY ONE read result is in the messages and it carries the ``lines: 1-100 of …`` extent header (asserted — the first range read landed as advertised): the SECOND range read on the SAME target — ``offset`` = :data:`TWO_RANGES_SECOND_OFFSET`, ``limit`` = :data:`TWO_RANGES_SECOND_LIMIT` (id ``call_1``). * ``("read1", target)`` — ``tools`` are offered and no tool result is in the messages yet: the FIRST range read on the parsed target — ``offset`` = :data:`TWO_RANGES_FIRST_OFFSET`, ``limit`` = :data:`TWO_RANGES_FIRST_LIMIT` (id ``call_0``). * ``("bad", result)`` — a tool result is in the messages that is not a well-keyed read result for the step (a refusal — the second range read was refused, the pre-phase bug — or an unkeyed extent header): the deterministic broken-premise echo — the offending result verbatim under ``Premise broke:`` (the read-cap flow's broken-premise convention — a suite fails loudly on it). * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the target is unparseable, or ``tools`` are not offered and no tool result is in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if TWO_RANGES_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _TWO_RANGES_CALL_RE.search(user) if match is None: return None target = match.group("arg") reads = _read_results(body) results = _tool_results(body) if len(reads) >= 2: extent = _two_ranges_extent(reads[1][1]) if extent is None or not extent.startswith( f"lines: {TWO_RANGES_SECOND_OFFSET}-" ): return ("bad", reads[1][1]) return ("answer", _same_file_quote(reads[1][1])) if len(reads) == 1: if len(results) > 1: # A NON-read tool result after the read result — a # refusal (the pre-phase bug: the second range read was # refused ``ALREADY_IN_CONTEXT``): the premise broke — # echo it so a suite fails loudly. return ("bad", results[-1]) extent = _two_ranges_extent(reads[0][1]) if extent is None or not extent.startswith( f"lines: {TWO_RANGES_FIRST_OFFSET}-{TWO_RANGES_FIRST_LIMIT} of " ): return ("bad", reads[0][1]) return ("read2", target) if results: # A tool result that is NOT a read result before any read # landed — the first read was refused: the premise broke. return ("bad", results[-1]) if not body.get("tools"): return None return ("read1", target) # --------------------------------------------------------------------------- # Phase 145 (task 04): the git-source-links' dedicated story suite — the # owner's "the answer points at the original git source" scenario. See # the module docstring. # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — #: drives the deterministic SCRIPTED ORIGIN-LINK flow (the #: git-source-links' story suite, #: ``tests/e2e/test_git_source_links.py`` — the owner's scenario, #: TODO.md L7: the answer grounded in a git-sourced document must #: carry the way to the complete original in ONE step). Checked #: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger #: phrases — the phase-71/72/94/118/131/139/141 ordering #: convention; the trigger needs the ```` section, so #: deflected turns never hit it); verified 2026-10-01: no #: existing E2E question or fixture file contains the phrase, so #: every other suite is unaffected. ORIGIN_LINK_TRIGGER = "original source link" #: The scripted call in the origin-link question (case-insensitive #: — the suite's questions capitalize the trigger's first #: letter): the verb (``read``) plus the target — a combined #: ``source/path``, parsed from the RAW user message so the target #: keeps its case. The target is a ``[a-z0-9_./-]`` run (case- #: insensitively), so the suite's `` — `` flavor separator (em #: dash) can never bleed into it (the read-cap / drill-down #: convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``). _ORIGIN_LINK_CALL_RE = re.compile( r"original source link:\s*read\s+(?P[a-z0-9_./-]+)", re.I, ) #: The flow's FINAL answer for a read result that carries the #: phase-145 ``origin:`` line — the DETERMINISTIC string carrying #: the repository URL VERBATIM (the model-side contract, owner- #: confirmed 2026-09-30: the answer includes the original source). #: ``{url}`` is parsed from the read result's ``origin:`` line BY #: THE MOCK ITSELF — the mock cannot fabricate it, so the URL in #: the answer is the server-side half's assertion (the real #: server's resolver put the line in the result the model saw). ORIGIN_LINK_ANSWER = ( "The full source lives at {url} — see the repo for the complete original." ) #: The flow's FINAL answer for a read result that carries NO #: ``origin:`` line (a local-sourced document — the byte-identity #: control): the deterministic no-origin sentence (its wording #: carries NO ``origin:`` substring — the suite's absence #: assertion keys off the line's exact prefix). ORIGIN_LINK_NO_ORIGIN = ( "No origin line in the read result — the document is not from a git source." ) #: The house scripted-turn echo header (the ``READ_CAP_TRIGGER`` #: flow's convention): the read result rides VERBATIM into the #: answer, so the rendered output is the E2E's lens on the #: ``origin:``/``branch:`` lines' presence (and their absence in #: the local-source control). _ORIGIN_LINK_ECHO_HEADER = "Here's what the read returned:" def _origin_link_url(result: str) -> str | None: """The read result's ``origin:`` line's URL (the phase-145 task-01 line — the sanitized repository URL the real server's resolver put in the result) — or ``None`` when the result carries no origin line (a local-sourced document, no git row, or an ambiguous basename). The search is position-independent (any line with the exact ``origin: `` prefix): the phase-145 lines sit between the ``date:`` line and the content, but the parse never drifts if the result's header shape changes. A document BODY line that merely starts with ``origin: `` would be a fixture defect (the suite's fixtures are pinned content — no such line), never a mock defect. """ for line in result.splitlines(): if line.startswith("origin: "): return line[len("origin: "):] return None def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None: """Classify a phase-145 scripted origin-link request (see the module docstring). The question carries the scripted call (``original source link: read source/path``); the step is then discriminated statelessly from the tool results, like the other marker flows: * ``("call", target, "call_0")`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: the scripted ``read`` on the parsed target (synthetic id ``call_0``). * ``("answer", url, result)`` — a ``tool``-role READ result (content starting with the agent's ``"Document "`` prefix — ``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` / ``branch:`` lines sit BELOW the header, so the first-line parsing is untouched) is in the messages: the deterministic FINAL answer — ``url`` the result's ``origin:`` line's URL (or ``""`` — the no-origin sentence, the local-source control; the empty-string sentinel keeps the return homogeneous ``tuple[str, …]``, the ``_read_lines_flow`` convention) and ``result`` the read result VERBATIM (the house scripted-turn echo — the rendered output is the lens on the origin lines). * ``("bad", result)`` — a tool result is in the messages that is NOT a read result (a refusal — the premise broke): the deterministic broken-premise echo — the offending result VERBATIM under ``Premise broke:`` (the two-ranges convention — the suite fails loudly on it). * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), the scripted call is unparseable, or ``tools`` are not offered and no tool results are in the messages yet (e.g. ``agent_max_rounds=0``). """ user = _user(body) if ORIGIN_LINK_TRIGGER not in user.lower(): return None if "" not in _system(body): return None match = _ORIGIN_LINK_CALL_RE.search(user) if match is None: return None target = match.group("arg") results = _tool_results(body) if results: last = results[-1] if not last.startswith(_READ_RESULT_PREFIX): return ("bad", last) # The empty-string sentinel (never a real URL — a URL is # always non-empty: the resolver returns the stored URL's # sanitized form, which is a non-empty string): keeps the # return homogeneous ``tuple[str, …]`` (pyright), the # ``_read_lines_flow`` convention. return ("answer", _origin_link_url(last) or "", last) if not body.get("tools"): return None return ("call", target, "call_0") def long_answer() -> str: """~900-word deterministic walkthrough (phase 11): numbered steps plus a unique final line that must survive the stream untruncated.""" lines = [ f"{i}. Step {i}: configure node-{i} with the homelab defaults and " f"verify that step {i} of the long walkthrough is complete before moving on." for i in range(1, LONG_ANSWER_LINES + 1) ] lines.append(LONG_ANSWER_END) return "\n".join(lines) #: First numbered note line of a ```` section (phase 15). _TUNING_BLOCK_RE = re.compile(r"\n(.*?)\n", re.S) _NOTE_LINE_RE = re.compile(r"^\d+\.\s*(.+)$") def first_tuning_note(system: str) -> str | None: """The first steering note in the system prompt, or ``None``. The prompt numbers notes 1..N oldest-first (see ``app.rag.prompts.build_steering_section``); the mock echoes the first one into its answer so prompt injection is observable in the UI. """ block = _TUNING_BLOCK_RE.search(system) if not block: return None for line in block.group(1).splitlines(): m = _NOTE_LINE_RE.match(line.strip()) if m: return m.group(1).strip() return None #: First ``-`` bullet line of a ```` section (phase 31). _KB_BLOCK_RE = re.compile(r"\n(.*?)\n", re.S) _KB_BULLET_RE = re.compile(r"^-(?:\s+(.*))?$") def first_kb_bullet(system: str) -> str | None: """The first outline bullet in the system prompt, or ``None``. The stored outline (phase 31) is ``-`` bullet lines (see ``app.rag.overview.OVERVIEW_INSTRUCTION``); the mock echoes the first one into its answer as `` (kb: )`` — the exact :func:`first_tuning_note` convention, so prompt injection of the ```` section is observable in the UI. """ block = _KB_BLOCK_RE.search(system) if not block: return None for line in block.group(1).splitlines(): m = _KB_BULLET_RE.match(line.strip()) if m: return (m.group(1) or "").strip() return None def folder_map_lines(system: str) -> list[str]: """The suggested-folder lines of the ```` section (phase 119, D3, LOCKED A4) — or ``[]`` when there are none. ``build_high_prompt`` lays the section body out as ``SUGGEST_INTRO`` (line 1), the folder lines (each on its own line, immediately after the intro line), a blank line, then the first ```` block — so the lines are exactly the non-blank lines between the intro line and the first `` str: """The phase-74 history echo (byte-stable, stateless over messages). ``history: N prior messages`` — N = the count of non-``system`` messages before the LAST ``user`` message (the client's phase-74 ``history`` block: the prior turns only, the current question itself excluded). ``last answer tail: `` — the LAST 24 chars of the most recent prior ``assistant`` message's content, or ``none`` when there is no prior assistant message (the cold-start pin: no phantom history). ``thinking: yes|no`` — ``yes`` iff that prior assistant message carries a non-empty ``reasoning_content`` field (A4: the client's prior thinking, mapped by ``app.rag.prompts.history_to_messages``), ``no`` otherwise. The invariant (see the module docstring): the client history is ``user``/``assistant``-only, so the last ``user`` message is always the current question and every earlier non-system message is a client-provided prior turn. """ msgs = _messages(body) last_user = max( (i for i, m in enumerate(msgs) if m.get("role") == "user"), default=-1, ) prior = [ m for i, m in enumerate(msgs) if i < last_user and m.get("role") != "system" ] tail = "none" thinking = "no" for m in reversed(prior): if m.get("role") == "assistant": tail = str(m.get("content") or "")[-24:] thinking = "yes" if str(m.get("reasoning_content") or "") else "no" break return ( f"history: {len(prior)} prior messages; " f"last answer tail: {tail}; " f"thinking: {thinking} " "(Deterministic mock answer for E2E.)" ) # --------------------------------------------------------------------------- # Phase 96 (task 04, one-shot resilience): the incident-shape injection # for the folder-summary one-shot path — see the module docstring # --------------------------------------------------------------------------- #: The folder-label triggers (phase 96, task 04): the ``FOLDER_SUMMARY_MODE`` #: branch's folder label (the user message's ``Folder: …`` tail — the #: ``FOLDER_HEADER_PREFIX`` parse) ending with these suffixes is THIS #: SUITE'S own fixture folder (``tests/e2e/test_oneshot_llm_retry.py`` #: seeds the names — no other E2E can hit them; they seed different #: trees, and the triggers carry the E2E prefix). ONESHOT_EMPTY_ONCE_SUFFIX = "/e2e_empty_once" ONESHOT_EMPTY_ALWAYS_SUFFIX = "/e2e_empty_always" #: Module-level per-label incident counter — the mock is single- #: conversation per e2e server (the phase-67 ``_fail_posts`` #: convention). Counts the non-stream folder-summary POSTs served per #: trigger label; the once-sequence resets after the success it guards #: (the first normal reply), so a second sync re-drives the sequence #: deterministically. _empty_once_posts: dict[str, int] = {} def _folder_summary_incident(body: dict[str, Any]) -> bool: """Should this NON-stream folder-summary POST answer with the 2026-09-11 incident envelope (``content=""`` + ``finish_reason="length"`` — the exact wire shape of the empty ``lite`` reply phase 96's one-shot retry targets)? * the system prompt lacks ``FOLDER_SUMMARY_MODE`` → never (only the folder-summary one-shot carries the marker — ``chat()`` is the mock's only non-streaming consumer of it, so the counter counts exactly the one-shot POSTs the app's retry policy drives); * the label ends with ``/e2e_empty_always`` → EVERY POST (the exhaustion path — 1 + ``BOR_LLM_RETRIES`` empty attempts, the per-folder fail-soft leaves the row absent); * the label ends with ``/e2e_empty_once`` → the FIRST non-stream POST for that label only — every later POST returns the normal line (the retry recovers the row), and the counter resets on that first normal reply (the phase-67 pattern). The folder label is the ``FOLDER_SUMMARY_MODE`` branch's existing parse (the user message's first line, the ``FOLDER_HEADER_PREFIX`` tail) — the injection is a pure function of the request plus the per-label counter (the house marker-flow convention). """ if "FOLDER_SUMMARY_MODE" not in _system(body): return False user = _user(body) header = user.splitlines()[0] if user else "" if not header.startswith(FOLDER_HEADER_PREFIX): return False label = header.removeprefix(FOLDER_HEADER_PREFIX).strip() if label.endswith(ONESHOT_EMPTY_ALWAYS_SUFFIX): return True if label.endswith(ONESHOT_EMPTY_ONCE_SUFFIX): n = _empty_once_posts.get(label, 0) + 1 _empty_once_posts[label] = n if n == 1: return True # the first POST: the incident envelope _empty_once_posts[label] = 0 # the retry went out — restart return False # --------------------------------------------------------------------------- # Phase 109 (task 03, the never-frozen-turn story suite): the # deterministic REPORTED-REPRO turn (delta → tool → # thinking-after-delta) — see the module docstring # --------------------------------------------------------------------------- #: A user message containing this substring (case-insensitive) — #: combined with the ```` section in the system prompt — drives #: the deterministic REPORTED-REPRO turn (TODO.md L3): the scripted #: first answer (no reasoning), the no-arg ``ls``, then the post-tool #: thinking → answer → FINAL thinking round — the owner's exact #: "the model responds, calls a tool, then continues thinking" sequence #: with baked-in delays (mid-turn windows of ≥1 s each — the #: ``slow_llm.py`` pacing precedent) so the story E2E's window #: assertions are race-free. Checked BEFORE the plain ``TOOLS_TRIGGER`` #: flow (disjoint trigger phrases — the phase-71/72/94 ordering #: convention); verified 2026-09-16: no existing E2E question or #: fixture file contains the phrase, so every other suite is #: unaffected. TURN_PROGRESS_TRIGGER = "answer first, then list, then think" #: Request 1's pre-delay (s) — model latency: the loader's start-state #: window (the turn is in flight, NO frame has arrived yet). TURN_PROGRESS_PREDELAY_S = 2.0 #: Request 2's frameless gap (s) — the silence after the ``tool`` frame #: before the first ``thinking`` frame. Deliberately past the phase-87 #: tool-line counter's 5 s gate (``TOOL_LINE_ELAPSED_AFTER_MS`` in #: app.js) with ≥2 s of margin: the line's ``(Ns)`` suffix appears #: (5 s tick) and ticks (6 s) BEFORE the first thinking frame settles #: the line, so the story E2E can pin the counter in the tool gap #: without a race. TURN_PROGRESS_TOOL_GAP_S = 7.0 #: Per-chunk delays (s) for the two streams of request 2. TURN_PROGRESS_THINK_DELAY_S = 0.3 # reasoning chunks (~10 × 0.3 s, ~3 × 0.3 s) TURN_PROGRESS_CONTENT_DELAY_S = 0.05 # the short content deltas (3 + 3 chunks) #: Chunk sizes (chars): the content deltas stay SHORT (2-3 chunks each), #: the reasoning streams run 10 × 30 and 3 × 30. TURN_PROGRESS_CONTENT_CHUNK = 40 TURN_PROGRESS_THINK_CHUNK = 30 #: Request 1's answer (byte-stable): 3 × 40-char content chunks, NO #: reasoning — the answer starts FIRST (the repro's "the model #: responds"). ``marker-progress-41`` is the sentinel the story E2E #: matches to prove call 1's content landed in the bubble. TURN_PROGRESS_FIRST_ANSWER = ( "Checking the listing first — opening answer: the kubernetes setup, " "short and sweet. marker-progress-41." ) #: Request 2's scratchpad (byte-stable): 10 × 30-char reasoning chunks — #: thinking AFTER the answer, the reported repro. ``marker-thought-42`` #: is the sentinel the story E2E matches in the re-opened block. TURN_PROGRESS_THINKING = ( "The listing just landed — now I can see which documents exist, so the " "answer can anchor to the kubernetes file first and the deployments note " "second, citing each fact by the exact path it came from. Hosts, " "versions, and ports stay exactly as the notes write them. " "marker-thought-42." ) #: Request 2's answer (byte-stable): 3 × 40-char content chunks ending in #: the DISTINCTIVE final sentence the story E2E matches on to pin the #: post-delta state (``marker-progress-42`` — the repro's second answer). TURN_PROGRESS_FINAL_ANSWER = ( "Here is the plan after the listing: step one, step two, step three — " "that is the whole of it. marker-progress-42." ) #: Request 2's FINAL reasoning (byte-stable): 3 × 30-char chunks — the #: thinking that arrives AFTER the answer's last delta. The story E2E #: waits for the re-opened block after the final sentence landed and #: matches ``marker-final-thought-42`` to prove the last round's #: thinking rendered in the scratchpad. TURN_PROGRESS_FINAL_THINKING = ( "Final check — hosts and ports are verbatim. marker-final-thought-42." ) def _turn_progress_flow(body: dict[str, Any]) -> str | None: """Classify a phase-109 reported-repro request (see the module docstring). Stateless over the messages, like the other marker flows: * ``"first"`` — ``tools`` are offered and no ``tool``-role result is in the messages yet: request 1 — the ~2 s pre-delay (model latency — the loader's start-state window), the short ``delta.content`` stream (3 chunks, NO reasoning — the answer starts first), then the ``ls`` tool call (synthetic id ``call_0``, no arguments), ``finish_reason: "tool_calls"``. * ``"second"`` — a ``tool``-role result is in the messages (the server ran the ``ls``): request 2 — the ~7 s frameless gap (the phase-87 window), the ``reasoning_content`` stream (10 × 0.3 s), the ``delta.content`` stream (3 chunks ending in the distinctive final sentence), and the FINAL ``reasoning_content`` chunks (3 × 0.3 s) — thinking AFTER the answer — then ``finish_reason: "stop"``. * ``None`` — not the flow: the trigger is absent, the ```` section is missing (deflected turns never carry it), or ``tools`` are not offered and no tool result is in the messages yet (e.g. ``agent_max_rounds=0``). """ if TURN_PROGRESS_TRIGGER not in _user(body).lower(): return None if "" not in _system(body): return None if _tool_results(body): return "second" if not body.get("tools"): return None return "first" def _turn_progress_stream(step: str) -> Any: """SSE frames for one phase-109 reported-repro request. ``"first"`` (model call 1): the ~2 s pre-delay, the short ``delta.content`` stream (3 chunks — the answer starts first, NO reasoning), the ``ls`` tool-call partial (synthetic id ``call_0``, no arguments — the phase-37 ``_tool_call_stream`` first-partial shape), and the ``finish_reason: "tool_calls"`` frame. The app is position-independent over the wire (``app/rag/llm.py``): the content chunks stream as ``delta`` SSE frames as they arrive and the tool call materializes AFTER the stream (the content-before- tools convention), so the request's SSE is exactly ``delta → tool``. ``"second"`` (model call 2 — after the server ran the ``ls``): the ~7 s frameless gap (the phase-87 window — the tool line's ``(Ns)`` counter appears and ticks before the first frame settles it), the ``reasoning_content`` stream (10 × 30 chars, 0.3 s each), the ``delta.content`` stream (3 chunks ending in the distinctive final sentence), the FINAL ``reasoning_content`` chunks (3 × 30 chars, 0.3 s each — thinking AFTER the answer, the reported repro), and the ``finish_reason: "stop"`` frame. """ model = "turbo" chunk_id = f"chatcmpl-{uuid.uuid4()}" def frame(delta: dict[str, Any], finish: str | None = None) -> str: return ( "data: " + json_dumps( { "id": chunk_id, "object": "chat.completion.chunk", "created": int(time.time()), "model": model, "choices": [{"index": 0, "delta": delta, "finish_reason": finish}], } ) + "\n\n" ) if step == "first": time.sleep(TURN_PROGRESS_PREDELAY_S) for piece in re.findall( rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FIRST_ANSWER, re.S ): yield frame({"content": piece}) time.sleep(TURN_PROGRESS_CONTENT_DELAY_S) yield frame( { "role": "assistant", "tool_calls": [ { "index": 0, "id": "call_0", "type": "function", "function": {"name": "ls", "arguments": "{}"}, } ], } ) time.sleep(0.1) yield frame({}, "tool_calls") yield "data: [DONE]\n\n" return time.sleep(TURN_PROGRESS_TOOL_GAP_S) for piece in re.findall( rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_THINKING, re.S ): yield frame({"reasoning_content": piece}) time.sleep(TURN_PROGRESS_THINK_DELAY_S) for piece in re.findall( rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FINAL_ANSWER, re.S ): yield frame({"content": piece}) time.sleep(TURN_PROGRESS_CONTENT_DELAY_S) for piece in re.findall( rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_FINAL_THINKING, re.S ): yield frame({"reasoning_content": piece}) time.sleep(TURN_PROGRESS_THINK_DELAY_S) yield frame({}, "stop") yield "data: [DONE]\n\n" def compose_answer(body: dict[str, Any]) -> str: system = _system(body) user = _user(body) if LONG_ANSWER_TRIGGER in user.lower(): answer = long_answer() elif "FOLDER_SUMMARY_MODE" in system: # Folder summary (phase 94, voice nudge phase 155 task 01): # the ``lite`` stand-in returns the deterministic one-liner # ``Documentation for .`` — is the # user message's ``Folder: …`` header line (the generator puts # the canonical source/folder identity there, imported as # ``FOLDER_HEADER_PREFIX`` so the mock can never drift from it). # The stored row therefore always names its folder with a # natural opener (never the forbidden "This folder contains…" # openers — phase 155's pin) — the drill-down E2E # (``test_ls_tree_drilldown.py``) asserts on it. # Checked BEFORE the ``SUMMARY_MODE`` branch: the folder marker # CONTAINS the summary marker as a substring (``FOLDER_`` + # ``SUMMARY_MODE``), so the summary branch would otherwise # shadow every folder-summary call. Checked BEFORE the # DEFLECT_MODE branch, like the other lite-mode markers (a # deflection prompt never carries one). # Phase 96 (task 04): the incident-shape injection keys on THIS # branch's label (``_folder_summary_incident`` — the module # docstring) and overrides the NON-STREAM response envelope in # ``chat_completions`` (``content=""`` + # ``finish_reason="length"``); the line below is what the later # / normal POSTs return. header = user.splitlines()[0] if user else "" folder = ( header.removeprefix(FOLDER_HEADER_PREFIX).strip() if header.startswith(FOLDER_HEADER_PREFIX) else "(unnamed folder)" ) answer = f"Documentation for {folder}." elif "SUMMARY_MODE" in system: # Document summaries (phase 30): the ``lite`` stand-in returns a # deterministic digest — the first 24 tokens of the user message # (the summarizer puts the capped document content there). Byte- # stable for a given fixture, so the summary chunk's retrieval # rank is a pure function of the fixture text. Checked BEFORE the # DEFLECT_MODE branch (task 06) so a deflection prompt that ever # carries the marker cannot shadow the summary call. answer = ( f"This document covers " f"{' '.join(TOKEN_RE.findall(user.lower())[:24])}." ) elif "KB_OVERVIEW_MODE" in system: # KB overview (phase 31): the ``lite`` stand-in returns the # deterministic outline — the first 8 tokens of the user message # (the generator puts the document list there). Byte-stable for a # given KB, so the stored row is a pure function of the fixture. # Checked BEFORE the DEFLECT_MODE branch, like SUMMARY_MODE, so a # prompt that ever carries both markers cannot shadow the # overview call. answer = "Knowledge base outline:\n- " + " ".join( TOKEN_RE.findall(user.lower())[:8] ) elif "IMAGE_DESCRIPTION_MODE" in user: # Phase 122 (image documents, LOCKED A3): the CHAT model's # (vision) image description — a NON-STREAMING multimodal user # message, so only this path ever sees it. ``_content_text`` # joins the text parts, so the marker (the first text part) # lands in ``user``. Checked before the user-trigger branches: # the marker is a fixed app constant, and a real question is # never expected to type it (the other user triggers are owner # phrasings a question might legitimately contain — this one # is an app-to-app marker). Phase 137 (task 05): the # document-context trigger — when the text part carries the # provenance header (``Extracted from `` — the importer's # block builder, tasks 02/03), the canned reply becomes the # context-aware :data:`IMAGE_DESCRIPTION_CONTEXT_ANSWER` # (the deterministic witness that the context REACHED the # vision model); the bare path (standalone images, the # pre-137 embedded shape) stays on the bare # :data:`IMAGE_DESCRIPTION_ANSWER` — byte-identical. answer = ( IMAGE_DESCRIPTION_CONTEXT_ANSWER if "Extracted from " in user else IMAGE_DESCRIPTION_ANSWER ) elif "PICTORIAL_PAGE_MODE" in user: # Phase 140 (pictorial PDF pages, LOCKED A2): the CHAT model's # vision description of ONE rendered sparse PDF page — a # NON-STREAMING multimodal user message, so only this path # ever sees it. ``_content_text`` joins the text parts, so the # marker (the first text part) lands in ``user``. The two # markers are disjoint fixed app constants (neither contains # the other), so the branch order vs ``IMAGE_DESCRIPTION_MODE`` # cannot shadow either flow; like that marker, it is an # app-to-app marker a real question is never expected to type. answer = PICTORIAL_PAGE_ANSWER elif TABLE_TRIGGER in user.lower(): # Markdown tables (phase 44, TODO.md L6): the story E2E's # deterministic table answer — a 3-column table, the # XSS probe line (it must survive the mock # byte-for-byte so the E2E can prove the renderer neutralizes # it), and a wide 5-column table (guarantees scrollWidth > # clientWidth inside the 72rem container column, phase 100). # Byte-stable. Checked # BEFORE the DEFLECT_MODE branch: a deflection prompt never # carries the marker (it lives in the user message, same # reasoning as SUMMARY_MODE), so a marker question always gets # the table answer, whatever the gate says; the E2E asks it # against an on-topic fixture, where the gate is HIGH, and # asserts non-deflection as part of the table test. answer = TABLE_ANSWER elif QUOTE_TRIGGER in user.lower(): # Markdown blockquotes (phase 138, TODO.md L3): the story E2E's # deterministic quote answer — the renderer's blockquote # contract in one byte-stable stream (single-line quote, # 3-line group, nested collapse, the