Files
brain-of-reese/tests/e2e/mock_llm.py
T
ducoterra 051f62deb7
Build and Push Containers / build-and-push-app (push) Successful in 1m55s
Build and Push Containers / build-and-push-db (push) Successful in 14s
phase: 161_reasoning_bubble
All gates verified. Final check — AGENTS.md rules against the project (commands, gates, protocols) — the quick-reference commands all exist and the rules (PLAN §2/§7/§9 references, phase protocol, No CDN, Debugpy, git gates) accurately describe the project; the reasoning bubble is a pure UI change with no new commands or env vars, so no AGENTS.md change is needed.

**Verification complete — full results:**

- **Verified** (implemented by prior task passes, all in working tree): `createReasoningWrap`/`ensureReasoningWrap` live container (D1), `settleReasoning` fold at the 3 settle sites + not at delta (D2), 320px `.reasoning-body` window + pinned-tail follow (D3), zero persistence change (D4), collapsed restore + shared-page parity with D6 step counts, `styles.css` family treatment + reduced-motion still, mock `REASONING_MIDROUND_TRIGGER` flow, 6-test story suite, phase-158 pin updates
- **Full suite:** `uv run pytest` → **3738 passed, 0 failed** · `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99% (>90%)** · `uv run ruff check .` → all checks passed · `uv run pyright` → **0 errors, 0 warnings**
- **E2E in isolation** (DB up; port-8123 collision with an unrelated `undercooked` service → `E2E_APP_PORT=8125`/`E2E_MOCK_PORT=8902`, no live change made): `test_reasoning_bubble.py` **6/6** · `test_thinking_timeline.py` ✓ · `test_thinking_display.py` ✓ · `test_tool_call_compact.py` ✓ · `test_big_read_progress.py` ✓ · `test_turn_progress_loader.py` ✓ · `test_share_chat.py` ✓ · `test_chat_persistence.py` ✓ · `test_smoke.py` ✓ (all exit 0, zero failures)
- **Completion criteria:** all 9 checked against the code + test outcomes — container open/live/nested ✓, stays open past mid-round delta ✓, folds at done ✓, restore + shared collapsed with "3 steps" ✓, plain turn = no container ✓, regression suites green ✓, suite/coverage/lint green ✓, diff scope limited to `frontend/assets/{app.js,shared.js,styles.css}` + `tests/` + `README.md` + phase files (no `app/` — D4) ✓
- **Documentation:** README.md "Thinking" section names the Reasoning bubble accurately (320px window, open-until-settle incl. mid-round deltas, "N steps", byte-identical no-container, collapsed on restore/shared) — verified consistent, no changes needed; AGENTS.md — verified consistent, no changes needed; all `.env.example` vars documented
- **Decisions:** commit + phase-dir move left to the harness (per pass rules — no git index/history commands executed); `undercooked` dev server on port 8123 untouched, E2E used the env port overrides
- **Next pending phase:** none — `161_reasoning_bubble` is the last in `todo/`
2026-10-06 06:36:09 -04:00

5240 lines
259 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Deterministic OpenAI-compatible mock for E2E tests (aipi stand-in).
Implements just enough of the aipi surface:
* ``GET /v1/models``
* ``POST /v1/embeddings`` — real bag-of-words vectors (768-dim, L2-normed).
Because similarity is *genuine token overlap*, the relevance threshold
behaves the same way it will in production: related questions score high,
unrelated ones score low and trigger honest deflection.
* ``POST /v1/chat/completions`` — streaming (SSE) or not. The content keys
off markers in the system prompt:
- user message containing ``write a long answer`` -> a ~900-word
deterministic numbered answer (long-answers story, phase 11)
- ``SUMMARY_MODE`` -> the deterministic summary digest: the first 24
tokens of the user message (the summarizer puts the capped document
content there) — byte-stable for a given fixture (document summaries,
phase 30)
- ``KB_OVERVIEW_MODE`` -> the deterministic outline: the first 8 tokens
of the user message (the generator puts the document list there) —
byte-stable for a given KB (KB overview, phase 31)
- ``FOLDER_SUMMARY_MODE`` -> the deterministic folder one-liner
``Documentation for <folder>.`` (phase 155 task 01: the new voice
— a natural opener naming the folder, never the forbidden
"This folder contains…" openers) — <folder> is the user
message's ``Folder: …`` header line (the generator names the
source/folder there, phase 94), so the stored row always names its
folder and an E2E can assert on it. Checked BEFORE the
``SUMMARY_MODE`` branch: the folder marker CONTAINS the summary
marker as a substring, so the summary branch would otherwise
shadow every folder-summary call
- ``IMAGE_DESCRIPTION_MODE`` (phase 122, image documents) -> the
fixed ``IMAGE_DESCRIPTION_ANSWER`` description: the CHAT model's
(vision) reply to ``summarizer.describe_image``'s multimodal user
message (the marker is its first text part; the call is
non-streaming, so only this answer path serves it). The reply is
the image document's whole content + summary, so it is byte-
stable and token-dense (its cosine against the story suite's
question clears the mock-calibrated gate)
- ``PICTORIAL_PAGE_MODE`` (phase 140, pictorial PDF pages) -> the
fixed ``PICTORIAL_PAGE_ANSWER`` description: the CHAT model's
(vision) reply to ``summarizer.describe_pdf_page``'s multimodal
user message (one rendered sparse PDF page — the marker is its
first text part; the call is non-streaming, so only this answer
path serves it). The reply is appended to the page's indexed
content block as ``[Page N illustration: …]`` (LOCKED A4), so it
is byte-stable: a re-sync with the same reply re-digests to the
same content and lands as ``unchanged``
- ``DEFLECT_MODE`` -> honest "I haven't done anything like that" answer
- otherwise -> upbeat answer quoting the provided document context
- user message containing ``pretend to think slowly`` -> 3s warm-up delay
(used by the loading-feedback story).
- user message containing ``think out loud`` -> the answer is preceded by
~2 700 chars of deterministic ``reasoning_content`` chunks (the
thinking-display story, phase 17; lengthened in phase 21 so the
rendered scratchpad overflows the 320px ``.thinking-text`` window)
- user message containing ``think out loud then hesitate`` -> the
``think out loud`` stream, then a 4s pause before the first content
frame (the sources-midstream story, phase 20 — a deterministic
"leave during pure thinking" navigation window).
- user message containing ``think in paragraphs`` (``THINK_PARAS_TRIGGER``)
-> the ``think out loud`` scratchpad WITH REAL paragraph breaks
("\n\n"), streamed at 60-char frames (vs the mock's 12-char default).
One frame renders several lines — a real-model-sized delta, the
condition under which a POST-render pin-state reading (the old
app.js) measured the chunk's height instead of the user's position
and the think-window follow died at the first 2-newline gap. The
regression pin for the pre-render capture in app.js (2026-08-29,
owner report). Checked BEFORE ``think out loud`` (it is the more
specific phrase); existing E2E questions carry neither, so every
other suite is unaffected.
- system prompt containing ``<tuning>`` (phase 15, steering notes) ->
the composed answer ends with `` (tuning: <first note line>)`` —
makes prompt injection observable in the UI deterministically.
- system prompt containing ``<knowledge_base>`` (phase 31, KB overview)
-> the composed answer ends with `` (kb: <first bullet line>)`` —
the same echo convention for the overview's prompt injection.
- user message containing ``show the end of your notes`` (phase 24;
phase 118 re-targeted — the story's dedicated suite
``tests/e2e/test_summary_seed_context.py``) -> the answer quotes
the **last 160 chars of the ``<documents>`` block** — a tail echo,
byte-stable across runs. (Phase 37: the HIGH prompt now ends with
a ``<tools>`` section after ``</documents>``, so the echo targets
the block itself; its tail still includes the closing tag.)
Phase 118 (A6): the block carries the suggested documents'
SUMMARIES (never full texts), so the echoed tail is the LAST
suggested document's SUMMARY tail (its digest + the
``Source: <source>/<path>`` pointer line) — a sentinel on a
document's *last line* appears in the rendered answer iff the
FULL content (not the summary) was in the prompt, which under the
summary-seed contract is only through a ``read`` tool result.
- user message containing ``use your tools`` (phase 37, agent document
tools; phase 70: the flow emits the harness-aligned names — ``ls``
/ ``read`` with the combined ``source/path`` identity; phase 94:
the flow DRILLS through the tree ``ls`` — the top-level listing
carries sources only, so the flow takes one drill step, ``ls``
scoped to the first source, before the first file line exists)
**and** the system prompt carries the ``<tools>`` section -> the
deterministic SINGLE-READ tool flow, discriminated statelessly from
the messages (the ``tools`` parameter gates the list/read steps —
a no-tools request with no tool results is not the flow):
* request 1 (``tools`` offered, no tool results yet): stream ONLY
``tool_calls`` deltas — ``ls`` (synthetic id ``call_0``, no
arguments), ``finish_reason: "tool_calls"``, no content;
* request 2 (the top-level source listing in the messages — the
agent's ``ls`` tree format, phase 94: no file lines yet):
stream a ``tool_calls`` delta — ``ls`` scoped to the FIRST
source of the listing (id ``call_1``) — the drill step (one
drill per flow — ``_drill_target`` skips already-drilled
sources, so the second listing is the folder level);
* request 3 (a ``tool``-role folder listing with file lines in
the messages): parse the FIRST file line (``source: X | path:
Y | title: Z`` — the labeled ``source:`` / ``path:`` fields,
phase 63) and stream a ``tool_calls`` delta calling ``read`` on
the JOINED combined ``source/path`` (the mock joins the two
labeled fields — the file-line format is unchanged, so this
join is the only parse change, phase 70) (id ``call_2``);
* request 4 (a ``tool``-role read result in the messages —
content starting with the agent's ``"Document <source/path>:"``
header): a content answer, deterministic: ``Read
<source/path>. <first 80 chars of the read document's
content>`` — so a suite can assert the read document reached
the model and landed in the answer. Reached regardless of the
``tools`` parameter (phase 45 keeps the tools offered until the
round cap).
The single-read flow stops at ONE read result; the MULTI-READ
variant below reads two.
- user message containing BOTH ``use your tools`` AND ``read two
documents`` (``MULTI_READ_TRIGGER``, phase 45 task 02) **and** the
system prompt carries the ``<tools>`` section -> the deterministic
MULTI-READ flow (list → drill → read #1 → read #2 → answer; phase
94: the drill step between the top-level listing and the first
read), classified by the COUNT of ``tool``-role read results
(content starting with the agent's ``"Document <source/path>:"``
prefix); phase 70: the same flow on the harness-aligned names —
``ls``, the drill ``ls`` scoped to the first source, then ``read``
on the JOINED combined ``source/path`` of each file line:
* 0 read results, no listing yet: ``ls`` (id ``call_0``);
* 0 read results, top-level listing only (no file lines yet):
the drill — ``ls`` scoped to the first source (id ``call_1``);
* 0 read results, file lines present: ``read`` on the JOINED
combined ``source/path`` of the FIRST file line (id ``call_2``);
* 1 read result: ``read`` on the JOINED combined ``source/path``
of the SECOND file line — the first listing line whose
``source/path`` differs from the one already read (id
``call_3``); a one-file listing degenerates to the
single-read answer (nothing second to read);
* 2 read results: the forced answer, byte-stable: the single-read
shape quoting the FIRST read result, plus the line ``I read
<sp1> and <sp2>.`` naming both read paths in read order — so a
suite can assert the model used BOTH documents.
All other requests (including the marker without a ``<tools>``
section, or with the tool conversation not yet started and no tools
offered — e.g. ``agent_max_rounds=0``) behave exactly as today.
``E2E_REAL_LLM=1`` ignores the mock entirely (the real model does
what it does).
- user message containing ``think out loud and use your tools``
(``THINK_TOOLS_TRIGGER``, phase 158 task 04 — the
thinking/tool-call timeline story's dedicated suite
``tests/e2e/test_thinking_timeline.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
INTERLEAVED tool flow: the phase-37 single-read shape (list →
drill → read → the quoted answer) with a DISTINCT
``reasoning_content`` run streamed AHEAD of every tool call and
ahead of the final answer — the wire order the client's
per-run timeline renders (Thinking → Tool call (1) → Thinking →
Tool call (1) → Thinking → Tool call (1) → Thinking → answer).
Discriminated statelessly from the messages (streaming only —
the app's chat path is the only streaming consumer; a
non-streaming marker request never issues and falls through to
the plain ``think out loud`` scratchpad answer, like the other
marker flows): the ``tools`` parameter gates the list/drill/read
steps (a no-tools request with no tool results is not the flow),
and the answer is reached regardless of the ``tools`` parameter
(phase 45 keeps the tools offered until the round cap):
* request 1 (``tools`` offered, no tool results yet): stream
``reasoning_content`` (``Step 1: list the documents to find
the reference server config.``) FIRST — the 12-char
``_sse_stream`` convention — then ONLY ``tool_calls``
deltas: ``ls`` (synthetic id ``call_0``, no arguments),
``finish_reason: "tool_calls"``;
* request 2 (the top-level source listing in the messages —
the agent's ``ls`` tree format, phase 94: no file lines
yet): ``Step 2: drill into the first source.`` first, then
``tool_calls`` — ``ls`` scoped to the FIRST source of the
listing (id ``call_1``) — the drill step (``_drill_target``
skips already-drilled sources, so the second listing is the
folder level);
* request 3 (a ``tool``-role folder listing with file lines in
the messages — the labeled ``source:`` / ``path:`` fields,
phase 63/70): ``Step 3: read the file line's document.``
first, then ``tool_calls`` — ``read`` on the JOINED
combined ``source/path`` of the FIRST file line (id
``call_2``);
* request 4 (a ``tool``-role read result in the messages —
content starting with the agent's ``"Document
<source/path>:"`` header): ``Step 4: compose the answer from
the read document.`` first, then the phase-37 content
answer, deterministic: ``Read <source/path>. <first 80
chars of the read document's content>`` — so a suite can
assert the read document reached the model and landed in
the answer.
Checked BEFORE the plain ``use your tools`` flow: the trigger
CONTAINS ``use your tools`` (and ``think out loud``), so the
plain tool flow must not see it (the more-specific-phrase-wins
convention — the ``SLOW_PRETOKEN_TRIGGER`` / ``THINK_PARAS_-
TRIGGER`` / ``SCAFFOLD_ALWAYS_TRIGGER`` precedent). Verified
2026-10-04: no existing E2E question or fixture file contains
the phrase, so every other suite is unaffected.
- user message containing ``think out loud, answer, then check
your work`` (``REASONING_MIDROUND_TRIGGER``, phase 161 task 04
— the reasoning-bubble story's dedicated suite
``tests/e2e/test_reasoning_bubble.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
MID-ROUND tool flow: the owner's 2026-10-05 TODO's stays-open
wire shape — a ``reasoning_content`` run, then a MID-ROUND
``content`` delta (the React-style partial answer, the content
stream BEFORE the tool call), then the tool call; the follow-up
request thinks, closes the answer, and FINISHES with a content
finish (no further tool call — the ``done`` convention).
Discriminated statelessly from the messages (streaming only —
the app's chat path is the only streaming consumer; a
non-streaming marker request never issues and falls through to
the plain ``think out loud`` scratchpad answer, like the other
marker flows): the ``tools`` parameter gates the first step (a
no-tools request with no tool results is not the flow), and the
final step is reached when a ``tool``-role result is in the
messages (regardless of the ``tools`` parameter):
* request 1 (``tools`` offered, no tool results yet): stream
``reasoning_content`` (``Step 1: I think I know the
answer — let me check the documents.``) FIRST — the 12-char
``_sse_stream`` convention — then the MID-ROUND ``content``
delta (``The reference server is ref-server-1.`` — the SAME
12-char / 0.02 s convention), and ONLY then
``tool_calls`` deltas: ``ls`` (synthetic id ``call_0``, no
arguments), ``finish_reason: "tool_calls"`` — the tool call
AFTER the delta (the exact mid-round ordering the TODO
describes);
* request 2 (a ``tool``-role result in the messages): stream
``reasoning_content`` (``Step 2: the listing confirms
it.``) FIRST, then the closing ``content`` delta (`` —
confirmed by the listing.`` — it concatenates onto the
mid-round delta, so the turn's full answer is byte-stable),
then ``finish_reason: "stop"`` + ``[DONE]`` — the flow ends
with a content finish (NO further tool call).
Checked BEFORE the phase-158 ``think out loud and use your
tools`` check: the phrases share only ``think out loud`` and
NEITHER is a substring of the other (verified 2026-10-06), and
no existing E2E question or fixture file contains the phrase
(verified 2026-10-06: every other suite is unaffected) — the
more-specific-phrase-wins convention (the ``SLOW_PRETOKEN_-
TRIGGER`` / ``THINK_PARAS_TRIGGER`` / ``SCAFFOLD_ALWAYS_TRIGGER``
precedent). The trigger needs the ``<tools>`` section, so
deflected turns never hit it.
- user message containing ``search your documents``
(``SEARCH_TRIGGER``, phase 68 search tool — renamed to the
harness-aligned ``grep`` in phase 70, same match/output contract)
**and** the system prompt carries the ``<tools>`` section -> the
deterministic SEARCH tool flow, discriminated statelessly from the
messages (streaming only):
* request 1 (``tools`` offered, no search result yet): stream
ONLY ``tool_calls`` deltas — ``grep`` with
``{"pattern": SEARCH_PATTERN}`` (id ``call_0``);
* request 2 (a ``tool``-role search result in the messages —
recognizable by its ``source/path:line: text`` match lines or
the sentinel in its content): the content answer, deterministic:
``Found <first matched line's content up to 80 chars>`` — so a
suite can assert the search result reached the model and landed
in the answer.
Checked BEFORE the plain ``use your tools`` flow (it is the more
specific phrase — same convention as ``think in paragraphs``); no
existing E2E question or fixture file contains the trigger, so
every other suite is unaffected.
- user message containing ``search within one document``
(``SCOPED_SEARCH_TRIGGER``, phase 146 task 02 — the search tool's
SCOPE: a re-search of ONE document, so the UI can show where the
search runs and repeated patterns read as distinct searches,
TODO.md L3) **and** the system prompt carries the ``<tools>``
section -> the deterministic SCOPED-SEARCH flow, discriminated
statelessly from the messages (streaming only; the bare-grep
``SEARCH_TRIGGER`` flow above is the byte-identity pin and stays
untouched):
* request 1 (``tools`` offered, no search result yet): stream
ONLY ``tool_calls`` deltas — ``grep`` with
``{"pattern": SEARCH_PATTERN, "path": SCOPED_SEARCH_PATH}``
(id ``call_0``) — the SCOPED form (pattern + the re-searched
document's combined ``source/path``);
* request 2 (a ``tool``-role search result in the messages —
the agent's REAL scoped grep matched the fixture's sentinel
line): the content answer, deterministic: ``Found <first
matched line's content up to 80 chars>`` (the same echo shape
as the bare flow).
Checked BEFORE the bare ``SEARCH_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72 ordering convention; the trigger needs
the ``<tools>`` section, so deflected turns never hit it); no
existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
- user message containing ``emit raw tool markup``
(``SCAFFOLD_TRIGGER``, phase 71, tool-scaffolding guardrails — the
2026-09-03 incident where a deflected round streamed the model's
raw ``<|tool_call_start|>…<|tool_call_end|>`` markup into the UI)
**or** ``always emit raw tool markup``
(``SCAFFOLD_ALWAYS_TRIGGER``, checked FIRST — it contains the
former phrase) -> the deterministic SCAFFOLDING flow, independent
of the ``<tools>`` marker (both grounded and deflected turns hit
it):
* ``SCAFFOLD_ALWAYS_TRIGGER``: EVERY request (the one bounded
recovery included) streams ONLY ``delta.content`` chunks
carrying the incident span ``SCAFFOLD_SPAN`` —
``<|tool_call_start|>[read(path='search_docs/reese-notes.md')]
<|tool_call_end|>`` — split across the mock's 12-char chunks
(the filter's boundary path), ``finish_reason: "stop"``, no
structured ``tool_calls``, no reasoning — the terminal
malformed-reply path.
* ``SCAFFOLD_TRIGGER``: request 1 (no ``CORRECTION_INSTRUCTION``
in the system prompt) streams the same scaffolding-only span;
request 2 (the system prompt carries the harness constant — a
stable substring of ``app.rag.agent.CORRECTION_INSTRUCTION``,
IMPORTED into this module so the mock can never drift from
it: the one bounded recovery, ``tools=None`` with the
correction folded into the single system prompt) streams the
clean ``SCAFFOLD_RECOVERY_ANSWER`` — the recovery path.
Checked BEFORE the ``SEARCH_TRIGGER`` / ``TOOLS_TRIGGER`` flows
(the trigger needs no ``<tools>`` section); no existing E2E
question or fixture file contains the phrase, so every other
suite is unaffected.
- user message containing ``list the files in this directory``
(``LS_TEACH_TRIGGER``, phase 72, teaching refusals — the
2026-09-03 incident where the harness-prior ``ls(path='.')``
misuse met the terse refusal and the model re-reasoned the same
paragraphs over and over) **and** the system prompt carries the
``<tools>`` section -> the deterministic LS-TEACHING flow,
discriminated statelessly from the messages (streaming only;
phase 94: the correction's top-level listing carries sources only,
so the flow drills one level before the first file line exists):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): stream ONLY ``tool_calls`` deltas — ``ls``
with ``{"path": "."}`` (synthetic id ``call_0``),
``finish_reason: "tool_calls"``, no content — the incident's
misuse, deterministic;
* request 2 (a ``tool``-role result present that is NOT a
listing with file lines — i.e. the teaching refusal): a
``tool_calls`` delta — ``ls`` with no arguments (id ``call_1``)
— the correction;
* request 3 (the top-level source listing in the messages —
file lines still absent): the drill — a ``tool_calls`` delta
— ``ls`` scoped to the FIRST source of the listing
(id ``call_2``);
* request 4 (a ``tool``-result with a ``source: X | path: Y |
title: Z`` file line in the messages — the folder listing):
a deterministic content answer — ``These are the indexed
documents: <first file line>`` (the line, parsed with the
``_CATALOG_LINE_RE`` machinery), ``finish_reason: "stop"`` —
the loop ended in ONE correction + ONE drill, not at the round
cap.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (the trigger
phrases are disjoint substrings — the phase-71 ordering
convention); no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``drill down the tree`` (``DRILL_TRIGGER``,
phase 94 task 04 — the drill-down ``ls``'s dedicated story suite
``tests/e2e/test_ls_tree_drilldown.py``) **and** the system prompt
carries the ``<tools>`` section -> the deterministic SCRIPTED
DRILL-DOWN flow: the question carries its own tool call after the
colon — ``drill down the tree: ls [target]`` (no target = the top
level; ``target`` = a source name or ``source/folder`` path) or
``drill down the tree: read source/path`` — parsed by
``_DRILL_CALL_RE`` from the RAW user message (the target keeps its
case), then discriminated statelessly from the tool results
(streaming only):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): the SCRIPTED call — ``ls`` with no arguments
for the top level, ``ls``/``read`` with the parsed target
otherwise (synthetic id ``call_0``);
* the last tool result is the NOT-A-FOLDER teaching refusal
(it carries ``"is not a folder"`` — the phase-94 task-03
teaching line, the refusal being VISIBLE to the model is what
fires this branch): the scripted one-round recovery — ``ls``
the target's SOURCE segment (id ``call_1``);
* the last tool result is a READ result (``"Document
<source/path>:…``): the deterministic echo answer ``Read
<source/path>. <first 80 chars of the read document's
content>`` (the phase-37 single-read shape — the grounded-
turn citation contract);
* the last tool result is the agent's ALREADY_IN_CONTEXT dedupe
refusal (phase 118: the read target is a document ALREADY
READ into full-text context earlier in the same turn — the
seeds are summaries, so a first read of any document
succeeds and only a re-read is refused): the model answers
FROM THE PROMPT — the deterministic answer ``Already in
context: Read <source/path>. <first 80 chars of the target
document's text as it appears in the ``<documents>`` block>``
(same citation shape as the read-result branch — the document
text reached the model either way, and the answer proves it;
the block it quotes now carries the document's SUMMARY).
No suite exercises this branch today (the drill questions
read once per turn) — it is kept for the still-real
already-read refusal;
* any other last result (a top-level or folder LISTING landed):
the deterministic ECHO answer ``Here's the level I listed:\n
<the listing, verbatim>`` — the mock echoes what it received
(the house scripted-turn way of asserting on tool results):
the suite asserts on the exact tree level — the ``— N
documents`` lines, the stored folder summaries, the
``source: X | path: Y | title: Z`` file lines, and the 50-line
cap + ``…and N more documents…`` note — through the rendered
answer, the only E2E lens on the LLM's context.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72 ordering convention; the trigger needs
the ``<tools>`` section, so deflected turns never hit it); no
existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
- user message containing ``read the capped document``
(``READ_CAP_TRIGGER``, phase 95 task 03 — the read cap's dedicated
story suite ``tests/e2e/test_read_truncation_cap.py``) **and** the
system prompt carries the ``<tools>`` section -> the deterministic
SCRIPTED CAPPED-READ flow: the question carries its own tool call
after the colon — ``read the capped document: read source/path`` —
parsed by ``_READ_CAP_CALL_RE`` from the RAW user message (the
target keeps its case), then discriminated statelessly from the
tool results (streaming only):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): the scripted call — ``read`` with the parsed
target (synthetic id ``call_0``);
* a ``tool``-role result is in the messages: the deterministic
ECHO — the answer carries the LAST tool result VERBATIM
(``Here's what the read returned:\n<result>``): a read result
(``"Document <source/path>:…``) lands in the answer with its
FULL content — the first cap chars + ``[…truncated…]`` + the
pinned grep-pointer notice when the cap fired, the plain body
byte-identical to the pre-phase-95 shape when it did not — and
a refusal (the premise broke) lands just as visibly, so the
suite fails loudly on it. The mock is the only E2E lens on the
LLM's context, so the echo is the assertion surface for both
the marker's presence AND its absence.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72/94 ordering convention; the trigger
needs the ``<tools>`` section, so deflected turns never hit it);
verified 2026-09-10: no existing E2E question or fixture file
contains the phrase, so every other suite is unaffected.
- user message containing ``read the suggested document``
(``SUMMARY_SEED_READ_TRIGGER``, phase 118 task 06 — the
summary-seed context's dedicated story suite
``tests/e2e/test_summary_seed_context.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
SCRIPTED SUMMARY-READ flow: the question carries its own tool call
after the colon — ``read the suggested document: read source/path``
— parsed by ``_SUMMARY_SEED_READ_CALL_RE`` from the RAW user
message (the target keeps its case), then discriminated
statelessly from the tool results (streaming only):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): the scripted call — ``read`` with the parsed
target (synthetic id ``call_0``);
* a ``tool``-role result is in the messages: the deterministic
ECHO — the answer carries the LAST tool result VERBATIM
(``Here's what the read returned:\n<result>``): under the
phase-118 summary-seed contract a first ``read`` of ANY
document succeeds (the seeds are summaries, not full text), so
a read result (``"Document <source/path>:…`` — header + the
phase-106 D5 ``date:`` line + the FULL content) lands in the
answer with its tail intact — the story suite's lens on the
full text the ``read`` tool delivered (a tail sentinel on the
document's last line appears in the answer iff the full content
reached the model through the read, not the seed); a refusal
(the premise broke) lands just as visibly, so the suite fails
loudly on it. The mock is the only E2E lens on the LLM's
context, so the echo is the assertion surface.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72/94 ordering convention; the trigger
needs the ``<tools>`` section, so deflected turns never hit it);
no existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
- user message containing ``what are the correct llama.cpp
arguments`` (``GREP_TEACH_TRIGGER``, the 2026-09-05 incident —
the harness prior is that grep takes a REGEX; this app's grep is a
case-insensitive fixed substring, owner-locked A5) **and** the
system prompt carries the ``<tools>`` section -> the deterministic
GREP-REGEX-TEACHING flow, discriminated statelessly from the
messages (streaming only):
* request 1 (``tools`` offered, no ``tool``-role result yet):
stream ONLY ``tool_calls`` deltas — ``grep`` with
``{"pattern": GREP_TEACH_PATTERN}`` (``qwen.*3\\.8``, id
``call_0``) — the incident's regex-shaped first grep, which a
fixed-substring grep can NEVER match;
* request 2 (the last tool result is the server's TEACHING
no-match line — it carries ``GREP_TEACH_MARKER``):
``grep`` with the plain form ``GREP_TEACH_PLAIN``
(``qwen3.8``, id ``call_1``) — the one-round correction;
* request 3 (the last tool result carries
``source/path:line: text`` match lines): ``read`` the FIRST
match line's document by its combined ``source/path`` (id
``call_2``);
* request 4 (the last tool result is a read result, the
``"Document <combined>:\n<content>"`` shape): the
deterministic echo answer ``Read <combined>. <first 80
chars>``, ``finish_reason: "stop"`` — the loop ended in ONE
correction, not at the round cap.
* A PLAIN no-match as the last result (no match line, no
teaching marker — e.g. the plain pattern genuinely absent) is
the deterministic terminal answer ``No matches — the knowledge
base has no such text.`` (the flow cannot loop on a
well-formed pattern).
Checked BEFORE the SEARCH / TOOLS_TRIGGER flows (disjoint trigger
phrases — the phase-72 ordering convention); no existing E2E
question or fixture file contains the phrase, so every other suite
is unaffected.
- user message containing ``show me a table`` (phase 44, markdown
tables, TODO.md L6) -> the fixed table answer (``TABLE_ANSWER``):
a 3-column service table, an ``<img onerror>`` XSS probe line, and
a deliberately wide 5-column table — byte-stable, so the story E2E
can assert the rendered ``<table class="md-table">`` shape, the
escaped XSS line, and the wrapper's horizontal scroll inside the
72rem container column (phase 100). Checked BEFORE the
``DEFLECT_MODE`` branch (a
deflection prompt never carries the marker, same reasoning as
``SUMMARY_MODE``), so a marker question always gets the table
answer; the E2E asks it against an on-topic fixture (HIGH gate) and
asserts non-deflection.
- user message containing ``show me the blockquote`` (phase 138,
markdown blockquotes, TODO.md L3) -> the fixed quote answer
(``QUOTE_ANSWER``): a single-line quote, a 3-line quote group, a
nested ``>>`` collapse line, an ``<script>`` XSS probe quote, a
``>`` inside a code fence, a ``>`` in a table cell, and a plain
closing line — byte-stable, so the story E2E can assert the four
rendered ``<blockquote class="md-quote">`` shapes, the inert XSS
line, the untouched fence and table-cell ``>``, and the restore
path's byte-identical re-render. Checked BEFORE the
``DEFLECT_MODE`` branch (a deflection prompt never carries the
marker, same reasoning as ``SUMMARY_MODE``), so a marker question
always gets the quote answer; the E2E asks it against an on-topic
fixture (HIGH gate) and asserts non-deflection.
- user message containing ``echo my history``
(``HISTORY_TRIGGER``, phase 74, TODO L4 — chat history with prior
thinking reaches the LLM) -> the deterministic HISTORY ECHO, derived
statelessly from the request messages and byte-stable:
``history: N prior messages; last answer tail: <tail>; thinking:
yes|no (Deterministic mock answer for E2E.)`` where N = the count
of non-``system`` messages before the LAST ``user`` message
(everything the client sent as prior turns — the current question
itself is excluded), <tail> = the LAST 24 chars of the most recent
prior ``assistant`` message's content (``none`` when there is no
prior assistant message), and thinking is ``yes`` iff that prior
``assistant`` message carries a non-empty ``reasoning_content``
field (the client's phase-74 history mapping of the brain record's
``thinking`` — A4). Checked BEFORE the ``DEFLECT_MODE`` branch
(like ``TABLE_TRIGGER`` — the marker lives in the user message, a
deflection prompt never carries it), so a marker question always
gets the echo whatever the honesty gate says; the story E2E
(``tests/e2e/test_llm_history.py``) asserts the wire contents
byte-exactly against the conversation record the client persisted.
Invariant the marker relies on: the client history contains ONLY
``user``/``assistant`` messages — never ``tool``-role ones (the
client never sends tool calls/results) — so every existing marker
flow (which classifies statelessly from TOOL results and the LAST
user message) is unaffected by the now-always-present history.
- user message containing ``answer first, then list, then think``
(``TURN_PROGRESS_TRIGGER``, phase 109 task 03 — the never-frozen-
turn story's dedicated suite
``tests/e2e/test_turn_progress_loader.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
REPORTED-REPRO turn (TODO.md L3: "the model responds, calls a
tool, then continues thinking without re-expanding the thinking
block"): the owner's exact sequence, deterministic, with baked-in
delays (mid-turn windows of ≥1 s each — the ``slow_llm.py``
pacing precedent) so the story E2E's window assertions are
race-free. Discriminated statelessly from the messages (streaming
only):
* request 1 (``tools`` offered, no tool results yet): ~2 s pre-
delay (model latency — the loader's start-state window), a
short ``delta.content`` stream (3 chunks, NO reasoning — the
answer starts FIRST), then the ``ls`` tool call (synthetic id
``call_0``, no arguments — the phase-37 pattern),
``finish_reason: "tool_calls"``;
* request 2 (a ``tool``-role result in the messages — the server
ran the ``ls``): a ~7 s frameless gap (long enough that the
phase-87 tool-line ``(Ns)`` counter — 5 s+ after the line's
own arm — appears and ticks BEFORE the first ``thinking``
frame settles the line), then the ``reasoning_content`` stream
(10 × 0.3 s), a ``delta.content`` stream (3 chunks ending in
the distinctive final sentence carrying
``marker-progress-42``), and the FINAL ``reasoning_content``
chunks (3 × 0.3 s — thinking AFTER the answer, the reported
repro), ``finish_reason: "stop"``.
The server is position-independent over the wire (each
``reasoning_content`` chunk → a ``thinking`` SSE frame, each
``content`` chunk → a ``delta`` frame — ``app/rag/llm.py``; the
tool call materializes after its request's stream — the
content-before-tools convention), so the turn's SSE is exactly
``delta → tool → thinking → delta → thinking → done``.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72/94 ordering convention); verified
2026-09-16: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``repeat your folder map``
(``FOLDER_MAP_TRIGGER``, phase 119, D3 — the suggested-folder
context's story suite ``tests/e2e/test_name_signal_read_chips.py``)
**and** the system prompt contains the ``<documents>`` section ->
the composed answer ends with `` (folders: <line 1>; <line 2>; …)``
echoing the suggested-folder lines VERBATIM — the plain lines
between ``SUGGEST_INTRO`` and the first ``<document `` in the
``<documents>`` section (``app.rag.prompts.build_high_prompt`` /
``app.rag.agent.suggested_folder_lines``, phase 119 LOCKED A4:
``<source>/<prefix>/: e1, e2, …``), joined with ``; ``. If no
folder lines are present (empty *folder_lines* — the
byte-identical phase-118 shape) the echo is omitted and the answer
is otherwise unchanged. The same prompt-injection-echo convention
as the ``<tuning>`` / ``<knowledge_base>`` triggers above (a SUFFIX
appended after both, so the folders suffix is the last thing
rendered): phase 119 D3's observability — the folder lines land in
the model's prompt only on grounded turns, and the echo is the
E2E's deterministic lens on their EXACT text (line identity shape
+ entry order + the owning-doc exclusion + the caps). It composes
with any base answer (it does not compete with a marker branch);
the trigger phrase is disjoint from every other trigger's; verified
2026-09-16: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``show me the picture of``
(``SHOW_IMAGE_TRIGGER``, phase 124 task 07 — the image-chips-and-
answers suite ``tests/e2e/test_answer_images.py``) **and** the
system prompt contains ``show_image`` (the tool clause's presence
— the ``BOR_ANSWER_IMAGES`` toggle's server-side marker: the
clause rides inside the HIGH prompt's ``<tools>`` section ONLY
when the toggle is on, so on a default-OFF app the trigger NEVER
fires and the mock behaves exactly as pre-phase for every other
suite) -> the deterministic SHOW_IMAGE flow, discriminated
statelessly from the tool results (streaming only): the mock
picks the IMAGE document seeded in the ``<documents>`` blocks —
the block whose ``path`` attribute ends in an image extension
(``app.rag.summarizer.IMAGE_MIMES``' family — the suite's
single-PNG fixture guarantees exactly one such block):
* request 1 (``tools`` offered, no confirmed show yet): stream
ONLY ``tool_calls`` deltas — ``show_image`` with
``{"path": <combined source/path>}`` (synthetic id
``call_show``, ``finish_reason: "tool_calls"``, no content) —
NO ``read`` (the ZERO-READ turn: the image doc's seed block
already carries its entire content, so a real model never
reads it — the zero-read shape is what makes the bug-fix's
chip assertion meaningful);
* the follow-up request (a ``tool``-role result carrying the
agent's confirming line ``Image shown to the user`` —
``_SHOW_CONFIRM_MARKER``) in the messages: the deterministic
FINAL answer — a short sentence ending with the ``<documents>``
block's tail echo (the phase-24 END_OF_NOTES echo convention —
it proves the seed was there), NO further tool calls.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrase — the phase-71/72/94/118 ordering convention); verified
2026-09-26: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``which diagram is in your notes``
(``DIAGRAM_TRIGGER``, phase 124 task 07 — the BUG-FIX-ONLY
trigger, same suite) -> the deterministic ZERO-READ answer: the
mock answers straight from the seed with NO tool calls at all
(no ``show_image``, no ``read`` — unlike the ``SHOW_IMAGE_
TRIGGER`` flow it needs NO clause, so it also fires on a default-
OFF app: the chips fix is not gated by ``BOR_ANSWER_IMAGES``,
it belongs to the ``BOR_IMAGES``-indexed docs) and QUOTES THE
TAIL of the seeded image doc's ``<document>`` block (the same
image-doc pick as above): proving the description WAS in the
prompt (for an image doc the seed block IS its entire content)
and that its chip appears with NO tool calls at all (the phase-
124 auto-cite — the fix for the owner's report). Checked BEFORE
the ``DEFLECT_MODE`` branch (the marker lives in the user message
— the ``TABLE_TRIGGER``/``HISTORY_TRIGGER`` convention), so a
marker question always gets the echo whatever the honesty gate
says. Verified 2026-09-26: no existing E2E question or fixture
file contains the phrase, so every other suite is unaffected.
- user message containing ``quote the image reference``
(``CONTEXT_LINE_TRIGGER``, phase 125 task 06 — the image-anchor
context suite ``tests/e2e/test_image_anchor_context.py``) -> the
deterministic ZERO-READ answer that QUOTES THE CONTEXT LINE of the
seeded image doc's ``<document>`` block (the same image-doc pick
as ``DIAGRAM_TRIGGER``): an ENRICHED image doc's block (phase 125
task 05, LOCKED A4; phase 137 task 04) leads with the one-line
context line BEFORE the description — the
``Referenced in <source>/<path>: "<alt>"`` reference rendering
(phase 125) or the ``From <source>/<pdf> (page N)`` provenance
rendering (phase 137 — a PDF-embedded image's block, a combined
context joining both with ``; `` in stored order) — so the
mock's quote — the block's exact first line — proves the line
WAS in the prompt (and that its text is the one-line rendering,
not the stored multi-line block).
A non-enriched (standalone) image doc's block is the bare
description (byte-identical pre-phase): its first line never
starts with ``Referenced in `` or ``From `` (the mock's fixed
description cannot begin that way — the context-aware reply
does, but only ever rides a doc whose stored context makes the
``From …`` line the block's first line), so the mock answers
that the image
carries no reference line — the byte-identity criterion's
deterministic witness. Checked BEFORE the ``DEFLECT_MODE``
branch (the marker lives in the user message — the
``TABLE_TRIGGER``/``HISTORY_TRIGGER`` convention), so a marker
question always gets the echo whatever the honesty gate says.
Verified 2026-09-27: no existing E2E question or fixture file
contains the phrase, so every other suite is unaffected.
- user message containing ``read the lines``
(``READ_LINES_TRIGGER``, phase 131 task 04 — the read
line-ranges' dedicated story suite
``tests/e2e/test_read_line_ranges.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
grep -> read(offset/limit) DANCE (the owner's stuck scenario,
TODO.md L7, end to end): classified statelessly by the tool
results in the messages (the house count pattern — streaming
only):
* request 1 (``tools`` offered, no tool results yet): stream
ONLY ``tool_calls`` deltas — ``grep`` with
``{"pattern": READ_LINES_PATTERN}`` (id ``call_0``), no
content, no reasoning;
* a ``tool``-role grep result with a ``source/path:line: text``
match line is in the messages (the FIRST match line's line
number is parsed): the RANGE READ — a ``tool_calls`` delta —
``read`` on the match line's combined ``source/path`` with
``offset`` = the parsed line number (an int) and ``limit`` =
``READ_LINES_LIMIT`` (30) (id ``call_1``) — the owner's exact
dance: the range the model names is the part it does NOT
have;
* a ``tool``-role read result (content starting with the
agent's ``"Document "`` prefix) is in the messages: the
forced answer, byte-stable: ``Found it at line N. <first 80
chars of the read result's content>`` — N = the offset the
model passed (parsed from the read call's arguments in the
assistant tool-call message — the agent loop echoes every
executed call back into the messages), the quote = the read
slice with the header lines (the phase-106 D5 ``date:`` line
+ the phase-131 ``lines: a-b of N`` extent) skipped, so the
quote is the pure range content (the phase-94 read-echo
convention — the RANGE reached the model's context);
* a tool result is in the messages that is neither a read
result nor a grep match (the grep found nothing — the
premise broke): the deterministic terminal answer ``No
matches — the knowledge base has no such text.`` (the
grep-teach ``nomatch`` convention — the suite fails loudly
on the missing marker).
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
trigger phrases — the phase-71/72/94/118 ordering convention;
the trigger needs the ``<tools>`` section, so deflected turns
never hit it); verified 2026-09-28: no existing E2E question or
fixture file contains the phrase, so every other suite is
unaffected.
- user message containing ``read the same file twice``
(``SAME_FILE_TRIGGER``, phase 139 task 03 — the read
range-tool-lines' dedicated story suite
``tests/e2e/test_read_range_tool_lines.py``) **and** the system
prompt carries the ``<tools>`` section -> the owner's exact
"same file over and over" scenario (TODO.md L4), end to end: the
model range-reads the SAME document TWICE with different
``offset``/``limit``, so the two Reading lines are distinguishable.
The question carries its own target after the colon (``read the
same file twice: read source/path`` — the question-carried-call
convention of ``READ_CAP_TRIGGER`` / ``DRILL_TRIGGER``), and the
step is then discriminated statelessly by the tool results in the
messages (the house count pattern — streaming only):
* request 1 (``tools`` offered, no read results yet): a
``tool_calls`` delta — ``read`` on the parsed target with
``offset`` = ``SAME_FILE_FIRST_OFFSET`` (10) and ``limit`` =
``SAME_FILE_FIRST_LIMIT`` (20) (id ``call_0``);
* a ``tool``-role read result is in the messages (the FIRST read
landed): the SECOND range read — a ``tool_calls`` delta —
``read`` on the SAME target with ``offset`` =
``SAME_FILE_SECOND_OFFSET`` (40) and ``limit`` =
``SAME_FILE_SECOND_LIMIT`` (15) (id ``call_1``) — the same
file again, a different range;
* TWO or more ``tool``-role read results are in the messages
(both range reads landed): the forced answer, byte-stable, in
the phase-131 read-echo shape — ``Read the same file twice —
lines 10–29: <q1>; lines 40–54: <q2>`` where each ``<qN>`` is
that read result's content with the header lines (the
phase-106 D5 ``date:`` line + the phase-131 ``lines: a-b of
N`` extent) skipped, first 80 chars (the phase-94 read-echo
convention — BOTH ranges reached the model's context). NOTE:
for the second read to PROCEED (not be refused
``ALREADY_IN_CONTEXT``) the document was required to already
be in the agent's truncated set — the story suite boots the
app under test with a ``BOR_READ_MAX_CHARS`` small enough
that the FIRST (20-line) range slice truncates, which is
exactly the large-document premise of the owner's confusion.
Phase 141 RETIRED that premise: the coverage-tracked dedupe
proceeds with a second range read whenever its lines are not
fully covered — the suite's small cap stays as the
large-document premise, but is no longer LOAD-BEARING for the
second read to proceed.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72/94/118/131 ordering convention; the
trigger needs the ``<tools>`` section, so deflected turns never
hit it); verified 2026-09-30: no existing E2E question or
fixture file contains the phrase, so every other suite is
unaffected.
- user message containing ``read two ranges``
(``TWO_RANGES_TRIGGER``, phase 141 task 04 — the read
range-dedupe fix's dedicated story suite
``tests/e2e/test_read_range_dedupe.py``) **and** the system
prompt carries the ``<tools>`` section -> the owner's stuck
scenario (TODO.md L3), end to end: the model range-reads ONE
document in TWO different 100-line ranges — pre-phase the
second read was refused ``ALREADY_IN_CONTEXT`` ("the document
is already in my context, but I haven't actually seen the full
text") while ONLY the first slice was in the prompt; the
coverage-tracked server proceeds with it, and the answer
echoes the SECOND range's content (the marker line lives in
range B, so an echo of it proves the slice reached the model).
The question carries its own target after the colon (``read
two ranges: read source/path`` — the question-carried-call
convention of ``READ_CAP_TRIGGER`` / ``SAME_FILE_TRIGGER``),
and the step is then discriminated statelessly by the tool
results in the messages (the house count pattern — streaming
only):
* request 1 (``tools`` offered, no tool results yet): a
``tool_calls`` delta — ``read`` on the parsed target with
``offset`` = ``TWO_RANGES_FIRST_OFFSET`` (1) and ``limit``
= ``TWO_RANGES_FIRST_LIMIT`` (100) (id ``call_0``);
* EXACTLY ONE ``tool``-role read result is in the messages
(content starting with the agent's ``"Document "`` prefix —
``_READ_RESULT_PREFIX``) AND it carries the ``lines: 1-100
of …`` extent header (the mock's own keying, asserted):
the SECOND range read — a ``tool_calls`` delta — ``read``
on the SAME target with ``offset`` =
``TWO_RANGES_SECOND_OFFSET`` (101) and ``limit`` =
``TWO_RANGES_SECOND_LIMIT`` (100) (id ``call_1``) — the
flow only works if the second read PROCEEDS (the pre-phase
server would have refused it);
* TWO or more read results are in the messages and the
SECOND carries the ``lines: 101-…`` header (asserted the
same way — the mock's keying stays honest): the forced
answer, byte-stable, in the existing read-echo shape —
``Both parts read. <first 80 chars of the second read
result's content, header lines skipped>`` (the
:func:`_same_file_quote` convention — the RANGE reached
the model's context);
* a tool result is in the messages that is not a well-keyed
read result for the step (a refusal — the second range
read was refused, the pre-phase bug — or an unkeyed extent
header): the deterministic broken-premise echo — the
offending result VERBATIM under a ``Premise broke:`` line
— so a suite fails loudly on it (the read-cap flow's
broken-premise convention).
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
trigger phrases — the phase-71/72/94/118/131/139 ordering
convention; the trigger needs the ``<tools>`` section, so
deflected turns never hit it); verified 2026-10-01: no existing
E2E question or fixture file contains the phrase, so every
other suite is unaffected.
- user message containing ``original source link``
(``ORIGIN_LINK_TRIGGER``, phase 145 task 04 — the git-source-
links' dedicated story suite
``tests/e2e/test_git_source_links.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
SCRIPTED ORIGIN-LINK flow (the owner's scenario, TODO.md L7:
the answer grounded in a git-sourced document must carry the
way to the complete original in ONE step): the question
carries its own tool call after the colon — ``original source
link: read source/path`` — the question-carried-call
convention (``_READ_CAP_CALL_RE``), then discriminated
statelessly from the tool results (streaming only):
* request 1 (``tools`` offered, no tool results yet): a
``tool_calls`` delta — ``read`` on the parsed target
(synthetic id ``call_0``);
* a ``tool``-role READ result is in the messages (content
starting with the agent's ``"Document "`` prefix —
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
``branch:`` lines sit BELOW the header, so the first-line
parsing is untouched): the deterministic FINAL answer —
the repository URL VERBATIM, parsed from the read
result's ``origin:`` line BY THE MOCK ITSELF (the mock
cannot fabricate it — the real server's resolver put the
line in the result, so the URL in the answer is the
server-side half's assertion): ``The full source lives at
<url> — see the repo for the complete original.``
(``ORIGIN_LINK_ANSWER``) followed by the house
scripted-turn echo — the read result VERBATIM under
``Here's what the read returned:`` (the rendered output
is the E2E's lens on the ``origin:``/``branch:`` lines'
presence);
* a tool result is in the messages that carries NO
``origin:`` line (a local-sourced document — the byte-
identity control): the deterministic NO-ORIGIN answer —
``No origin line in the read result — the document is not
from a git source.`` (``ORIGIN_LINK_NO_ORIGIN``) + the
same verbatim echo (the suite asserts the line's ABSENCE
through it);
* a tool result is in the messages that is NOT a read
result (a refusal — the premise broke): the
deterministic broken-premise echo — the offending result
VERBATIM under ``Premise broke:`` (the two-ranges
convention — the suite fails loudly on it).
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
trigger phrases — the phase-71/72/94/118/131/139/141 ordering
convention; the trigger needs the ``<tools>`` section, so
deflected turns never hit it); verified 2026-10-01: no
existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
- user message containing ``run out of tool calls``
(``ROUND_CAP_TRIGGER``, phase 160 task 05 — the
tool-call-budget telemetry's dedicated suite
``tests/e2e/test_tool_budget_telemetry.py``) — detected on
the LAST user message only (the mock's existing invariant:
the last ``user`` message is the current question, the
earlier non-system messages are prior turns):
* while ``tools`` are offered (the request carries
``tools`` — the app issues such a request only while the
round budget is unspent, i.e. no tool result answers the
last emitted call yet) and the flow is active: stream
ONLY ``tool_calls`` deltas — ``ls`` with no arguments,
incrementing synthetic ids ``call_cap_1``,
``call_cap_2``, … (the next id is derived from the
``call_cap_N`` ids already in the messages' ``assistant``
tool calls), ``finish_reason: "tool_calls"``, no content —
the model NEVER stops asking, so the turn is guaranteed
to hit ``BOR_AGENT_MAX_ROUNDS``;
* on the request WITHOUT ``tools`` (the forced final — the
round-cap path; the request still carries the full
history, so the trigger stays the last user message):
stream a ``tool_calls`` delta ANYWAY — ``read`` on the
SEED path (the system prompt's ``<documents>`` section's
FIRST ``<document>`` block, the joined ``source/path`` —
the ``read`` tool's combined identity, phase 70), id
``call_cap_dump``, NO content — the EXACT dumped-call
shape the phase kills (and the shape ``SCAFFOLD_TRIGGER``
does not cover — that one emits textual markup, this one
is structured). The app REJECTS it (never executed, never
shown, counted in ``AgentHolder.rejected_calls``);
* the ``continue`` question (the last user message is
EXACTLY ``continue`` — the UI's ``CONTINUE_QUESTION`` —
and an EARLIER user message carries the trigger; the
client history rides in the stateless A10 request): the
stable ``ROUND_CAP_CONTINUE_ANSWER`` (a fixed sentence
naming the flow's seed document — the
``SCOPED_SEARCH_PATH`` suite-constant convention: the
literal "continue" shares no tokens with the KB, so the
continuation turn deflects and its prompt carries no
``<documents>`` block; the sentence names the document
directly), so a suite asserts a real continuation answer
rather than a tool call.
Streaming only (the app's chat path is the only streaming
consumer; a non-streaming marker request never issues and
falls through to the regular answer, like the other marker
flows). Checked BEFORE the plain ``TOOLS_TRIGGER`` flow
(disjoint trigger phrases — the phase-71/72/94/118/131/
139/141/145 ordering convention); verified 2026-10-05: no
existing E2E question or fixture file contains the phrase
(or sends a ``continue`` question with the trigger in the
history), so every other suite is unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
test_llm_retry.py``). The mock is single-conversation per e2e server, so
the sequences are driven by module-level counters that reset per
trigger phrase after the success they guard (a second question with
the same trigger re-drives the sequence from zero):
- user message containing ``fail then answer`` (``RETRY_TRIGGER``):
the first ``RETRY_DEAD_ATTEMPTS`` (2) app-level streaming attempts
respond 500 (JSON body, like a dead proxy) and the third streams
the normal composed answer — 2 = 1 original attempt + 1 retry under
the default ``BOR_LLM_RETRIES=3``, so a suite exercises a real
retry without waiting for the 4-attempt exhaustion. Counted in
APP-LEVEL attempts, not raw HTTP POSTs: while the endpoint stays
dead, the openai SDK's default policy (max_retries=2 — the app's
``LLMClient`` keeps it) re-POSTs a 500'd streaming request twice
before surfacing the error, so each dead attempt costs exactly 3
POSTs (``_HTTPS_PER_DEAD_ATTEMPT``).
- user message containing ``always fail``
(``ALWAYS_FAIL_TRIGGER``): EVERY streaming chat/completions request
responds 500 — the retry-budget exhaustion path (the terminal
error banner in the UI).
- embeddings request whose input contains ``embed fail once``
(``EMBED_FAIL_TRIGGER``): the FIRST such request responds 500, the
next returns the normal bag-of-words vector — the endpoint's
pre-stream embedding retry loop. Raw httpx on the client side (no
SDK-level retries), so one POST per attempt: the counter is per
POST here, unlike the chat counter above.
Non-streaming requests (document summaries, KB overview) never 500 —
the retry scope is the chat turn only (owner-locked A1). The one
non-streaming injection is phase 96's incident shape below (it
answers 200 with EMPTY content — the semantic failure class, not a
dead endpoint).
Failure injection (phase 96, one-shot resilience, task 04) — the
2026-09-11 incident shape for the folder-summary one-shot path
(``tests/e2e/test_oneshot_llm_retry.py``): a NON-stream
``chat/completions`` request whose system prompt carries
``FOLDER_SUMMARY_MODE`` (the folder-summary marker — ``chat()`` is
the mock's only non-streaming consumer of it) and whose user
message's ``Folder: …`` header (the branch's existing parse, the
``FOLDER_HEADER_PREFIX`` tail) labels this suite's own fixture
folders:
- the label ends with ``/e2e_empty_once``: the FIRST non-stream POST
for that label answers the incident envelope — the mock's normal
OpenAI chat-completion shape with ``choices[0].message.content =
""`` and ``choices[0].finish_reason = "length"`` (the exact wire
shape of the empty ``lite`` reply: the budget spent in
``reasoning_content``) — and every later POST returns the normal
``Documentation for <folder>.`` line (the one-shot retry,
phase 96 task 01, recovers the row).
- the label ends with ``/e2e_empty_always``: EVERY non-stream POST
for that label answers the empty envelope (the 1 +
``BOR_LLM_RETRIES`` exhaustion → per-folder fail-soft → the row
stays absent while the sync stays green, phase 94 contract).
The once-sequence is driven by a module-level per-label counter that
resets after the success it guards (the phase-67 ``_fail_posts``
pattern — the mock is single-conversation per e2e server), so a
second sync re-drives the sequence deterministically. The trigger
strings are this suite's own folder names, so no other E2E can hit
them (they seed different trees).
Failure injection (phase 120, failed-turn retry, TODO.md L3–4) —
deterministic failures for the failed-turn E2E suite
(``tests/e2e/test_failed_turn_retry.py``), reusing the phase-67
counter machinery:
- user message containing ``fail first turn``
(``FAIL_FIRST_TURN_TRIGGER``): EVERY app-level attempt of the FIRST
turn responds 500 — the full forced-default budget (conftest pins
``BOR_LLM_RETRIES`` to the code default 3 → 4 app-level attempts =
12 POSTs, ``FAIL_FIRST_TURN_DEAD_ATTEMPTS`` ×
``_HTTPS_PER_DEAD_ATTEMPT``) — the retry-budget exhaustion with
ZERO frames (the terminal ``error`` before any delta) — and the
SECOND turn's first request streams the normal composed answer (the
banner/in-bubble Retry's re-ask). The sequence resets after that
success, so a second question carrying the trigger re-drives the
failure from zero (the phase-67 convention).
- user message containing ``partial then fail``
(``PARTIAL_FAIL_TRIGGER``): the FIRST matching streaming request
sends ``PARTIAL_FAIL_TEXT`` in normal 12-char ``delta.content``
chunks and then the generator RAISES — a genuine mid-stream
connection reset (the body ends without ``finish_reason`` /
``[DONE]``; a mid-stream failure never re-POSTs — the body was
already flowing). The app's ``chat_stream_retried`` re-raises the
wrapped ``LLMError`` (a piece already emitted — the locked
retry-before-first-frame rule) and the chat endpoint settles the
turn with the terminal SSE ``error`` frame AFTER the partial
deltas (the partial-then-error wire: the partial bubble keeps its
text + the error note + the Retry). The SECOND request streams the
normal composed answer (the retry's re-ask) and resets the
sequence.
``max_tokens`` is honored deterministically (token ≈ whitespace word),
like a real endpoint: an answer longer than the cap is truncated. This
is what makes the phase-11 truncation regression observable.
"""
from __future__ import annotations
import hashlib
import json
import math
import os
import re
import signal
import threading
import time
import uuid
from typing import Any
from fastapi import FastAPI
from fastapi.responses import JSONResponse, StreamingResponse
from app.rag.agent import ALREADY_IN_CONTEXT, CORRECTION_INSTRUCTION
from app.rag.folder_summaries import FOLDER_HEADER_PREFIX # phase 94: the mock's key
from app.rag.summarizer import IMAGE_MIMES # phase 124: the image-doc extension family
app = FastAPI()
DIM = 768
TOKEN_RE = re.compile(r"[a-z0-9]+")
def embed_text(text: str) -> list[float]:
vec = [0.0] * DIM
for tok in TOKEN_RE.findall(text.lower()):
idx = int(hashlib.md5(tok.encode()).hexdigest(), 16) % DIM
vec[idx] += 1.0
norm = math.sqrt(sum(v * v for v in vec)) or 1.0
return [v / norm for v in vec]
def _messages(body: dict[str, Any]) -> list[dict[str, str]]:
return body.get("messages", [])
def _content_text(content: Any) -> str:
"""The TEXT of one message's content (phase 122 list safety).
String content passes through byte-identical (as does an absent
value or an explicit ``None`` — the ``or ""`` semantics the
:func:`_context` docstring pins). A multimodal part LIST — the
phase-122 image description's ``[{type: "text", …},
{type: "image_url", …}]``, the only list-content message the app
produces (``app.rag.summarizer.describe_image``) — contributes its
text parts joined (the ``image_url`` part carries no text): the
trigger checks and marker branches read the text, and no existing
string-content request is affected.
"""
if isinstance(content, list):
return " ".join(
part.get("text", "")
for part in content
if isinstance(part, dict) and part.get("type") == "text"
)
return content or ""
def _system(body: dict[str, Any]) -> str:
return " ".join(
_content_text(m.get("content"))
for m in _messages(body)
if m.get("role") == "system"
)
def _user(body: dict[str, Any]) -> str:
parts = [_content_text(m.get("content")) for m in _messages(body) if m.get("role") == "user"]
return parts[-1] if parts else ""
def _context(body: dict[str, Any]) -> str:
"""The document context is the longest system/user message in practice.
``_content_text(m.get("content"))`` (the ``or ""`` semantics, NOT
``m.get("content", "")``): a well-formed
OpenAI tool-call message carries ``content: None`` EXPLICITLY (the app's
agent loop appends exactly that — ``app/rag/agent.py``), and a forced
final answer after a tool round (the round-cap path) reaches this helper
with those messages in play. The default-value form returns ``None`` for
an explicit ``None`` and crashes ``len()`` with a 500 (phase 93 task 04
caught it via the deterministic single-read flow's ALREADY_IN_CONTEXT
loop); ``or ""`` treats absent and explicit-None alike, so the fallback
composes deterministically instead of traceback-ing. Phase 122: a
multimodal part list maps to its text parts (see
:func:`_content_text`), so ``len()``/slicing never meet a list.
"""
msgs = _messages(body)
return max((_content_text(m.get("content")) for m in msgs), key=len)
LONG_ANSWER_TRIGGER = "write a long answer"
#: ~920 words — comfortably past the old hard 700-token cap (where the
#: tail would be cut) yet short enough to stream in ~8s at the mock's
#: per-chunk pacing.
LONG_ANSWER_LINES = 40
LONG_ANSWER_END = "LONG-ANSWER-END"
#: Phase 17 (thinking-display story): a user message containing this
#: substring (case-insensitive) is answered with a deterministic
#: ``reasoning_content`` stream ahead of the content — same convention as
#: the other user-message triggers above. Existing E2E questions do not
#: contain the substring, so every other suite is unaffected.
THINKING_TRIGGER = "think out loud"
#: Phase 158 (task 04 — the thinking/tool-call timeline story): a
#: user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt —
#: drives the deterministic INTERLEAVED flow documented in the module
#: docstring: a DISTINCT ``reasoning_content`` run ahead of EVERY
#: tool call and ahead of the final answer (the ``Step N`` markers
#: make the runs individually assertable). Verified 2026-10-04: no
#: existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected. NOTE: the phrase CONTAINS
#: ``THINKING_TRIGGER`` ("think out loud") and ``TOOLS_TRIGGER``
#: ("use your tools") — the more-specific-phrase-wins convention
#: (``SLOW_PRETOKEN_TRIGGER`` / ``THINK_PARAS_TRIGGER`` /
#: ``SCAFFOLD_ALWAYS_TRIGGER``): the streaming dispatch checks the
#: flow BEFORE the plain ``_tool_flow`` (so the plain tool flow
#: never sees the marker). A non-streaming marker request (never
#: issued by the app — the chat path is the only streaming consumer)
#: falls through to the plain ``think out loud`` scratchpad answer,
#: exactly like the other marker flows.
THINK_TOOLS_TRIGGER = "think out loud and use your tools"
#: Phase 161 (task 04 — the reasoning-bubble story): a user message
#: containing this substring (case-insensitive) — combined with the
#: ``<tools>`` section in the system prompt — drives the
#: deterministic MID-ROUND flow documented in the module docstring:
#: a ``reasoning_content`` run, then a MID-ROUND ``delta`` (the
#: React-style partial answer — the content stream BEFORE the tool
#: call, the exact mid-round ordering the owner's 2026-10-05 TODO
#: describes), then the tool call; the follow-up request (a
#: ``tool``-role result in the messages) thinks, emits the closing
#: ``delta``, and FINISHES with a content finish (NO further tool
#: call — the ``done`` convention). Verified 2026-10-06: no
#: existing E2E question or fixture file contains the phrase, and
#: the dispatch is disjoint from the other ``think out loud``
#: triggers — the phrase shares only ``think out loud`` with
#: ``THINKING_TRIGGER`` and ``THINK_TOOLS_TRIGGER`` ("think out
#: loud and use your tools"): NEITHER ``THINK_TOOLS_TRIGGER`` nor
#: the new phrase is a substring of the other (the existing
#: continues ``…and use your tools``, the new continues ``…answer,
#: then check…``). NOTE: the phrase CONTAINS ``THINKING_TRIGGER``
#: ("think out loud") — the more-specific-phrase-wins convention
#: (``SLOW_PRETOKEN_TRIGGER`` / ``THINK_PARAS_TRIGGER`` /
#: ``SCAFFOLD_ALWAYS_TRIGGER``): the streaming dispatch checks the
#: flow BEFORE the phase-158 check and before the generic
#: ``think out loud`` path (so the plain scratchpad never sees the
#: marker). A non-streaming marker request (never issued by the app
#: — the chat path is the only streaming consumer) falls through to
#: the plain ``think out loud`` scratchpad answer, exactly like the
#: other marker flows.
REASONING_MIDROUND_TRIGGER = "think out loud, answer, then check your work"
#: Regression pin (2026-08-29, owner report): a user message containing
#: this substring (case-insensitive) gets the phase-17 scratchpad WITH
#: REAL paragraph breaks ("\n\n"), streamed at ``THINK_PARAS_CHUNK``
#: chars/frame — a single frame renders several lines (a real-model-sized
#: delta), which is the condition under which the old POST-render pin
#: reading in app.js died at the first 2-newline gap. Existing E2E
#: questions do not contain the phrase, so every other suite is
#: unaffected (checked before ``THINKING_TRIGGER`` — the more specific
#: phrase wins).
THINK_PARAS_TRIGGER = "think in paragraphs"
#: 60-char thinking frames for the paragraph trigger (the mock default is
#: 12 — a 12-char frame renders at most one line ≈ 22px, always inside
#: the 32px think-window band, which is why the old code passed the
#: 12-char suites while the real model's larger deltas killed the pin).
THINK_PARAS_CHUNK = 60
#: Phase 20 (sources-midstream bug): a user message containing this
#: substring (case-insensitive) gets the phase-17 thinking stream followed
#: by a multi-second pause before the FIRST content frame — the
#: navigation window for the "leave during pure thinking" scenario
#: (owner-confirmed A1.2: nothing brain-side may be persisted then).
#: Strictly longer than ``THINKING_TRIGGER``, so the phase-17 suite's
#: questions are unaffected.
SLOW_PRETOKEN_TRIGGER = "think out loud then hesitate"
PRE_CONTENT_PAUSE_S = 4.0
#: Phase 24 (whole-document-context story): a user message containing this
#: substring (case-insensitive) gets an answer quoting the TAIL of the
#: document context (see the module docstring). Verified 2026-08-24: no
#: existing E2E question or fixture file contains the phrase, so every
#: other suite is unaffected.
END_OF_NOTES_TRIGGER = "show the end of your notes"
#: The ``<documents>`` block of the system prompt (phase 37: the HIGH
#: prompt ends with the ``<tools>`` section after ``</documents>``, so the
#: phase-24 tail echo targets the block, not the raw message tail).
_DOCUMENTS_BLOCK_RE = re.compile(r"<documents>.*?</documents>", re.S)
#: Phase 37 (agent-document-tools story; phase 70: the flow emits the
#: harness-aligned names): a user message containing this substring
#: (case-insensitive) — combined with the ``<tools>`` section in the
#: system prompt — drives the deterministic tool flow documented in the
#: module docstring (ls → read on the first catalog line's combined
#: ``source/path`` → the quoted answer). Existing E2E questions do not
#: contain the phrase, so every other suite is unaffected.
TOOLS_TRIGGER = "use your tools"
#: Phase 45 (agent-unlimited-tools story, task 02): a user message
#: containing BOTH ``TOOLS_TRIGGER`` and this substring (case-insensitive
#: — the check lowercases the user message) drives the deterministic
#: MULTI-READ tool flow (list → read #1 → read #2 → the forced answer
#: naming both read paths) — see the module docstring. The existing
#: phase-37 E2E question carries ``TOOLS_TRIGGER`` but not this phrase,
#: so the 3-step flow is untouched.
MULTI_READ_TRIGGER = "read two documents"
#: Phase 68 (search tool, TODO.md L4; phase 70: renamed to the
#: harness-aligned ``grep``): a user message containing this substring
#: (case-insensitive) — combined with the ``<tools>`` section in the
#: system prompt — drives the deterministic SEARCH tool flow (grep for
#: ``SEARCH_PATTERN`` → the "Found …" answer), documented in the module
#: docstring. Checked BEFORE ``TOOLS_TRIGGER``
#: (the more specific phrase wins — the same convention as
#: ``THINK_PARAS_TRIGGER``); verified 2026-09-01: no existing E2E
#: question or fixture file contains the phrase, so every other suite
#: is unaffected.
SEARCH_TRIGGER = "search your documents"
#: The sentinel the search flow greps for: the e2e fixture document
#: (``tests/fixtures/search_docs/reese-notes.md``) carries exactly one
#: line containing it, so the search result — and the "Found …" answer
#: that quotes its first matched line — is byte-stable (the sentinel
#: convention of ``END_OF_NOTES_TRIGGER``).
SEARCH_PATTERN = "reese-sentinel-42"
#: Phase 146 (task 02 — the search tool's scope, TODO.md L3): a user
#: message containing this substring (case-insensitive) — combined with
#: the ``<tools>`` section in the system prompt — drives the
#: deterministic SCOPED-SEARCH flow (``grep`` with ``pattern`` +
#: ``path`` → the "Found …" answer), documented in the module
#: docstring. Disjoint from ``SEARCH_TRIGGER`` (the bare-grep flow —
#: the byte-identity pin) and from every other marker; verified 2026-10:
#: no existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected.
SCOPED_SEARCH_TRIGGER = "search within one document"
#: The scoped flow's target — the SAME fixture document the bare-grep
#: flow already greps (``tests/fixtures/search_docs/`` → source
#: ``search_docs``, doc ``reese-notes.md``): the agent's REAL scoped
#: grep finds its sentinel line, so the follow-up round carries a tool
#: result. The exact string the UI renders is the model's ``path``
#: argument — the fixture's real combined identity.
SCOPED_SEARCH_PATH = "search_docs/reese-notes.md"
#: Phase 44 (markdown-tables story, TODO.md L6): a user message
#: containing this substring (case-insensitive) gets the fixed table
#: answer (``TABLE_ANSWER`` below) — a 3-column table, an XSS probe
#: line, and a deliberately wide table (see the module docstring).
#: Existing E2E questions do not contain the phrase, so every other
#: suite is unaffected.
TABLE_TRIGGER = "show me a table"
#: The fixed table answer (phase 44) — byte-stable on purpose: the story
#: E2E asserts the rendered table shape, the escaped ``<img onerror>``
#: line (the XSS payload must survive the mock byte-for-byte), and the
#: wide table's ``scrollWidth > clientWidth`` inside the 72rem container
#: column (phase 100).
#: Phase 74 (chat history, TODO L4): a user message containing this
#: substring (case-insensitive) gets the deterministic HISTORY ECHO
#: (``_history_echo`` below — see the module docstring): the prior-turn
#: count, the last 24 chars of the most recent prior answer, and
#: whether that prior answer carried ``reasoning_content``. Verified
#: 2026-09-08: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
HISTORY_TRIGGER = "echo my history"
#: Phase 119 (D3, the suggested-folder context's story suite): a user
#: message containing this substring (case-insensitive) — combined
#: with the ``<documents>`` section in the system prompt (a grounded
#: turn) — gets the composed answer suffixed with the folder lines
#: VERBATIM (`` (folders: <line 1>; <line 2>; …)``, joined with
#: ``; ``; omitted when the section carries no folder lines) — see the
#: module docstring. Disjoint from every other trigger phrase; verified
#: 2026-09-16: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
FOLDER_MAP_TRIGGER = "repeat your folder map"
#: Phase 160 (task 05, tool-call budget telemetry, TODO.md L4 —
#: the dedicated suite
#: ``tests/e2e/test_tool_budget_telemetry.py``): a user message
#: containing this substring (case-insensitive), detected on the
#: LAST user message only (the mock's existing invariant — the
#: last ``user`` message is the current question, the earlier
#: non-system messages are prior turns), drives the deterministic
#: ROUND-CAP flow: while ``tools`` are offered the mock NEVER
#: answers (``ls`` calls, incrementing ``call_cap_N`` ids — the
#: turn is guaranteed to hit ``BOR_AGENT_MAX_ROUNDS``), and the
#: forced final (no ``tools``) streams a structured ``read`` call
#: anyway (id ``call_cap_dump``, the seed path, NO content) — the
#: exact dumped-call shape the phase kills. The ``continue``
#: question (exactly ``continue``, the trigger in an EARLIER user
#: message) gets :data:`ROUND_CAP_CONTINUE_ANSWER`. Disjoint from
#: every other trigger phrase; verified 2026-10-05: no existing
#: E2E question or fixture file contains the phrase, so every
#: other suite is unaffected.
ROUND_CAP_TRIGGER = "run out of tool calls"
#: Phase 160 (task 05): the ROUND-CAP flow's CONTINUATION answer
#: — the fixed sentence the ``continue`` question gets. The
#: literal "continue" shares no tokens with the KB, so the
#: continuation turn DEFLECTS (A8 LOW) and its prompt carries no
#: ``<documents>`` block — the sentence therefore names the
#: flow's seed document DIRECTLY (the
#: :data:`SCOPED_SEARCH_PATH` suite-constant convention; the
#: suite seeds exactly that document, so the name IS the
#: seed's). Byte-stable on purpose: the suite asserts on it, so
#: it must not drift.
ROUND_CAP_CONTINUE_ANSWER = (
"Picking up where the tool budget ran out — still digging into "
"docs/homelab/kubernetes.md: it says the cluster runs Talos "
"Linux on three nodes, and Cilium handles networking."
)
#: Phase 122 (image documents, LOCKED A3): the fixed description the
#: mock's vision (CHAT) model returns for an ``IMAGE_DESCRIPTION_MODE``
#: request — the multimodal user message
#: ``[{type: "text", …IMAGE_DESCRIPTION_MODE…}, {type: "image_url",
#: …}]`` (``app.rag.summarizer.describe_image``, the app's only
#: list-content message). It becomes the image document's WHOLE
#: ``content`` AND ``summary`` (the only embedded text — the embedding
#: model never sees pixels), so the text is byte-stable across runs and
#: token-dense: the story suite's question ("What is shown in the
#: homelab network diagram?", cosine ≈0.38 against the mock's
#: token-overlap embeddings) clears the mock-calibrated gate (0.30)
#: and the citation usefulness floor (0.15).
IMAGE_DESCRIPTION_ANSWER = (
"A network diagram of the homelab server room: a core router on top, "
"a core switch in the middle, and three labeled subnets at the bottom — "
"VLAN 10 office, VLAN 20 lab, and VLAN 30 storage — with a legend of "
"cable runs. Title: Homelab Network Map."
)
#: The phase-137 witness — the mock's vision reply when the
#: ``IMAGE_DESCRIPTION_MODE`` message's text part carries the
#: document-context block (its ``Extracted from …`` header). It
#: names the document + page (the context reached the model — the
#: deterministic proof) and keeps the diagram wording of
#: :data:`IMAGE_DESCRIPTION_ANSWER` so the story question's
#: token-overlap retrieval still grounds on the image doc.
IMAGE_DESCRIPTION_CONTEXT_ANSWER = (
"From report.pdf, page 2: a network diagram of the homelab "
"server room: a core router on top, a core switch in the "
"middle, and three labeled subnets at the bottom — VLAN 10 "
"office, VLAN 20 lab, and VLAN 30 storage — with a legend of "
"cable runs. Title: Homelab Network Map."
)
#: Phase 140 (pictorial PDF pages, LOCKED A2): the fixed description
#: the mock's vision (CHAT) model returns for a
#: ``PICTORIAL_PAGE_MODE`` request — the multimodal user message
#: ``[{type: "text", …PICTORIAL_PAGE_MODE…}, {type: "image_url",
#: …}]`` (``app.rag.summarizer.describe_pdf_page`` — one rendered
#: sparse PDF page, the page's own text as context). The reply is
#: appended to the page's indexed content block as
#: ``[Page N illustration: …]`` (LOCKED A4), so it is byte-stable and
#: retrieval-friendly (distinctive + token-dense on the assembly
#: vocabulary the story E2E's question targets).
PICTORIAL_PAGE_ANSWER = (
"Pictorial page: the illustration shows two parts being joined "
"with an arrow."
)
TABLE_ANSWER = (
"Here's the shape, in a table:\n"
"\n"
"| Service | Port | Host |\n"
"|---|---|---|\n"
"| Caddy | 80 | homelab-gw |\n"
"| GitLab | 8929 | homelab-git |\n"
"| ntfy | 2087 | homelab-ntfy |\n"
"\n"
"<img src=x onerror=alert(1)>\n"
"\n"
"And the wide one:\n"
"\n"
"| A very long column header to force overflow | Second column with "
"some padding text | Third column | Fourth | Fifth |\n"
"|---|---|---|---|---|\n"
"| value-one | value-two | value-three | value-four | value-five |"
)
#: Phase 138 (markdown blockquotes, TODO.md L3): a user message
#: containing this substring (case-insensitive) gets the fixed quote
#: answer (``QUOTE_ANSWER`` below) — the renderer's whole blockquote
#: contract in one byte-stable answer (see the module docstring).
#: Existing E2E questions do not contain the phrase, so every other
#: suite is unaffected.
QUOTE_TRIGGER = "show me the blockquote"
#: The fixed quote answer (phase 138) — byte-stable on purpose: the
#: story E2E asserts the four rendered <blockquote class="md-quote">
#: shapes (single line, 3-line group, nested collapse, XSS probe),
#: the untouched fence line and table cell, and the plain closing
#: line (the stream-completion marker — it lands last).
QUOTE_ANSWER = (
"Here's the shape, in quotes:\n"
"\n"
"> a short single-line quote\n"
"\n"
"> the first line\n"
"> the second line\n"
"> the third line\n"
"\n"
">> nested quotes collapse to one level\n"
"\n"
"> <script>alert(1)</script>\n"
"\n"
"```\n"
"> inside a fence stays fence content\n"
"```\n"
"\n"
"| Cell |\n"
"|---|\n"
"| > x stays a cell |\n"
"\n"
"And the quote ends here — a plain closing line."
)
# ---------------------------------------------------------------------------
# Phase 67 (LLM retry, TODO.md L3): deterministic failure injection
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) gets
#: ``RETRY_DEAD_ATTEMPTS`` dead streaming attempts (500, JSON body) before
#: the normal composed answer streams — 1 original attempt + 1 retry under
#: the default ``BOR_LLM_RETRIES=3`` (see the module docstring).
RETRY_TRIGGER = "fail then answer"
#: App-level attempts the endpoint stays dead for before the answer.
RETRY_DEAD_ATTEMPTS = 2
#: A user message containing this substring (case-insensitive) makes
#: EVERY streaming chat/completions request respond 500 — the
#: retry-budget exhaustion path (the terminal error banner in the UI).
ALWAYS_FAIL_TRIGGER = "always fail"
#: An embeddings request whose input contains this substring
#: (case-insensitive) 500s on its FIRST POST; the next returns the normal
#: bag-of-words vector — the endpoint's pre-stream embedding retry loop.
EMBED_FAIL_TRIGGER = "embed fail once"
# ---------------------------------------------------------------------------
# Phase 120 (failed-turn retry, TODO.md L3–4): deterministic failure
# injections for the failed-turn E2E suite (tests/e2e/
# test_failed_turn_retry.py) — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) dies for
#: the ENTIRE first turn — every app-level attempt 500s (the
#: retry-budget exhaustion, the zero-frame terminal ``error``) — and
#: streams the normal composed answer from the SECOND turn on (the
#: banner/in-bubble Retry's re-ask, phase 111/120). The dead window is
#: exactly ONE turn: ``FAIL_FIRST_TURN_DEAD_ATTEMPTS`` = the e2e
#: forced-default budget (conftest pins ``BOR_LLM_RETRIES`` to the code
#: default 3 → 4 app-level attempts = 12 POSTs at
#: ``_HTTPS_PER_DEAD_ATTEMPT``). Each sequence resets after the success
#: it guards, so a second question carrying the trigger re-drives the
#: failure from zero (the phase-67 ``_fail_posts`` convention).
FAIL_FIRST_TURN_TRIGGER = "fail first turn"
#: One full turn under the forced-default budget (3 retries → 4
#: attempts): the whole first turn dies, the second turn's first
#: attempt streams (the Retry's re-ask).
FAIL_FIRST_TURN_DEAD_ATTEMPTS = 4
#: A user message containing this substring (case-insensitive) streams
#: ``PARTIAL_FAIL_TEXT`` as ordinary ``delta.content`` chunks on its
#: FIRST matching request and then the mock's generator RAISES (a
#: mid-stream connection reset — the SSE body ends without
#: ``finish_reason``/``[DONE]``; a mid-stream failure never re-POSTs).
#: The app's ``chat_stream_retried`` re-raises the wrapped ``LLMError``
#: (a piece already emitted — the locked retry-before-first-frame rule)
#: and the chat endpoint settles with the terminal SSE ``error`` frame
#: AFTER the partial deltas. The SECOND request streams the normal
#: composed answer (the retry's re-ask) and resets the sequence.
PARTIAL_FAIL_TRIGGER = "partial then fail"
#: Matching requests that partial-then-die (one POST — see above).
PARTIAL_FAIL_DEAD_REQUESTS = 1
#: The deterministic partial answer (byte-stable — the E2E asserts the
#: bubble KEEPS exactly this text, with the error note appended).
PARTIAL_FAIL_TEXT = "Here is the start of the answer that never finished."
# ---------------------------------------------------------------------------
# Phase 71 (tool-scaffolding guardrails, 2026-09-03 incident):
# deterministic raw-markup flows — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) drives
#: the scaffolding flow: request 1 streams ONLY the incident's raw tool
#: markup as ``delta.content``; the follow-up request carrying the
#: harness correction in the system prompt (the one bounded recovery)
#: streams the clean answer. Independent of the ``<tools>`` marker —
#: both grounded and deflected turns hit it. Existing E2E questions do
#: not contain the phrase, so every other suite is unaffected.
SCAFFOLD_TRIGGER = "emit raw tool markup"
#: A user message containing this substring (checked BEFORE
#: ``SCAFFOLD_TRIGGER`` — it contains that phrase) streams the
#: scaffolding-only span on EVERY request, recovery included — the
#: terminal malformed-reply path (the dedicated error frame, no done).
SCAFFOLD_ALWAYS_TRIGGER = "always emit raw tool markup"
#: The incident span (2026-09-03): the model's chat-template tool
#: syntax, emitted as plain ``delta.content`` although no tools were
#: offered. Streamed through the mock's 12-char chunking, so it always
#: spans ≥2 wire chunks (the filter's boundary path).
SCAFFOLD_SPAN = (
"<|tool_call_start|>[read(path='search_docs/reese-notes.md')]"
"<|tool_call_end|>"
)
#: The clean answer the one bounded recovery produces (byte-stable —
#: the dedicated E2E suite asserts the recovered bubble and the wire's
#: delta text against it).
SCAFFOLD_RECOVERY_ANSWER = "Here is the plain-text answer the recovery produced."
#: The stable substring of the harness-owned correction constant the
#: recovery request carries in its system prompt. Keyed on a substring
#: (not the whole constant) so a re-wrap of the constant cannot silently
#: re-route the mock; the module-level assert below fails loudly if the
#: substring ever leaves the constant (the mock must never drift from
#: ``app.rag.agent.CORRECTION_INSTRUCTION``).
_CORRECTION_MARKER = "no tool syntax"
assert _CORRECTION_MARKER in CORRECTION_INSTRUCTION, (
"mock drift: the correction marker left CORRECTION_INSTRUCTION"
)
# ---------------------------------------------------------------------------
# Phase 72 (teaching refusals — the 2026-09-03 incident's ls misuse):
# the deterministic LS-TEACH self-correction flow — see the module
# docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic LS-TEACHING flow (the incident's
#: ``ls(path='.')`` misuse → the teaching refusal → the corrected
#: no-arg ``ls()`` → the catalog answer). Checked BEFORE the plain
#: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71
#: ordering convention); verified: no existing E2E question or fixture
#: file contains the phrase, so every other suite is unaffected.
LS_TEACH_TRIGGER = "list the files in this directory"
#: The deterministic GREP-REGEX-TEACH flow (the 2026-09-05 "Qwen 3.8"
#: incident — the harness prior is that grep takes a REGEX; this app's
#: grep is a case-insensitive fixed substring, owner-locked A5, so a
#: regex-shaped pattern can NEVER match, and the bare no-match line
#: made the turbo model trust the miss and end the turn with a wrong
#: "I searched the entire knowledge base" refusal). The flow pins the
#: self-correction on the SSE wire: the regex-shaped first grep → the
#: server's TEACHING no-match line (``GREP_TEACH_MARKER``) → the
#: plain-form retry grep → the match → the read → the deterministic
#: echo answer. Checked BEFORE the SEARCH / TOOLS_TRIGGER flows
#: (disjoint trigger phrases — the phase-71/72 ordering convention);
#: verified: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
GREP_TEACH_TRIGGER = "what are the correct llama.cpp arguments"
#: The incident's regex-shaped first grep (it can never match a
#: fixed-substring grep — that is the point of the flow).
GREP_TEACH_PATTERN = "qwen.*3\\.8"
#: The plain-form retry — the server's teaching line hands over exactly
#: this hint (``app.rag.agent.plain_form(GREP_TEACH_PATTERN)``).
GREP_TEACH_PLAIN = "qwen3.8"
#: The marker of the agent's teaching no-match line (app.rag.agent
#: ``NO_MATCHES_REGEX`` / ``NO_MATCHES_REGEX_SCOPED``) — the mock's
#: plain step keys on it (a plain no-match line carries it not).
GREP_TEACH_MARKER = "grep matches a plain substring"
# ---------------------------------------------------------------------------
# Phase 94 (task 04, the drill-down ls's dedicated story suite):
# the deterministic SCRIPTED drill-down turns — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the scripted DRILL-DOWN flow: the question carries its own tool call
#: after the colon (``drill down the tree: ls [target]`` / ``drill
#: down the tree: read source/path`` — the suite's scripted turns,
#: ``tests/e2e/test_ls_tree_drilldown.py``). Checked BEFORE the plain
#: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72
#: ordering convention); verified: no existing E2E question or fixture
#: file contains the phrase, so every other suite is unaffected.
DRILL_TRIGGER = "drill down the tree"
#: The scripted call in the drill-down question (case-insensitive — the
#: suite's questions capitalize the trigger's first letter): the verb
#: (``ls`` / ``read``) plus the optional target — a source name or
#: ``source/folder`` path (``ls``) or a combined ``source/path``
#: (``read``), parsed from the RAW user message so the target keeps its
#: case. The target is a ``[a-z0-9_./-]`` run (case-insensitively), so
#: the suite's `` — `` flavor separator (em dash) can never bleed into
#: it; no run = the top-level ``ls`` (no ``path`` argument).
_DRILL_CALL_RE = re.compile(
r"drill down the tree:\s*(?P<verb>ls|read)(?:\s+(?P<arg>[a-z0-9_./-]+))?",
re.I,
)
#: The agent's NOT-A-FOLDER teaching refusal marker (app.rag.agent
#: ``NOT_A_FOLDER`` — ``'{arg}' is not a folder — {parent} has:
#: {subfolders}``): the drill-down flow's scripted recovery keys on it
#: in the LAST tool result (the refusal being visible to the model is
#: exactly what triggers the one-round recovery — the phase-72
#: self-correction contract carrying the tree's teaching line).
_NOT_A_FOLDER_MARKER = "is not a folder"
#: The stable substring of the harness-owned dedupe refusal the
#: drill-down flow's ``ctx_answer`` branch keys on (phase 118: a read
#: target ALREADY READ into full-text context in the same turn — the
#: seeds are summaries, so a first read of any document succeeds and
#: only a re-read is refused; the mock then answers from the document's
#: summary in the ``<documents>`` prompt block, exactly what the
#: refusal instructs). Keyed on a substring (not the whole
#: constant) so a re-wrap of the constant cannot silently re-route the
#: mock; the module-level assert below fails loudly if the substring
#: ever leaves the constant (the mock must never drift from
#: ``app.rag.agent.ALREADY_IN_CONTEXT``).
_ALREADY_IN_CONTEXT_MARKER = "Already in your context"
assert _ALREADY_IN_CONTEXT_MARKER in ALREADY_IN_CONTEXT, (
"mock drift: the dedupe marker left ALREADY_IN_CONTEXT"
)
#: One ``<document>`` block of the HIGH prompt's ``<documents>``
#: section (``app.rag.prompts.build_high_prompt``): the block is the
#: document identity (``source``/``path``/``title`` attributes — plus,
#: since phase 106 D5, the ``date`` attribute, the row's ``created_at``
#: UTC date part, APPENDED after ``title``) plus the document's
#: SUMMARY (phase 118, A6 — the summary-seed contract: the seeded
#: blocks are summaries, never full text; full text enters the context
#: only through the capped ``read`` tool) between the tags. The
#: ``date`` group is OPTIONAL so the mock tolerates the pre- and
#: post-phase block shapes (house rule: the marker/regex lands with
#: the prompt change).
_DOCUMENT_BLOCK_RE = re.compile(
r'<document source="(?P<source>[^"]+)" path="(?P<path>[^"]+)" '
r'title="[^"]*"(\sdate="[^"]*")?>\n(?P<content>.*?)\n</document>',
re.S,
)
def _document_block(system: str, source: str, path: str) -> str | None:
"""The stored text of one ``<document>`` block (or ``None``).
The drill-down flow's ``ctx_answer`` branch: when the agent's read
of a document ALREADY READ in the same turn gets the
ALREADY_IN_CONTEXT dedupe (phase 118 — the seeds are summaries, so
a first read of any document succeeds and only a re-read is
refused), the mock (the model) extracts the block's text by the
identity attributes and quotes it, answering from the prompt as
the refusal instructs. The block carries the document's SUMMARY
(phase 118 A6) — no suite exercises the branch today; it is kept
for the still-real already-read refusal.
"""
for block in _DOCUMENT_BLOCK_RE.finditer(system):
if block.group("source") == source and block.group("path") == path:
return block.group("content")
return None
#: The agent's drill-down ``ls`` result shapes (app.rag.agent
#: ``render_ls_top`` / ``render_folder_listing``, phase 94): the
#: top-level header ``"N sources:"`` and the per-source block line
#: ``"<source> — N documents"`` (summary lines are indented — they
#: never match); the folder-level header ``"<identity> — N documents,
#: M folders:"`` (identity = the source name or ``source/folder``).
_SOURCE_HEADER_RE = re.compile(r"^\d+ sources:$")
_SOURCE_LINE_RE = re.compile(r"^(?P<source>.+?) — (?P<n>\d+) documents$")
_FOLDER_HEADER_RE = re.compile(r"^(?P<identity>.+?) — \d+ documents, \d+ folders:$")
#: One DEAD app-level chat attempt costs exactly this many HTTP POSTs
#: while the endpoint stays down: the openai SDK's default policy
#: (max_retries=2 — the app's ``LLMClient`` keeps it) re-POSTs a 500'd
#: streaming request twice before surfacing the error to
#: ``chat_stream_retried``. The failure counters below therefore count
#: app-level attempts (groups of this size), not raw POSTs — the visible
#: sequence (one SSE ``retry`` frame after each dead attempt, the answer
#: on the third) stays deterministic regardless of the SDK's internal
#: backoff pacing.
_HTTPS_PER_DEAD_ATTEMPT = 3
#: Module-level failure counters — the mock is single-conversation per
#: e2e server. Keyed by trigger phrase (reset per trigger): the number
#: of matching POSTs served so far. Each sequence resets after the
#: success it guards, so a second question carrying the same trigger
#: re-drives the failure sequence from zero.
_fail_posts: dict[str, int] = {}
#: Phase 123 (task 04 — question images): a small ring buffer of the
#: recent ``/v1/chat/completions`` request bodies, exposed on the
#: ``/v1/e2e/captured`` pair below. Purely OBSERVATIONAL (the buffer
#: never influences an answer — determinism is untouched): it lets a
#: story suite assert on the EXACT request the app built — e.g. that a
#: question-image turn delivered the multimodal user content list
#: (text part + ``image_url`` data URL) to the model.
_CAPTURED: list[dict[str, Any]] = []
_CAPTURE_MAX = 100
def _llm_500(why: str) -> JSONResponse:
"""A dead-proxy 500 with a JSON error body (phase 67 injection)."""
return JSONResponse(
status_code=500,
content={
"error": {
"message": f"upstream connection reset ({why})",
"type": "proxy_error",
}
},
)
def _bump_fail(key: str) -> int:
n = _fail_posts.get(key, 0) + 1
_fail_posts[key] = n
return n
def _chat_dead(key: str, dead_attempts: int) -> bool:
"""Bump *key*'s counter; True while the endpoint stays dead.
Counted in app-level attempts (see ``_HTTPS_PER_DEAD_ATTEMPT``): the
first ``dead_attempts * _HTTPS_PER_DEAD_ATTEMPT`` POSTs 500 and the
next attempt's first POST streams (the caller resets the counter on
the success).
"""
return _bump_fail(key) <= dead_attempts * _HTTPS_PER_DEAD_ATTEMPT
def _partial_fail_stream() -> Any:
"""The phase-120 partial-then-fail stream (``PARTIAL_FAIL_TRIGGER``):
``PARTIAL_FAIL_TEXT`` in 12-char ``delta.content`` chunks (the
mock's default cadence — NO ``finish_reason``, NO ``[DONE]``), then
a RAISE that ends the SSE body abruptly (a connection reset
mid-stream — the client SDK sees a truncated body, the app wraps it
in ``LLMError`` after the pieces already emitted, and the chat
endpoint's terminal ``error`` frame lands AFTER the partial deltas
on the client's stream)."""
model = "turbo"
chunk_id = f"chatcmpl-{uuid.uuid4()}"
for piece in re.findall(r".{1,12}", PARTIAL_FAIL_TEXT, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{"index": 0, "delta": {"content": piece}, "finish_reason": None}
],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
raise RuntimeError("e2e mid-stream failure (phase 120 partial-then-fail)")
#: The agent's ``read`` tool-result prefix (app.rag.agent
#: ``_execute_tool``): ``"Document <source/path>:\n<content>"``.
_READ_RESULT_PREFIX = "Document "
#: One line of the agent's ``ls`` output (app.rag.agent
#: ``_execute_tool``, phase 63): labeled, pipe-delimited fields —
#: ``source: X | path: Y | title: Z`` — unambiguous for LLM parsing even
#: when the path contains ``/`` characters.
_CATALOG_LINE_RE = re.compile(
r"^source: (?P<source>.+?) \| path: (?P<path>.+?) \| title: .+$"
)
def _read_results(body: dict[str, Any]) -> list[tuple[str, str]]:
"""The read results in the messages, in order: ``(source/path, content)``.
A read result is a ``tool``-role message whose content starts with
the agent's read-result prefix (``app.rag.agent`` ``_execute_tool``):
``"Document <source/path>:\n<content>"``. The header is stripped of
the prefix AND the trailing colon so the path stays clean.
"""
out: list[tuple[str, str]] = []
for m in _messages(body):
if m.get("role") != "tool":
continue
content = str(m.get("content") or "")
if content.startswith(_READ_RESULT_PREFIX):
header, _, doc_content = content.partition("\n")
sp = header[len(_READ_RESULT_PREFIX):].strip().removesuffix(":")
out.append((sp, doc_content))
return out
def _catalog_docs(body: dict[str, Any]) -> list[tuple[str, str]]:
"""Every ``(source, path)`` in the catalog tool result, in listing order.
Catalog lines are ``source: X | path: Y | title: Z`` (the agent's
``ls`` output — phase 63: labeled, pipe-delimited
fields, unambiguous even for paths full of ``/``): the line-level
regex recovers the ``source`` and ``path`` fields directly. The
``"N documents:"`` header line matches no line and is skipped;
read-result messages are full documents, not listings, and are
skipped too.
"""
docs: list[tuple[str, str]] = []
for m in _messages(body):
if m.get("role") != "tool":
continue
content = str(m.get("content") or "")
if content.startswith(_READ_RESULT_PREFIX):
continue
for line in content.splitlines():
match = _CATALOG_LINE_RE.match(line)
if match:
docs.append((match.group("source"), match.group("path")))
return docs
def _tool_results(body: dict[str, Any]) -> list[str]:
"""Every ``tool``-role result content in the messages, in order.
(Phase 72, LS-TEACH flow: the flow is discriminated statelessly
from the tool results — a catalog listing vs the teaching
refusal vs none yet.)
"""
return [
str(m.get("content") or "")
for m in _messages(body)
if m.get("role") == "tool"
]
def _first_file_line(body: dict[str, Any]) -> str | None:
"""The first file line (``source: X | path: Y | title: Z``) across
the ``tool``-role results in the messages, in message order (read
results — full documents, not listings — skipped; the ``_CATALOG_-
LINE_RE`` machinery).
Phase 94: the drill-down ``ls`` carries file lines only at folder
levels — the top-level source listing and the teaching refusals
have none, so ``None`` here means "no folder level reached yet".
"""
for content in _tool_results(body):
if content.startswith(_READ_RESULT_PREFIX):
continue
for line in content.splitlines():
if _CATALOG_LINE_RE.match(line):
return line
return None
def _drill_target(body: dict[str, Any]) -> str | None:
"""The next source to drill into (phase 94, the drill step).
The drill-down ``ls`` top level lists SOURCES only (no file lines),
so a deterministic flow that needs a file line must drill one level:
``ls`` scoped to a source. The target is the FIRST source of the
top-level listing (registry order — the listing's own order) that
does not yet have a folder-level listing in the messages (a folder
header whose identity is the source or ``source/…``); ``None`` when
no top-level listing is in the messages yet (nothing to drill from)
or every listed source has been drilled (an empty KB — the flow
degenerates to the old re-list loop, settling at the round cap
exactly like the phase-70 empty-catalog case).
"""
sources: list[str] = []
drilled: set[str] = set()
for content in _tool_results(body):
lines = content.splitlines()
if not lines:
continue
if _SOURCE_HEADER_RE.match(lines[0]):
for line in lines[1:]:
match = _SOURCE_LINE_RE.match(line)
if match:
sources.append(match.group("source"))
else:
folder = _FOLDER_HEADER_RE.match(lines[0])
if folder:
drilled.add(folder.group("identity"))
for source in sources:
if not any(
identity == source or identity.startswith(source + "/")
for identity in drilled
):
return source
return None
#: One line of the agent's ``grep`` output (app.rag.agent
#: ``_execute_tool``, phase 68 — phase 70 renamed the tool, the line
#: format is unchanged): ``source/path:LINE: text``. The
#: non-greedy prefix keeps nested paths (``/`` in the path) intact.
_SEARCH_LINE_RE = re.compile(r"^(?P<sp>.+?):(?P<line>\d+): (?P<text>.*)$")
def _search_result_line(body: dict[str, Any]) -> str | None:
"""The first matched line's text of a search result in the messages.
A search result is a ``tool``-role message — never a read result
(those start with the agent's ``"Document "`` prefix) — that either
carries ``source/path:LINE: text`` match lines (the agent's
``grep`` output, phase 68) or the sentinel pattern
itself (its no-match line quotes the pattern). Returns the first
match line's ``text`` part (already 200-char-capped server-side),
or the message's first line in the sentinel-only shape, or ``None``
when no search result is in the messages yet.
"""
sentinel = SEARCH_PATTERN.lower()
for m in _messages(body):
if m.get("role") != "tool":
continue
content = str(m.get("content") or "")
if content.startswith(_READ_RESULT_PREFIX):
continue
for line in content.splitlines():
match = _SEARCH_LINE_RE.match(line)
if match:
return match.group("text")
if sentinel in content.lower():
lines = content.splitlines()
return lines[0] if lines else ""
return None
def _search_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a SEARCH_TRIGGER request into a step of the search flow.
* ``("search",)`` — ``tools`` are offered and no search result is
in the messages yet: the model greps the whole KB for
``SEARCH_PATTERN`` (id ``call_0``).
* ``("found", first_line)`` — a ``tool``-role search result is in
the messages: the model answers, quoting the first matched line
(``Found <first matched line's content up to 80 chars>``). Reached
regardless of the ``tools`` parameter (phase 45 keeps the tools
offered until the round cap).
* ``None`` — not the search flow: the trigger is absent, the
``<tools>`` section is missing (deflected turns never carry it),
or ``tools`` are not offered and no search result is in the
messages yet (e.g. ``agent_max_rounds=0``).
"""
if SEARCH_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
first_line = _search_result_line(body)
if first_line is not None:
return ("found", first_line)
if not body.get("tools"):
return None
return ("search",)
def _scoped_search_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a SCOPED_SEARCH_TRIGGER request into a step.
Phase 146 (task 02): the SCOPED search flow — the owner's "looks
like the LLM is repeating itself" scenario (TODO.md L3): a re-search
of ONE document (``grep(pattern, path)``) so the UI can show WHERE
the search runs. The same scripted-turn idiom as ``_search_flow``
(the phase-70/72 grep branches): stateless classification from the
messages (streaming only).
* ``("search",)`` — ``tools`` are offered and no search result is
in the messages yet: the model greps the ONE fixture document
(``SCOPED_SEARCH_PATH``) for ``SEARCH_PATTERN`` (id ``call_0``) —
the scoped form (``pattern`` + ``path``).
* ``("found", first_line)`` — a ``tool``-role search result is in
the messages (the agent's REAL scoped grep matched the fixture's
sentinel line): the model answers, quoting the first matched
line (``Found <first matched line's content up to 80 chars>``).
Reached regardless of the ``tools`` parameter (phase 45 keeps the
tools offered until the round cap).
* ``None`` — not the scoped flow: the trigger is absent, the
``<tools>`` section is missing (deflected turns never carry it),
or ``tools`` are not offered and no search result is in the
messages yet (e.g. ``agent_max_rounds=0``).
"""
if SCOPED_SEARCH_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
first_line = _search_result_line(body)
if first_line is not None:
return ("found", first_line)
if not body.get("tools"):
return None
return ("search",)
def _scaffold_flow(body: dict[str, Any]) -> str | None:
"""Classify a phase-71 scaffolding request (see the module docstring).
* ``"scaffold"`` — stream ONLY the incident span
(``SCAFFOLD_SPAN``) as ``delta.content`` chunks: ``finish_reason:
"stop"``, no structured ``tool_calls``, no reasoning. EVERY
request for ``SCAFFOLD_ALWAYS_TRIGGER`` (the recovery included),
and the FIRST request of ``SCAFFOLD_TRIGGER`` (no correction in
the system prompt yet).
* ``"recovery"`` — ``SCAFFOLD_TRIGGER`` whose system prompt carries
the harness correction (the one bounded recovery: ``tools=None``,
the constant folded into the single system prompt by
``app.api.chat`` / ``app.rag.agent``): stream the clean
``SCAFFOLD_RECOVERY_ANSWER``.
* ``None`` — not the scaffolding flow. The discrimination is
stateless, like the other marker flows: the trigger phrase in
the user message plus the correction's presence in the system
prompt.
"""
user = _user(body).lower()
if SCAFFOLD_ALWAYS_TRIGGER in user: # checked FIRST — it contains SCAFFOLD_TRIGGER
return "scaffold"
if SCAFFOLD_TRIGGER in user:
if _CORRECTION_MARKER in _system(body):
return "recovery"
return "scaffold"
return None
def _tool_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a marker request into one step of the tool flow.
Single-read (phase 37 — the user message carries ``TOOLS_TRIGGER``
only):
* ``("list", "", "")`` — ``tools`` are offered and no tool results
are in the messages yet: the model lists the catalog.
* ``("read", source, path, "call_1")`` — a ``tool``-role catalog
result is in the messages: the model reads its FIRST
``source: X | path: Y | title: Z`` line (the labeled
``source:`` / ``path:`` fields, phase 63), emitted as ``read`` on
the JOINED combined ``source/path`` (phase 70: the mock joins
the two fields — the canonical document identity).
* ``("answer", "source/path", content)`` — a ``tool``-role read
result (``"Document <source/path>:\n<content>"``) is in the
messages: the model answers, quoting the read document. Reached
regardless of the ``tools`` parameter (phase 45 keeps the tools
offered until the round cap).
Multi-read (phase 45 task 02 — the user message carries BOTH
``TOOLS_TRIGGER`` and ``MULTI_READ_TRIGGER``), classified by the
count of ``tool``-role read results:
* 0 read results: ``("list", "", "")`` (no listing yet),
``("drill", source, "call_1")`` when the top-level source
listing is in the messages but no file line yet (phase 94: the
drill — the top level carries sources only), or
``("read", source, path, "call_2")`` on the FIRST file-line doc.
* 1 read result: ``("read", source, path, "call_3")`` on the SECOND
file-line doc — the first listing line whose ``source/path``
differs from the one already read. A one-file listing
degenerates to the single-read ``("answer", ...)`` shape (nothing
second to read).
* 2 read results: ``("multi_answer", "", text)`` — the forced
answer, byte-stable: the single-read shape quoting the FIRST read
result, plus ``I read <sp1> and <sp2>.`` (both read paths, read
order). The second element is unused.
* ``None`` — not the marker flow: the request behaves exactly as
before (marker absent, no ``<tools>`` section, or a no-tools
request with no tool results — e.g. ``agent_max_rounds=0``).
"""
user = _user(body).lower()
if TOOLS_TRIGGER not in user:
return None
if "<tools>" not in _system(body):
return None
reads = _read_results(body)
if MULTI_READ_TRIGGER in user:
if not reads:
if not body.get("tools"):
return None
docs = _catalog_docs(body)
if docs:
return ("read", docs[0][0], docs[0][1], "call_2")
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_1")
return ("list", "", "")
if len(reads) == 1:
skip = reads[0][0]
second = next(
(d for d in _catalog_docs(body) if f"{d[0]}/{d[1]}" != skip), None
)
if second is None:
# One-file listing: nothing second to read — the
# single-read answer shape (deterministic degenerate).
return ("answer", reads[0][0], reads[0][1])
return ("read", second[0], second[1], "call_3")
(sp1, c1), (sp2, _c2) = reads[0], reads[1]
answer = f"Read {sp1}. {c1[:80]} I read {sp1} and {sp2}."
return ("multi_answer", "", answer)
# Phase-37 single-read flow (phase 94: the drill step between the
# top-level listing and the first file line).
if reads:
return ("answer", reads[0][0], reads[0][1])
if not body.get("tools"):
return None
docs = _catalog_docs(body)
if docs:
return ("read", docs[0][0], docs[0][1], "call_2")
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_1")
return ("list", "", "")
def _thinking_tools_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-158 marker request into one step of the
INTERLEAVED thinking + tool flow (see the module docstring).
The phase-37 single-read shape — list → drill → read → the
quoted answer — with a DISTINCT ``reasoning_content`` run
streamed AHEAD of every tool call and ahead of the final answer
(the ``Step N`` markers make each run individually assertable —
the per-run timeline the story E2E pins):
* ``("list", <step 1 thinking>)`` — ``tools`` are offered and no
tool results are in the messages yet: the model thinks, then
lists the catalog (``ls``, id ``call_0``, no arguments).
* ``("drill", source, <step 2 thinking>)`` — the top-level
source listing is in the messages but no file line yet (phase
94: the drill — ``_drill_target`` skips already-drilled
sources, so the second listing is the folder level): the model
thinks, then drills into the first source (id ``call_1``).
* ``("read", "source/path", <step 3 thinking>)`` — a
``tool``-role folder listing with file lines: the model
thinks, then reads the FIRST file line's document — the
labeled ``source:`` / ``path:`` fields (phase 63) joined into
the combined ``source/path`` (phase 70), id ``call_2``.
* ``("answer", "source/path", content, <step 4 thinking>)`` — a
``tool``-role read result (``"Document <source/path>:\n
<content>"``): the model thinks, then answers, quoting the
read document (the phase-37 shape). Reached regardless of the
``tools`` parameter (phase 45 keeps the tools offered until
the round cap).
* ``None`` — not the marker flow: the marker is absent, the
``<tools>`` section is not in the system prompt, or a
no-tools request with no tool results (e.g. ``agent_max_
rounds=0``) — the request behaves exactly as before.
"""
user = _user(body).lower()
if THINK_TOOLS_TRIGGER not in user:
return None
if "<tools>" not in _system(body):
return None
reads = _read_results(body)
if reads:
sp, content = reads[0]
return (
"answer",
sp,
content,
"Step 4: compose the answer from the read document.",
)
if not body.get("tools"):
return None
first = _first_file_line(body)
if first:
match = _CATALOG_LINE_RE.match(first)
if match:
return (
"read",
f"{match.group('source')}/{match.group('path')}",
"Step 3: read the file line's document.",
)
return None # a malformed line (never at runtime — defensive)
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "Step 2: drill into the first source.")
return ("list", "Step 1: list the documents to find the reference server config.")
def _reasoning_midround_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-161 marker request into one step of the
MID-ROUND thinking + delta + tool flow (see the module
docstring).
The two-request flow — the owner's 2026-10-05 TODO's
stays-open wire shape (the mid-round ``delta`` BEFORE the tool
call is what the client's stays-open container contract needs).
Discriminated statelessly from the messages, like the other
marker flows:
* ``("midround", <step 1 thinking>, <mid-round delta>)`` —
``tools`` are offered and no tool results are in the
messages yet: the model thinks, then emits a MID-ROUND
``delta`` (the content stream BEFORE the tool call), then
calls ``ls`` (no arguments, id ``call_0``) — the tool call
AFTER the delta.
* ``("final", <step 2 thinking>, <closing delta>)`` — a
``tool``-role result is in the messages: the model thinks,
then emits the closing ``delta`` and FINISHES with a content
finish (``finish_reason: "stop"`` — the ``done`` convention;
NO further tool call). Reached regardless of the ``tools``
parameter (phase 45 keeps the tools offered until the round
cap).
* ``None`` — not the marker flow: the marker is absent, the
``<tools>`` section is not in the system prompt, or a
no-tools request with no tool results (e.g. ``agent_max_
rounds=0``) — the request behaves exactly as before.
"""
user = _user(body).lower()
if REASONING_MIDROUND_TRIGGER not in user:
return None
if "<tools>" not in _system(body):
return None
if _tool_results(body):
return (
"final",
"Step 2: the listing confirms it.",
" — confirmed by the listing.",
)
if not body.get("tools"):
return None
return (
"midround",
"Step 1: I think I know the answer — let me check the documents.",
"The reference server is ref-server-1.",
)
def _ls_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-72 LS-TEACH request (see the module docstring).
* ``("misuse",)`` — ``tools`` are offered and no ``tool``-role
result is in the messages yet: the incident's misuse — ``ls``
with ``{"path": "."}`` (id ``call_0``), ``finish_reason:
"tool_calls"``, no content.
* ``("correct",)`` — a ``tool``-role result is in the messages and
no file line exists yet (the teaching refusal): the correction —
``ls`` with no arguments (id ``call_1``).
* ``("drill", source, "call_2")`` — the top-level source listing
is in the messages (the correction ran) but no file line yet
(phase 94: the top level carries sources only): the drill —
``ls`` scoped to the first source of the listing.
* ``("answer", line)`` — a ``source: X | path: Y | title: Z`` file
line is in the messages (the folder listing): the deterministic
content answer ``These are the indexed documents: <line>`` (the
first file line), ``finish_reason: "stop"`` — the loop settled
in ONE correction + ONE drill, not at the round cap.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or
``tools`` are not offered and no tool results are in the
messages yet (e.g. ``agent_max_rounds=0``).
"""
if LS_TEACH_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
line = _first_file_line(body)
if line is not None:
return ("answer", line)
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_2")
if _tool_results(body):
return ("correct",)
if not body.get("tools"):
return None
return ("misuse",)
def _grep_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a GREP-TEACH request (the 2026-09-05 incident — see the
``GREP_TEACH_*`` constants). Stateless over the messages, like the
other marker flows:
* ``("regex",)`` — ``tools`` are offered and no ``tool``-role result
is in the messages yet: the incident's regex-shaped first grep —
``grep`` with ``{"pattern": GREP_TEACH_PATTERN}`` (id ``call_0``).
* ``("plain",)`` — the LAST tool result is the server's TEACHING
no-match line (it carries ``GREP_TEACH_MARKER``): the one-round
correction — ``grep`` with the plain form (id ``call_1``).
* ``("read", combined, "call_2")`` — the last tool result carries
``source/path:line: text`` match lines: ``read`` the FIRST match
line's document by its combined ``source/path`` identity.
* ``("answer", combined, content)`` — the last tool result is a
read result (``"Document <combined>:\n<content>"``): the
deterministic echo answer ``Read <combined>. <first 80 chars>``.
* ``("nomatch",)`` — the last tool result is a PLAIN no-match (no
match line, no teaching marker): the deterministic terminal
``No matches — the knowledge base has no such text.`` answer.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or ``tools``
are not offered and no tool results are in the messages yet (e.g.
``agent_max_rounds=0``).
"""
user = _user(body).lower()
if GREP_TEACH_TRIGGER not in user:
return None
if "<tools>" not in _system(body):
return None
results = [
str(m.get("content") or "")
for m in _messages(body)
if m.get("role") == "tool"
]
if not results:
if not body.get("tools"):
return None
return ("regex",)
last = results[-1]
if last.startswith(_READ_RESULT_PREFIX):
head, _, content = last.partition("\n")
# The read result is ``"Document <combined>:\n<content>"`` — the
# head carries the server's appended ``:`` (removed here; a
# document path never legitimately ends with one).
return ("answer", head[len(_READ_RESULT_PREFIX):].removesuffix(":"), content)
if GREP_TEACH_MARKER in last:
return ("plain",)
for line in last.splitlines():
m = _SEARCH_LINE_RE.match(line)
if m:
return ("read", m.group("sp"), "call_2")
return ("nomatch",)
def _drill_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-94 scripted drill-down request (see the module
docstring). The question carries the scripted call
(``drill down the tree: ls [target]`` / ``drill down the tree: read
source/path``); the step is then discriminated statelessly from the
tool results, like the other marker flows:
* ``("call", verb, target, "call_0")`` — ``tools`` are offered and
no ``tool``-role result is in the messages yet: the scripted call
(``ls`` with NO path for the top level — empty target —, ``ls``
with the target for source/folder levels, ``read`` with the
combined ``source/path``).
* ``("recover", source)`` — the LAST tool result is the NOT-A-FOLDER
teaching refusal (it carries :data:`_NOT_A_FOLDER_MARKER`): the
scripted one-round recovery — ``ls`` the target's SOURCE segment
(the refusal listing the parent's subfolders is what lets the
model — and this mock — self-correct in the next round, the
phase-72 contract).
* ``("read_answer", combined, quote)`` — the LAST tool result is a
read result (``"Document <source/path>:\n<content>"``): the
deterministic echo answer ``Read <source/path>. <first 80 chars>
`` (the phase-37 single-read shape — the grounded-turn citation
contract the suite asserts).
* ``("ctx_answer", target, quote)`` — the scripted call was a
``read`` and the LAST tool result is the ALREADY_IN_CONTEXT
dedupe refusal (the target is already a top-2 retrieval document
— deterministic for the drill fixture: the read question names
the file's path, so the file self-matches the hybrid gate): the
model answers FROM THE PROMPT — ``Already in context: Read
<target>. <first 80 chars of the target document's text in the
``<documents>`` block>`` (the same citation shape as the
read-result branch — the document text reached the model either
way). Falls through to the echo when the target is not a
``<document>`` block (the premise broke — the suite fails
loudly on the answer).
* ``("echo", listing)`` — a listing landed (top-level or folder —
any other last result): the deterministic ECHO — the answer
carries the listing VERBATIM (``Here's the level I listed:\n
<listing>``), so the suite asserts on the exact tree level the
model saw (source ``— N documents`` lines + stored summaries,
subfolder lines, the ``source: X | path: Y | title: Z`` file
lines, the 50-line cap + note) through the rendered answer — the
house scripted-turn way of asserting on tool results (the mock is
the only E2E lens on the LLM's context).
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), the scripted
call is unparseable, or ``tools`` are not offered and no tool
results are in the messages yet (e.g. ``agent_max_rounds=0``).
"""
user = _user(body)
if DRILL_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _DRILL_CALL_RE.search(user)
if match is None:
return None
verb = match.group("verb")
target = match.group("arg") or ""
results = _tool_results(body)
if not results:
if not body.get("tools"):
return None
return ("call", verb, target, "call_0")
last = results[-1]
if last.startswith(_READ_RESULT_PREFIX):
head, _, content = last.partition("\n")
# The read result is ``"Document <source/path>:\ndate: …\n<content>"``
# — the head carries the server's appended ``:`` (removed here;
# a document path never legitimately ends with one); the
# phase-106 D5 ``date:`` line sits between the header and the
# document text — skipped so the quote stays pure document
# content (the suite's byte-identical pin).
combined = head[len(_READ_RESULT_PREFIX):].strip().removesuffix(":")
if content.startswith("date: "):
content = content.partition("\n")[2]
return ("read_answer", combined, content[:80])
if _ALREADY_IN_CONTEXT_MARKER in last and verb == "read" and "/" in target:
# Phase 118: the dedupe fires only for a document ALREADY READ
# in the same turn (the seeds are summaries — a first read of
# a suggested document succeeds). Answer from the prompt (the
# refusal's instruction), quoting the block's text (now the
# document's summary).
src, _, p = target.partition("/")
content = _document_block(_system(body), src, p)
if content is not None:
return ("ctx_answer", target, content[:80])
if _NOT_A_FOLDER_MARKER in last:
return ("recover", target.split("/")[0])
return ("echo", last)
# ---------------------------------------------------------------------------
# Phase 95 (task 03, the read cap's dedicated story suite):
# the deterministic SCRIPTED capped read — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the scripted CAPPED-READ flow (the read cap's story suite,
#: ``tests/e2e/test_read_truncation_cap.py``): the question carries its
#: own tool call after the colon — ``read the capped document: read
#: source/path`` — the mock emits the scripted ``read``, then ECHOES
#: the ENTIRE tool result into its answer (the house scripted-turn lens
#: on the LLM's context: the truncated shape — first cap chars +
#: ``[…truncated…]`` + the pinned grep-pointer notice — or the plain
#: shape, byte-identical to the pre-phase-95 result, lands in the
#: rendered answer, and the suite asserts both directions through it).
#: Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
#: phrases — the phase-71/72/94 ordering convention); verified
#: 2026-09-10: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
READ_CAP_TRIGGER = "read the capped document"
#: The scripted call in the read-cap question (case-insensitive — the
#: suite's questions capitalize the trigger's first letter): the verb
#: (``read``) plus the target — a combined ``source/path``, parsed from
#: the RAW user message so the target keeps its case. The target is a
#: ``[a-z0-9_./-]`` run (case-insensitively), so the suite's `` — ``
#: flavor separator (em dash) can never bleed into it (the drill-down
#: convention, ``_DRILL_CALL_RE``).
_READ_CAP_CALL_RE = re.compile(
r"read the capped document:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
def _read_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-95 scripted read-cap request (see the module
docstring). The question carries the scripted call (``read the
capped document: read source/path``); the step is then discriminated
statelessly from the tool results, like the other marker flows:
* ``("call", target, "call_0")`` — ``tools`` are offered and no
``tool``-role result is in the messages yet: the scripted
``read`` on the parsed target (synthetic id ``call_0``).
* ``("echo", result)`` — a ``tool``-role result is in the messages:
the deterministic ECHO — the answer carries the LAST tool result
VERBATIM (``Here's what the read returned:\n<result>``): a read
result (``"Document <source/path>:…``) lands in the answer with
its full content — the truncation marker + the pinned notice when
the cap fired, the plain body when it did not — and a refusal
(the premise broke) lands just as visibly, so the suite fails
loudly on it.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), the scripted
call is unparseable, or ``tools`` are not offered and no tool
results are in the messages yet (e.g. ``agent_max_rounds=0``).
"""
user = _user(body)
if READ_CAP_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _READ_CAP_CALL_RE.search(user)
if match is None:
return None
results = _tool_results(body)
if not results:
if not body.get("tools"):
return None
return ("call", match.group("arg"), "call_0")
return ("echo", results[-1])
# ---------------------------------------------------------------------------
# Phase 118 (task 06, the summary-seed context's dedicated story suite):
# the deterministic SCRIPTED summary read — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the scripted SUMMARY-READ flow (the summary-seed context's story
#: suite, ``tests/e2e/test_summary_seed_context.py``): the question
#: carries its own tool call after the colon — ``read the suggested
#: document: read source/path`` — the mock emits the scripted ``read``
#: (phase 118: the seeds are summaries, so a first read of a suggested
#: document SUCCEEDS — the full text arrives through the read), then
#: ECHOES the ENTIRE tool result into its answer (the house
#: scripted-turn lens on the LLM's context — the full content's tail
#: reaches the rendered answer iff the read delivered it). Checked
#: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases —
#: the phase-71/72/94 ordering convention); no existing E2E question
#: or fixture file contains the phrase, so every other suite is
#: unaffected.
SUMMARY_SEED_READ_TRIGGER = "read the suggested document"
#: The scripted call in the summary-read question (case-insensitive —
#: the suite's questions capitalize the trigger's first letter): the
#: verb (``read``) plus the target — a combined ``source/path``, parsed
#: from the RAW user message so the target keeps its case. The target
#: is a ``[a-z0-9_./-]`` run (case-insensitively), so the suite's
#: `` — `` flavor separator (em dash) can never bleed into it (the
#: read-cap/drill convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``).
_SUMMARY_SEED_READ_CALL_RE = re.compile(
r"read the suggested document:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
# ---------------------------------------------------------------------------
# Phase 124 (task 07, image chips and answers, TODO L3): the scripted
# SHOW_IMAGE flow + the zero-read bug-fix trigger — see the module
# docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``show_image`` prompt clause in the system
#: prompt (the ``BOR_ANSWER_IMAGES`` toggle's server-side marker —
#: the clause rides inside the HIGH prompt's ``<tools>`` section ONLY
#: when the toggle is on) — drives the deterministic SHOW_IMAGE flow
#: (the mock shows the seeded image document and answers straight
#: from the seed — ZERO ``read`` calls). On a default-OFF app the
#: clause is absent, so the trigger NEVER fires there and the mock
#: behaves exactly as pre-phase for every other suite. Disjoint from
#: every other trigger phrase — the phase-71/72/94/118 ordering
#: convention; verified 2026-09-26: no existing E2E question or
#: fixture file contains the phrase, so every other suite is
#: unaffected.
SHOW_IMAGE_TRIGGER = "show me the picture of"
#: The phase-124 BUG-FIX-ONLY trigger (case-insensitive): the mock
#: answers straight from the seed with ZERO tool calls and QUOTES THE
#: TAIL of the seeded image doc's ``<document>`` block — proving the
#: description was in the prompt (for an image doc the seed block IS
#: its entire content) and that its chip appears with NO tool calls
#: at all (the phase-124 auto-cite). Unlike ``SHOW_IMAGE_TRIGGER``
#: it needs NO clause — the chips fix is not gated by
#: ``BOR_ANSWER_IMAGES`` (it belongs to the ``BOR_IMAGES``-indexed
#: docs), so it also fires on a default-OFF app. Checked BEFORE the
#: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker
#: lives in the user message — the TABLE/HISTORY convention), so a
#: marker question always gets the echo whatever the honesty gate
#: says.
DIAGRAM_TRIGGER = "which diagram is in your notes"
#: The phase-125 (task 06, image-anchor context, LOCKED A4) trigger
#: (case-insensitive): the mock QUOTES THE CONTEXT LINE of the seeded
#: image doc's ``<document>`` block (the same image-doc pick as
#: :data:`DIAGRAM_TRIGGER`). For an ENRICHED image doc the block
#: leads with the one-line ``Referenced in <source>/<path>: "<alt>"``
#: line (the task-05 rendering of the stored ``image_context``,
#: BEFORE the description) — quoting the block's first line proves
#: the line reached the prompt; for a non-enriched (standalone) doc
#: the block is the bare description (byte-identical pre-phase) and
#: its first line never starts with ``Referenced in `` (the mock's
#: fixed description cannot begin that way) — the mock then answers
#: that the image carries no reference line. Checked BEFORE the
#: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker
#: lives in the user message — the TABLE/HISTORY convention), so a
#: marker question always gets the echo whatever the honesty gate
#: says. Disjoint from every other trigger phrase; verified
#: 2026-09-27: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
CONTEXT_LINE_TRIGGER = "quote the image reference"
#: The agent's ``show_image`` SUCCESS result's confirming first line
#: (``app.rag.agent``'s ``show_image`` branch — the line ``read``'s
#: result never carries): the SHOW_IMAGE flow's follow-up request
#: keys on it in a ``tool``-role result (stateless discrimination —
#: the house marker-flow convention).
_SHOW_CONFIRM_MARKER = "Image shown to the user"
#: The image-doc extension family (``app.rag.summarizer.IMAGE_MIMES``
#: — phase 122's ext→mime map, imported so the mock can never drift
#: from the app's image family): a seeded ``<documents>`` block is
#: the IMAGE document when its ``path`` attribute ends in one of
#: these (the suite's single-PNG fixture guarantees exactly one such
#: block — the pick is deterministic).
_IMAGE_PATH_SUFFIXES = tuple(IMAGE_MIMES)
def _seeded_image_doc(system: str) -> tuple[str, str, str] | None:
"""The seeded image doc's ``(source, path, content)`` — or ``None``.
Phase 124 (task 07): the ``<documents>`` block (the
:data:`_DOCUMENT_BLOCK_RE` machinery) whose ``path`` attribute
ends in an image extension (:data:`_IMAGE_PATH_SUFFIXES`) is the
seeded image document — for an image doc the block's content IS
the document's entire text (the vision description — content ==
summary, phase 122 task 03). Both of this phase's triggers pick
the doc this way (the suite's single-PNG fixture guarantees
exactly one such block; ``None`` = a text-only KB, the premise
broke — the caller degrades loudly).
"""
for block in _DOCUMENT_BLOCK_RE.finditer(system):
if block.group("path").lower().endswith(_IMAGE_PATH_SUFFIXES):
return (block.group("source"), block.group("path"), block.group("content"))
return None
def _seed_context_line(block_content: str) -> str | None:
"""The context line of a seeded image doc's block (phase 125
task 06, LOCKED A4; phase 137 task 04/05) — or ``None``.
An ENRICHED image doc's ``<document>`` block leads with the
one-line context rendering (the task-05/task-04
:func:`app.rag.prompts._extract_context_line` output, BEFORE the
description): the one-line ``Referenced in …`` reference shape
(phase 125) or the one-line ``From <source>/<pdf> (page N)``
provenance shape (phase 137 — a PDF-embedded image, whose block
leads with the document-context clause, a combined context
joining both with ``; ``). Its first line, verbatim, is the line
the mock quotes (the prompt-injection echo convention). A
non-enriched block is the bare description — its first line never
starts with ``Referenced in `` or ``From `` (the mock's fixed
:data:`IMAGE_DESCRIPTION_ANSWER` begins "A network diagram" — the
context-aware :data:`IMAGE_DESCRIPTION_CONTEXT_ANSWER` begins
"From " but only ever rides a doc whose stored context makes the
``From …`` CONTEXT LINE the block's first line, so the bare
description is never the first line for it), so ``None`` pins the
byte-identical pre-phase shape.
"""
first = block_content.split("\n", 1)[0].strip()
if first.startswith("Referenced in ") or first.startswith("From "):
return first
return None
def _show_image_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-124 SHOW_IMAGE request (see the module
docstring). Stateless over the messages, like the other marker
flows:
* ``("call", combined)`` — the system prompt carries the
``show_image`` clause (the toggle-ON gate) and no ``tool``-role
result has confirmed the show yet: stream ONLY the
``show_image`` tool-call delta on the seeded image doc's
combined ``source/path`` (synthetic id ``call_show``,
``finish_reason: "tool_calls"``, no content) — NO ``read`` (the
zero-read turn).
* ``("answer",)`` — a ``tool``-role result carrying the agent's
confirming line (:data:`_SHOW_CONFIRM_MARKER`) is in the
messages: the deterministic FINAL answer — a short sentence
ending with the ``<documents>`` block's tail echo (the phase-24
END_OF_NOTES echo convention — it proves the seed was there),
NO further tool calls.
* ``None`` — not the flow: the trigger phrase is absent, the
clause is absent (a default-OFF app — the mock behaves exactly
as pre-phase), the turn is deflected (no ``<documents>`` seed
— a deflection prompt never carries one), or no seeded image
doc exists (a text-only KB — the premise broke; the suite fails
loudly on the missing figure).
"""
if SHOW_IMAGE_TRIGGER not in _user(body).lower():
return None
system = _system(body)
if "show_image" not in system:
return None
doc = _seeded_image_doc(system)
if doc is None:
return None
if any(_SHOW_CONFIRM_MARKER in r for r in _tool_results(body)):
return ("answer",)
if not body.get("tools"):
return None
return ("call", f"{doc[0]}/{doc[1]}")
def _summary_seed_read_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-118 scripted summary-read request (see the
module docstring). The question carries the scripted call (``read
the suggested document: read source/path``); the step is then
discriminated statelessly from the tool results, like the other
marker flows:
* ``("call", target, "call_0")`` — ``tools`` are offered and no
``tool``-role result is in the messages yet: the scripted
``read`` on the parsed target (synthetic id ``call_0``).
* ``("echo", result)`` — a ``tool``-role result is in the messages:
the deterministic ECHO — the answer carries the LAST tool result
VERBATIM (``Here's what the read returned:\n<result>``): a read
result (``"Document <source/path>:…`` — header + date line +
FULL content) lands in the answer with its tail intact (the
full text the ``read`` tool delivered — the phase-118 contract:
a first read of a suggested document succeeds, the seed was a
summary); a refusal (the premise broke) lands just as visibly,
so the suite fails loudly on it.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), the scripted
call is unparseable, or ``tools`` are not offered and no tool
results are in the messages yet (e.g. ``agent_max_rounds=0``).
"""
user = _user(body)
if SUMMARY_SEED_READ_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _SUMMARY_SEED_READ_CALL_RE.search(user)
if match is None:
return None
results = _tool_results(body)
if not results:
if not body.get("tools"):
return None
return ("call", match.group("arg"), "call_0")
return ("echo", results[-1])
# ---------------------------------------------------------------------------
# Phase 131 (task 04, the read line-ranges' dedicated story suite):
# the deterministic grep -> read(offset/limit) dance — see the module
# docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic RANGE-READ flow (the read line-ranges' story
#: suite, ``tests/e2e/test_read_line_ranges.py``): the owner's stuck
#: scenario (TODO.md L7) — the huge document: ``grep`` finds the
#: line, the ``read`` lands exactly that part with ``offset`` /
#: ``limit``, and the answer echoes the read slice. Checked BEFORE
#: the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the
#: phase-71/72/94/118 ordering convention); verified 2026-09-28: no
#: existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected.
READ_LINES_TRIGGER = "read the lines"
#: The canned pattern the flow greps for (the suite's fixture document
#: carries exactly one line containing it — the flow's read target
#: line; the sentinel convention of ``SEARCH_PATTERN``).
READ_LINES_PATTERN = "LINE_MARKER"
#: The line count the flow's range read requests (a PART of the
#: document, not the whole — the owner's dance, TODO.md L7).
READ_LINES_LIMIT = 30
def _read_call_offset(body: dict[str, Any]) -> int:
"""The offset the model's ``read`` call passed (phase 131, task 04).
The agent loop echoes every executed call back into the messages
(the assistant ``tool_calls`` message, the arguments as a JSON
string — ``app/rag/agent.py``), so the range-read request
(request 3 of the flow) carries the call it was made from: this
parses the FIRST assistant ``read`` call's ``offset``. ``0`` when
absent or unparseable (the premise broke — the suite fails
loudly on ``line 0``).
"""
for m in _messages(body):
if m.get("role") != "assistant":
continue
# The agent loop's echoed call (the ``tool_calls`` LIST on the
# assistant message — ``app/rag/agent.py``; :func:`_messages`
# is typed for string content, so the list comes back as
# ``Any`` here — the house mock convention over the wire
# shape).
raw_calls: Any = m.get("tool_calls")
for call in (raw_calls if isinstance(raw_calls, list) else []):
if not isinstance(call, dict):
continue
fn = call.get("function") or {}
if not isinstance(fn, dict) or fn.get("name") != "read":
continue
try:
args = json.loads(fn.get("arguments") or "{}")
except json.JSONDecodeError:
continue
offset = args.get("offset") if isinstance(args, dict) else None
if isinstance(offset, int) and not isinstance(offset, bool):
return offset
return 0
def _read_lines_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-131 range-read request (see the module
docstring). Classified statelessly by the tool results in the
messages (the house count pattern), like the other marker flows:
* ``("answer", offset, quote)`` — a ``tool``-role read result
(content starting with the agent's ``"Document "`` prefix) is in
the messages: the forced answer, byte-stable — ``Found it at
line N. <quote>`` with N the offset the model passed (parsed
from the read call's arguments in the assistant tool-call
message — a string on this return for the homogeneous
``tuple[str, ...]`` shape; the dispatch formats it into the
answer, the display surface) and quote the read result's
content with the header lines (the phase-106 D5 ``date:`` line
+ the phase-131 ``lines: a-b of N`` extent) skipped — the pure
read slice, first 80 chars (the phase-94 read-echo convention:
the RANGE reached the model's context).
* ``("read", combined, line, "call_1")`` — a ``tool``-role grep
result carrying ``source/path:line: text`` match lines is in the
messages: the RANGE READ on the FIRST match line's combined
``source/path`` — ``offset`` = the parsed line number (the
grep's own string here — the dispatch parses the int for the
wire) and ``limit`` = :data:`READ_LINES_LIMIT`.
* ``("nomatch",)`` — a tool result is in the messages but it is
neither a read result nor a grep match (the grep found nothing
— the premise broke): the deterministic terminal answer (the
grep-teach ``nomatch`` line — the suite fails loudly on the
missing marker).
* ``("grep",)`` — ``tools`` are offered and no tool results are in
the messages yet: the first step — ``grep`` with
``{"pattern": READ_LINES_PATTERN}`` (id ``call_0``).
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or
``tools`` are not offered and no tool results are in the
messages yet (e.g. ``agent_max_rounds=0``).
"""
if READ_LINES_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
reads = _read_results(body)
if reads:
_sp, content = reads[-1]
# The read result's header lines sit between the ``Document
# …:`` header and the document text — the phase-106 D5
# ``date:`` line always, the phase-131 ``lines: a-b of N``
# extent on a range read (a plain read carries no extent
# line) — skipped (the drill-flow convention) so the quote is
# the pure read content (the suite's marker-line assertion).
if content.startswith("date: "):
content = content.partition("\n")[2]
if content.startswith("lines: "):
content = content.partition("\n")[2]
return ("answer", str(_read_call_offset(body)), content[:80])
for result in _tool_results(body):
if result.startswith(_READ_RESULT_PREFIX):
continue
for line in result.splitlines():
match = _SEARCH_LINE_RE.match(line)
if match:
return (
"read",
match.group("sp"),
match.group("line"),
"call_1",
)
if _tool_results(body):
return ("nomatch",)
if not body.get("tools"):
return None
return ("grep",)
# ---------------------------------------------------------------------------
# Phase 139 (task 03): the owner's "same file over and over" scenario —
# the model range-reads the SAME document TWICE with different
# offset/limit, so the two Reading lines are distinguishable. See the
# module docstring.
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic SAME-FILE-TWICE flow (the read range-tool-lines'
#: story suite, ``tests/e2e/test_read_range_tool_lines.py`` — the
#: owner's "same file over and over" confusion, TODO.md L4). Checked
#: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases —
#: the phase-71/72/94/118/131 ordering convention); verified
#: 2026-09-30: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
SAME_FILE_TRIGGER = "read the same file twice"
#: The scripted target in the same-file-twice question (case-
#: insensitive — the suite's question capitalizes the trigger's first
#: letter): the verb (``read``) plus the target — a combined
#: ``source/path``, parsed from the RAW user message so the target
#: keeps its case. The target is a ``[a-z0-9_./-]`` run (case-
#: insensitively), so a `` — `` flavor separator can never bleed into
#: it (the read-cap / drill-down convention, ``_READ_CAP_CALL_RE`` /
#: ``_DRILL_CALL_RE``).
_SAME_FILE_CALL_RE = re.compile(
r"read the same file twice:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
#: The FIRST range read's requested extent — the model's REQUEST
#: (ASSUMPTION A1: the suffix shows the request, not the actual slice
#: extent). ``offset`` 10 + ``limit`` 20 renders `` (lines 10–29)``.
SAME_FILE_FIRST_OFFSET = 10
SAME_FILE_FIRST_LIMIT = 20
#: The SECOND range read's requested extent — the SAME file again, a
#: different range (the owner's confusion made visible). ``offset`` 40
#: + ``limit`` 15 renders `` (lines 40–54)``.
SAME_FILE_SECOND_OFFSET = 40
SAME_FILE_SECOND_LIMIT = 15
def _same_file_quote(doc_content: str) -> str:
"""The phase-94/131 read-echo quote of ONE read result's content.
The read result's header lines sit between the ``Document …:``
header and the document text — the phase-106 D5 ``date:`` line
always, the phase-131 ``lines: a-b of N`` extent on a range read
(a plain read carries no extent line) — skipped (the drill-flow
convention) so the quote is the pure read content, first 80 chars
(the RANGE reached the model's context). Mirrors the phase-131
``_read_lines_flow`` quote logic so the two flows' echo shapes
stay byte-consistent.
"""
content = doc_content
if content.startswith("date: "):
content = content.partition("\n")[2]
if content.startswith("lines: "):
content = content.partition("\n")[2]
return content[:80]
def _same_file_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-139 same-file-twice request (see the module
docstring). The question carries its own target (``read the same
file twice: read source/path``); the step is then discriminated
statelessly by the READ results in the messages (the house count
pattern — the flow only ever issues ``read`` calls, so counting
read results is reliable):
* ``("answer", quote1, quote2)`` — TWO or more ``tool``-role read
results are in the messages (both range reads landed): the
forced answer, byte-stable, in the phase-131 read-echo shape —
``Read the same file twice — lines 10–29: <q1>; lines 40–54:
<q2>`` where each ``<qN>`` is that read result's pure content
(header lines skipped, first 80 chars — the :func:
``_same_file_quote`` convention).
* ``("read2", target)`` — EXACTLY ONE read result is in the
messages (the first read landed): the SECOND range read on the
SAME target — ``offset`` = :data:`SAME_FILE_SECOND_OFFSET`,
``limit`` = :data:`SAME_FILE_SECOND_LIMIT` (id ``call_1``).
* ``("read1", target)`` — ``tools`` are offered and no read result
is in the messages yet: the FIRST range read on the parsed
target — ``offset`` = :data:`SAME_FILE_FIRST_OFFSET`, ``limit``
= :data:`SAME_FILE_FIRST_LIMIT` (id ``call_0``).
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), the
target is unparseable, or ``tools`` are not offered and no read
result is in the messages yet (e.g. ``agent_max_rounds=0``).
"""
user = _user(body)
if SAME_FILE_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _SAME_FILE_CALL_RE.search(user)
if match is None:
return None
target = match.group("arg")
reads = _read_results(body)
if len(reads) >= 2:
return (
"answer",
_same_file_quote(reads[0][1]),
_same_file_quote(reads[1][1]),
)
if len(reads) == 1:
return ("read2", target)
if not body.get("tools"):
return None
return ("read1", target)
# ---------------------------------------------------------------------------
# Phase 141 (task 04): the owner's stuck scenario — the model
# range-reads ONE document in TWO different ranges; the
# coverage-tracked server proceeds with the second (the pre-phase
# pair-set dedupe refused it ``ALREADY_IN_CONTEXT`` while only the
# first slice was in the prompt). See the module docstring.
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic TWO-RANGES flow (the read range-dedupe fix's
#: story suite, ``tests/e2e/test_read_range_dedupe.py`` — the
#: owner's stuck scenario, TODO.md L3). Checked BEFORE the plain
#: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the
#: phase-71/72/94/118/131/139 ordering convention); verified
#: 2026-10-01: no existing E2E question or fixture file contains
#: the phrase, so every other suite is unaffected.
TWO_RANGES_TRIGGER = "read two ranges"
#: The scripted target in the two-ranges question (case-
#: insensitive — the suite's question capitalizes the trigger's
#: first letter): the verb (``read``) plus the target — a combined
#: ``source/path``, parsed from the RAW user message so the target
#: keeps its case (the ``_SAME_FILE_CALL_RE`` convention; the target
#: is a ``[a-z0-9_./-]`` run, so a `` — `` flavor separator can
#: never bleed into it).
_TWO_RANGES_CALL_RE = re.compile(
r"read two ranges:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
#: The FIRST range read's requested extent — range A (lines 1–100
#: of the suite's 200-line fixture document).
TWO_RANGES_FIRST_OFFSET = 1
TWO_RANGES_FIRST_LIMIT = 100
#: The SECOND range read's requested extent — range B (lines 101–
#: …): the SAME file again, a different range. Pre-phase the server
#: refused this call ``ALREADY_IN_CONTEXT`` (the false "full text is
#: in your prompt" claim — the owner's stuck turn); the coverage-
#: tracked server proceeds with it (its lines are NOT covered by
#: the first read's span).
TWO_RANGES_SECOND_OFFSET = 101
TWO_RANGES_SECOND_LIMIT = 100
def _two_ranges_extent(content: str) -> str | None:
"""The ``lines: a-b of N`` extent line of one read result's
content (the ``Document …:`` header already stripped by
:func:`_read_results`, the phase-106 D5 ``date:`` line skipped)
— ``None`` when the result carries no extent line (a plain
read). The flow keys its step on this line (the mock's own
keying, asserted — a refusal or a differently-shaped result is
the broken premise, never a step input).
"""
body = content
if body.startswith("date: "):
body = body.partition("\n")[2]
if body.startswith("lines: "):
return body.partition("\n")[0]
return None
def _two_ranges_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-141 two-ranges request (see the module
docstring). The question carries its own target (``read two
ranges: read source/path``); the step is then discriminated
statelessly by the tool results in the messages (the house count
pattern — the flow only ever issues ``read`` calls, so counting
read results is reliable):
* ``("answer", quote)`` — TWO or more ``tool``-role read results
are in the messages and the SECOND carries the ``lines: 101-
…`` extent header (asserted — the second range read landed as
advertised): the forced answer, byte-stable, in the existing
read-echo shape — ``Both parts read. <quote>`` with quote the
second result's pure content (header lines skipped, first 80
chars — the :func:`_same_file_quote` convention; the marker
line lives in range B, so an echo of it proves the slice
reached the model).
* ``("read2", target)`` — EXACTLY ONE read result is in the
messages and it carries the ``lines: 1-100 of …`` extent
header (asserted — the first range read landed as advertised):
the SECOND range read on the SAME target — ``offset`` =
:data:`TWO_RANGES_SECOND_OFFSET`, ``limit`` =
:data:`TWO_RANGES_SECOND_LIMIT` (id ``call_1``).
* ``("read1", target)`` — ``tools`` are offered and no tool
result is in the messages yet: the FIRST range read on the
parsed target — ``offset`` = :data:`TWO_RANGES_FIRST_OFFSET`,
``limit`` = :data:`TWO_RANGES_FIRST_LIMIT` (id ``call_0``).
* ``("bad", result)`` — a tool result is in the messages that is
not a well-keyed read result for the step (a refusal — the
second range read was refused, the pre-phase bug — or an
unkeyed extent header): the deterministic broken-premise echo
— the offending result verbatim under ``Premise broke:`` (the
read-cap flow's broken-premise convention — a suite fails
loudly on it).
* ``None`` — not the flow: the trigger is absent, the
``<tools>`` section is missing (deflected turns never carry
it), the target is unparseable, or ``tools`` are not offered
and no tool result is in the messages yet (e.g.
``agent_max_rounds=0``).
"""
user = _user(body)
if TWO_RANGES_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _TWO_RANGES_CALL_RE.search(user)
if match is None:
return None
target = match.group("arg")
reads = _read_results(body)
results = _tool_results(body)
if len(reads) >= 2:
extent = _two_ranges_extent(reads[1][1])
if extent is None or not extent.startswith(
f"lines: {TWO_RANGES_SECOND_OFFSET}-"
):
return ("bad", reads[1][1])
return ("answer", _same_file_quote(reads[1][1]))
if len(reads) == 1:
if len(results) > 1:
# A NON-read tool result after the read result — a
# refusal (the pre-phase bug: the second range read was
# refused ``ALREADY_IN_CONTEXT``): the premise broke —
# echo it so a suite fails loudly.
return ("bad", results[-1])
extent = _two_ranges_extent(reads[0][1])
if extent is None or not extent.startswith(
f"lines: {TWO_RANGES_FIRST_OFFSET}-{TWO_RANGES_FIRST_LIMIT} of "
):
return ("bad", reads[0][1])
return ("read2", target)
if results:
# A tool result that is NOT a read result before any read
# landed — the first read was refused: the premise broke.
return ("bad", results[-1])
if not body.get("tools"):
return None
return ("read1", target)
# ---------------------------------------------------------------------------
# Phase 145 (task 04): the git-source-links' dedicated story suite — the
# owner's "the answer points at the original git source" scenario. See
# the module docstring.
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt —
#: drives the deterministic SCRIPTED ORIGIN-LINK flow (the
#: git-source-links' story suite,
#: ``tests/e2e/test_git_source_links.py`` — the owner's scenario,
#: TODO.md L7: the answer grounded in a git-sourced document must
#: carry the way to the complete original in ONE step). Checked
#: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
#: phrases — the phase-71/72/94/118/131/139/141 ordering
#: convention; the trigger needs the ``<tools>`` section, so
#: deflected turns never hit it); verified 2026-10-01: no
#: existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected.
ORIGIN_LINK_TRIGGER = "original source link"
#: The scripted call in the origin-link question (case-insensitive
#: — the suite's questions capitalize the trigger's first
#: letter): the verb (``read``) plus the target — a combined
#: ``source/path``, parsed from the RAW user message so the target
#: keeps its case. The target is a ``[a-z0-9_./-]`` run (case-
#: insensitively), so the suite's `` — `` flavor separator (em
#: dash) can never bleed into it (the read-cap / drill-down
#: convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``).
_ORIGIN_LINK_CALL_RE = re.compile(
r"original source link:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
#: The flow's FINAL answer for a read result that carries the
#: phase-145 ``origin:`` line — the DETERMINISTIC string carrying
#: the repository URL VERBATIM (the model-side contract, owner-
#: confirmed 2026-09-30: the answer includes the original source).
#: ``{url}`` is parsed from the read result's ``origin:`` line BY
#: THE MOCK ITSELF — the mock cannot fabricate it, so the URL in
#: the answer is the server-side half's assertion (the real
#: server's resolver put the line in the result the model saw).
ORIGIN_LINK_ANSWER = (
"The full source lives at {url} — see the repo for the complete original."
)
#: The flow's FINAL answer for a read result that carries NO
#: ``origin:`` line (a local-sourced document — the byte-identity
#: control): the deterministic no-origin sentence (its wording
#: carries NO ``origin:`` substring — the suite's absence
#: assertion keys off the line's exact prefix).
ORIGIN_LINK_NO_ORIGIN = (
"No origin line in the read result — the document is not from a git source."
)
#: The house scripted-turn echo header (the ``READ_CAP_TRIGGER``
#: flow's convention): the read result rides VERBATIM into the
#: answer, so the rendered output is the E2E's lens on the
#: ``origin:``/``branch:`` lines' presence (and their absence in
#: the local-source control).
_ORIGIN_LINK_ECHO_HEADER = "Here's what the read returned:"
def _origin_link_url(result: str) -> str | None:
"""The read result's ``origin:`` line's URL (the phase-145
task-01 line — the sanitized repository URL the real server's
resolver put in the result) — or ``None`` when the result
carries no origin line (a local-sourced document, no git row,
or an ambiguous basename).
The search is position-independent (any line with the exact
``origin: `` prefix): the phase-145 lines sit between the
``date:`` line and the content, but the parse never drifts if
the result's header shape changes. A document BODY line that
merely starts with ``origin: `` would be a fixture defect (the
suite's fixtures are pinned content — no such line), never a
mock defect.
"""
for line in result.splitlines():
if line.startswith("origin: "):
return line[len("origin: "):]
return None
def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-145 scripted origin-link request (see the
module docstring). The question carries the scripted call
(``original source link: read source/path``); the step is then
discriminated statelessly from the tool results, like the
other marker flows:
* ``("call", target, "call_0")`` — ``tools`` are offered and
no ``tool``-role result is in the messages yet: the scripted
``read`` on the parsed target (synthetic id ``call_0``).
* ``("answer", url, result)`` — a ``tool``-role READ result
(content starting with the agent's ``"Document "`` prefix —
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
``branch:`` lines sit BELOW the header, so the first-line
parsing is untouched) is in the messages: the deterministic
FINAL answer — ``url`` the result's ``origin:`` line's URL
(or ``""`` — the no-origin sentence, the local-source
control; the empty-string sentinel keeps the return
homogeneous ``tuple[str, …]``, the ``_read_lines_flow``
convention) and ``result`` the read result VERBATIM (the
house scripted-turn echo — the rendered output is the lens
on the origin lines).
* ``("bad", result)`` — a tool result is in the messages that
is NOT a read result (a refusal — the premise broke): the
deterministic broken-premise echo — the offending result
VERBATIM under ``Premise broke:`` (the two-ranges convention
— the suite fails loudly on it).
* ``None`` — not the flow: the trigger is absent, the
``<tools>`` section is missing (deflected turns never carry
it), the scripted call is unparseable, or ``tools`` are not
offered and no tool results are in the messages yet (e.g.
``agent_max_rounds=0``).
"""
user = _user(body)
if ORIGIN_LINK_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _ORIGIN_LINK_CALL_RE.search(user)
if match is None:
return None
target = match.group("arg")
results = _tool_results(body)
if results:
last = results[-1]
if not last.startswith(_READ_RESULT_PREFIX):
return ("bad", last)
# The empty-string sentinel (never a real URL — a URL is
# always non-empty: the resolver returns the stored URL's
# sanitized form, which is a non-empty string): keeps the
# return homogeneous ``tuple[str, …]`` (pyright), the
# ``_read_lines_flow`` convention.
return ("answer", _origin_link_url(last) or "", last)
if not body.get("tools"):
return None
return ("call", target, "call_0")
#: Phase 160 (task 05): the round-cap flow's incrementing call ids
#: — ``call_cap_1``, ``call_cap_2``, … The NEXT id is derived from
#: the ids already in the messages' ``assistant`` tool calls (the
#: flow is stateless over the message history — a retry that
#: re-issues the same request re-derives the same next id).
_CAP_CALL_ID_RE = re.compile(r"^call_cap_(\d+)$")
def _round_cap_next_call_id(body: dict[str, Any]) -> str:
"""The next incrementing ``call_cap_N`` id (stateless).
The mock's ``ls`` calls in the round-cap flow carry
incrementing synthetic ids — the NEXT id is one past the
highest ``call_cap_N`` id already in the messages'
``assistant`` tool calls (``call_cap_1`` on the first
request — no ids yet).
"""
n = 0
for m in _messages(body):
if m.get("role") != "assistant":
continue
for call in m.get("tool_calls") or []:
if not isinstance(call, dict):
continue
match = _CAP_CALL_ID_RE.match(str(call.get("id") or ""))
if match:
n = max(n, int(match.group(1)))
return f"call_cap_{n + 1}"
def _round_cap_seed_path(body: dict[str, Any]) -> str:
"""The seed document's combined ``source/path`` — or the
sentinel (the premise broke: no ``<documents>`` block; the
suite fails loudly on the sentinel in the dump call).
Phase 160 (task 05): the FORCED FINAL's request (the
round-cap path) still carries the grounded system prompt —
the ``<documents>`` section's FIRST ``<document>`` block is
the seed document (the suite seeds exactly one). The mock
joins the labeled ``source`` / ``path`` fields (the
``read`` tool's combined identity, phase 70).
"""
block = _DOCUMENT_BLOCK_RE.search(_system(body))
if not block:
return "seed/unknown.md"
return f"{block.group('source')}/{block.group('path')}"
def _round_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Phase 160 (task 05): the ROUND-CAP marker flow — or ``None``.
Discriminated on the LAST user message (the mock's existing
invariant: the last ``user`` message is the current
question, the earlier non-system messages are prior turns —
the ``_history_echo`` ``last_user`` scan convention):
* the last user message carries :data:`ROUND_CAP_TRIGGER`:
- the request offers ``tools`` (the app issues such a
request only while the round budget is unspent — no
tool result answers the last emitted call yet):
``("call", call_cap_N)`` — the model NEVER stops
asking (``ls``, no arguments, the next incrementing
id) — the turn is guaranteed to hit
``BOR_AGENT_MAX_ROUNDS``;
- the request carries NO ``tools`` (the forced final —
the round-cap path; the trigger is still the last user
message): ``("dump", <seed path>)`` — a ``read`` on
the seed path (id ``call_cap_dump``, NO content) — the
exact dumped-call shape the phase kills;
* the last user message is EXACTLY ``continue`` (the UI's
``CONTINUE_QUESTION``) and an EARLIER user message
carries the trigger (the client history rides in the
stateless A10 request): ``("continue",)`` — the stable
:data:`ROUND_CAP_CONTINUE_ANSWER` (a fixed sentence
naming the flow's seed document — the
:data:`SCOPED_SEARCH_PATH` suite-constant convention: the
literal "continue" shares no tokens with the KB, so the
continuation turn deflects and its prompt carries no
``<documents>`` block).
"""
users = [
_content_text(m.get("content"))
for m in _messages(body)
if m.get("role") == "user"
]
if not users:
return None
last_user = users[-1].strip().lower()
if ROUND_CAP_TRIGGER in last_user:
if body.get("tools"):
return ("call", _round_cap_next_call_id(body))
return ("dump", _round_cap_seed_path(body))
if (
last_user == "continue"
and any(ROUND_CAP_TRIGGER in u.lower() for u in users[:-1])
):
return ("continue",)
return None
def long_answer() -> str:
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
a unique final line that must survive the stream untruncated."""
lines = [
f"{i}. Step {i}: configure node-{i} with the homelab defaults and "
f"verify that step {i} of the long walkthrough is complete before moving on."
for i in range(1, LONG_ANSWER_LINES + 1)
]
lines.append(LONG_ANSWER_END)
return "\n".join(lines)
#: First numbered note line of a ``<tuning>`` section (phase 15).
_TUNING_BLOCK_RE = re.compile(r"<tuning>\n(.*?)\n</tuning>", re.S)
_NOTE_LINE_RE = re.compile(r"^\d+\.\s*(.+)$")
def first_tuning_note(system: str) -> str | None:
"""The first steering note in the system prompt, or ``None``.
The prompt numbers notes 1..N oldest-first (see
``app.rag.prompts.build_steering_section``); the mock echoes the first
one into its answer so prompt injection is observable in the UI.
"""
block = _TUNING_BLOCK_RE.search(system)
if not block:
return None
for line in block.group(1).splitlines():
m = _NOTE_LINE_RE.match(line.strip())
if m:
return m.group(1).strip()
return None
#: First ``-`` bullet line of a ``<knowledge_base>`` section (phase 31).
_KB_BLOCK_RE = re.compile(r"<knowledge_base>\n(.*?)\n</knowledge_base>", re.S)
_KB_BULLET_RE = re.compile(r"^-(?:\s+(.*))?$")
def first_kb_bullet(system: str) -> str | None:
"""The first outline bullet in the system prompt, or ``None``.
The stored outline (phase 31) is ``-`` bullet lines (see
``app.rag.overview.OVERVIEW_INSTRUCTION``); the mock echoes the first
one into its answer as `` (kb: <bullet>)`` — the exact
:func:`first_tuning_note` convention, so prompt injection of the
``<knowledge_base>`` section is observable in the UI.
"""
block = _KB_BLOCK_RE.search(system)
if not block:
return None
for line in block.group(1).splitlines():
m = _KB_BULLET_RE.match(line.strip())
if m:
return (m.group(1) or "").strip()
return None
def folder_map_lines(system: str) -> list[str]:
"""The suggested-folder lines of the ``<documents>`` section
(phase 119, D3, LOCKED A4) — or ``[]`` when there are none.
``build_high_prompt`` lays the section body out as ``SUGGEST_INTRO``
(line 1), the folder lines (each on its own line, immediately after
the intro line), a blank line, then the first ``<document>`` block
— so the lines are exactly the non-blank lines between the intro
line and the first ``<document `` line of the block. With no folder
lines (the byte-identical phase-118 shape) there is only the blank
line there, and this returns ``[]`` (the echo is then omitted — the
answer is otherwise unchanged).
"""
block = _DOCUMENTS_BLOCK_RE.search(system)
if not block:
return []
lines = block.group(0).splitlines()
out: list[str] = []
for line in lines[2:]: # skip the tag line + the SUGGEST_INTRO line
if line.lstrip().startswith("<document"):
break
if line.strip():
out.append(line)
return out
def _history_echo(body: dict[str, Any]) -> str:
"""The phase-74 history echo (byte-stable, stateless over messages).
``history: N prior messages`` — N = the count of non-``system``
messages before the LAST ``user`` message (the client's phase-74
``history`` block: the prior turns only, the current question
itself excluded). ``last answer tail: <tail>`` — the LAST 24 chars
of the most recent prior ``assistant`` message's content, or
``none`` when there is no prior assistant message (the cold-start
pin: no phantom history). ``thinking: yes|no`` — ``yes`` iff that
prior assistant message carries a non-empty ``reasoning_content``
field (A4: the client's prior thinking, mapped by
``app.rag.prompts.history_to_messages``), ``no`` otherwise.
The invariant (see the module docstring): the client history is
``user``/``assistant``-only, so the last ``user`` message is always
the current question and every earlier non-system message is a
client-provided prior turn.
"""
msgs = _messages(body)
last_user = max(
(i for i, m in enumerate(msgs) if m.get("role") == "user"),
default=-1,
)
prior = [
m
for i, m in enumerate(msgs)
if i < last_user and m.get("role") != "system"
]
tail = "none"
thinking = "no"
for m in reversed(prior):
if m.get("role") == "assistant":
tail = str(m.get("content") or "")[-24:]
thinking = "yes" if str(m.get("reasoning_content") or "") else "no"
break
return (
f"history: {len(prior)} prior messages; "
f"last answer tail: {tail}; "
f"thinking: {thinking} "
"(Deterministic mock answer for E2E.)"
)
# ---------------------------------------------------------------------------
# Phase 96 (task 04, one-shot resilience): the incident-shape injection
# for the folder-summary one-shot path — see the module docstring
# ---------------------------------------------------------------------------
#: The folder-label triggers (phase 96, task 04): the ``FOLDER_SUMMARY_MODE``
#: branch's folder label (the user message's ``Folder: …`` tail — the
#: ``FOLDER_HEADER_PREFIX`` parse) ending with these suffixes is THIS
#: SUITE'S own fixture folder (``tests/e2e/test_oneshot_llm_retry.py``
#: seeds the names — no other E2E can hit them; they seed different
#: trees, and the triggers carry the E2E prefix).
ONESHOT_EMPTY_ONCE_SUFFIX = "/e2e_empty_once"
ONESHOT_EMPTY_ALWAYS_SUFFIX = "/e2e_empty_always"
#: Module-level per-label incident counter — the mock is single-
#: conversation per e2e server (the phase-67 ``_fail_posts``
#: convention). Counts the non-stream folder-summary POSTs served per
#: trigger label; the once-sequence resets after the success it guards
#: (the first normal reply), so a second sync re-drives the sequence
#: deterministically.
_empty_once_posts: dict[str, int] = {}
def _folder_summary_incident(body: dict[str, Any]) -> bool:
"""Should this NON-stream folder-summary POST answer with the
2026-09-11 incident envelope (``content=""`` +
``finish_reason="length"`` — the exact wire shape of the empty
``lite`` reply phase 96's one-shot retry targets)?
* the system prompt lacks ``FOLDER_SUMMARY_MODE`` → never (only the
folder-summary one-shot carries the marker — ``chat()`` is the
mock's only non-streaming consumer of it, so the counter counts
exactly the one-shot POSTs the app's retry policy drives);
* the label ends with ``/e2e_empty_always`` → EVERY POST (the
exhaustion path — 1 + ``BOR_LLM_RETRIES`` empty attempts, the
per-folder fail-soft leaves the row absent);
* the label ends with ``/e2e_empty_once`` → the FIRST non-stream
POST for that label only — every later POST returns the normal
line (the retry recovers the row), and the counter resets on
that first normal reply (the phase-67 pattern).
The folder label is the ``FOLDER_SUMMARY_MODE`` branch's existing
parse (the user message's first line, the ``FOLDER_HEADER_PREFIX``
tail) — the injection is a pure function of the request plus the
per-label counter (the house marker-flow convention).
"""
if "FOLDER_SUMMARY_MODE" not in _system(body):
return False
user = _user(body)
header = user.splitlines()[0] if user else ""
if not header.startswith(FOLDER_HEADER_PREFIX):
return False
label = header.removeprefix(FOLDER_HEADER_PREFIX).strip()
if label.endswith(ONESHOT_EMPTY_ALWAYS_SUFFIX):
return True
if label.endswith(ONESHOT_EMPTY_ONCE_SUFFIX):
n = _empty_once_posts.get(label, 0) + 1
_empty_once_posts[label] = n
if n == 1:
return True # the first POST: the incident envelope
_empty_once_posts[label] = 0 # the retry went out — restart
return False
# ---------------------------------------------------------------------------
# Phase 109 (task 03, the never-frozen-turn story suite): the
# deterministic REPORTED-REPRO turn (delta → tool →
# thinking-after-delta) — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic REPORTED-REPRO turn (TODO.md L3): the scripted
#: first answer (no reasoning), the no-arg ``ls``, then the post-tool
#: thinking → answer → FINAL thinking round — the owner's exact
#: "the model responds, calls a tool, then continues thinking" sequence
#: with baked-in delays (mid-turn windows of ≥1 s each — the
#: ``slow_llm.py`` pacing precedent) so the story E2E's window
#: assertions are race-free. Checked BEFORE the plain ``TOOLS_TRIGGER``
#: flow (disjoint trigger phrases — the phase-71/72/94 ordering
#: convention); verified 2026-09-16: no existing E2E question or
#: fixture file contains the phrase, so every other suite is
#: unaffected.
TURN_PROGRESS_TRIGGER = "answer first, then list, then think"
#: Request 1's pre-delay (s) — model latency: the loader's start-state
#: window (the turn is in flight, NO frame has arrived yet).
TURN_PROGRESS_PREDELAY_S = 2.0
#: Request 2's frameless gap (s) — the silence after the ``tool`` frame
#: before the first ``thinking`` frame. Deliberately past the phase-87
#: tool-line counter's 5 s gate (``TOOL_LINE_ELAPSED_AFTER_MS`` in
#: app.js) with ≥2 s of margin: the line's ``(Ns)`` suffix appears
#: (5 s tick) and ticks (6 s) BEFORE the first thinking frame settles
#: the line, so the story E2E can pin the counter in the tool gap
#: without a race.
TURN_PROGRESS_TOOL_GAP_S = 7.0
#: Per-chunk delays (s) for the two streams of request 2.
TURN_PROGRESS_THINK_DELAY_S = 0.3 # reasoning chunks (~10 × 0.3 s, ~3 × 0.3 s)
TURN_PROGRESS_CONTENT_DELAY_S = 0.05 # the short content deltas (3 + 3 chunks)
#: Chunk sizes (chars): the content deltas stay SHORT (2-3 chunks each),
#: the reasoning streams run 10 × 30 and 3 × 30.
TURN_PROGRESS_CONTENT_CHUNK = 40
TURN_PROGRESS_THINK_CHUNK = 30
#: Request 1's answer (byte-stable): 3 × 40-char content chunks, NO
#: reasoning — the answer starts FIRST (the repro's "the model
#: responds"). ``marker-progress-41`` is the sentinel the story E2E
#: matches to prove call 1's content landed in the bubble.
TURN_PROGRESS_FIRST_ANSWER = (
"Checking the listing first — opening answer: the kubernetes setup, "
"short and sweet. marker-progress-41."
)
#: Request 2's scratchpad (byte-stable): 10 × 30-char reasoning chunks —
#: thinking AFTER the answer, the reported repro. ``marker-thought-42``
#: is the sentinel the story E2E matches in the re-opened block.
TURN_PROGRESS_THINKING = (
"The listing just landed — now I can see which documents exist, so the "
"answer can anchor to the kubernetes file first and the deployments note "
"second, citing each fact by the exact path it came from. Hosts, "
"versions, and ports stay exactly as the notes write them. "
"marker-thought-42."
)
#: Request 2's answer (byte-stable): 3 × 40-char content chunks ending in
#: the DISTINCTIVE final sentence the story E2E matches on to pin the
#: post-delta state (``marker-progress-42`` — the repro's second answer).
TURN_PROGRESS_FINAL_ANSWER = (
"Here is the plan after the listing: step one, step two, step three — "
"that is the whole of it. marker-progress-42."
)
#: Request 2's FINAL reasoning (byte-stable): 3 × 30-char chunks — the
#: thinking that arrives AFTER the answer's last delta. The story E2E
#: waits for the re-opened block after the final sentence landed and
#: matches ``marker-final-thought-42`` to prove the last round's
#: thinking rendered in the scratchpad.
TURN_PROGRESS_FINAL_THINKING = (
"Final check — hosts and ports are verbatim. marker-final-thought-42."
)
def _turn_progress_flow(body: dict[str, Any]) -> str | None:
"""Classify a phase-109 reported-repro request (see the module
docstring). Stateless over the messages, like the other marker
flows:
* ``"first"`` — ``tools`` are offered and no ``tool``-role result
is in the messages yet: request 1 — the ~2 s pre-delay (model
latency — the loader's start-state window), the short
``delta.content`` stream (3 chunks, NO reasoning — the answer
starts first), then the ``ls`` tool call (synthetic id
``call_0``, no arguments), ``finish_reason: "tool_calls"``.
* ``"second"`` — a ``tool``-role result is in the messages (the
server ran the ``ls``): request 2 — the ~7 s frameless gap
(the phase-87 window), the ``reasoning_content`` stream (10 ×
0.3 s), the ``delta.content`` stream (3 chunks ending in the
distinctive final sentence), and the FINAL
``reasoning_content`` chunks (3 × 0.3 s) — thinking AFTER the
answer — then ``finish_reason: "stop"``.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or
``tools`` are not offered and no tool result is in the messages
yet (e.g. ``agent_max_rounds=0``).
"""
if TURN_PROGRESS_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
if _tool_results(body):
return "second"
if not body.get("tools"):
return None
return "first"
def _turn_progress_stream(step: str) -> Any:
"""SSE frames for one phase-109 reported-repro request.
``"first"`` (model call 1): the ~2 s pre-delay, the short
``delta.content`` stream (3 chunks — the answer starts first, NO
reasoning), the ``ls`` tool-call partial (synthetic id ``call_0``,
no arguments — the phase-37 ``_tool_call_stream`` first-partial
shape), and the ``finish_reason: "tool_calls"`` frame. The app is
position-independent over the wire (``app/rag/llm.py``): the
content chunks stream as ``delta`` SSE frames as they arrive and
the tool call materializes AFTER the stream (the content-before-
tools convention), so the request's SSE is exactly ``delta →
tool``.
``"second"`` (model call 2 — after the server ran the ``ls``): the
~7 s frameless gap (the phase-87 window — the tool line's
``(Ns)`` counter appears and ticks before the first frame settles
it), the ``reasoning_content`` stream (10 × 30 chars, 0.3 s each),
the ``delta.content`` stream (3 chunks ending in the distinctive
final sentence), the FINAL ``reasoning_content`` chunks (3 × 30
chars, 0.3 s each — thinking AFTER the answer, the reported
repro), and the ``finish_reason: "stop"`` frame.
"""
model = "turbo"
chunk_id = f"chatcmpl-{uuid.uuid4()}"
def frame(delta: dict[str, Any], finish: str | None = None) -> str:
return (
"data: "
+ json_dumps(
{
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}],
}
)
+ "\n\n"
)
if step == "first":
time.sleep(TURN_PROGRESS_PREDELAY_S)
for piece in re.findall(
rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FIRST_ANSWER, re.S
):
yield frame({"content": piece})
time.sleep(TURN_PROGRESS_CONTENT_DELAY_S)
yield frame(
{
"role": "assistant",
"tool_calls": [
{
"index": 0,
"id": "call_0",
"type": "function",
"function": {"name": "ls", "arguments": "{}"},
}
],
}
)
time.sleep(0.1)
yield frame({}, "tool_calls")
yield "data: [DONE]\n\n"
return
time.sleep(TURN_PROGRESS_TOOL_GAP_S)
for piece in re.findall(
rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_THINKING, re.S
):
yield frame({"reasoning_content": piece})
time.sleep(TURN_PROGRESS_THINK_DELAY_S)
for piece in re.findall(
rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FINAL_ANSWER, re.S
):
yield frame({"content": piece})
time.sleep(TURN_PROGRESS_CONTENT_DELAY_S)
for piece in re.findall(
rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_FINAL_THINKING, re.S
):
yield frame({"reasoning_content": piece})
time.sleep(TURN_PROGRESS_THINK_DELAY_S)
yield frame({}, "stop")
yield "data: [DONE]\n\n"
def compose_answer(body: dict[str, Any]) -> str:
system = _system(body)
user = _user(body)
if LONG_ANSWER_TRIGGER in user.lower():
answer = long_answer()
elif "FOLDER_SUMMARY_MODE" in system:
# Folder summary (phase 94, voice nudge phase 155 task 01):
# the ``lite`` stand-in returns the deterministic one-liner
# ``Documentation for <folder>.`` — <folder> is the
# user message's ``Folder: …`` header line (the generator puts
# the canonical source/folder identity there, imported as
# ``FOLDER_HEADER_PREFIX`` so the mock can never drift from it).
# The stored row therefore always names its folder with a
# natural opener (never the forbidden "This folder contains…"
# openers — phase 155's pin) — the drill-down E2E
# (``test_ls_tree_drilldown.py``) asserts on it.
# Checked BEFORE the ``SUMMARY_MODE`` branch: the folder marker
# CONTAINS the summary marker as a substring (``FOLDER_`` +
# ``SUMMARY_MODE``), so the summary branch would otherwise
# shadow every folder-summary call. Checked BEFORE the
# DEFLECT_MODE branch, like the other lite-mode markers (a
# deflection prompt never carries one).
# Phase 96 (task 04): the incident-shape injection keys on THIS
# branch's label (``_folder_summary_incident`` — the module
# docstring) and overrides the NON-STREAM response envelope in
# ``chat_completions`` (``content=""`` +
# ``finish_reason="length"``); the line below is what the later
# / normal POSTs return.
header = user.splitlines()[0] if user else ""
folder = (
header.removeprefix(FOLDER_HEADER_PREFIX).strip()
if header.startswith(FOLDER_HEADER_PREFIX)
else "(unnamed folder)"
)
answer = f"Documentation for {folder}."
elif "SUMMARY_MODE" in system:
# Document summaries (phase 30): the ``lite`` stand-in returns a
# deterministic digest — the first 24 tokens of the user message
# (the summarizer puts the capped document content there). Byte-
# stable for a given fixture, so the summary chunk's retrieval
# rank is a pure function of the fixture text. Checked BEFORE the
# DEFLECT_MODE branch (task 06) so a deflection prompt that ever
# carries the marker cannot shadow the summary call.
answer = (
f"This document covers "
f"{' '.join(TOKEN_RE.findall(user.lower())[:24])}."
)
elif "KB_OVERVIEW_MODE" in system:
# KB overview (phase 31): the ``lite`` stand-in returns the
# deterministic outline — the first 8 tokens of the user message
# (the generator puts the document list there). Byte-stable for a
# given KB, so the stored row is a pure function of the fixture.
# Checked BEFORE the DEFLECT_MODE branch, like SUMMARY_MODE, so a
# prompt that ever carries both markers cannot shadow the
# overview call.
answer = "Knowledge base outline:\n- " + " ".join(
TOKEN_RE.findall(user.lower())[:8]
)
elif "IMAGE_DESCRIPTION_MODE" in user:
# Phase 122 (image documents, LOCKED A3): the CHAT model's
# (vision) image description — a NON-STREAMING multimodal user
# message, so only this path ever sees it. ``_content_text``
# joins the text parts, so the marker (the first text part)
# lands in ``user``. Checked before the user-trigger branches:
# the marker is a fixed app constant, and a real question is
# never expected to type it (the other user triggers are owner
# phrasings a question might legitimately contain — this one
# is an app-to-app marker). Phase 137 (task 05): the
# document-context trigger — when the text part carries the
# provenance header (``Extracted from `` — the importer's
# block builder, tasks 02/03), the canned reply becomes the
# context-aware :data:`IMAGE_DESCRIPTION_CONTEXT_ANSWER`
# (the deterministic witness that the context REACHED the
# vision model); the bare path (standalone images, the
# pre-137 embedded shape) stays on the bare
# :data:`IMAGE_DESCRIPTION_ANSWER` — byte-identical.
answer = (
IMAGE_DESCRIPTION_CONTEXT_ANSWER
if "Extracted from " in user
else IMAGE_DESCRIPTION_ANSWER
)
elif "PICTORIAL_PAGE_MODE" in user:
# Phase 140 (pictorial PDF pages, LOCKED A2): the CHAT model's
# vision description of ONE rendered sparse PDF page — a
# NON-STREAMING multimodal user message, so only this path
# ever sees it. ``_content_text`` joins the text parts, so the
# marker (the first text part) lands in ``user``. The two
# markers are disjoint fixed app constants (neither contains
# the other), so the branch order vs ``IMAGE_DESCRIPTION_MODE``
# cannot shadow either flow; like that marker, it is an
# app-to-app marker a real question is never expected to type.
answer = PICTORIAL_PAGE_ANSWER
elif TABLE_TRIGGER in user.lower():
# Markdown tables (phase 44, TODO.md L6): the story E2E's
# deterministic table answer — a 3-column table, the
# <img onerror> XSS probe line (it must survive the mock
# byte-for-byte so the E2E can prove the renderer neutralizes
# it), and a wide 5-column table (guarantees scrollWidth >
# clientWidth inside the 72rem container column, phase 100).
# Byte-stable. Checked
# BEFORE the DEFLECT_MODE branch: a deflection prompt never
# carries the marker (it lives in the user message, same
# reasoning as SUMMARY_MODE), so a marker question always gets
# the table answer, whatever the gate says; the E2E asks it
# against an on-topic fixture, where the gate is HIGH, and
# asserts non-deflection as part of the table test.
answer = TABLE_ANSWER
elif QUOTE_TRIGGER in user.lower():
# Markdown blockquotes (phase 138, TODO.md L3): the story E2E's
# deterministic quote answer — the renderer's blockquote
# contract in one byte-stable stream (single-line quote,
# 3-line group, nested collapse, the <script> XSS probe, the
# fence-protected >, the table-cell >, the plain close).
# Checked BEFORE the DEFLECT_MODE branch, like TABLE_TRIGGER:
# the marker lives in the user message, so a marker question
# always gets the quote answer whatever the gate says.
answer = QUOTE_ANSWER
elif HISTORY_TRIGGER in user.lower():
# Phase 74 (TODO L4, chat history): the deterministic history
# echo — proves on the wire that the client's prior turns (and
# the prior thinking, as ``reasoning_content`` on the assistant
# messages) reached the model. Checked BEFORE the DEFLECT_MODE
# branch, like TABLE_TRIGGER: the marker lives in the user
# message, a deflection prompt never carries it, so a marker
# question always gets the echo whatever the gate says.
answer = _history_echo(body)
elif DIAGRAM_TRIGGER in user.lower():
# Phase 124 (task 07): the BUG-FIX-ONLY trigger — the mock
# answers straight from the seed with ZERO tool calls (no
# ``show_image``, no ``read`` — the clause is NOT required,
# so the flow also fires on a default-OFF app: the chips fix
# belongs to the ``BOR_IMAGES``-indexed docs) and quotes the
# TAIL of the seeded image doc's ``<document>`` block (the
# same pick as the SHOW_IMAGE flow — the block's content is
# the image doc's entire text, the vision description):
# proving the description WAS in the prompt and that the chip
# appears with NO tool calls at all (the phase-124 auto-cite
# — the fix for the owner's report). Checked BEFORE the
# DEFLECT_MODE branch (the marker lives in the user message —
# the TABLE/HISTORY convention), so a marker question always
# gets the echo whatever the honesty gate says. A text-only KB
# (no seeded image doc) degrades to the plain line — the
# suite's premise broke; it fails loudly on the missing quote.
image_doc = _seeded_image_doc(system)
answer = (
f"Your notes carry that diagram — it reads: “{image_doc[2][-160:]}” "
"(Deterministic mock answer for E2E.)"
if image_doc
else "Your notes don't carry that diagram — there is nothing to show. "
"(Deterministic mock answer for E2E.)"
)
elif CONTEXT_LINE_TRIGGER in user.lower():
# Phase 125 (task 06, image-anchor context, LOCKED A4): the
# ZERO-READ context-line echo — the mock QUOTES THE CONTEXT
# LINE of the seeded image doc's <document> block (the block's
# FIRST line — the task-05 one-line rendering for an enriched
# doc; a standalone doc's block is the bare description, whose
# first line never matches — the "no reference line" answer
# is the byte-identity witness). Checked BEFORE the
# DEFLECT_MODE branch (the marker lives in the user message —
# the TABLE/HISTORY convention), so a marker question always
# gets the echo whatever the honesty gate says.
image_doc = _seeded_image_doc(system)
if image_doc is None:
answer = (
"Your notes don't carry that image — there is nothing to reference. "
"(Deterministic mock answer for E2E.)"
)
else:
context_line = _seed_context_line(image_doc[2])
answer = (
f"The notes reference the image thus: {context_line} "
"(Deterministic mock answer for E2E.)"
if context_line is not None
else "The image stands alone — the notes carry no reference line for it. "
"(Deterministic mock answer for E2E.)"
)
elif "DEFLECT_MODE" in system:
answer = (
"Ah — I haven't done anything like that, so I don't want to make stuff up! "
"You're thinking bigger than my notes for a second. Try asking about "
"kubernetes, backups, or deploying a new service — I know those inside out. "
"You've got this!"
)
elif END_OF_NOTES_TRIGGER in user.lower():
# Phase 24 (whole-document context) — phase 118 re-targeted
# (the summary-seed context's story suite): echo the tail of
# the <documents> block. Byte-stable across runs — under the
# phase-118 summary-seed contract the block carries the
# suggested docs' SUMMARIES, so the echoed tail is the LAST
# suggested doc's summary tail (digest + Source: pointer line),
# and a sentinel on a document's last line appears in the
# answer only if the FULL content reached the model (through a
# read tool result — never the seed). (The tail includes the
# closing </documents> — harmless for the E2E sentinel
# assertions.) Phase 37: the HIGH prompt ends with the <tools>
# section after </documents>, so the echo targets the
# <documents> block itself.
block = _DOCUMENTS_BLOCK_RE.search(_system(body))
tail_source = block.group(0) if block else _context(body)
answer = (
f"…and the very end of my notes reads: “{tail_source[-160:]}” "
"(Deterministic mock answer for E2E.)"
)
else:
ctx = _context(body)
snippet = ctx[:220].replace("\n", " ").strip()
answer = (
f"Great question — you've absolutely got this! Here's what my notes say about "
f"“{user.strip()[:80]}”: {snippet}… That's the gist from the docs; happy to "
"dig into any of it. (Deterministic mock answer for E2E.)"
)
# Steering (phase 15): when the system prompt carries <tuning>, the
# answer ends with the first note — deterministically observable.
note = first_tuning_note(system)
if note:
answer = f"{answer} (tuning: {note})"
# KB overview (phase 31): when the system prompt carries
# <knowledge_base>, the answer ends with the first outline bullet —
# mirrors the steering echo exactly (appended after it, so the kb
# suffix is the last thing rendered).
bullet = first_kb_bullet(system)
if bullet:
answer = f"{answer} (kb: {bullet})"
# Suggested-folder context (phase 119, D3, LOCKED A4): when the user
# message carries FOLDER_MAP_TRIGGER and the system prompt carries
# the <documents> section, the answer ends with the folder lines
# VERBATIM (joined with "; ") — the E2E's deterministic lens on the
# exact lines build_high_prompt seeded (D3 observability; the same
# prompt-injection-echo convention as the two echoes above). Omitted
# when the section carries no folder lines (the byte-identical
# phase-118 shape) — the answer is otherwise unchanged.
if FOLDER_MAP_TRIGGER in user.lower() and "<documents>" in system:
folder_lines = folder_map_lines(system)
if folder_lines:
answer = f"{answer} (folders: {'; '.join(folder_lines)})"
return answer
def compose_thinking(body: dict[str, Any]) -> str:
"""Deterministic reasoning scratchpad (thinking-display story, phase 17;
lengthened in phase 21).
A fixed "Step 1… Step 4" template interleaved with a "Scratch" deep-dive
block, quoting the first ~60 chars of the user question: unique per
question, byte-stable across runs, ~2 700 chars total (≈ 230 frames at
the mock's 12-char/0.02s pacing). The length is deliberate (phase 21,
thinking-no-scroll story): rendered in the 320px ``.thinking-text``
window it overflows by ~2x, so the live-tail clip and the no-user-scroll
contract are observable in E2E. The ``Step 2: Check my notes`` line
fragment (phase 17) and the ``nothing is invented`` tail (phase 20's
THINKING_TAIL) are what the E2E assertions key off — both are preserved.
"""
q = _user(body).strip()[:60]
return _thinking_template(q)
def _thinking_template(q: str) -> str:
"""The fixed Step/Scratch scratchpad (``compose_thinking`` and its
paragraph variant share the exact same text — only the line
separators differ)."""
return (
f"Step 1: Read the question carefully — “{q}” — and figure out what kind of "
"answer it wants (a how-to, a lookup, or a design decision) before touching "
"the docs, so I don't over- or under-answer.\n"
"Step 2: Check my notes for the closest match. The homelab kubernetes file "
"is the obvious candidate, but I should also consider whether a deployments "
"note covers the same ground better.\n"
"Scratch 1: the kubernetes file is organized by component — control plane, "
"worker nodes, ingress, storage — so I can map each part of the question to "
"a section instead of summarizing the whole file at once, and keep the "
"answer anchored to the structure the notes actually use.\n"
"Scratch 2: I should check whether the deployments note duplicates any of "
"that ground; if it does, I will prefer the homelab file because the "
"question is phrased around the cluster itself, and I will say which file "
"each fact came from so the citation is honest.\n"
"Scratch 3: versions and ports are the facts most likely to be stale in my "
"memory — the etcd backup schedule, the ingress controller port, the "
"registry mirror address — so I will re-read those lines verbatim before "
"writing a single one of them into the answer.\n"
"Scratch 4: if the answer needs a sequence, for example how a node joins the "
"cluster or how the load balancer fronts the control plane, I will keep the "
"order exactly as the notes write it rather than re-deriving it from general "
"kubernetes knowledge that may not match this setup.\n"
"Scratch 5: anything I cannot find in the notes — a host I do not recognize, "
"a version I am not sure about, a schedule I cannot place — gets left out of "
"the answer instead of guessed, because the honesty rule beats a longer "
"answer every single time.\n"
"Scratch 6: one more pass over the question wording to make sure I am "
"answering the cluster setup, not some other homelab topic that shares the "
"same vocabulary, and I will stay on the specific the question asked about.\n"
"Scratch 7: I will also verify that the file describes the current setup — "
"if the notes mention a migration from an older cluster, I should answer "
"from the post-migration section and not mix in the old host names or the "
"old port numbers that no longer apply.\n"
"Scratch 8: final shape check before I commit — short paragraphs, a few "
"bullets at most, the document path cited where the fact came from, and no "
"invented facts anywhere in the draft.\n"
"Step 3: Re-read the relevant sections top to bottom so every specific — "
"hosts, versions, ports, schedules — is exact as written rather than "
"remembered, and note which document each fact comes from.\n"
"Step 4: Draft the answer around those specifics, keep it tight with short "
"paragraphs and bullets where it helps, cite the documents by path, and "
"double-check that nothing is invented."
)
def compose_thinking_paragraphs(body: dict[str, Any]) -> str:
"""The phase-17 scratchpad with REAL paragraph breaks (\n\n, 2026-08-29
regression pin): the same deterministic text as ``compose_thinking``,
with a blank line inserted after scratchpad lines 2 and 6 (0-based) —
two genuine \"2-newline gaps\" in the rendered scratchpad. Unique per
question, byte-stable across runs (same length contract + 2 chars)."""
base = compose_thinking(body)
lines = base.split("\n")
out: list[str] = []
for i, line in enumerate(lines):
out.append(line)
if i in (2, 6):
out.append("") # blank line -> a real \"\n\n\" gap
return "\n".join(out)
@app.get("/v1/e2e/captured")
def e2e_captured() -> list[dict[str, Any]]:
"""Phase 123 (task 04): the recent chat-completions request bodies
(oldest first, capped at ``_CAPTURED_MAX``) — the mock's capture
for request-shape assertions (see ``_CAPTURED``)."""
return _CAPTURED
@app.post("/v1/e2e/captured/reset")
def e2e_captured_reset() -> dict[str, int]:
"""Phase 123 (task 04): clear the capture so a suite can assert on
exactly the requests of the turn it is about to drive (e.g. the
toggle-off rejection proves ZERO chat calls — an empty capture)."""
_CAPTURED.clear()
return {"cleared": True}
@app.post("/__shutdown__")
def shutdown() -> dict[str, Any]:
"""Test hook (loading-feedback story): terminate this mock process to
simulate an LLM outage. The E2E fixture restores a fresh instance on
the same port afterwards, so the rest of the session keeps working."""
def _die() -> None:
time.sleep(0.1) # let the HTTP response flush before we exit
os.kill(os.getpid(), signal.SIGTERM)
threading.Thread(target=_die, daemon=True).start()
return {"status": "shutting down"}
@app.get("/v1/models")
def models() -> dict[str, Any]:
return {
"object": "list",
"data": [
{"id": "turbo", "object": "model"},
{"id": "embed", "object": "model"},
{"id": "lite", "object": "model"},
],
}
@app.post("/v1/embeddings")
def embeddings(body: dict[str, Any]) -> Any: # dict, or a 500 (phase 67)
raw = body.get("input")
if isinstance(raw, str):
raw = [raw]
inputs: list[Any] = list(raw) if isinstance(raw, list) else []
# Phase 67 (embedding retry): the first embeddings request whose
# input carries the marker 500s; the next returns the normal
# bag-of-words vector (see the module docstring). Raw httpx on the
# client side — no SDK-level retries — so one POST per app attempt:
# the counter is per POST here (unlike the chat counter below).
joined = " ".join(str(t) for t in inputs if isinstance(t, str)).lower()
if EMBED_FAIL_TRIGGER in joined:
n = _bump_fail(EMBED_FAIL_TRIGGER)
if n == 1:
return _llm_500(EMBED_FAIL_TRIGGER)
_fail_posts[EMBED_FAIL_TRIGGER] = 0 # the vector went out — restart
data = [
{"object": "embedding", "index": i, "embedding": embed_text(t)}
for i, t in enumerate(inputs)
]
return {
"object": "list",
"data": data,
"model": body.get("model", "embed"),
"usage": {"prompt_tokens": 8, "total_tokens": 8},
}
def _sse_stream(
answer: str,
delay: float,
thinking: str = "",
pre_content_delay: float = 0.0,
chunk: int = 12,
) -> Any:
"""SSE frames for one chat completion (phase 17: + reasoning).
``chunk`` (default 12) is the slice size for BOTH the thinking and
the content frames — the ``think in paragraphs`` trigger raises it
to ``THINK_PARAS_CHUNK`` (60) so a single frame renders past the
32px think-window band (see ``THINK_PARAS_TRIGGER``). At 12 the
output is byte-identical to the original.
When ``thinking`` is non-empty its ``chunk``-sized slices go out FIRST as
``delta.reasoning_content`` frames — same 0.02s cadence and envelope
as the content frames, the aipi wire convention (reasoning before
content). Without ``thinking`` the output is byte-identical to the
content-only stream, so the other story suites are unaffected.
``pre_content_delay`` (phase 20) inserts a silence gap between the end
of the thinking stream and the first content frame — the client stays
in its pre-token "thinking" state the whole time (0.02s cadence and
frame shapes are unchanged, so 0.0 is byte-identical to before).
"""
model = "turbo"
chunk_id = f"chatcmpl-{uuid.uuid4()}"
if delay:
time.sleep(delay)
for piece in re.findall(rf".{{1,{chunk}}}", thinking, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{"index": 0, "delta": {"reasoning_content": piece}, "finish_reason": None}
],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
if pre_content_delay:
time.sleep(pre_content_delay)
for piece in re.findall(rf".{{1,{chunk}}}", answer, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": {"content": piece}, "finish_reason": None}],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
yield (
"data: "
+ json_dumps(
{
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
}
)
+ "\n\n"
)
yield "data: [DONE]\n\n"
def json_dumps(obj: dict[str, Any]) -> str:
return json.dumps(obj)
def _apply_max_tokens(answer: str, max_tokens: Any) -> str:
"""Deterministic stand-in for the endpoint's output cap: one token ≈
one whitespace-separated word. Answers within the cap pass through
byte-identical, so existing (short) answers are unaffected."""
if not isinstance(max_tokens, int) or max_tokens <= 0:
return answer
words = answer.split()
if len(words) <= max_tokens:
return answer
return " ".join(words[:max_tokens])
def _tool_call_stream(
name: str,
arguments: dict[str, Any],
call_id: str,
thinking: str = "",
) -> Any:
"""SSE frames for one tool-call-only chat completion (phase 37).
The OpenAI wire convention the app accumulates (``app/rag/llm.py``):
the first partial of index 0 carries ``id`` + ``type`` +
``function.name`` plus the first ``function.arguments`` fragment;
the remaining fragments (deterministic 16-char split — so the
multi-fragment accumulation path is exercised) arrive on later
chunks; the final chunk carries ``finish_reason: "tool_calls"``.
No ``content`` / ``reasoning_content`` frames — the turn asked for a
tool instead of answering.
Pacing: 0.1 s per frame — deliberately SLOWER than the content
stream's 0.02 s, so the UI's transient "calling tool" state (held
from the first ``tool`` frame until the first answer ``delta``) is a
comfortable observation window for the story E2E (~1 s across the
two tool requests).
Phase 158 (task 04 — the phase-158 flow): the optional fourth
parameter — when ``thinking`` is non-empty its 12-char
``_sse_stream``-convention slices go out FIRST as
``delta.reasoning_content`` frames (the 0.02 s cadence, the
aipi wire convention — reasoning before the tool call), then the
tool-call frames + ``finish_reason: "tool_calls"`` below are
UNCHANGED. Default ``""`` → every pre-phase 3-arg caller
streams BYTE-IDENTICAL (the other marker flows' byte-
identity pins stay green).
"""
model = "turbo"
chunk_id = f"chatcmpl-{uuid.uuid4()}"
if thinking:
for piece in re.findall(r".{1,12}", thinking, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{"index": 0, "delta": {"reasoning_content": piece}, "finish_reason": None}
],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
raw_args = json_dumps(arguments) if arguments else "{}"
frags = [raw_args[i : i + 16] for i in range(0, len(raw_args), 16)] or ["{}"]
for i, frag in enumerate(frags):
tc: dict[str, Any] = {"index": 0, "function": {"arguments": frag}}
delta: dict[str, Any] = {"tool_calls": [tc]}
if i == 0:
tc = {
"index": 0,
"id": call_id,
"type": "function",
"function": {"name": name, "arguments": frag},
}
delta = {"role": "assistant", "tool_calls": [tc]}
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": None}],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.1)
yield (
"data: "
+ json_dumps(
{
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": {}, "finish_reason": "tool_calls"}],
}
)
+ "\n\n"
)
yield "data: [DONE]\n\n"
def _midround_tool_stream(
thinking: str,
mid_round_delta: str,
name: str,
arguments: dict[str, Any],
call_id: str,
) -> Any:
"""SSE frames for one MID-ROUND tool completion (phase 161, task
04 — the stays-open contract's wire shape).
The three-part ordering the owner's 2026-10-05 TODO describes:
the ``reasoning_content`` run FIRST (the 12-char
``_tool_call_stream`` convention — the 0.02 s cadence, the
aipi wire convention: reasoning before content), then the
MID-ROUND ``content`` delta (the SAME 12-char / 0.02 s
convention — the React-style partial answer, the content stream
BEFORE the tool call), and ONLY then the ``_tool_call_stream``
tool-call frames (the deterministic 16-char argument split, the
0.1 s pacing) + ``finish_reason: "tool_calls"`` + ``[DONE]``.
The app accumulates exactly this (``app/rag/llm.py`` — the
content-before-tools flush: the mid-round content reaches the
client as a ``delta`` frame BEFORE the ``tool`` frame; a rare
stream with both content and a tool call keeps the content AND
runs the tool, ``app/rag/agent.py``), so the client sees
thinking → mid-round delta → tool call → next round — the D2
ordering the stays-open container contract needs.
"""
model = "turbo"
chunk_id = f"chatcmpl-{uuid.uuid4()}"
for piece in re.findall(r".{1,12}", thinking, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{"index": 0, "delta": {"reasoning_content": piece}, "finish_reason": None}
],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
for piece in re.findall(r".{1,12}", mid_round_delta, re.S):
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": {"content": piece}, "finish_reason": None}],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.02)
raw_args = json_dumps(arguments) if arguments else "{}"
frags = [raw_args[i : i + 16] for i in range(0, len(raw_args), 16)] or ["{}"]
for i, frag in enumerate(frags):
tc: dict[str, Any] = {"index": 0, "function": {"arguments": frag}}
delta: dict[str, Any] = {"tool_calls": [tc]}
if i == 0:
tc = {
"index": 0,
"id": call_id,
"type": "function",
"function": {"name": name, "arguments": frag},
}
delta = {"role": "assistant", "tool_calls": [tc]}
payload = {
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": delta, "finish_reason": None}],
}
yield f"data: {json_dumps(payload)}\n\n"
time.sleep(0.1)
yield (
"data: "
+ json_dumps(
{
"id": chunk_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [{"index": 0, "delta": {}, "finish_reason": "tool_calls"}],
}
)
+ "\n\n"
)
yield "data: [DONE]\n\n"
@app.post("/v1/chat/completions")
def chat_completions(body: dict[str, Any]) -> Any:
# Phase 123 (task 04): the OBSERVATIONAL capture (the ring buffer —
# recorded before any flow decision, so a 500-injected request is
# captured too; the buffer never touches the answer path).
_CAPTURED.append(body)
if len(_CAPTURED) > _CAPTURE_MAX:
del _CAPTURED[: len(_CAPTURED) - _CAPTURE_MAX]
user_lower = _user(body).lower()
# Phase 37 (agent document tools): the deterministic marker flow.
# The app's chat path is the only streaming consumer of this mock, so
# the flow handles streaming requests; a non-streaming marker request
# (never issued by the app) falls through to the regular answer.
if body.get("stream"):
# Phase 67 (LLM retry): deterministic failure injection — see
# the module docstring. Checked before the marker tool flow: the
# injection markers never combine with the tool-flow markers in
# any suite, and a dead endpoint answers nothing (no flow).
if ALWAYS_FAIL_TRIGGER in user_lower:
return _llm_500(ALWAYS_FAIL_TRIGGER)
if RETRY_TRIGGER in user_lower:
if _chat_dead(RETRY_TRIGGER, RETRY_DEAD_ATTEMPTS):
return _llm_500(RETRY_TRIGGER)
_fail_posts[RETRY_TRIGGER] = 0 # the answer streamed — restart
# Phase 120 (failed-turn retry): the failed-turn suite's two
# injections — the whole-first-turn death (the zero-frame
# exhaustion whose Retry's re-ask the mock then answers) and the
# partial-then-die (the partial deltas + terminal error frame).
if FAIL_FIRST_TURN_TRIGGER in user_lower:
if _chat_dead(FAIL_FIRST_TURN_TRIGGER, FAIL_FIRST_TURN_DEAD_ATTEMPTS):
return _llm_500(FAIL_FIRST_TURN_TRIGGER)
_fail_posts[FAIL_FIRST_TURN_TRIGGER] = 0 # the answer streamed — restart
if PARTIAL_FAIL_TRIGGER in user_lower:
if _bump_fail(PARTIAL_FAIL_TRIGGER) <= PARTIAL_FAIL_DEAD_REQUESTS:
return StreamingResponse(
_partial_fail_stream(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
_fail_posts[PARTIAL_FAIL_TRIGGER] = 0 # the answer streamed — restart
# Phase 71 (tool-scaffolding guardrails): the deterministic raw-
# markup flow — checked BEFORE the search/tool marker flows (the
# trigger is independent of the ``<tools>`` marker, so both
# grounded and deflected turns hit it; SCAFFOLD_ALWAYS_TRIGGER
# is checked first inside the classifier — the more specific
# phrase wins, same convention as THINK_PARAS_TRIGGER).
scaffold_flow = _scaffold_flow(body)
if scaffold_flow is not None:
# Request 1 (or EVERY request for the ALWAYS trigger): the
# incident span as plain delta.content, 12-char chunks
# (the span always spans ≥2 chunks — the filter's boundary
# path), finish_reason "stop", no tool_calls, no reasoning.
# Request 2 of the recovery trigger: the clean answer.
answer = (
SCAFFOLD_RECOVERY_ANSWER
if scaffold_flow == "recovery"
else SCAFFOLD_SPAN
)
return StreamingResponse(
_sse_stream(answer, 0.0),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# 2026-09-05 (the "Qwen 3.8" incident — the grep regex prior):
# the deterministic GREP-TEACH self-correction flow — checked
# BEFORE the SEARCH / TOOLS_TRIGGER flows (disjoint trigger
# phrases — the phase-72 ordering convention; the trigger needs
# the ``<tools>`` section, so deflected turns never hit it).
grep_teach = _grep_teach_flow(body)
if grep_teach is not None:
if grep_teach[0] == "regex":
# The incident's misuse, deterministic: the regex-shaped
# pattern (it can never match a fixed-substring grep).
stream = _tool_call_stream(
"grep", {"pattern": GREP_TEACH_PATTERN}, "call_0"
)
elif grep_teach[0] == "plain":
# The one-round correction: the plain-form retry (the
# teaching line handed over exactly this hint).
stream = _tool_call_stream(
"grep", {"pattern": GREP_TEACH_PLAIN}, "call_1"
)
elif grep_teach[0] == "read":
stream = _tool_call_stream(
"read", {"path": grep_teach[1]}, grep_teach[2]
)
elif grep_teach[0] == "nomatch":
stream = _sse_stream(
_apply_max_tokens(
"No matches — the knowledge base has no such text.",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — quote the read document (first 80 chars)
stream = _sse_stream(
_apply_max_tokens(
f"Read {grep_teach[1]}. {grep_teach[2][:80]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 146 (task 02): the deterministic SCOPED search marker
# flow — checked BEFORE the phase-68 search flow (disjoint
# trigger phrases — the phase-71/72 ordering convention; the
# trigger needs the ``<tools>`` section, so deflected turns
# never hit it). The scoped form (pattern + path) is the wire
# subject: the app's tool frame carries the scope as its
# additive ``path`` field, and the UI shows WHERE the search
# runs (the "repeating itself" confusion, TODO.md L3).
scoped_search = _scoped_search_flow(body)
if scoped_search is not None:
if scoped_search[0] == "search":
stream = _tool_call_stream(
"grep",
{"pattern": SEARCH_PATTERN, "path": SCOPED_SEARCH_PATH},
"call_0",
)
else: # "found" — quote the first matched line (80 chars)
answer = _apply_max_tokens(
f"Found {scoped_search[1][:80]}", body.get("max_tokens")
)
stream = _sse_stream(answer, 0.0)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 68 (search tool): the deterministic search marker flow —
# checked BEFORE the phase-37 tool flow (the more specific
# trigger phrase wins, same convention as THINK_PARAS_TRIGGER).
search_flow = _search_flow(body)
if search_flow is not None:
if search_flow[0] == "search":
stream = _tool_call_stream(
"grep", {"pattern": SEARCH_PATTERN}, "call_0"
)
else: # "found" — quote the first matched line (80 chars)
answer = _apply_max_tokens(
f"Found {search_flow[1][:80]}", body.get("max_tokens")
)
stream = _sse_stream(answer, 0.0)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 72 (teaching refusals): the deterministic LS-TEACH
# self-correction flow — checked BEFORE the plain
# TOOLS_TRIGGER flow (disjoint trigger phrases — the phase-71
# ordering convention; the trigger needs the ``<tools>``
# section, so deflected turns never hit it).
ls_teach = _ls_teach_flow(body)
if ls_teach is not None:
if ls_teach[0] == "misuse":
# The incident's misuse, deterministic: ls(path='.').
stream = _tool_call_stream("ls", {"path": "."}, "call_0")
elif ls_teach[0] == "correct":
# The one-round correction: the no-arg top-level
# listing (phase 94: sources only — the drill follows).
stream = _tool_call_stream("ls", {}, "call_1")
elif ls_teach[0] == "drill":
# Phase 94: the top level lists sources only — drill
# one level into the first source for the file lines.
stream = _tool_call_stream(
"ls", {"path": ls_teach[1]}, ls_teach[2]
)
else: # "answer" — quote the first file line
answer = _apply_max_tokens(
f"These are the indexed documents: {ls_teach[1]}",
body.get("max_tokens"),
)
stream = _sse_stream(answer, 0.0)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 94 (task 04): the deterministic SCRIPTED drill-down
# turns (sources → folders → files → the grounded read; the
# 50-line cap; the NOT-A-FOLDER teaching + scripted recovery) —
# checked BEFORE the plain TOOLS_TRIGGER flow (disjoint trigger
# phrases — the phase-71/72 ordering convention; the trigger
# needs the ``<tools>`` section, so deflected turns never hit
# it).
drill = _drill_flow(body)
if drill is not None:
if drill[0] == "call":
# The scripted call: ls with no path at the top level
# (empty target), ls/read with the parsed target.
stream = _tool_call_stream(
drill[1], {"path": drill[2]} if drill[2] else {}, drill[3]
)
elif drill[0] == "recover":
# The scripted one-round recovery after the NOT-A-FOLDER
# teaching: ls the target's source (the parent level).
stream = _tool_call_stream("ls", {"path": drill[1]}, "call_1")
elif drill[0] == "read_answer":
# Quote the read document (first 80 chars) — the
# phase-37 single-read shape.
stream = _sse_stream(
_apply_max_tokens(
f"Read {drill[1]}. {drill[2]}", body.get("max_tokens")
),
0.0,
)
elif drill[0] == "ctx_answer":
# The dedupe fired: the target's full text is in the
# <documents> prompt — answer from it (same citation
# shape, prefixed so the suite pins the dedupe path).
stream = _sse_stream(
_apply_max_tokens(
f"Already in context: Read {drill[1]}. {drill[2]}",
body.get("max_tokens"),
),
0.0,
)
else: # "echo" — the listing verbatim (the suite's lens)
stream = _sse_stream(
_apply_max_tokens(
f"Here's the level I listed:\n{drill[1]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 95 (task 03): the deterministic SCRIPTED capped read
# (the question carries its own call — ``read the capped
# document: read source/path``): the scripted ``read``, then the
# answer that ECHOES the whole tool result (the marker + notice
# when the cap fired, the plain shape when it did not — the
# suite's lens on the LLM's context). Checked BEFORE the plain
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
# phase-71/72/94 ordering convention; the trigger needs the
# ``<tools>`` section, so deflected turns never hit it).
read_cap = _read_cap_flow(body)
if read_cap is not None:
if read_cap[0] == "call":
stream = _tool_call_stream(
"read", {"path": read_cap[1]}, read_cap[2]
)
else: # "echo" — the last tool result verbatim (the lens)
stream = _sse_stream(
_apply_max_tokens(
f"Here's what the read returned:\n{read_cap[1]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 118 (task 06): the deterministic SCRIPTED summary read
# (the question carries its own call — ``read the suggested
# document: read source/path``): the scripted ``read`` (phase
# 118: a first read of a suggested document succeeds — the
# seeds are summaries), then the answer that ECHOES the whole
# tool result (the full text's tail reaches the rendered answer
# iff the read delivered it — the suite's lens on the LLM's
# context). Checked BEFORE the plain TOOLS_TRIGGER flow
# (disjoint trigger phrases — the phase-71/72/94 ordering
# convention; the trigger needs the ``<tools>`` section, so
# deflected turns never hit it).
seed_read = _summary_seed_read_flow(body)
if seed_read is not None:
if seed_read[0] == "call":
stream = _tool_call_stream(
"read", {"path": seed_read[1]}, seed_read[2]
)
else: # "echo" — the last tool result verbatim (the lens)
stream = _sse_stream(
_apply_max_tokens(
f"Here's what the read returned:\n{seed_read[1]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 131 (task 04): the deterministic grep ->
# read(offset/limit) DANCE (the read line-ranges' story suite,
# tests/e2e/test_read_line_ranges.py — the owner's stuck
# scenario, TODO.md L7): grep finds the line, the range read
# lands the part the model does NOT have, the answer echoes
# the read slice. Checked BEFORE the plain TOOLS_TRIGGER flow
# (disjoint trigger phrases — the phase-71/72/94/118 ordering
# convention; the trigger needs the ``<tools>`` section, so
# deflected turns never hit it).
read_lines = _read_lines_flow(body)
if read_lines is not None:
if read_lines[0] == "grep":
stream = _tool_call_stream(
"grep", {"pattern": READ_LINES_PATTERN}, "call_0"
)
elif read_lines[0] == "read":
# The owner's dance, deterministic: ``offset`` = the
# grep-reported line number (parsed from the FIRST
# match line — the classifier keeps it as the grep's
# own string for the homogeneous return; the int
# here is the wire's), ``limit`` = READ_LINES_LIMIT
# (a part of the document, not the whole).
stream = _tool_call_stream(
"read",
{
"path": read_lines[1],
"offset": int(read_lines[2]),
"limit": READ_LINES_LIMIT,
},
read_lines[3],
)
elif read_lines[0] == "nomatch":
# The premise broke (the grep found nothing) — the
# deterministic terminal answer (the grep-teach
# ``nomatch`` convention — the suite fails loudly on
# the missing marker).
stream = _sse_stream(
_apply_max_tokens(
"No matches — the knowledge base has no such text.",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — the forced read-echo (phase-94 shape)
stream = _sse_stream(
_apply_max_tokens(
f"Found it at line {read_lines[1]}. {read_lines[2]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 139 (task 03): the owner's "same file over and over"
# scenario (the read range-tool-lines' story suite,
# tests/e2e/test_read_range_tool_lines.py — TODO.md L4): the
# model range-reads the SAME document TWICE with different
# offset/limit, so the two Reading lines are distinguishable.
# Checked BEFORE the plain TOOLS_TRIGGER flow (disjoint trigger
# phrase — the phase-71/72/94/118/131 ordering convention; the
# trigger needs the ``<tools>`` section, so deflected turns
# never hit it).
same_file = _same_file_flow(body)
if same_file is not None:
if same_file[0] == "read1":
# The FIRST range read on the parsed target — the
# model's REQUEST (ASSUMPTION A1: the suffix shows the
# request, not the actual slice extent).
stream = _tool_call_stream(
"read",
{
"path": same_file[1],
"offset": SAME_FILE_FIRST_OFFSET,
"limit": SAME_FILE_FIRST_LIMIT,
},
"call_0",
)
elif same_file[0] == "read2":
# The SECOND range read — the SAME file again, a
# different range (the owner's confusion made visible).
stream = _tool_call_stream(
"read",
{
"path": same_file[1],
"offset": SAME_FILE_SECOND_OFFSET,
"limit": SAME_FILE_SECOND_LIMIT,
},
"call_1",
)
else: # "answer" — the forced read-echo of BOTH ranges
stream = _sse_stream(
_apply_max_tokens(
f"Read the same file twice — lines "
f"{SAME_FILE_FIRST_OFFSET}–"
f"{SAME_FILE_FIRST_OFFSET + SAME_FILE_FIRST_LIMIT - 1}: "
f"{same_file[1]}; lines "
f"{SAME_FILE_SECOND_OFFSET}–"
f"{SAME_FILE_SECOND_OFFSET + SAME_FILE_SECOND_LIMIT - 1}: "
f"{same_file[2]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 141 (task 04): the owner's stuck scenario (the read
# range-dedupe fix's story suite,
# tests/e2e/test_read_range_dedupe.py — TODO.md L3): the
# model range-reads ONE document in TWO different ranges —
# the second PROCEEDS on the coverage-tracked server (the
# pre-phase pair-set dedupe refused it ``ALREADY_IN_
# CONTEXT``), and the answer echoes the second range's
# content. Checked BEFORE the plain TOOLS_TRIGGER flow
# (disjoint trigger phrases — the phase-71/72/94/118/131/
# 139 ordering convention; the trigger needs the ``<tools>``
# section, so deflected turns never hit it).
two_ranges = _two_ranges_flow(body)
if two_ranges is not None:
if two_ranges[0] == "read1":
# The FIRST range read on the parsed target — range
# A (lines 1–100 of the suite's document).
stream = _tool_call_stream(
"read",
{
"path": two_ranges[1],
"offset": TWO_RANGES_FIRST_OFFSET,
"limit": TWO_RANGES_FIRST_LIMIT,
},
"call_0",
)
elif two_ranges[0] == "read2":
# The SECOND range read — the SAME file again, a
# different range (range B, lines 101–…). It only
# PROCEEDS on the coverage-tracked server: its lines
# are NOT covered by the first read's span (the pre-
# phase pair-set dedupe refused it — the owner's
# stuck turn).
stream = _tool_call_stream(
"read",
{
"path": two_ranges[1],
"offset": TWO_RANGES_SECOND_OFFSET,
"limit": TWO_RANGES_SECOND_LIMIT,
},
"call_1",
)
elif two_ranges[0] == "bad":
# The premise broke (a refusal — the second range
# read was refused — or an unkeyed extent header):
# echo the offending result verbatim so a suite
# fails loudly on it (the read-cap flow's broken-
# premise convention).
stream = _sse_stream(
_apply_max_tokens(
f"Premise broke:\n{two_ranges[1]}",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — the forced read-echo of range B
# (the marker line lives in range B — the echo of it
# proves the second slice reached the model).
stream = _sse_stream(
_apply_max_tokens(
f"Both parts read. {two_ranges[1]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 145 (task 04): the owner's "the answer points at the
# original git source" scenario (the git-source-links' story
# suite, tests/e2e/test_git_source_links.py — TODO.md L7):
# the scripted read, then the final answer that carries the
# repository URL parsed from the read result's ``origin:``
# line (the server-side half — the mock cannot fabricate it)
# + the result VERBATIM (the lens on the origin lines'
# presence/absence). Checked BEFORE the plain TOOLS_TRIGGER
# flow (disjoint trigger phrases — the phase-71/72/94/118/
# 131/139/141 ordering convention; the trigger needs the
# ``<tools>`` section, so deflected turns never hit it).
origin_link = _origin_link_flow(body)
if origin_link is not None:
if origin_link[0] == "call":
stream = _tool_call_stream(
"read", {"path": origin_link[1]}, origin_link[2]
)
elif origin_link[0] == "bad":
# The premise broke (a refusal — the scripted read
# was refused): echo the offending result verbatim
# so a suite fails loudly on it (the two-ranges
# convention).
stream = _sse_stream(
_apply_max_tokens(
f"Premise broke:\n{origin_link[1]}",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — the final answer (tests 1–3's model-
# side contract) + the verbatim echo (the lens on
# the origin lines' presence/absence). ``url`` is
# the ``origin:`` line's URL or the "" sentinel
# (the no-origin sentence — the local-source
# control).
url = origin_link[1]
sentence = (
ORIGIN_LINK_ANSWER.format(url=url) if url else ORIGIN_LINK_NO_ORIGIN
)
stream = _sse_stream(
_apply_max_tokens(
f"{sentence}\n{_ORIGIN_LINK_ECHO_HEADER}\n{origin_link[2]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 160 (task 05, tool-call budget telemetry,
# TODO.md L4): the deterministic ROUND-CAP marker flow
# (the budget-telemetry suite's dedicated flow — see the
# module docstring). Checked BEFORE the plain
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
# phase-71/72/94/118/131/139/141/145 ordering
# convention).
round_cap = _round_cap_flow(body)
if round_cap is not None:
if round_cap[0] == "call":
# The model never stops asking: ``ls`` with no
# arguments, the next incrementing id, NO content
# — the turn is guaranteed to hit
# ``BOR_AGENT_MAX_ROUNDS``.
stream = _tool_call_stream("ls", {}, round_cap[1])
elif round_cap[0] == "dump":
# The forced final (no ``tools``) — the model
# dumps a structured ``read`` call anyway (the
# seed path, id ``call_cap_dump``, NO content) —
# the exact shape the phase kills (the app
# REJECTS it — never executed, never shown).
stream = _tool_call_stream(
"read", {"path": round_cap[1]}, "call_cap_dump"
)
else: # "continue" — the stable continuation answer
stream = _sse_stream(
_apply_max_tokens(
ROUND_CAP_CONTINUE_ANSWER, body.get("max_tokens")
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 124 (task 07, image chips and answers, TODO L3): the
# scripted SHOW_IMAGE flow — the mock shows the seeded image
# document (the ``<documents>`` block with the image-
# extension path) and answers straight from the seed: ZERO
# ``read`` calls (the zero-read turn that makes the bug-fix's
# chip assertion meaningful). Clause-gated — the ``show_image``
# teaching line in the system prompt is the
# ``BOR_ANSWER_IMAGES`` toggle's presence marker, so on a
# default-OFF app the trigger never fires and the mock stays
# byte-identical to pre-phase for every other suite. Checked
# BEFORE the plain TOOLS_TRIGGER flow (disjoint trigger
# phrase — the phase-71/72/94/118 ordering convention; the
# trigger needs the clause, so deflected turns never hit it).
show_image = _show_image_flow(body)
if show_image is not None:
if show_image[0] == "call":
stream = _tool_call_stream(
"show_image", {"path": show_image[1]}, "call_show"
)
else: # "answer" — the final answer + the tail echo
block = _DOCUMENTS_BLOCK_RE.search(_system(body))
tail = (block.group(0) if block else _context(body))[-160:]
stream = _sse_stream(
_apply_max_tokens(
"Here is the picture you asked about — …and the very "
f"end of my notes reads: “{tail}” "
"(Deterministic mock answer for E2E.)",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 109 (task 03, never-frozen-turn story suite): the
# deterministic REPORTED-REPRO turn (delta → tool →
# thinking-after-delta — TODO.md L3): the scripted first answer,
# the no-arg ls, then the post-tool thinking → answer → FINAL
# thinking round — the owner's exact repro with baked-in delays
# (see the module docstring). Checked BEFORE the plain
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
# phase-71/72/94 ordering convention; the trigger needs the
# ``<tools>`` section, so deflected turns never hit it).
turn_progress = _turn_progress_flow(body)
if turn_progress is not None:
stream = _turn_progress_stream(turn_progress)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 161 (task 04 — the reasoning-bubble story): the
# deterministic MID-ROUND flow — a thinking run, then a
# MID-ROUND ``delta`` (the React-style partial answer — the
# content stream BEFORE the tool call, the exact ordering
# the owner's 2026-10-05 TODO describes), then the tool
# call; the follow-up request thinks + the closing
# ``delta`` + the content finish (see the module
# docstring). Checked BEFORE the phase-158 check: the
# phrases share only ``think out loud`` and NEITHER is a
# substring of the other (verified 2026-10-06), and no
# existing E2E question carries the phrase (verified
# 2026-10-06: every other suite is unaffected) — the
# more-specific-phrase-wins convention (the SLOW_PRETOKEN /
# THINK_PARAS / SCAFFOLD_ALWAYS precedent). The trigger
# needs the ``<tools>`` section, so deflected turns never
# hit it.
midround = _reasoning_midround_flow(body)
if midround is not None:
if midround[0] == "midround":
# The mid-round ordering: the thinking run, then the
# MID-ROUND content delta, then the tool call — the
# ``_midround_tool_stream`` three-part shape.
stream = _midround_tool_stream(
midround[1],
midround[2],
"ls",
{},
"call_0",
)
else: # "final" — think, the closing delta, the content finish
stream = _sse_stream(
midround[2],
0.0,
thinking=midround[1],
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 158 (task 04 — the thinking/tool-call timeline
# story): the deterministic INTERLEAVED flow — a DISTINCT
# reasoning run ahead of every tool call and ahead of the
# final answer (see the module docstring). Checked BEFORE
# the plain TOOLS_TRIGGER flow: the trigger CONTAINS
# ``use your tools`` (and ``think out loud``), so the plain
# tool flow must not see it (the more-specific-phrase-wins
# convention — the SLOW_PRETOKEN / THINK_PARAS / SCAFFOLD_
# ALWAYS precedent; verified 2026-10-04: no existing E2E
# question contains the phrase, so every other suite is
# unaffected). The trigger needs the ``<tools>`` section, so
# deflected turns never hit it.
think_tools = _thinking_tools_flow(body)
if think_tools is not None:
if think_tools[0] == "list":
stream = _tool_call_stream("ls", {}, "call_0", think_tools[1])
elif think_tools[0] == "drill":
stream = _tool_call_stream(
"ls", {"path": think_tools[1]}, "call_1", think_tools[2]
)
elif think_tools[0] == "read":
stream = _tool_call_stream(
"read", {"path": think_tools[1]}, "call_2", think_tools[2]
)
else: # "answer" — the phase-37 shape, thinking first
answer = _apply_max_tokens(
f"Read {think_tools[1]}. {think_tools[2][:80]}",
body.get("max_tokens"),
)
stream = _sse_stream(answer, 0.0, thinking=think_tools[3])
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
flow = _tool_flow(body)
if flow is not None:
if flow[0] == "list":
stream = _tool_call_stream("ls", {}, "call_0")
elif flow[0] == "drill":
# Phase 94: the drill step — ls scoped to the first
# source of the top-level listing (flow[1], flow[2] is
# the synthetic call id).
stream = _tool_call_stream("ls", {"path": flow[1]}, flow[2])
elif flow[0] == "read":
# flow[3] is the synthetic call id — "call_1" for the
# single-read flow and the multi-read first read,
# "call_2" for the multi-read second read (phase 45,
# task 02). Phase 70: the harness-aligned shape — one
# combined ``source/path`` argument (the mock joins the
# two catalog fields; the catalog format is unchanged).
stream = _tool_call_stream(
"read", {"path": f"{flow[1]}/{flow[2]}"}, flow[3]
)
elif flow[0] == "multi_answer":
# Phase 45 (task 02): the multi-read forced answer —
# computed in _tool_flow, byte-stable.
stream = _sse_stream(_apply_max_tokens(flow[2], body.get("max_tokens")), 0.0)
else: # "answer" — quote the read document (first 80 chars)
answer = _apply_max_tokens(
f"Read {flow[1]}. {flow[2][:80]}", body.get("max_tokens")
)
stream = _sse_stream(answer, 0.0)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
answer = _apply_max_tokens(compose_answer(body), body.get("max_tokens"))
delay = 3.0 if "pretend to think slowly" in _user(body) else 0.0
# ``think in paragraphs`` wins over ``think out loud`` (more specific):
# the same scratchpad WITH real "\n\n" paragraph breaks, at 60-char
# frames (real-model-sized deltas — the 32px-band regression pin).
if THINK_PARAS_TRIGGER in user_lower:
thinking = compose_thinking_paragraphs(body)
chunk = THINK_PARAS_CHUNK
elif THINKING_TRIGGER in user_lower:
thinking = compose_thinking(body)
chunk = 12
else:
thinking = ""
chunk = 12
pre_content = (
PRE_CONTENT_PAUSE_S if SLOW_PRETOKEN_TRIGGER in user_lower else 0.0
)
if not body.get("stream"):
# Phase 96 (task 04): the incident-shape injection (the module
# docstring) — the trigger-labelled folder summary answers the
# exact 2026-09-11 envelope: the mock's normal OpenAI
# chat-completion shape with ``content=""`` and
# ``finish_reason="length"`` (the budget spent in
# ``reasoning_content``). ``chat()`` is the mock's only
# non-streaming folder-summary consumer, so this is the one-shot
# retry path (``app.rag.llm.LLMClient.chat``, phase 96 task 01)
# and nothing else — every other non-stream response is
# byte-identical to pre-phase-96.
if _folder_summary_incident(body):
return {
"id": f"chatcmpl-{uuid.uuid4()}",
"object": "chat.completion",
"created": int(time.time()),
"model": body.get("model", "turbo"),
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": ""},
"finish_reason": "length",
}
],
"usage": {
"prompt_tokens": 100,
"completion_tokens": 2048,
"total_tokens": 2148,
},
}
message: dict[str, Any] = {"role": "assistant", "content": answer}
if thinking:
# Harmless future-proofing: the app only uses streaming, but a
# non-streaming client that reads the field gets the reasoning.
message["reasoning_content"] = thinking
return {
"id": f"chatcmpl-{uuid.uuid4()}",
"object": "chat.completion",
"created": int(time.time()),
"model": body.get("model", "turbo"),
"choices": [
{"index": 0, "message": message, "finish_reason": "stop"}
],
"usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150},
}
return StreamingResponse(
_sse_stream(
answer,
delay,
thinking=thinking,
pre_content_delay=pre_content,
chunk=chunk,
),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)