Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 94_ls_tree_drilldown
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Details
Build and Push Containers / build-and-push-db (push) Successful in 25s
Details

Browse Source 
All green. Verification complete.

**Phase 94 — `ls` drill-down tree: final verification pass (all 5 tasks were already complete; verified, nothing to fix)**

- Verified `ls` 3-level tree (`app/rag/agent.py`): `ls()` sources + summaries, `ls(source)`/`ls(source/folder)` drill-down, 50-line file cap + grep-pointer note, NOT-A-FOLDER teaching refusal
- Verified `folder_summaries` (migration 0017, model, `app/rag/folder_summaries.py` generator: `FOLDER_SUMMARY_MODE` marker, fail-soft per folder, ≥2-doc scope + prune) wired change-gated in both sync paths
- Verified 10-turn fixture battery verdict recorded in `TOOL_CALLING_TESTING.md` §9 (2026-09-11): turbo PASS 19/19 contract, 98.7 s (−12.5…−13.2 % vs baseline); lite PASS 18/18, 43.6 s (+7.7 %) — accuracy at/above baseline, gate met
- `uv run pytest --cov=app --cov-report=term-missing` → 1939 passed, 0 failed; TOTAL coverage **99 %** (folder_summaries.py 100 %)
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- E2E in isolation: `test_ls_tree_drilldown.py` 3 passed; `test_agent_document_tools` 4, `test_agent_unlimited_tools` 4, `test_harness_aligned_tools` 3, `test_search_tool` 3, `test_grep_regex_teaching` 2, `test_response_to_docs` 4 — all passed (read/grep contracts untouched)
- Dedicated folder-summary tests (fail-soft, prune, both sync paths, migration): 46 passed
- Completion criteria: all 6 met; working tree holds only phase-94 changes (commit left to harness per protocol)

**Next pending phase:** `95_read_truncation_cap`
This commit is contained in:
ducoterra
2026-09-11 00:59:35 -04:00
parent 9188be259b
commit d4943b4822
61 changed files with 6289 additions and 666 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/e2e/mock_llm.py
+456 -73
View File
@@ -18,6 +18,14 @@ Implements just enough of the aipi surface:
- ``KB_OVERVIEW_MODE`` -> the deterministic outline: the first 8 tokens
of the user message (the generator puts the document list there) —
byte-stable for a given KB (KB overview, phase 31)
- ``FOLDER_SUMMARY_MODE`` -> the deterministic folder one-liner
``Fixture folder summary for <folder>.`` — <folder> is the user
message's ``Folder: …`` header line (the generator names the
source/folder there, phase 94), so the stored row always names its
folder and an E2E can assert on it. Checked BEFORE the
``SUMMARY_MODE`` branch: the folder marker CONTAINS the summary
marker as a substring, so the summary branch would otherwise
shadow every folder-summary call
- ``DEFLECT_MODE`` -> honest "I haven't done anything like that" answer
- otherwise -> upbeat answer quoting the provided document context
- user message containing ``pretend to think slowly`` -> 3s warm-up delay
@@ -57,22 +65,31 @@ Implements just enough of the aipi surface:
closing tag — same sentinel semantics.)
- user message containing ``use your tools`` (phase 37, agent document
tools; phase 70: the flow emits the harness-aligned names — ``ls``
/ ``read`` with the combined ``source/path`` identity) **and** the
system prompt carries the ``<tools>`` section -> the deterministic
SINGLE-READ tool flow, discriminated statelessly from the messages
(the ``tools`` parameter gates the list/read steps — a no-tools
request with no tool results is not the flow):
/ ``read`` with the combined ``source/path`` identity; phase 94:
the flow DRILLS through the tree ``ls`` — the top-level listing
carries sources only, so the flow takes one drill step, ``ls``
scoped to the first source, before the first file line exists)
**and** the system prompt carries the ``<tools>`` section -> the
deterministic SINGLE-READ tool flow, discriminated statelessly from
the messages (the ``tools`` parameter gates the list/read steps —
a no-tools request with no tool results is not the flow):
* request 1 (``tools`` offered, no tool results yet): stream ONLY
``tool_calls`` deltas — ``ls`` (synthetic id ``call_0``, no
arguments), ``finish_reason: "tool_calls"``, no content;
* request 2 (a ``tool``-role catalog result in the messages):
parse the FIRST catalog line (``source: X | path: Y | title: Z``
— the labeled ``source:`` / ``path:`` fields, phase 63) and
stream a ``tool_calls`` delta calling ``read`` on the JOINED
combined ``source/path`` (the mock joins the two labeled fields
— the catalog format is unchanged, so this join is the only
parse change, phase 70) (id ``call_1``);
* request 3 (a ``tool``-role read result in the messages —
* request 2 (the top-level source listing in the messages — the
agent's ``ls`` tree format, phase 94: no file lines yet):
stream a ``tool_calls`` delta — ``ls`` scoped to the FIRST
source of the listing (id ``call_1``) — the drill step (one
drill per flow — ``_drill_target`` skips already-drilled
sources, so the second listing is the folder level);
* request 3 (a ``tool``-role folder listing with file lines in
the messages): parse the FIRST file line (``source: X | path:
Y | title: Z`` — the labeled ``source:`` / ``path:`` fields,
phase 63) and stream a ``tool_calls`` delta calling ``read`` on
the JOINED combined ``source/path`` (the mock joins the two
labeled fields — the file-line format is unchanged, so this
join is the only parse change, phase 70) (id ``call_2``);
* request 4 (a ``tool``-role read result in the messages —
content starting with the agent's ``"Document <source/path>:"``
header): a content answer, deterministic: ``Read
<source/path>. <first 80 chars of the read document's
@@ -85,19 +102,22 @@ Implements just enough of the aipi surface:
- user message containing BOTH ``use your tools`` AND ``read two
documents`` (``MULTI_READ_TRIGGER``, phase 45 task 02) **and** the
system prompt carries the ``<tools>`` section -> the deterministic
MULTI-READ flow (list → read #1 → read #2 → answer), classified by
the COUNT of ``tool``-role read results (content starting with the
agent's ``"Document <source/path>:"`` prefix); phase 70: the same
flow on the harness-aligned names — ``ls``, then ``read`` on the
JOINED combined ``source/path`` of each catalog line:
* 0 read results, no catalog yet: ``ls`` (id ``call_0``);
* 0 read results, catalog present: ``read`` on the JOINED
combined ``source/path`` of the FIRST catalog line
(id ``call_1``);
MULTI-READ flow (list → drill → read #1 → read #2 → answer; phase
94: the drill step between the top-level listing and the first
read), classified by the COUNT of ``tool``-role read results
(content starting with the agent's ``"Document <source/path>:"``
prefix); phase 70: the same flow on the harness-aligned names —
``ls``, the drill ``ls`` scoped to the first source, then ``read``
on the JOINED combined ``source/path`` of each file line:
* 0 read results, no listing yet: ``ls`` (id ``call_0``);
* 0 read results, top-level listing only (no file lines yet):
the drill — ``ls`` scoped to the first source (id ``call_1``);
* 0 read results, file lines present: ``read`` on the JOINED
combined ``source/path`` of the FIRST file line (id ``call_2``);
* 1 read result: ``read`` on the JOINED combined ``source/path``
of the SECOND catalog line — the first listing line whose
of the SECOND file line — the first listing line whose
``source/path`` differs from the one already read (id
``call_2``); a one-document catalog degenerates to the
``call_3``); a one-file listing degenerates to the
single-read answer (nothing second to read);
* 2 read results: the forced answer, byte-stable: the single-read
shape quoting the FIRST read result, plus the line ``I read
@@ -162,27 +182,83 @@ Implements just enough of the aipi surface:
misuse met the terse refusal and the model re-reasoned the same
paragraphs over and over) **and** the system prompt carries the
``<tools>`` section -> the deterministic LS-TEACHING flow,
discriminated statelessly from the messages (streaming only):
discriminated statelessly from the messages (streaming only;
phase 94: the correction's top-level listing carries sources only,
so the flow drills one level before the first file line exists):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): stream ONLY ``tool_calls`` deltas — ``ls``
with ``{"path": "."}`` (synthetic id ``call_0``),
``finish_reason: "tool_calls"``, no content — the incident's
misuse, deterministic;
* request 2 (a ``tool``-role result present that is NOT a
catalog listing — i.e. the teaching refusal): a ``tool_calls``
delta — ``ls`` with no arguments (id ``call_1``) — the
correction;
* request 3 (a ``tool``-role result whose first line matches the
``^\\d+ documents:`` catalog header): a deterministic content
answer — ``These are the indexed documents: <first catalog
line>`` (the ``source: X | path: Y | title: Z`` line, parsed
with the ``_CATALOG_LINE_RE`` machinery), ``finish_reason:
"stop"`` — the loop ended in ONE correction, not at the round
listing with file lines — i.e. the teaching refusal): a
``tool_calls`` delta — ``ls`` with no arguments (id ``call_1``)
— the correction;
* request 3 (the top-level source listing in the messages —
file lines still absent): the drill — a ``tool_calls`` delta
— ``ls`` scoped to the FIRST source of the listing
(id ``call_2``);
* request 4 (a ``tool``-result with a ``source: X | path: Y |
title: Z`` file line in the messages — the folder listing):
a deterministic content answer — ``These are the indexed
documents: <first file line>`` (the line, parsed with the
``_CATALOG_LINE_RE`` machinery), ``finish_reason: "stop"`` —
the loop ended in ONE correction + ONE drill, not at the round
cap.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (the trigger
phrases are disjoint substrings — the phase-71 ordering
convention); no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``drill down the tree`` (``DRILL_TRIGGER``,
phase 94 task 04 — the drill-down ``ls``'s dedicated story suite
``tests/e2e/test_ls_tree_drilldown.py``) **and** the system prompt
carries the ``<tools>`` section -> the deterministic SCRIPTED
DRILL-DOWN flow: the question carries its own tool call after the
colon — ``drill down the tree: ls [target]`` (no target = the top
level; ``target`` = a source name or ``source/folder`` path) or
``drill down the tree: read source/path`` — parsed by
``_DRILL_CALL_RE`` from the RAW user message (the target keeps its
case), then discriminated statelessly from the tool results
(streaming only):
* request 1 (``tools`` offered, no ``tool``-role result in the
messages yet): the SCRIPTED call — ``ls`` with no arguments
for the top level, ``ls``/``read`` with the parsed target
otherwise (synthetic id ``call_0``);
* the last tool result is the NOT-A-FOLDER teaching refusal
(it carries ``"is not a folder"`` — the phase-94 task-03
teaching line, the refusal being VISIBLE to the model is what
fires this branch): the scripted one-round recovery — ``ls``
the target's SOURCE segment (id ``call_1``);
* the last tool result is a READ result (``"Document
<source/path>:…``): the deterministic echo answer ``Read
<source/path>. <first 80 chars of the read document's
content>`` (the phase-37 single-read shape — the grounded-
turn citation contract);
* the last tool result is the agent's ALREADY_IN_CONTEXT dedupe
refusal (the read target is already a top-2 retrieval
document — with the drill fixture that is DETERMINISTIC:
the read question names the file's path, so the file
self-matches the hybrid gate and its FULL text is in the
``<documents>`` prompt): the model answers FROM THE PROMPT —
the deterministic answer ``Already in context: Read
<source/path>. <first 80 chars of the target document's text
as it appears in the ``<documents>`` block>`` (same citation
shape as the read-result branch — the document text reached
the model either way, and the answer proves it);
* any other last result (a top-level or folder LISTING landed):
the deterministic ECHO answer ``Here's the level I listed:\n
<the listing, verbatim>`` — the mock echoes what it received
(the house scripted-turn way of asserting on tool results):
the suite asserts on the exact tree level — the ``— N
documents`` lines, the stored folder summaries, the
``source: X | path: Y | title: Z`` file lines, and the 50-line
cap + ``…and N more documents…`` note — through the rendered
answer, the only E2E lens on the LLM's context.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrases — the phase-71/72 ordering convention; the trigger needs
the ``<tools>`` section, so deflected turns never hit it); no
existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
- user message containing ``what are the correct llama.cpp
arguments`` (``GREP_TEACH_TRIGGER``, the 2026-09-05 incident —
the harness prior is that grep takes a REGEX; this app's grep is a
@@ -302,7 +378,8 @@ from typing import Any
from fastapi import FastAPI
from fastapi.responses import JSONResponse, StreamingResponse
from app.rag.agent import CORRECTION_INSTRUCTION # phase 71: the harness constant
from app.rag.agent import ALREADY_IN_CONTEXT, CORRECTION_INSTRUCTION
from app.rag.folder_summaries import FOLDER_HEADER_PREFIX # phase 94: the mock's key
app = FastAPI()
@@ -588,9 +665,92 @@ GREP_TEACH_PLAIN = "qwen3.8"
#: plain step keys on it (a plain no-match line carries it not).
GREP_TEACH_MARKER = "grep matches a plain substring"
#: The agent's ``ls`` listing header (app.rag.agent ``_execute_tool``):
#: ``"N documents:"`` — the first line of every catalog tool result.
_CATALOG_HEADER_RE = re.compile(r"^\d+ documents:")
# ---------------------------------------------------------------------------
# Phase 94 (task 04, the drill-down ls's dedicated story suite):
# the deterministic SCRIPTED drill-down turns — see the module docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the scripted DRILL-DOWN flow: the question carries its own tool call
#: after the colon (``drill down the tree: ls [target]`` / ``drill
#: down the tree: read source/path`` — the suite's scripted turns,
#: ``tests/e2e/test_ls_tree_drilldown.py``). Checked BEFORE the plain
#: ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the phase-71/72
#: ordering convention); verified: no existing E2E question or fixture
#: file contains the phrase, so every other suite is unaffected.
DRILL_TRIGGER = "drill down the tree"
#: The scripted call in the drill-down question (case-insensitive — the
#: suite's questions capitalize the trigger's first letter): the verb
#: (``ls`` / ``read``) plus the optional target — a source name or
#: ``source/folder`` path (``ls``) or a combined ``source/path``
#: (``read``), parsed from the RAW user message so the target keeps its
#: case. The target is a ``[a-z0-9_./-]`` run (case-insensitively), so
#: the suite's `` — `` flavor separator (em dash) can never bleed into
#: it; no run = the top-level ``ls`` (no ``path`` argument).
_DRILL_CALL_RE = re.compile(
r"drill down the tree:\s*(?P<verb>ls|read)(?:\s+(?P<arg>[a-z0-9_./-]+))?",
re.I,
)
#: The agent's NOT-A-FOLDER teaching refusal marker (app.rag.agent
#: ``NOT_A_FOLDER`` — ``'{arg}' is not a folder — {parent} has:
#: {subfolders}``): the drill-down flow's scripted recovery keys on it
#: in the LAST tool result (the refusal being visible to the model is
#: exactly what triggers the one-round recovery — the phase-72
#: self-correction contract carrying the tree's teaching line).
_NOT_A_FOLDER_MARKER = "is not a folder"
#: The stable substring of the harness-owned dedupe refusal the
#: drill-down flow's ``ctx_answer`` branch keys on (a read target that
#: is already a top-2 retrieval document — the mock then answers from
#: the document's text in the ``<documents>`` prompt block, exactly
#: what the refusal instructs). Keyed on a substring (not the whole
#: constant) so a re-wrap of the constant cannot silently re-route the
#: mock; the module-level assert below fails loudly if the substring
#: ever leaves the constant (the mock must never drift from
#: ``app.rag.agent.ALREADY_IN_CONTEXT``).
_ALREADY_IN_CONTEXT_MARKER = "Already in your context"
assert _ALREADY_IN_CONTEXT_MARKER in ALREADY_IN_CONTEXT, (
"mock drift: the dedupe marker left ALREADY_IN_CONTEXT"
)
#: One ``<document>`` block of the HIGH prompt's ``<documents>``
#: section (``app.rag.prompts.build_high_prompt``): the block is the
#: document identity (``source``/``path``/``title`` attributes) plus
#: the document's FULL text (never truncated on the retrieval path,
#: owner-locked A7) between the tags.
_DOCUMENT_BLOCK_RE = re.compile(
r'<document source="(?P<source>[^"]+)" path="(?P<path>[^"]+)" '
r'title="[^"]*">\n(?P<content>.*?)\n</document>',
re.S,
)
def _document_block(system: str, source: str, path: str) -> str | None:
"""The stored text of one ``<document>`` block (or ``None``).
The drill-down flow's ``ctx_answer`` branch: when the agent's read
of a top-2 retrieval document gets the ALREADY_IN_CONTEXT dedupe,
the document's full text is in the ``<documents>`` prompt — the
mock (the model) extracts it by the block's identity attributes
and quotes it, answering from the prompt as the refusal instructs.
"""
for block in _DOCUMENT_BLOCK_RE.finditer(system):
if block.group("source") == source and block.group("path") == path:
return block.group("content")
return None
#: The agent's drill-down ``ls`` result shapes (app.rag.agent
#: ``render_ls_top`` / ``render_folder_listing``, phase 94): the
#: top-level header ``"N sources:"`` and the per-source block line
#: ``"<source> — N documents"`` (summary lines are indented — they
#: never match); the folder-level header ``"<identity> — N documents,
#: M folders:"`` (identity = the source name or ``source/folder``).
_SOURCE_HEADER_RE = re.compile(r"^\d+ sources:$")
_SOURCE_LINE_RE = re.compile(r"^(?P<source>.+?) — (?P<n>\d+) documents$")
_FOLDER_HEADER_RE = re.compile(r"^(?P<identity>.+?) — \d+ documents, \d+ folders:$")
#: One DEAD app-level chat attempt costs exactly this many HTTP POSTs
#: while the endpoint stays down: the openai SDK's default policy
@@ -713,26 +873,60 @@ def _tool_results(body: dict[str, Any]) -> list[str]:
]
def _first_catalog_line(body: dict[str, Any]) -> str | None:
"""The first catalog line of a catalog listing in the messages.
def _first_file_line(body: dict[str, Any]) -> str | None:
"""The first file line (``source: X | path: Y | title: Z``) across
the ``tool``-role results in the messages, in message order (read
results — full documents, not listings — skipped; the ``_CATALOG_-
LINE_RE`` machinery).
A catalog listing is a ``tool``-role result whose FIRST line is the
agent's ``"N documents:"`` header (``_CATALOG_HEADER_RE``); its
first ``source: X | path: Y | title: Z`` line (the
``_CATALOG_LINE_RE`` machinery) is returned. ``None`` when no
catalog listing is in the messages — e.g. while only the teaching
refusal is there (the phase-72 LS-TEACH flow's request-2 state).
An empty listing (``"0 documents:"`` with no lines) returns
``""`` — the listing is present, it is just empty.
Phase 94: the drill-down ``ls`` carries file lines only at folder
levels — the top-level source listing and the teaching refusals
have none, so ``None`` here means "no folder level reached yet".
"""
for content in _tool_results(body):
lines = content.splitlines()
if not lines or not _CATALOG_HEADER_RE.match(lines[0]):
if content.startswith(_READ_RESULT_PREFIX):
continue
for line in lines[1:]:
for line in content.splitlines():
if _CATALOG_LINE_RE.match(line):
return line
return ""
return None
def _drill_target(body: dict[str, Any]) -> str | None:
"""The next source to drill into (phase 94, the drill step).
The drill-down ``ls`` top level lists SOURCES only (no file lines),
so a deterministic flow that needs a file line must drill one level:
``ls`` scoped to a source. The target is the FIRST source of the
top-level listing (registry order — the listing's own order) that
does not yet have a folder-level listing in the messages (a folder
header whose identity is the source or ``source/…``); ``None`` when
no top-level listing is in the messages yet (nothing to drill from)
or every listed source has been drilled (an empty KB — the flow
degenerates to the old re-list loop, settling at the round cap
exactly like the phase-70 empty-catalog case).
"""
sources: list[str] = []
drilled: set[str] = set()
for content in _tool_results(body):
lines = content.splitlines()
if not lines:
continue
if _SOURCE_HEADER_RE.match(lines[0]):
for line in lines[1:]:
match = _SOURCE_LINE_RE.match(line)
if match:
sources.append(match.group("source"))
else:
folder = _FOLDER_HEADER_RE.match(lines[0])
if folder:
drilled.add(folder.group("identity"))
for source in sources:
if not any(
identity == source or identity.startswith(source + "/")
for identity in drilled
):
return source
return None
@@ -853,11 +1047,14 @@ def _tool_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
``TOOLS_TRIGGER`` and ``MULTI_READ_TRIGGER``), classified by the
count of ``tool``-role read results:
* 0 read results: ``("list", "", "")`` (no catalog yet) or
``("read", source, path, "call_1")`` on the FIRST catalog doc.
* 1 read result: ``("read", source, path, "call_2")`` on the SECOND
catalog doc — the first listing line whose ``source/path``
differs from the one already read. A one-document catalog
* 0 read results: ``("list", "", "")`` (no listing yet),
``("drill", source, "call_1")`` when the top-level source
listing is in the messages but no file line yet (phase 94: the
drill — the top level carries sources only), or
``("read", source, path, "call_2")`` on the FIRST file-line doc.
* 1 read result: ``("read", source, path, "call_3")`` on the SECOND
file-line doc — the first listing line whose ``source/path``
differs from the one already read. A one-file listing
degenerates to the single-read ``("answer", ...)`` shape (nothing
second to read).
* 2 read results: ``("multi_answer", "", text)`` — the forced
@@ -880,30 +1077,37 @@ def _tool_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
if not body.get("tools"):
return None
docs = _catalog_docs(body)
if not docs:
return ("list", "", "")
return ("read", docs[0][0], docs[0][1], "call_1")
if docs:
return ("read", docs[0][0], docs[0][1], "call_2")
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_1")
return ("list", "", "")
if len(reads) == 1:
skip = reads[0][0]
second = next(
(d for d in _catalog_docs(body) if f"{d[0]}/{d[1]}" != skip), None
)
if second is None:
# One-document catalog: nothing second to read — the
# One-file listing: nothing second to read — the
# single-read answer shape (deterministic degenerate).
return ("answer", reads[0][0], reads[0][1])
return ("read", second[0], second[1], "call_2")
return ("read", second[0], second[1], "call_3")
(sp1, c1), (sp2, _c2) = reads[0], reads[1]
answer = f"Read {sp1}. {c1[:80]} I read {sp1} and {sp2}."
return ("multi_answer", "", answer)
# Phase-37 single-read flow — byte-identical to the original.
# Phase-37 single-read flow (phase 94: the drill step between the
# top-level listing and the first file line).
if reads:
return ("answer", reads[0][0], reads[0][1])
if not body.get("tools"):
return None
docs = _catalog_docs(body)
if docs:
return ("read", docs[0][0], docs[0][1], "call_1")
return ("read", docs[0][0], docs[0][1], "call_2")
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_1")
return ("list", "", "")
@@ -915,13 +1119,17 @@ def _ls_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
with ``{"path": "."}`` (id ``call_0``), ``finish_reason:
"tool_calls"``, no content.
* ``("correct",)`` — a ``tool``-role result is in the messages and
it is NOT a catalog listing (the teaching refusal): the
correction — ``ls`` with no arguments (id ``call_1``).
* ``("answer", line)`` — a ``tool``-role result whose first line
is the ``"N documents:"`` catalog header: the deterministic
no file line exists yet (the teaching refusal): the correction —
``ls`` with no arguments (id ``call_1``).
* ``("drill", source, "call_2")`` — the top-level source listing
is in the messages (the correction ran) but no file line yet
(phase 94: the top level carries sources only): the drill —
``ls`` scoped to the first source of the listing.
* ``("answer", line)`` — a ``source: X | path: Y | title: Z`` file
line is in the messages (the folder listing): the deterministic
content answer ``These are the indexed documents: <line>`` (the
first catalog line), ``finish_reason: "stop"`` — the loop
settled in ONE correction, not at the round cap.
first file line), ``finish_reason: "stop"`` — the loop settled
in ONE correction + ONE drill, not at the round cap.
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or
``tools`` are not offered and no tool results are in the
@@ -931,9 +1139,12 @@ def _ls_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
return None
if "<tools>" not in _system(body):
return None
line = _first_catalog_line(body)
line = _first_file_line(body)
if line is not None:
return ("answer", line)
drill = _drill_target(body)
if drill is not None:
return ("drill", drill, "call_2")
if _tool_results(body):
return ("correct",)
if not body.get("tools"):
@@ -996,6 +1207,92 @@ def _grep_teach_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
return ("nomatch",)
def _drill_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-94 scripted drill-down request (see the module
docstring). The question carries the scripted call
(``drill down the tree: ls [target]`` / ``drill down the tree: read
source/path``); the step is then discriminated statelessly from the
tool results, like the other marker flows:
* ``("call", verb, target, "call_0")`` — ``tools`` are offered and
no ``tool``-role result is in the messages yet: the scripted call
(``ls`` with NO path for the top level — empty target —, ``ls``
with the target for source/folder levels, ``read`` with the
combined ``source/path``).
* ``("recover", source)`` — the LAST tool result is the NOT-A-FOLDER
teaching refusal (it carries :data:`_NOT_A_FOLDER_MARKER`): the
scripted one-round recovery — ``ls`` the target's SOURCE segment
(the refusal listing the parent's subfolders is what lets the
model — and this mock — self-correct in the next round, the
phase-72 contract).
* ``("read_answer", combined, quote)`` — the LAST tool result is a
read result (``"Document <source/path>:\n<content>"``): the
deterministic echo answer ``Read <source/path>. <first 80 chars>
`` (the phase-37 single-read shape — the grounded-turn citation
contract the suite asserts).
* ``("ctx_answer", target, quote)`` — the scripted call was a
``read`` and the LAST tool result is the ALREADY_IN_CONTEXT
dedupe refusal (the target is already a top-2 retrieval document
— deterministic for the drill fixture: the read question names
the file's path, so the file self-matches the hybrid gate): the
model answers FROM THE PROMPT — ``Already in context: Read
<target>. <first 80 chars of the target document's text in the
``<documents>`` block>`` (the same citation shape as the
read-result branch — the document text reached the model either
way). Falls through to the echo when the target is not a
``<document>`` block (the premise broke — the suite fails
loudly on the answer).
* ``("echo", listing)`` — a listing landed (top-level or folder —
any other last result): the deterministic ECHO — the answer
carries the listing VERBATIM (``Here's the level I listed:\n
<listing>``), so the suite asserts on the exact tree level the
model saw (source ``— N documents`` lines + stored summaries,
subfolder lines, the ``source: X | path: Y | title: Z`` file
lines, the 50-line cap + note) through the rendered answer — the
house scripted-turn way of asserting on tool results (the mock is
the only E2E lens on the LLM's context).
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), the scripted
call is unparseable, or ``tools`` are not offered and no tool
results are in the messages yet (e.g. ``agent_max_rounds=0``).
"""
user = _user(body)
if DRILL_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _DRILL_CALL_RE.search(user)
if match is None:
return None
verb = match.group("verb")
target = match.group("arg") or ""
results = _tool_results(body)
if not results:
if not body.get("tools"):
return None
return ("call", verb, target, "call_0")
last = results[-1]
if last.startswith(_READ_RESULT_PREFIX):
head, _, content = last.partition("\n")
# The read result is ``"Document <source/path>:\n<content>"`` —
# the head carries the server's appended ``:`` (removed here;
# a document path never legitimately ends with one).
combined = head[len(_READ_RESULT_PREFIX):].strip().removesuffix(":")
return ("read_answer", combined, content[:80])
if _ALREADY_IN_CONTEXT_MARKER in last and verb == "read" and "/" in target:
# The dedupe fired: the read target is already a top-2
# retrieval document, so its FULL text is in the
# ``<documents>`` prompt — answer from the prompt (the
# refusal's instruction), quoting the block's text.
src, _, p = target.partition("/")
content = _document_block(_system(body), src, p)
if content is not None:
return ("ctx_answer", target, content[:80])
if _NOT_A_FOLDER_MARKER in last:
return ("recover", target.split("/")[0])
return ("echo", last)
def long_answer() -> str:
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
a unique final line that must survive the stream untruncated."""
@@ -1103,6 +1400,28 @@ def compose_answer(body: dict[str, Any]) -> str:
user = _user(body)
if LONG_ANSWER_TRIGGER in user.lower():
answer = long_answer()
elif "FOLDER_SUMMARY_MODE" in system:
# Folder summary (phase 94, TODO.md L4): the ``lite`` stand-in
# returns the deterministic one-liner
# ``Fixture folder summary for <folder>.`` — <folder> is the
# user message's ``Folder: …`` header line (the generator puts
# the canonical source/folder identity there, imported as
# ``FOLDER_HEADER_PREFIX`` so the mock can never drift from it).
# The stored row therefore always names its folder — the drill-
# down E2E (``test_ls_tree_drilldown.py``) asserts on it.
# Checked BEFORE the ``SUMMARY_MODE`` branch: the folder marker
# CONTAINS the summary marker as a substring (``FOLDER_`` +
# ``SUMMARY_MODE``), so the summary branch would otherwise
# shadow every folder-summary call. Checked BEFORE the
# DEFLECT_MODE branch, like the other lite-mode markers (a
# deflection prompt never carries one).
header = user.splitlines()[0] if user else ""
folder = (
header.removeprefix(FOLDER_HEADER_PREFIX).strip()
if header.startswith(FOLDER_HEADER_PREFIX)
else "(unnamed folder)"
)
answer = f"Fixture folder summary for {folder}."
elif "SUMMARY_MODE" in system:
# Document summaries (phase 30): the ``lite`` stand-in returns a
# deterministic digest — the first 24 tokens of the user message
@@ -1594,9 +1913,16 @@ def chat_completions(body: dict[str, Any]) -> Any:
# The incident's misuse, deterministic: ls(path='.').
stream = _tool_call_stream("ls", {"path": "."}, "call_0")
elif ls_teach[0] == "correct":
# The one-round correction: the no-arg full listing.
# The one-round correction: the no-arg top-level
# listing (phase 94: sources only — the drill follows).
stream = _tool_call_stream("ls", {}, "call_1")
else: # "answer" — quote the first catalog line
elif ls_teach[0] == "drill":
# Phase 94: the top level lists sources only — drill
# one level into the first source for the file lines.
stream = _tool_call_stream(
"ls", {"path": ls_teach[1]}, ls_teach[2]
)
else: # "answer" — quote the first file line
answer = _apply_max_tokens(
f"These are the indexed documents: {ls_teach[1]}",
body.get("max_tokens"),
@@ -1607,10 +1933,67 @@ def chat_completions(body: dict[str, Any]) -> Any:
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 94 (task 04): the deterministic SCRIPTED drill-down
# turns (sources → folders → files → the grounded read; the
# 50-line cap; the NOT-A-FOLDER teaching + scripted recovery) —
# checked BEFORE the plain TOOLS_TRIGGER flow (disjoint trigger
# phrases — the phase-71/72 ordering convention; the trigger
# needs the ``<tools>`` section, so deflected turns never hit
# it).
drill = _drill_flow(body)
if drill is not None:
if drill[0] == "call":
# The scripted call: ls with no path at the top level
# (empty target), ls/read with the parsed target.
stream = _tool_call_stream(
drill[1], {"path": drill[2]} if drill[2] else {}, drill[3]
)
elif drill[0] == "recover":
# The scripted one-round recovery after the NOT-A-FOLDER
# teaching: ls the target's source (the parent level).
stream = _tool_call_stream("ls", {"path": drill[1]}, "call_1")
elif drill[0] == "read_answer":
# Quote the read document (first 80 chars) — the
# phase-37 single-read shape.
stream = _sse_stream(
_apply_max_tokens(
f"Read {drill[1]}. {drill[2]}", body.get("max_tokens")
),
0.0,
)
elif drill[0] == "ctx_answer":
# The dedupe fired: the target's full text is in the
# <documents> prompt — answer from it (same citation
# shape, prefixed so the suite pins the dedupe path).
stream = _sse_stream(
_apply_max_tokens(
f"Already in context: Read {drill[1]}. {drill[2]}",
body.get("max_tokens"),
),
0.0,
)
else: # "echo" — the listing verbatim (the suite's lens)
stream = _sse_stream(
_apply_max_tokens(
f"Here's the level I listed:\n{drill[1]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
flow = _tool_flow(body)
if flow is not None:
if flow[0] == "list":
stream = _tool_call_stream("ls", {}, "call_0")
elif flow[0] == "drill":
# Phase 94: the drill step — ls scoped to the first
# source of the top-level listing (flow[1], flow[2] is
# the synthetic call id).
stream = _tool_call_stream("ls", {"path": flow[1]}, flow[2])
elif flow[0] == "read":
# flow[3] is the synthetic call id — "call_1" for the
# single-read flow and the multi-read first read,