phase: 95_read_truncation_cap
All gates are green. Final verification is complete — no fixes were needed; the phase is fully implemented and passing. **Phase 95 final verification report** - Verified all 3 tasks already implemented (task files in `complete/`): read cap + notice + `ToolResultPiece` (task 01), `tool_result` SSE + live/saved/shared marker (task 02), E2E suite (task 03). - Code checks passed: `BOR_READ_MAX_CHARS` (default 128 000, validator), pinned `READ_TRUNCATION_NOTICE` + `TRUNCATION_MARKER`, byte-identical at/under-cap read, holder `read_truncations` (no `tool_calls` impact), piece order (after `tool`, before next round), `ChatToolResultEvent`, `ToolCall` back-compat fields, `app.js` live/restore handlers, `shared.js`, `.truncated-note` CSS, `.env.example` entry, mock-LLM echo script. - **Tests:** `uv run pytest` → 1966 passed; `uv run pytest --cov=app --cov-report=term-missing` → all green, TOTAL **99%** (>90% gate). - **E2E:** `uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov` → **3 passed** (frame order + live marker + LLM notice via echo; save→shared fidelity; short-read control). - **Regression (isolated):** `test_agent_document_tools` 4 ✓, `test_chat_history` 5 ✓, `test_share_chat` 4 ✓, `test_big_read_progress` 4 ✓, `test_stop_generation` 3 ✓. - **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings. **Completion criteria:** ① over-cap read → first-cap-chars + marker + pinned notice — ✓ (unit-pinned: at-cap/cap+1/notice tests); ② user marker live/saved/shared — ✓ (E2E + frontend tests); ③ at/under cap byte-identical, no frame — ✓ (unit + control E2E); ④ top-2 `<documents>` retrieval untouched — ✓ (`app/rag/retriever.py` unmodified vs HEAD); ⑤ suite green, >90% coverage, ruff+pyright clean — ✓; ⑥ no completed-phase behavior change — ✓ (all gates green; commit left to harness per pass rules). - No defects found; no changes made this pass. Next pending phase: none in `todo/` (96 is the next free number).
This commit is contained in:
@@ -259,6 +259,34 @@ Implements just enough of the aipi surface:
|
||||
the ``<tools>`` section, so deflected turns never hit it); no
|
||||
existing E2E question or fixture file contains the phrase, so
|
||||
every other suite is unaffected.
|
||||
- user message containing ``read the capped document``
|
||||
(``READ_CAP_TRIGGER``, phase 95 task 03 — the read cap's dedicated
|
||||
story suite ``tests/e2e/test_read_truncation_cap.py``) **and** the
|
||||
system prompt carries the ``<tools>`` section -> the deterministic
|
||||
SCRIPTED CAPPED-READ flow: the question carries its own tool call
|
||||
after the colon — ``read the capped document: read source/path`` —
|
||||
parsed by ``_READ_CAP_CALL_RE`` from the RAW user message (the
|
||||
target keeps its case), then discriminated statelessly from the
|
||||
tool results (streaming only):
|
||||
* request 1 (``tools`` offered, no ``tool``-role result in the
|
||||
messages yet): the scripted call — ``read`` with the parsed
|
||||
target (synthetic id ``call_0``);
|
||||
* a ``tool``-role result is in the messages: the deterministic
|
||||
ECHO — the answer carries the LAST tool result VERBATIM
|
||||
(``Here's what the read returned:\n<result>``): a read result
|
||||
(``"Document <source/path>:…``) lands in the answer with its
|
||||
FULL content — the first cap chars + ``[…truncated…]`` + the
|
||||
pinned grep-pointer notice when the cap fired, the plain body
|
||||
byte-identical to the pre-phase-95 shape when it did not — and
|
||||
a refusal (the premise broke) lands just as visibly, so the
|
||||
suite fails loudly on it. The mock is the only E2E lens on the
|
||||
LLM's context, so the echo is the assertion surface for both
|
||||
the marker's presence AND its absence.
|
||||
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
|
||||
phrases — the phase-71/72/94 ordering convention; the trigger
|
||||
needs the ``<tools>`` section, so deflected turns never hit it);
|
||||
verified 2026-09-10: no existing E2E question or fixture file
|
||||
contains the phrase, so every other suite is unaffected.
|
||||
- user message containing ``what are the correct llama.cpp
|
||||
arguments`` (``GREP_TEACH_TRIGGER``, the 2026-09-05 incident —
|
||||
the harness prior is that grep takes a REGEX; this app's grep is a
|
||||
@@ -1293,6 +1321,79 @@ def _drill_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
return ("echo", last)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 95 (task 03, the read cap's dedicated story suite):
|
||||
# the deterministic SCRIPTED capped read — see the module docstring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A user message containing this substring (case-insensitive) —
|
||||
#: combined with the ``<tools>`` section in the system prompt — drives
|
||||
#: the scripted CAPPED-READ flow (the read cap's story suite,
|
||||
#: ``tests/e2e/test_read_truncation_cap.py``): the question carries its
|
||||
#: own tool call after the colon — ``read the capped document: read
|
||||
#: source/path`` — the mock emits the scripted ``read``, then ECHOES
|
||||
#: the ENTIRE tool result into its answer (the house scripted-turn lens
|
||||
#: on the LLM's context: the truncated shape — first cap chars +
|
||||
#: ``[…truncated…]`` + the pinned grep-pointer notice — or the plain
|
||||
#: shape, byte-identical to the pre-phase-95 result, lands in the
|
||||
#: rendered answer, and the suite asserts both directions through it).
|
||||
#: Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
|
||||
#: phrases — the phase-71/72/94 ordering convention); verified
|
||||
#: 2026-09-10: no existing E2E question or fixture file contains the
|
||||
#: phrase, so every other suite is unaffected.
|
||||
READ_CAP_TRIGGER = "read the capped document"
|
||||
|
||||
#: The scripted call in the read-cap question (case-insensitive — the
|
||||
#: suite's questions capitalize the trigger's first letter): the verb
|
||||
#: (``read``) plus the target — a combined ``source/path``, parsed from
|
||||
#: the RAW user message so the target keeps its case. The target is a
|
||||
#: ``[a-z0-9_./-]`` run (case-insensitively), so the suite's `` — ``
|
||||
#: flavor separator (em dash) can never bleed into it (the drill-down
|
||||
#: convention, ``_DRILL_CALL_RE``).
|
||||
_READ_CAP_CALL_RE = re.compile(
|
||||
r"read the capped document:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
|
||||
re.I,
|
||||
)
|
||||
|
||||
|
||||
def _read_cap_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
"""Classify a phase-95 scripted read-cap request (see the module
|
||||
docstring). The question carries the scripted call (``read the
|
||||
capped document: read source/path``); the step is then discriminated
|
||||
statelessly from the tool results, like the other marker flows:
|
||||
|
||||
* ``("call", target, "call_0")`` — ``tools`` are offered and no
|
||||
``tool``-role result is in the messages yet: the scripted
|
||||
``read`` on the parsed target (synthetic id ``call_0``).
|
||||
* ``("echo", result)`` — a ``tool``-role result is in the messages:
|
||||
the deterministic ECHO — the answer carries the LAST tool result
|
||||
VERBATIM (``Here's what the read returned:\n<result>``): a read
|
||||
result (``"Document <source/path>:…``) lands in the answer with
|
||||
its full content — the truncation marker + the pinned notice when
|
||||
the cap fired, the plain body when it did not — and a refusal
|
||||
(the premise broke) lands just as visibly, so the suite fails
|
||||
loudly on it.
|
||||
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
|
||||
section is missing (deflected turns never carry it), the scripted
|
||||
call is unparseable, or ``tools`` are not offered and no tool
|
||||
results are in the messages yet (e.g. ``agent_max_rounds=0``).
|
||||
"""
|
||||
user = _user(body)
|
||||
if READ_CAP_TRIGGER not in user.lower():
|
||||
return None
|
||||
if "<tools>" not in _system(body):
|
||||
return None
|
||||
match = _READ_CAP_CALL_RE.search(user)
|
||||
if match is None:
|
||||
return None
|
||||
results = _tool_results(body)
|
||||
if not results:
|
||||
if not body.get("tools"):
|
||||
return None
|
||||
return ("call", match.group("arg"), "call_0")
|
||||
return ("echo", results[-1])
|
||||
|
||||
|
||||
def long_answer() -> str:
|
||||
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
|
||||
a unique final line that must survive the stream untruncated."""
|
||||
@@ -1985,6 +2086,34 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 95 (task 03): the deterministic SCRIPTED capped read
|
||||
# (the question carries its own call — ``read the capped
|
||||
# document: read source/path``): the scripted ``read``, then the
|
||||
# answer that ECHOES the whole tool result (the marker + notice
|
||||
# when the cap fired, the plain shape when it did not — the
|
||||
# suite's lens on the LLM's context). Checked BEFORE the plain
|
||||
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
|
||||
# phase-71/72/94 ordering convention; the trigger needs the
|
||||
# ``<tools>`` section, so deflected turns never hit it).
|
||||
read_cap = _read_cap_flow(body)
|
||||
if read_cap is not None:
|
||||
if read_cap[0] == "call":
|
||||
stream = _tool_call_stream(
|
||||
"read", {"path": read_cap[1]}, read_cap[2]
|
||||
)
|
||||
else: # "echo" — the last tool result verbatim (the lens)
|
||||
stream = _sse_stream(
|
||||
_apply_max_tokens(
|
||||
f"Here's what the read returned:\n{read_cap[1]}",
|
||||
body.get("max_tokens"),
|
||||
),
|
||||
0.0,
|
||||
)
|
||||
return StreamingResponse(
|
||||
stream,
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
flow = _tool_flow(body)
|
||||
if flow is not None:
|
||||
if flow[0] == "list":
|
||||
|
||||
Reference in New Issue
Block a user