phase: 131_read_line_ranges
Build and Push Containers / build-and-push-app (push) Successful in 2m30s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All 44 progress lines are pure dots, final line at 100%, exit 0. Everything is verified. Final report:

**Phase 131 — final verification pass (all 4 tasks already complete in working tree)**

- **Verified implemented:** `read(offset, limit)` schema + teaching descriptions; rewritten truncation notice; full 6-row dedupe matrix + range validation + `lines: a-b of N` header + slice cap in `_execute_tool`; wire-level `offset`/`limit` on `ChatToolEvent` (absent keys dropped → non-range frames byte-identical); mock-LLM `read the lines` flow; handbook repro (wording + line numbers derived from the real PDF at test time)
- **Tests:** `uv run pytest --no-cov` → 3097 passed, exit 0; `uv run pytest --cov=app` → **99% on app/** (>90%); `uv run pytest tests/e2e/test_read_line_ranges.py -v --no-cov` → 2/2 in isolation (47.6 s); `tests/integration/test_read_line_ranges_handbook.py` → 2/2 (3.5 s); 10-file targeted E2E batch pinning pre-phase surfaces (grep/ls/plain-read/ALREADY_IN_CONTEXT/truncation) → 31/31; `ruff check .` clean; `pyright` 0 errors
- **Completion criteria:** all 5 green — (1) exact range lines + `lines:` header, plain reads byte-identical (existing pins pass unchanged) ✅ (2) truncated-then-range succeeds end-to-end on the real handbook; other re-reads get teaching refusals ✅ (3) find-traps answered from real PDF text, no hardcoded wording ✅ (4) coverage + isolated E2E ✅ (5) no behavior change in completed phases ✅
- **Docs:** README + AGENTS reviewed in full. Fixed 3 inconsistencies: README missing `BOR_HISTORY_MAX_TURNS`/`BOR_HISTORY_MAX_CHARS` rows (pre-existing phase-74 gap — env-tunable, unit-pinned, undocumented); `.env.example` missing the same two vars (added); `.env.example` stale `BOR_READ_MAX_CHARS` comment (still said "grep-pointer notice (phase 95)" — updated to the phase-131 workflow notice). AGENTS.md verified consistent — no changes needed. All 67 `.env.example` vars documented, all defaults match `app/config.py`, pages/auth tables match live routes
- **Note:** PLAN.md §7.1 still states the retired 46rem chat column (superseded by phase 100, which AGENTS.md rule 5 records correctly) — PLAN.md is off-limits for this pass, flagged only
- **Next pending phase:** `132_image_upload_feedback`
This commit is contained in:
2026-09-28 11:49:55 -04:00
parent c88c65641a
commit 101156bada
39 changed files with 3521 additions and 108 deletions
+239 -2
View File
@@ -539,6 +539,50 @@ Implements just enough of the aipi surface:
question always gets the echo whatever the honesty gate says.
Verified 2026-09-27: no existing E2E question or fixture file
contains the phrase, so every other suite is unaffected.
- user message containing ``read the lines``
(``READ_LINES_TRIGGER``, phase 131 task 04 — the read
line-ranges' dedicated story suite
``tests/e2e/test_read_line_ranges.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
grep -> read(offset/limit) DANCE (the owner's stuck scenario,
TODO.md L7, end to end): classified statelessly by the tool
results in the messages (the house count pattern — streaming
only):
* request 1 (``tools`` offered, no tool results yet): stream
ONLY ``tool_calls`` deltas — ``grep`` with
``{"pattern": READ_LINES_PATTERN}`` (id ``call_0``), no
content, no reasoning;
* a ``tool``-role grep result with a ``source/path:line: text``
match line is in the messages (the FIRST match line's line
number is parsed): the RANGE READ — a ``tool_calls`` delta —
``read`` on the match line's combined ``source/path`` with
``offset`` = the parsed line number (an int) and ``limit`` =
``READ_LINES_LIMIT`` (30) (id ``call_1``) — the owner's exact
dance: the range the model names is the part it does NOT
have;
* a ``tool``-role read result (content starting with the
agent's ``"Document "`` prefix) is in the messages: the
forced answer, byte-stable: ``Found it at line N. <first 80
chars of the read result's content>`` — N = the offset the
model passed (parsed from the read call's arguments in the
assistant tool-call message — the agent loop echoes every
executed call back into the messages), the quote = the read
slice with the header lines (the phase-106 D5 ``date:`` line
+ the phase-131 ``lines: a-b of N`` extent) skipped, so the
quote is the pure range content (the phase-94 read-echo
convention — the RANGE reached the model's context);
* a tool result is in the messages that is neither a read
result nor a grep match (the grep found nothing — the
premise broke): the deterministic terminal answer ``No
matches — the knowledge base has no such text.`` (the
grep-teach ``nomatch`` convention — the suite fails loudly
on the missing marker).
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
trigger phrases — the phase-71/72/94/118 ordering convention;
the trigger needs the ``<tools>`` section, so deflected turns
never hit it); verified 2026-09-28: no existing E2E question or
fixture file contains the phrase, so every other suite is
unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
@@ -638,6 +682,7 @@ test_llm_retry.py``). The mock is single-conversation per e2e server, so
from __future__ import annotations
import hashlib
import json
import math
import os
import re
@@ -2020,6 +2065,144 @@ def _summary_seed_read_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
return ("echo", results[-1])
# ---------------------------------------------------------------------------
# Phase 131 (task 04, the read line-ranges' dedicated story suite):
# the deterministic grep -> read(offset/limit) dance — see the module
# docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt — drives
#: the deterministic RANGE-READ flow (the read line-ranges' story
#: suite, ``tests/e2e/test_read_line_ranges.py``): the owner's stuck
#: scenario (TODO.md L7) — the huge document: ``grep`` finds the
#: line, the ``read`` lands exactly that part with ``offset`` /
#: ``limit``, and the answer echoes the read slice. Checked BEFORE
#: the plain ``TOOLS_TRIGGER`` flow (disjoint trigger phrases — the
#: phase-71/72/94/118 ordering convention); verified 2026-09-28: no
#: existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected.
READ_LINES_TRIGGER = "read the lines"
#: The canned pattern the flow greps for (the suite's fixture document
#: carries exactly one line containing it — the flow's read target
#: line; the sentinel convention of ``SEARCH_PATTERN``).
READ_LINES_PATTERN = "LINE_MARKER"
#: The line count the flow's range read requests (a PART of the
#: document, not the whole — the owner's dance, TODO.md L7).
READ_LINES_LIMIT = 30
def _read_call_offset(body: dict[str, Any]) -> int:
"""The offset the model's ``read`` call passed (phase 131, task 04).
The agent loop echoes every executed call back into the messages
(the assistant ``tool_calls`` message, the arguments as a JSON
string — ``app/rag/agent.py``), so the range-read request
(request 3 of the flow) carries the call it was made from: this
parses the FIRST assistant ``read`` call's ``offset``. ``0`` when
absent or unparseable (the premise broke — the suite fails
loudly on ``line 0``).
"""
for m in _messages(body):
if m.get("role") != "assistant":
continue
# The agent loop's echoed call (the ``tool_calls`` LIST on the
# assistant message — ``app/rag/agent.py``; :func:`_messages`
# is typed for string content, so the list comes back as
# ``Any`` here — the house mock convention over the wire
# shape).
raw_calls: Any = m.get("tool_calls")
for call in (raw_calls if isinstance(raw_calls, list) else []):
if not isinstance(call, dict):
continue
fn = call.get("function") or {}
if not isinstance(fn, dict) or fn.get("name") != "read":
continue
try:
args = json.loads(fn.get("arguments") or "{}")
except json.JSONDecodeError:
continue
offset = args.get("offset") if isinstance(args, dict) else None
if isinstance(offset, int) and not isinstance(offset, bool):
return offset
return 0
def _read_lines_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-131 range-read request (see the module
docstring). Classified statelessly by the tool results in the
messages (the house count pattern), like the other marker flows:
* ``("answer", offset, quote)`` — a ``tool``-role read result
(content starting with the agent's ``"Document "`` prefix) is in
the messages: the forced answer, byte-stable — ``Found it at
line N. <quote>`` with N the offset the model passed (parsed
from the read call's arguments in the assistant tool-call
message — a string on this return for the homogeneous
``tuple[str, ...]`` shape; the dispatch formats it into the
answer, the display surface) and quote the read result's
content with the header lines (the phase-106 D5 ``date:`` line
+ the phase-131 ``lines: a-b of N`` extent) skipped — the pure
read slice, first 80 chars (the phase-94 read-echo convention:
the RANGE reached the model's context).
* ``("read", combined, line, "call_1")`` — a ``tool``-role grep
result carrying ``source/path:line: text`` match lines is in the
messages: the RANGE READ on the FIRST match line's combined
``source/path`` — ``offset`` = the parsed line number (the
grep's own string here — the dispatch parses the int for the
wire) and ``limit`` = :data:`READ_LINES_LIMIT`.
* ``("nomatch",)`` — a tool result is in the messages but it is
neither a read result nor a grep match (the grep found nothing
— the premise broke): the deterministic terminal answer (the
grep-teach ``nomatch`` line — the suite fails loudly on the
missing marker).
* ``("grep",)`` — ``tools`` are offered and no tool results are in
the messages yet: the first step — ``grep`` with
``{"pattern": READ_LINES_PATTERN}`` (id ``call_0``).
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
section is missing (deflected turns never carry it), or
``tools`` are not offered and no tool results are in the
messages yet (e.g. ``agent_max_rounds=0``).
"""
if READ_LINES_TRIGGER not in _user(body).lower():
return None
if "<tools>" not in _system(body):
return None
reads = _read_results(body)
if reads:
_sp, content = reads[-1]
# The read result's header lines sit between the ``Document
# …:`` header and the document text — the phase-106 D5
# ``date:`` line always, the phase-131 ``lines: a-b of N``
# extent on a range read (a plain read carries no extent
# line) — skipped (the drill-flow convention) so the quote is
# the pure read content (the suite's marker-line assertion).
if content.startswith("date: "):
content = content.partition("\n")[2]
if content.startswith("lines: "):
content = content.partition("\n")[2]
return ("answer", str(_read_call_offset(body)), content[:80])
for result in _tool_results(body):
if result.startswith(_READ_RESULT_PREFIX):
continue
for line in result.splitlines():
match = _SEARCH_LINE_RE.match(line)
if match:
return (
"read",
match.group("sp"),
match.group("line"),
"call_1",
)
if _tool_results(body):
return ("nomatch",)
if not body.get("tools"):
return None
return ("grep",)
def long_answer() -> str:
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
a unique final line that must survive the stream untruncated."""
@@ -2844,8 +3027,6 @@ def _sse_stream(
def json_dumps(obj: dict[str, Any]) -> str:
import json
return json.dumps(obj)
@@ -3186,6 +3367,62 @@ def chat_completions(body: dict[str, Any]) -> Any:
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 131 (task 04): the deterministic grep ->
# read(offset/limit) DANCE (the read line-ranges' story suite,
# tests/e2e/test_read_line_ranges.py — the owner's stuck
# scenario, TODO.md L7): grep finds the line, the range read
# lands the part the model does NOT have, the answer echoes
# the read slice. Checked BEFORE the plain TOOLS_TRIGGER flow
# (disjoint trigger phrases — the phase-71/72/94/118 ordering
# convention; the trigger needs the ``<tools>`` section, so
# deflected turns never hit it).
read_lines = _read_lines_flow(body)
if read_lines is not None:
if read_lines[0] == "grep":
stream = _tool_call_stream(
"grep", {"pattern": READ_LINES_PATTERN}, "call_0"
)
elif read_lines[0] == "read":
# The owner's dance, deterministic: ``offset`` = the
# grep-reported line number (parsed from the FIRST
# match line — the classifier keeps it as the grep's
# own string for the homogeneous return; the int
# here is the wire's), ``limit`` = READ_LINES_LIMIT
# (a part of the document, not the whole).
stream = _tool_call_stream(
"read",
{
"path": read_lines[1],
"offset": int(read_lines[2]),
"limit": READ_LINES_LIMIT,
},
read_lines[3],
)
elif read_lines[0] == "nomatch":
# The premise broke (the grep found nothing) — the
# deterministic terminal answer (the grep-teach
# ``nomatch`` convention — the suite fails loudly on
# the missing marker).
stream = _sse_stream(
_apply_max_tokens(
"No matches — the knowledge base has no such text.",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — the forced read-echo (phase-94 shape)
stream = _sse_stream(
_apply_max_tokens(
f"Found it at line {read_lines[1]}. {read_lines[2]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 124 (task 07, image chips and answers, TODO L3): the
# scripted SHOW_IMAGE flow — the mock shows the seeded image
# document (the ``<documents>`` block with the image-
+655
View File
@@ -0,0 +1,655 @@
"""Phase 131 task 04 E2E (Playwright, mock-only): the read line-ranges'
dedicated story suite — the owner's stuck scenario, end to end.
The whole TODO.md L7 item (``131_read_line_ranges`` task 04 is the
story's gate): with the owner's ``BOR_READ_MAX_CHARS=24000`` in force,
the agent deterministically performs the grep → read(offset/limit)
dance on a ~100 KB document — the grep finds the marker line and
reports its line number, the range read lands exactly that part (the
part the model did NOT have), the SSE ``tool`` frames carry the REAL
``offset``/``limit`` ints, and the final answer is grounded in the
document's tail (the marker line). The second turn pins the other half
of the fix: a plain read of the same large document truncates at
24000, and the surfaced truncation carries the REWRITTEN notice that
teaches the grep → read(offset/limit) workflow (the pre-phase dead-end
wording is gone).
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_read_line_ranges.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real
``turbo`` does whatever it does with the tools, while this story's
gate is the deterministic SCRIPTED flows in
``tests/e2e/mock_llm.py`` (the mock is the only E2E lens on the
LLM's context):
* the range turn — ``READ_LINES_TRIGGER``: a user message containing
``read the lines`` **and** the system prompt carrying the HIGH
prompt's ``<tools>`` section. Request 1 greps for
``READ_LINES_PATTERN``; request 2 (a ``tool``-role grep result with
a ``source/path:line:`` match line) reads the match line's document
with ``offset`` = the parsed line number and ``limit`` =
``READ_LINES_LIMIT`` (30); request 3 (the read result present)
answers ``Found it at line N. <first 80 chars of the read slice>``
— N the offset the model passed (the phase-94 read-echo
convention: the RANGE reached the context).
* the plain-read turn — the mock's EXISTING ``READ_CAP_TRIGGER`` flow
(``read the capped document: read source/path``, the
``test_read_truncation_cap.py`` convention) reused on THIS suite's
large document: the mock ECHOES the ENTIRE read tool result into
its final answer (the house scripted-turn way of asserting on tool
results), so the truncated shape — first cap chars +
``[…truncated…]`` + the REWRITTEN notice — lands on the rendered
answer and the suite asserts the notice's presence AND the old
dead-end's absence through it.
The app under test boots with the OWNER'S cap (the
env-override-for-the-app-under-test pattern — this suite's module app,
as in ``test_read_truncation_cap.py``): ``BOR_READ_MAX_CHARS=24000``
(the owner's own setting, TODO.md L7) — small enough that the ~99 KB
fixture document's plain read truncates deterministically at 24000,
big enough that the 30-line range slice (~1 300 chars) never does.
KB fixture — ONE large deterministic markdown document
(``linedoc/big.md``, ~99 KB): this module generates it — 2 000
numbered filler lines (``Line <i>: …`` — the line numbers are exact)
under 200 ``##`` headings (one heading per 10 lines), with the marker
line ``LINE_MARKER the trap components are documented here`` at the
1 500th numbered line (= line 1 652 of the document — a known line
near ~1500, FAR beyond the 24 000-char cap: the document's first
24 000 chars hold NO marker). The marker's true line is computed from
the very string written to the fixture (``BIG_DOC.splitlines().index(
…) + 1``) — the assertions use that value, never a literal, so the
suite stays true if the fixture is regenerated.
The app under test is mock-calibrated (conftest pattern):
``BOR_RELEVANCE_THRESHOLD=0.30`` + ``BOR_LEXICAL_SUPPORT_FLOOR=0.15``
+ ``BOR_SOURCE_USEFULNESS_FLOOR=0.15`` — both questions' best-chunk
cosine (~0.23 on the mock's token-overlap distribution, the marker
chunk wins) clears the floor with FTS hits (``trap`` / ``components``
in the marker chunk), so every scripted turn stays GROUNDED (the
``<tools>`` section is present), and the read document clears the
usefulness bar into ``done.sources``.
Test → observable mapping (Playwright Mapping Rule; the story is the
owner's TODO item — one Playwright file per story, A16):
1. ``test_range_read_flow_over_sse`` — a question containing
``read the lines`` → the SSE ``tool`` frames, IN ORDER: the
``grep`` call (arg ``pattern="LINE_MARKER"``), then the ``read``
call (asserted on the FRAME's args, not a substring: ``path`` =
the doc's combined ``source/path``, ``offset`` an int == the
marker's true line, ``limit`` == 30 — the wire's assertion
surface for the owner's dance); the ``done`` frame is
non-deflected with the read document cited (phase 119 A1); the
final answer is the GROUNDED echo — ``Found it at line <N>``
(N == the marker's true line) + the marker text itself (the range
reached the model's context); NO truncation-notice dead-end
(neither the ``[…truncated…]`` marker nor the notice is in the
answer — the range read did not truncate, so no ``tool_result``
frame either).
2. ``test_plain_read_truncation_notice_teaches_the_workflow`` — a
question carrying the mock's existing plain-read trigger (``read
the capped document: read linedoc/big.md`` — the
``READ_CAP_TRIGGER`` flow on this suite's large document) → the
plain read truncates at 24 000: the SSE ``tool_result`` frame
(``truncated: true``, ``chars_shown: 24000``, ``chars_total:`` the
document's true length) lands after the ``tool`` frame and before
the answer's first delta (the pinned order); the surfaced
truncation — the mock's echo of the read result, the LLM-side
surface the notice rides on — carries the REWRITTEN notice
naming ``offset`` and ``limit`` (the pinned substrings — the
pre-phase-131 dead-end wording "to locate what you need — grep
searches the whole document" is gone).
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Locator, Page, expect
from sqlalchemy import select, text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document
from app.rag.agent import READ_TRUNCATION_NOTICE
from app.rag.retriever import TRUNCATION_MARKER
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
from e2e.mock_llm import READ_LINES_LIMIT, READ_LINES_PATTERN
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): the conftest session app owns
# its port in a combined run — this module app binds its own port
# instead (a same-port second uvicorn dies on bind and would drive
# the wrong server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_READLINES", "8148"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The owner's read cap (TODO.md L7 — the env under which the owner
#: hit the stuck scenario): the app under test boots with it.
READ_CAP = 24000
SOURCE = "linedoc" # the local directory's basename = the source name
BIG_REL = "big.md"
BIG_SP = f"{SOURCE}/{BIG_REL}" # the combined read identity
#: The marker line the flow greps for: the mock's canned pattern
#: (``READ_LINES_PATTERN`` — imported so the mock can never drift
#: from the fixture) + the owner's question wording. The fixture
#: document carries exactly one line containing it — the range
#: read's target; the echo assertion asserts on this very string.
MARKER_LINE_TEXT = f"{READ_LINES_PATTERN} the trap components are documented here"
# The fixture's layout (deterministic — see the module docstring):
# 2 000 numbered lines under 200 headings (10 lines each), the
# marker replacing the 1 500th numbered line.
NUMBERED_LINES = 2000
LINES_PER_BLOCK = 10
MARKER_NUMBERED = 1500
_FILLER_WORDS = [
"quiet", "mirror", "vault", "raid", "pool", "chain", "slot", "cycle",
"guard", "probe", "audit", "manifest", "retention", "rotation",
"label", "window", "image", "snapshot", "archive", "copy", "verify",
"steady", "order", "exact", "clean", "whole", "night", "weekly",
"plan", "track", "mark", "note", "file", "page", "entry", "index",
"column", "row", "cell",
]
def _big_doc() -> str:
"""The ~99 KB fixture document (deterministic — module docstring).
The H1, then 200 ``##`` headings (one per 10 numbered lines), each
followed by its 10 numbered lines (``Line <i>: …``) — the marker
line REPLACES the ``MARKER_NUMBERED``-th numbered line. The line
numbers are exact: the marker's true line is computed from the
returned string (``splitlines().index(…) + 1``), never a literal.
"""
lines: list[str] = ["# Lined document", ""]
numbered = 0
wi = 0
for block in range(1, NUMBERED_LINES // LINES_PER_BLOCK + 1):
lines.append(f"## Lines of the big lined read {block:04d}")
for _ in range(LINES_PER_BLOCK):
numbered += 1
if numbered == MARKER_NUMBERED:
lines.append(MARKER_LINE_TEXT)
else:
words = _FILLER_WORDS
lines.append(
f"Line {numbered}: "
+ " ".join(words[wi % len(words) : wi % len(words) + 6])
)
wi += 6
return "\n".join(lines)
BIG_DOC = _big_doc()
BIG_DOC_TOTAL = len(BIG_DOC)
#: The marker's true (1-based) line — computed from the very string
#: written to the fixture (the determinism pin: the assertions use
#: this, never a literal).
MARKER_LINE_NO = BIG_DOC.splitlines().index(MARKER_LINE_TEXT) + 1
# The suite's length contracts (the task's pinned-count assumptions —
# the assertions use ``len(BIG_DOC)`` of the very string written to
# the fixture, and ``synced_kb`` pins the stored content
# byte-identical to it): the document is OVER the owner's cap (the
# plain read truncates), the marker is FAR past the cap (the first
# cap chars hold NO marker — the stuck premise), the marker is unique
# (the grep has exactly one match line), and the range slice is UNDER
# the cap (the range read does not truncate — no ``tool_result``
# frame on the range turn).
assert BIG_DOC_TOTAL > READ_CAP, BIG_DOC_TOTAL
assert MARKER_LINE_TEXT not in BIG_DOC[:READ_CAP], (
f"the first {READ_CAP} chars must hold no marker"
)
assert BIG_DOC.count(MARKER_LINE_TEXT) == 1
_slice = "\n".join(
BIG_DOC.splitlines()[MARKER_LINE_NO - 1 : MARKER_LINE_NO - 1 + READ_LINES_LIMIT]
)
assert len(_slice) < READ_CAP, len(_slice)
# The REWRITTEN notice for this fixture's counts (the app.rag.agent
# constant). The bubble renders the raw LLM-side text through
# renderMarkdown (escape-first, markdown.js) — the notice's inline-
# code backticks (`` `offset` `` / `` `limit` ``) become <code>
# elements, so the DOM text_content() carries the notice with the
# backticks stripped (the test_read_truncation_cap convention).
NOTICE = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=BIG_DOC_TOTAL)
NOTICE_RENDERED = NOTICE.replace("`", "")
# The pinned substrings (task 04): the notice names BOTH range
# arguments — the workflow teaching.
assert "offset" in NOTICE and "limit" in NOTICE
# The pre-phase-131 dead-end wording (must be GONE from the
# surfaced truncation).
OLD_DEAD_END_1 = "to locate what you need"
OLD_DEAD_END_2 = "grep searches the whole document"
# The scripted turns. Q_RANGE carries the mock's ``READ_LINES_TRIGGER``
# (the range dance); Q_PLAIN carries the mock's existing
# ``READ_CAP_TRIGGER`` (``read the capped document: read
# source/path`` — the plain-read flow's scripted call, the
# test_read_truncation_cap convention) on this suite's large
# document. The shared ``trap components`` words keep both turns
# GROUNDED on the single-document KB (the mock-calibrated gate — see
# the module docstring).
Q_RANGE = f"Read the lines of {BIG_SP} — what are the trap components?"
Q_PLAIN = f"Read the capped document: read {BIG_SP} — the trap components, plain read check"
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def linedoc_dirs(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The story's local-directory source: one host temp dir (the app
server runs on the same host, so the path is visible to it)
holding the ONE generated large document. The directory's
basename is the source name (``kind=local``, phase 38)."""
root = tmp_path_factory.mktemp("bor_read_lines") / SOURCE
root.mkdir()
(root / BIG_REL).write_text(BIG_DOC, encoding="utf-8")
assert (root / BIG_REL).read_text(encoding="utf-8") == BIG_DOC
assert not (root / ".git").exists() # the local-kind story: NOT git
return root
@pytest.fixture(scope="module")
def app_server(mock_llm: int, linedoc_dirs: Path) -> Iterator[str]:
"""The real app under test — per-module env (the conftest
pattern, cf. ``test_read_truncation_cap.py``): NO
``BOR_GIT_SOURCES`` (the env fallback is git-only — the source
here is a DB-registered local directory), the mock LLM, the
mock-calibrated gate (threshold 0.30 + lexical floor 0.15 — the
conftest pair; the code-default floor 0.35 would refuse boot
against the 0.30 threshold, phase 112) — and the SUBJECT: the
owner's cap ``BOR_READ_MAX_CHARS=24000`` (TODO.md L7). The
session app is never started in this isolated run, so no port
clash."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern): both scripted turns
# stay GROUNDED (the questions' best-chunk cosine ~0.23 clears
# the 0.15 floor with FTS hits — the marker chunk wins).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
# The read document must clear the citation bar into
# done.sources (phase 119 A1 — the range turn asserts it).
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget (the
# conftest leak-guard pattern).
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env file carries the owner's settings (the app reads
# it from cwd) — override the source list with an EMPTY value (the
# env var beats the .env file): the registry must hold EXACTLY the
# local directory this suite registers (a leftover env git list
# would pollute the KB the scripted turns run against).
env["BOR_GIT_SOURCES"] = ""
# Leak guards (conftest pattern): an operator's local (gitignored)
# .env cannot leak corpus-specific settings into the app under
# test.
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
# THE SUBJECT: the owner's cap (TODO.md L7 — the env under which
# the owner hit the stuck scenario; the production default of
# 128 000 stays pinned in tests/unit/test_config.py).
env["BOR_READ_MAX_CHARS"] = str(READ_CAP)
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E
suites share one Postgres, so a leftover git_sources row or
document would pollute the retrieval the scripted turns run
against (the single-document KB's gate margins are pinned
against EXACTLY this one document)."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the (cookie-authenticated) status endpoint until the run
reaches a terminal state (the test_ls_tree_drilldown pattern, over
plain httpx — this fixture has no browser page yet)."""
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
return body
time.sleep(0.5)
raise AssertionError(f"sync did not reach a terminal state: {body}")
@pytest.fixture(scope="module")
def synced_kb(app_server: str, linedoc_dirs: Path) -> None:
"""The story's precondition: the one-source KB synced under the
deterministic mock. Registers the temp directory through the
authenticated API (the ``test_local_directory_sources.py``
pattern), runs the REAL in-process sync (``POST /api/sync`` —
walk → chunk → embed → overview → version bump), pins the stored
content byte-identical to the fixture string (the ``chars_total``
every assertion below uses is ``len(BIG_DOC)`` of the very string
written to the directory)."""
_truncate_all()
with httpx.Client(base_url=app_server, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
r = client.post(
"/api/git-sources", json={"kind": "local", "path": str(linedoc_dirs)}
)
assert r.status_code == 201, r.text
r = client.post("/api/sync")
assert r.status_code == 202, r.text
body = _wait_sync_done_http(client)
assert body["state"] == "success", body
detail = body["detail"]
assert detail["added"] == 1, detail
assert detail["pruned"] == 0, detail
# The import stored the fixture string BYTE-IDENTICALLY — the
# ``chars_total`` every assertion below uses is ``len()`` of the
# very string written to the directory.
with SessionLocal() as db:
stored = db.scalar(
select(Document.content).where(
Document.source == SOURCE, Document.path == BIG_REL
)
)
assert stored == BIG_DOC, "stored content drifted from the fixture"
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped —
the scripted turns never change it, so the registry persists
across the tests of this module)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# Page helpers (the test_read_truncation_cap / test_ls_tree_drilldown
# house patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
#: (a response clone read in the background) — wire-level assertions
#: for the ``tool`` / ``tool_result`` frames, independent of the UI
#: rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page, deadline_s: float = 120.0) -> list[dict]:
"""The SSE frames captured since the last submit (``_submit``
clears the buffer), once the hook's background read settles. The
default deadline is generous: the plain-read turn's echo streams
the whole 24 000-char truncated result (the mock's 12-char /
0.02 s pacing ≈ 40 s)."""
deadline = time.monotonic() + deadline_s
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _tool_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool"]
def _result_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool_result"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page, timeout_ms: int = 120_000) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (the phase-48 settle wait; the timeout is a parameter —
the plain-read turn's 24 000-char echo streams for ~40 s)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text(
"", timeout=timeout_ms
)
expect(page.locator("#send-btn")).to_be_enabled(timeout=timeout_ms)
expect(page.locator("#send-label")).to_have_text("Send", timeout=timeout_ms)
def _last_brain(page: Page) -> Locator:
return page.locator(".msg.brain").last
# --------------------------------------------------------------------------
# 1. The range turn: the owner's grep → read(offset/limit) dance on
# the wire, grounded in the large doc's tail
# --------------------------------------------------------------------------
def test_range_read_flow_over_sse(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_RANGE)
_wait_settled(page)
frames = _frames(page)
# The owner's dance on the wire — IN ORDER, asserted on the
# frames' ARGS (not a substring): the grep (the canned pattern),
# then the range read — path = the doc's combined identity,
# offset = the marker's TRUE line (the mock parsed it from the
# grep result's first match line — the mock does the owner's
# exact dance), limit = READ_LINES_LIMIT. The range fields ride
# the SAME frame (the phase-131 wire surface — absent keys on
# every non-range call).
assert _tool_frames(frames) == [
{"type": "tool", "name": "grep", "argument": READ_LINES_PATTERN},
{
"type": "tool",
"name": "read",
"argument": BIG_SP,
"offset": MARKER_LINE_NO,
"limit": READ_LINES_LIMIT,
},
], _tool_frames(frames)
# The range read did NOT truncate (30 lines ≈ 1 300 chars << the
# 24 000 cap) — it streams NO tool_result frame (one frame = one
# noteworthy event, the phase-95 additive contract).
assert _result_frames(frames) == [], _result_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source (phase 119 A1 —
# done.sources is the READ docs).
assert any(
s["path"] == BIG_REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
# The GROUNDED echo — the range reached the model's context and
# landed in the answer: ``Found it at line <N>`` (N == the
# marker's true line) + the marker text itself (the answer is
# grounded in the large doc's TAIL — far past the 24 000-char
# cap a plain read could reach)…
text = _last_brain(page).locator(".bubble").text_content() or ""
assert f"Found it at line {MARKER_LINE_NO}." in text, text
assert MARKER_LINE_TEXT in text, text
# …and NO truncation-notice dead-end (the pre-phase stuck shape —
# "the rest is NOT shown, use grep" — is not the answer: the
# range read landed).
assert TRUNCATION_MARKER not in text, text
assert "TRUNCATED —" not in text, text
# --------------------------------------------------------------------------
# 2. The plain-read turn: the truncation still fires at the owner's
# cap, and the surfaced notice teaches the workflow (the dead-end
# is gone)
# --------------------------------------------------------------------------
def test_plain_read_truncation_notice_teaches_the_workflow(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(150_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_PLAIN)
_wait_settled(page)
frames = _frames(page)
# The plain read executed — the tool frame carries NO range keys
# (a plain read: the phase-131 additive fields are ABSENT, never
# null — the pre-phase frame shape, byte-identical).
assert _tool_frames(frames) == [
{"type": "tool", "name": "read", "argument": BIG_SP}
], _tool_frames(frames)
# EXACTLY ONE tool_result frame — the pinned counts (the
# owner's cap: showing the first READ_CAP of the fixture's true
# length).
assert _result_frames(frames) == [
{
"type": "tool_result",
"name": "read",
"argument": BIG_SP,
"truncated": True,
"chars_shown": READ_CAP,
"chars_total": BIG_DOC_TOTAL,
}
], _result_frames(frames)
# Frame order: the marker lands AFTER the call is shown and
# BEFORE the answer's first delta (the phase-37/48 tool-line
# timing is untouched — the beat between the two is the
# truncation).
i_tool = next(i for i, f in enumerate(frames) if f.get("type") == "tool")
i_result = next(
i for i, f in enumerate(frames) if f.get("type") == "tool_result"
)
i_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
assert i_tool < i_result < i_delta, (i_tool, i_result, i_delta)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The surfaced truncation — the mock's echo of the read result
# (the LLM-side surface the notice rides on — the house
# scripted-turn lens): the REWRITTEN notice landed in the model's
# context and in the rendered answer, naming BOTH range
# arguments (the workflow teaching: grep reports line numbers,
# then read with `offset` + `limit`)…
text = _last_brain(page).locator(".bubble").text_content() or ""
assert TRUNCATION_MARKER in text, text
assert NOTICE_RENDERED in text, text
# …and the pre-phase-131 dead-end wording is GONE.
assert OLD_DEAD_END_1 not in text, text
assert OLD_DEAD_END_2 not in text, text
+8 -2
View File
@@ -222,6 +222,12 @@ assert SHORT_DOC.count(SHORT_SENTINEL) == 1
# app.rag.agent constant, formatted for this fixture's counts).
MARKER = f" (truncated — showing {READ_CAP} of {CAP_DOC_TOTAL} chars)"
NOTICE = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=CAP_DOC_TOTAL)
# The bubble renders the raw LLM-side text through renderMarkdown
# (escape-first, markdown.js) — the phase-131 notice's inline-code
# backticks (`` `offset` `` / `` `limit` ``) become <code> elements, so
# the DOM text_content() carries the notice with the backticks
# stripped.
NOTICE_RENDERED = NOTICE.replace("`", "")
# The scripted turns (the mock's ``READ_CAP_TRIGGER`` questions — each
# carries its own tool call after the colon; the turn-specific flavor
@@ -679,7 +685,7 @@ def test_truncated_read_frame_order_live_marker_and_llm_notice(
bubble = _last_brain(page).locator(".bubble")
text = bubble.text_content() or ""
assert TRUNCATION_MARKER in text, text
assert NOTICE in text, text
assert NOTICE_RENDERED in text, text
# The first cap chars reached the model…
assert HEAD_SENTINEL in text, text
# …and the past-the-cap tail did NOT (the cut is real, not cosmetic).
@@ -761,7 +767,7 @@ def test_truncated_read_save_and_shared_fidelity(
# told about (same stored text, same renderer).
anon_text = anon.locator(".msg.brain .bubble").first.text_content() or ""
assert TRUNCATION_MARKER in anon_text, anon_text
assert NOTICE in anon_text, anon_text
assert NOTICE_RENDERED in anon_text, anon_text
assert HEAD_SENTINEL in anon_text, anon_text
assert TAIL_SENTINEL not in anon_text, anon_text
anon_ctx.close()
+4 -2
View File
@@ -197,8 +197,10 @@ def test_agent_tools_offers_the_harness_aligned_surface() -> None:
assert set(ls["properties"]) == {"path"}
read = by_name["read"]["function"]["parameters"]
assert read["type"] == "object"
assert read["required"] == ["path"]
assert set(read["properties"]) == {"path"}
assert read["required"] == ["path"] # offset/limit optional (131)
assert set(read["properties"]) == {"path", "offset", "limit"}
assert read["properties"]["offset"]["type"] == "integer"
assert read["properties"]["limit"]["type"] == "integer"
grep = by_name["grep"]["function"]["parameters"]
assert grep["type"] == "object"
assert grep["required"] == ["pattern"]
@@ -0,0 +1,495 @@
"""Integration: phase 131 (task 03) — the owner's handbook repro.
The exact incident from ``TODO.md`` L7: the agent was asked "What are
the components of the find traps spell?" over the real D&D player's
handbook (``test_sources/players_handbook.pdf`` — the untracked fixture
from ``pdf_tests.tar``), it ``grep``'d the text and FOUND it "including
line number", and then dead-ended: "However, read takes a path, not a
line number." This suite proves the fixed loop closes the dance on the
REAL document: the handbook is indexed through the REAL importer
(:func:`app.rag.importer.import_sources` — the deterministic in-process
:class:`~tests.fakes.FakeEmbedder` stands in for the embed/summary
calls, so the ~1200-chunk index build stays fast and offline), and a
scripted fake LLM (the ``tests/unit/test_agent.py`` round pattern)
drives the turn with the owner's ``BOR_READ_MAX_CHARS=24000`` in force:
* the **grep-first** repro — round 1 a ``grep`` (``pattern="find
traps"``, lowercase — grep is case-insensitive), round 2 a ``read``
with ``offset`` = the line the REAL grep result reported (parsed,
never hardcoded) + ``limit`` = 40, round 3 the answer: the read
result's content, verbatim (the phase-94 "the read document reached
the context" echo convention — the loop's reply IS the slice).
Exactly 3 rounds, ``tool_calls == 2`` (the owner's stuck loop made 2
and dead-ended — the fixed loop closes in 3).
* the **plain-read-first** variant — round 1 a plain ``read`` (the
handbook truncates at 24000 with the REWRITTEN phase-131 notice,
which names ``offset``/``limit`` — the dead-end is gone), round 2
the grep, round 3 the range read of the ALREADY-TRUNCATED document
(the phase-131 dedupe matrix's truncated-then-range row, end-to-end
on the real document — it PROCEEDS; the ``REREAD_TRUNCATED``
teaching is never needed because the model went straight to the
range), round 4 the echo answer.
The expected find-traps text is DERIVED from the fixture at test time
(extract the PDF's text with the phase-126
:func:`app.rag.pdf.extract_pdf_text`, grep it the same way the model
does — :func:`app.rag.agent.grep_document`'s case-insensitive fixed
substring — and take the passage's first 60 chars): no hardcoded
handbook wording, so the test stays true if the fixture is
re-extracted.
House KB pattern (``test_agent_tools.py`` / ``test_importer_e2e.py``):
the ``documents`` table is truncated around each test, so the whole-KB
``grep`` sees EXACTLY the handbook — the grep's line numbers are then
the extracted text's line numbers, one-to-one. The untracked fixture
is only ever COPIED into a temp source dir, never mutated.
Requires: podman compose up -d db
"""
from __future__ import annotations
import asyncio
import re
import shutil
from collections.abc import AsyncIterator, Iterator
from copy import deepcopy
from pathlib import Path
from typing import TYPE_CHECKING, Any, cast
import pytest
from sqlalchemy import delete, select, text
from sqlalchemy.orm import Session
from app.config import Settings
from app.db import SessionLocal
from app.models import Document, GitSource
from app.rag import agent
from app.rag.agent import (
AGENT_TOOLS,
READ_TRUNCATION_NOTICE,
REREAD_TRUNCATED,
AgentHolder,
run_agent,
)
from app.rag.importer import import_sources
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
)
from app.rag.pdf import extract_pdf_text
from app.rag.retriever import TRUNCATION_MARKER
from tests.fakes import FakeEmbedder
if TYPE_CHECKING:
from app.rag.scaffolding import ScaffoldingFilter
#: The repo root (this file lives in ``tests/integration/``).
REPO_ROOT = Path(__file__).resolve().parents[2]
#: The owner's real fixture (untracked — unpacked from
#: ``test_sources/pdf_tests.tar``): the D&D player's handbook.
HANDBOOK = REPO_ROOT / "test_sources" / "players_handbook.pdf"
#: The imported identity: the source dir is named ``test_sources``
#: (the importer's ``source = root.name``) and the PDF sits directly
#: inside it.
SOURCE = "test_sources"
PDF_NAME = "players_handbook.pdf"
IDENTITY = f"{SOURCE}/{PDF_NAME}"
#: The owner's exact question (``TODO.md`` L7) — the turn's user
#: message; the scripted loop answers it through the tools.
QUESTION = "What are the components of the find traps spell?"
#: The owner's env: ``BOR_READ_MAX_CHARS=24000`` (the handbook is far
#: longer, so a plain read of it truncates).
READ_CAP = 24_000
#: The owner's pattern — lowercase (grep is case-insensitive).
PATTERN = "find traps"
#: The scripted range read's line count (both variants).
RANGE_LIMIT = 40
pytestmark = pytest.mark.skipif(
not HANDBOOK.is_file(),
reason=(
"test_sources/players_handbook.pdf is missing — unpack "
"test_sources/pdf_tests.tar (the untracked owner fixture)"
),
)
def _settings() -> Settings:
"""The turn's settings: the owner's read cap in force and the code
default import extensions plus ``pdf`` (the owner's
``BOR_IMPORT_EXTENSIONS`` names it — the handbook is an indexed
source). ``_env_file=None`` keeps an operator's local ``.env`` out
of the pins."""
defaults = Settings.model_fields["import_extensions"].default
kwargs: dict[str, Any] = {
"_env_file": None,
"import_extensions": f"{defaults},pdf",
"read_max_chars": READ_CAP,
}
return Settings(**kwargs) # pyright: ignore[reportCallIssue]
def _expected_passage() -> str:
"""The first 60 chars of the handbook's find-traps passage, DERIVED
from the fixture at test time (the task's no-hardcoded-wording
rule): the PDF's extracted text (the same text the importer stored)
is grepped the same way the model does —
:func:`app.rag.agent.grep_document`, the tool's case-insensitive
fixed-substring semantics — and the passage is the
:data:`RANGE_LIMIT` lines from the FIRST grep match (the exact
window the scripted loop reads). If the fixture is re-extracted the
line numbers and wording are re-derived, so the assertion stays
true."""
content = extract_pdf_text(HANDBOOK)
first_line, _ = agent.grep_document(content, PATTERN)[0]
lines = content.splitlines()
return "\n".join(lines[first_line - 1 : first_line - 1 + RANGE_LIMIT])[:60]
def _first_grep_line(messages: list[dict[str, Any]], identity: str) -> int:
"""The first line number the loop's grep reported for *identity* —
parsed from the REAL grep result the loop produced (the task's
no-hardcoded-line rule). The last ``tool`` message carries the
grep result, one ``source/path:line: text`` line per match; the
first match line for the handbook names the line the scripted
range read takes."""
last_tool = next(m for m in reversed(messages) if m.get("role") == "tool")
content = last_tool["content"]
assert isinstance(content, str)
for line in content.splitlines():
if line.startswith(f"{identity}:"):
match = re.match(rf"^{re.escape(identity)}:(\d+):", line)
assert match is not None, f"unparseable grep line: {line!r}"
return int(match.group(1))
raise AssertionError(f"no grep result line for {identity!r}")
class HandbookAgentLLM:
"""The scripted owner-scenario loop (the ``test_agent.py`` round
pattern) — deterministic, and it parses the REAL line number from
the grep result the loop produced (never a hardcoded line):
``plain_first=False`` (the grep-first repro — test 1):
* round 1 → ``grep`` (``pattern="find traps"``);
* round 2 → ``read`` (``path`` = the handbook identity, ``offset``
= the first grep-reported line, ``limit`` = 40);
* round 3 → the answer: the read result's content, verbatim (the
phase-94 echo convention — the loop's reply IS the slice).
``plain_first=True`` (the plain-read-first variant — test 2):
* round 1 → plain ``read`` (``path``);
* round 2 → ``grep`` (``pattern="find traps"``);
* round 3 → ``read`` (``path``, ``offset`` = the first grep-
reported line, ``limit`` = 40) — the dedupe matrix's
truncated-then-range row on the real document;
* round 4 → the answer: the read result's content, verbatim.
Every ``chat_stream`` request's messages and tools are recorded
(``requests``); the scripted :class:`ToolCallPiece` s, in call
order, in ``calls``. The canned answer pieces are fed through the
round's :class:`ScaffoldingFilter` exactly like
``LLMClient.chat_stream`` (the ``test_agent.py`` ScriptedLLM
pattern)."""
def __init__(self, identity: str, *, plain_first: bool) -> None:
self.identity = identity
self.plain_first = plain_first
self.requests: list[
tuple[list[dict[str, Any]], list[dict[str, Any]] | None]
] = []
self.calls: list[ToolCallPiece] = []
def _next_call(self, index: int, messages: list[dict[str, Any]]) -> ToolCallPiece | None:
"""The scripted call for request *index* — ``None`` on the
final (answer) round. The range read's ``offset`` is parsed
from the grep result just delivered."""
if self.plain_first:
if index == 0:
return ToolCallPiece(id="call_1", name="read", arguments={"path": self.identity})
if index == 1:
return ToolCallPiece(id="call_2", name="grep", arguments={"pattern": PATTERN})
if index == 2:
return ToolCallPiece(
id="call_3",
name="read",
arguments={
"path": self.identity,
"offset": _first_grep_line(messages, self.identity),
"limit": RANGE_LIMIT,
},
)
else:
if index == 0:
return ToolCallPiece(id="call_1", name="grep", arguments={"pattern": PATTERN})
if index == 1:
return ToolCallPiece(
id="call_2",
name="read",
arguments={
"path": self.identity,
"offset": _first_grep_line(messages, self.identity),
"limit": RANGE_LIMIT,
},
)
return None
async def chat_stream(
self,
messages: list[dict[str, str]],
tools: list[dict[str, Any]] | None = None,
scaffolding: ScaffoldingFilter | None = None,
) -> AsyncIterator[StreamPiece | ToolCallPiece]:
self.requests.append((deepcopy(messages), deepcopy(tools)))
call = self._next_call(len(self.requests) - 1, messages)
if call is not None:
self.calls.append(call)
yield call
return
# The answer round: the last tool result (the range read's
# result) — the loop's reply IS the slice, verbatim.
last_tool = next(m for m in reversed(messages) if m.get("role") == "tool")
content = last_tool["content"]
assert isinstance(content, str)
if scaffolding is None:
yield StreamPiece("content", content)
return
cleaned = scaffolding.feed(content)
if cleaned:
yield StreamPiece("content", cleaned)
tail = scaffolding.flush()
if tail:
yield StreamPiece("content", tail)
async def _consume(
llm: HandbookAgentLLM,
holder: AgentHolder,
settings: Settings,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
"""Consume one scripted ``run_agent`` turn (the owner's question,
no seed docs). SEC-14-04: a short-lived session per tool call —
the same pattern as production and the phase-70 integration suite."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async for piece in run_agent(
cast("LLMClient", llm),
lambda: SessionLocal(),
system_prompt="SYSTEM_PROMPT",
user_message=QUESTION,
seed_docs=[],
settings=settings,
holder=holder,
):
out.append(piece)
return out
def _answer(pieces: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]) -> str:
"""The turn's final answer text (the content pieces)."""
return "".join(
piece.text for piece in pieces if isinstance(piece, StreamPiece) and piece.kind == "content"
)
@pytest.fixture()
def handbook_kb(db: Session, tmp_path: Path) -> Iterator[Document]:
"""The real handbook, indexed once per test through the REAL
importer: a temp source dir containing ``test_sources/
players_handbook.pdf`` (a COPY — the untracked fixture is never
mutated), registered as a local source, imported with the
deterministic in-process :class:`~tests.fakes.FakeEmbedder` (the
~1200-chunk embed is instant — the whole test stays well under the
task's ~60 s bar). Yields the indexed
:class:`~app.models.Document` row.
The house KB pattern: ``chunks``/``documents``/``folder_summaries``
are truncated around the test, so the whole-KB ``grep`` sees
EXACTLY the handbook (its line numbers are the extracted text's
line numbers, one-to-one)."""
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
db.commit()
root = tmp_path / SOURCE
root.mkdir()
shutil.copy2(HANDBOOK, root / PDF_NAME)
row = GitSource(url=str(root), kind="local", path=str(root))
db.add(row)
db.commit()
fake = FakeEmbedder(dim=768)
fake.settings = _settings() # the owner's cap + the pdf token
asyncio.run(import_sources([root], fake, session=db))
doc = db.scalar(
select(Document).where(Document.source == SOURCE, Document.path == PDF_NAME)
)
assert doc is not None, "the handbook did not import"
assert len(doc.content) > READ_CAP, "the handbook must outgrow the owner's read cap"
yield doc
db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
db.execute(delete(GitSource).where(GitSource.id == row.id))
db.commit()
db.expire_all()
def test_grep_then_range_read_answers_the_find_traps_question(
handbook_kb: Document,
) -> None:
"""The owner's stuck loop, fixed — grep → range read → answer:
exactly 3 rounds with ``tool_calls == 2`` (the owner's stuck loop
made 2 and dead-ended), the read call carries the grep-reported
``offset`` + ``limit`` (parsed, not literal), and the final answer
— the read result, verbatim — contains the handbook's REAL
find-traps text (derived from the fixture at test time)."""
doc = handbook_kb
expected = _expected_passage()
llm = HandbookAgentLLM(IDENTITY, plain_first=False)
holder = AgentHolder()
pieces = asyncio.run(_consume(llm, holder, _settings()))
# Exactly 3 rounds: grep → range read → answer (the fixed loop
# closes where the owner's stuck loop dead-ended).
assert len(llm.requests) == 3
assert [call.name for call in llm.calls] == ["grep", "read"]
# The tools stay offered on every request (phase 45 — no budgets).
assert llm.requests[0][1] == AGENT_TOOLS
assert llm.requests[2][1] == AGENT_TOOLS
# Round 1: the lowercase pattern (grep is case-insensitive).
assert llm.calls[0].arguments == {"pattern": PATTERN}
# The grep result the loop produced: the handbook's REAL match
# lines (``source/path:line: text``), the first naming the line
# the range read must take.
grep_result = llm.requests[1][0][-1]
assert grep_result["role"] == "tool"
assert isinstance(grep_result["content"], str)
assert grep_result["content"].startswith(f"{IDENTITY}:")
grep_line = _first_grep_line(llm.requests[1][0], IDENTITY)
# Round 2: the read call carried offset/limit — offset is the
# grep-reported line (parsed from the real grep result, never a
# literal), limit 40.
read_call = llm.calls[1]
assert read_call.arguments == {
"path": IDENTITY,
"offset": grep_line,
"limit": RANGE_LIMIT,
}
# The range read's result: the byte-identical header lines, the
# true extent on the third line, the slice below.
read_result = llm.requests[2][0][-1]
assert read_result["role"] == "tool"
assert isinstance(read_result["content"], str)
total_lines = len(doc.content.splitlines())
assert read_result["content"].startswith(f"Document {IDENTITY}:\n")
assert (
f"lines: {grep_line}-{grep_line + RANGE_LIMIT - 1} of {total_lines}\n"
in read_result["content"]
)
# Round 3: the answer IS the read result (the phase-94 echo
# convention) and it carries the REAL find-traps text.
answer = _answer(pieces)
assert answer == read_result["content"]
assert expected in answer
# Executed-call bookkeeping: both calls counted; the doc read into
# the context exactly once.
assert holder.tool_calls == 2
assert [d.id for d in holder.read_docs] == [doc.id]
assert holder.read_truncations == [] # the range slice fits the cap
def test_plain_read_truncates_then_range_read_answers_the_find_traps_question(
handbook_kb: Document,
) -> None:
"""The plain-read-first variant on the REAL document: round 1's
plain read truncates at the owner's 24000 cap with the REWRITTEN
phase-131 notice (it names ``offset``/``limit`` — the dead-end is
gone) and surfaces a truncation signal; round 2 greps; round 3's
range read of the ALREADY-TRUNCATED document PROCEEDS (the dedupe
matrix's truncated-then-range row, end-to-end — the
``REREAD_TRUNCATED`` teaching is never needed because the model
went straight to the range); round 4 answers with the slice."""
doc = handbook_kb
expected = _expected_passage()
llm = HandbookAgentLLM(IDENTITY, plain_first=True)
holder = AgentHolder()
pieces = asyncio.run(_consume(llm, holder, _settings()))
# Four rounds: plain read → grep → range read → answer.
assert len(llm.requests) == 4
assert [call.name for call in llm.calls] == ["read", "grep", "read"]
# Round 1: the PLAIN read (no range arguments at all).
assert llm.calls[0].arguments == {"path": IDENTITY}
# Round 1's result: cut at the owner's cap, the shared marker +
# the REWRITTEN notice (``{total}`` = the handbook's true length,
# ``{shown}`` = the cap) — the notice names the offset/limit
# workflow the loop then follows.
plain_result = llm.requests[1][0][-1]
assert plain_result["role"] == "tool"
assert isinstance(plain_result["content"], str)
notice = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=len(doc.content))
assert plain_result["content"].startswith(f"Document {IDENTITY}:\n")
assert plain_result["content"].endswith(f"\n{TRUNCATION_MARKER}\n{notice}")
# The truncation surfaced to the API layer as a ToolResultPiece
# (the SSE ``tool_result`` frame's wire value).
truncations = [p for p in pieces if isinstance(p, ToolResultPiece)]
assert [(p.argument, p.chars_shown, p.chars_total) for p in truncations] == [
(IDENTITY, READ_CAP, len(doc.content))
]
# Round 3: the range read of the truncated doc — offset is the
# grep-reported line (parsed), limit 40 — and the dedupe matrix
# lets it PROCEED (no ``REREAD_TRUNCATED`` teaching anywhere in
# the loop's tool results).
assert llm.calls[1].arguments == {"pattern": PATTERN}
grep_line = _first_grep_line(llm.requests[2][0], IDENTITY)
read_call = llm.calls[2]
assert read_call.arguments == {
"path": IDENTITY,
"offset": grep_line,
"limit": RANGE_LIMIT,
}
reread = REREAD_TRUNCATED.format(shown=READ_CAP, source=SOURCE, path=PDF_NAME)
tool_results = [
m["content"]
for messages, _tools in llm.requests
for m in messages
if m.get("role") == "tool" and isinstance(m["content"], str)
]
assert reread not in tool_results
# The range read's result: the true extent on the third line.
read_result = llm.requests[3][0][-1]
assert read_result["role"] == "tool"
assert isinstance(read_result["content"], str)
total_lines = len(doc.content.splitlines())
assert read_result["content"].startswith(f"Document {IDENTITY}:\n")
assert (
f"lines: {grep_line}-{grep_line + RANGE_LIMIT - 1} of {total_lines}\n"
in read_result["content"]
)
# Round 4: the answer IS the range read's result and it carries
# the REAL find-traps text.
answer = _answer(pieces)
assert answer == read_result["content"]
assert expected in answer
# All three calls executed (the truncated plain read, the grep,
# the range read); the doc is committed to the context exactly
# once; the holder's truncation record is the plain read's tuple.
assert holder.tool_calls == 3
assert [d.id for d in holder.read_docs] == [doc.id]
assert holder.read_truncations == [(IDENTITY, READ_CAP, len(doc.content))]
+755 -28
View File
@@ -17,7 +17,18 @@ unknown-source refusal that counts nothing, the NOT-A-FOLDER teaching
with the parent's subfolders, the 50-file cap + grep-pointer note),
``read`` on the canonical combined form (split at the
FIRST slash, full content, the bare-source-name refusal, the
already-in-context dedupe, missing-args refusals), the phase-68 ``grep``
already-in-context dedupe, missing-args refusals), the phase-131
(task 02) range-read execution (the optional ``offset``/``limit``:
:func:`agent._read_range_args` validation — whole ints ≥ 1, bools
excluded — with the pinned :data:`agent.BAD_RANGE_ARGS` echo, the
six-row dedupe matrix (a plain re-read of a TRUNCATED doc gets the
pinned :data:`agent.REREAD_TRUNCATED` teaching; the truncated-then-
range row PROCEEDS — the stuck-scenario fix —; every other re-read of
a fully-read doc stays ``ALREADY_IN_CONTEXT``),
:data:`agent.OFFSET_PAST_END` (the true line count named), the slice
math + the ``lines: a-b of N`` header shape (plain reads byte-
identical to pre-phase-131), the cap on the slice with the slice-
length tuple ``chars_total``), the phase-68 ``grep``
contract under its new name (the locked A5 pins: fixed substring,
case-insensitive, 20-cap in catalog order, 200-char lines, locator-only
— ``read_docs`` untouched, no-match lines counted), the round cap
@@ -227,9 +238,12 @@ def test_agent_tools_names_and_parameters() -> None:
# the prompt yet); read adds one of them (or any other document)
# to the context; only an already-read document is refused. The
# combined-identity handoff to the `ls` output / the <documents>
# blocks stays; the truncation-notice paragraph (phase 95) and
# the one-call-at-a-time discipline clause (task 05) survive the
# rewrite byte-identical.
# blocks stays; the one-call-at-a-time discipline clause (task
# 05) survives byte-identical. Phase 131 (task 01): the old
# truncation dead-end paragraph ("Follow it and use `grep`
# (pattern) to locate the rest") is replaced by the working
# workflow — grep reports line numbers, then read the part with
# `offset` (1-based first line) + `limit` (number of lines).
assert read["description"] == (
"The <documents> section shows the SUMMARIES of the "
"top-ranked documents — their full texts are NOT in "
@@ -240,17 +254,20 @@ def test_agent_tools_names_and_parameters() -> None:
"document you have already read — its full text is "
"already in your prompt. Very large documents are "
"truncated: you receive the first part plus a TRUNCATED "
"notice naming how many more characters exist — the "
"notice is authoritative, the document did NOT end "
"where it stopped. Follow it and use `grep` (pattern) "
"to locate the rest — it searches the whole document. "
"notice naming how many more characters exist. To read "
"the rest: use `grep` (pattern) to find where your text "
"is — it reports line numbers — then call `read` again "
"on the same document with `offset` (1-based first line) "
"and `limit` (number of lines) to read exactly that "
"part. Do not re-read a document you have already read "
"in full — its full text is already in your prompt. "
"Call one tool at a time — wait for this result before "
"your next call."
)
# The byte-preserved contracts inside the rewrite, pinned as
# substrings: the combined source/path identity, the
# truncation-notice paragraph, the one-call-at-a-time
# discipline sentence.
# phase-131 grep → read(offset, limit) workflow paragraph, the
# one-call-at-a-time discipline sentence.
assert (
"by its combined `source/path` string, exactly as shown in "
"the `ls` output or the <documents> blocks"
@@ -258,10 +275,11 @@ def test_agent_tools_names_and_parameters() -> None:
assert (
"Very large documents are truncated: you receive the first "
"part plus a TRUNCATED notice naming how many more "
"characters exist — the notice is authoritative, the "
"document did NOT end where it stopped. Follow it and use "
"`grep` (pattern) to locate the rest — it searches the "
"whole document."
"characters exist. To read the rest: use `grep` (pattern) "
"to find where your text is — it reports line numbers — "
"then call `read` again on the same document with "
"`offset` (1-based first line) and `limit` (number of "
"lines) to read exactly that part."
) in read["description"]
assert (
"Call one tool at a time — wait for this result before "
@@ -269,9 +287,30 @@ def test_agent_tools_names_and_parameters() -> None:
) in read["description"]
read_params = read["parameters"]
assert read_params["type"] == "object"
# Phase 131 (task 01, LOCKED A2): offset/limit are OPTIONAL —
# ``required`` stays ``["path"]``.
assert read_params["required"] == ["path"]
assert set(read_params["properties"]) == {"path"}
assert set(read_params["properties"]) == {"path", "offset", "limit"}
assert read_params["properties"]["path"]["type"] == "string"
# The line-range parameters (phase 131, task 01, LOCKED A2):
# BOTH integer, 1-based (``offset`` = first line, ``limit`` =
# line count) — the pi-harness ``read`` shape the model was
# trained on. The descriptions carry the grep → read(offset,
# limit) teaching (pinned copy).
assert read_params["properties"]["offset"]["type"] == "integer"
assert read_params["properties"]["limit"]["type"] == "integer"
assert read_params["properties"]["offset"]["description"] == (
"The first line to read, 1-based. Omit it to read from the "
"beginning of the document. Use it to read a specific part "
"of a large document — grep reports the line numbers of "
"what you found."
)
assert read_params["properties"]["limit"]["description"] == (
"The maximum number of lines to read, starting at offset. "
"Omit it to read as many lines as fit under the character "
"cap. Keep it small — you are reading a part, not the "
"document."
)
# The combined source/path string is the canonical document identity
# (phase 70) — the description pins it with a worked example; the
# bare-path contract is stated up front (phase 72, task 02). Phase
@@ -303,13 +342,14 @@ def test_agent_tools_names_and_parameters() -> None:
"Search the indexed documents for an exact string "
"(case-insensitive) and return up to 20 matching lines "
"as `source/path:line: text` — a locator, not a "
"context-adder: read the winner with `read`. The "
"pattern is a plain substring, NEVER a regex — if a "
"pattern with regex syntax (like '.*' or '\\.') comes "
"back with no matches, retry with the plain text you "
"expect to see. For a normal search pass ONLY `pattern` "
"— it searches every document and that is how you "
"search the knowledge base; never pass a source name "
"context-adder: read the winner with `read` — for a "
"large document, use the reported line number as "
"`offset`. The pattern is a plain substring, NEVER a "
"regex — if a pattern with regex syntax (like '.*' or "
"'\\.') comes back with no matches, retry with the plain "
"text you expect to see. For a normal search pass ONLY "
"`pattern` — it searches every document and that is how "
"you search the knowledge base; never pass a source name "
"as `path` (a source name is not a document). Call one "
"tool at a time — wait for this result before your "
"next call."
@@ -1956,16 +1996,27 @@ def test_read_at_cap_plus_one_truncates_with_marker_and_notice(
def test_read_truncation_notice_is_pinned() -> None:
"""Phase 95: the notice copy is pinned — it names the true length
(``{total}``), the cap kept (``{shown}``), states the rest is NOT
shown (the document did not end where it stopped), and points at
``grep`` (which searches the whole document)."""
"""Phase 95 (REWRITTEN phase 131, task 01): the notice copy is
pinned — it names the true length (``{total}``), the cap kept
(``{shown}``), states the rest is NOT shown (the document did
not end where it stopped), and teaches the working workflow —
the old dead-end is gone: ``grep`` reports line numbers, then
read that part with ``offset`` / ``limit`` (BOTH parameters
named in the notice itself)."""
assert READ_TRUNCATION_NOTICE.format(shown=100, total=250) == (
"TRUNCATED — this document is 250 characters; only the first "
"100 are in your context. The rest is NOT shown. Use grep "
"(pattern) to locate what you need — grep searches the whole "
"document."
"(pattern) to find what you need — it reports line numbers — "
"then read that part with `offset` (1-based first line) and "
"`limit` (number of lines)."
)
# Both line-range parameters are named in the notice — the model
# stuck at the truncation sees the exact follow-up call.
assert "`offset`" in READ_TRUNCATION_NOTICE
assert "`limit`" in READ_TRUNCATION_NOTICE
# The old dead-end sentence (no way to act on the line number)
# is gone.
assert "to locate what you need" not in READ_TRUNCATION_NOTICE
def test_run_agent_yields_tool_result_after_tool_frame_before_next_round(
@@ -2081,6 +2132,682 @@ def test_read_truncation_paths_with_suggested_seed(
assert holder2.tool_calls == 0 and holder2.read_docs == []
# ---------- phase 131 (task 02): the range-read execution (offset/limit) ----------
def _lines_doc(source: str, path: str, n_lines: int, prefix: str = "line ") -> Document:
"""A fixture document of EXACTLY *n_lines* lines (``prefix{i}``)."""
content = "\n".join(f"{prefix}{i}" for i in range(1, n_lines + 1))
return _doc(source, path, path, content)
def test_bad_range_args_is_pinned() -> None:
"""Phase 131 (task 02): the range-argument validation refusal is
pinned — the contract (whole line numbers ≥ 1) with the bad values
echoed (``!r`` — an absent value prints as ``None``)."""
assert agent.BAD_RANGE_ARGS.format(offset=2.5, limit=None) == (
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
"(you passed offset=2.5, limit=None)."
)
assert agent.BAD_RANGE_ARGS.format(offset=None, limit="10") == (
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
"(you passed offset=None, limit='10')."
)
assert agent.BAD_RANGE_ARGS.format(offset=True, limit=None) == (
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
"(you passed offset=True, limit=None)."
)
def test_reread_truncated_is_pinned() -> None:
"""Phase 131 (task 02): the plain re-read teaching for a TRUNCATED
document is pinned — the shown count, the resolved identity, and
the working workflow (grep reports line numbers → read that part
with ``offset`` + ``limit``)."""
assert agent.REREAD_TRUNCATED.format(shown=100, source="S", path="big.md") == (
"You already have the first 100 characters of S/big.md in your "
"context. To read the rest, find where your text is with grep "
"(it reports line numbers), then call read with `offset` "
"(1-based first line) and `limit` (lines)."
)
def test_offset_past_end_is_pinned() -> None:
"""Phase 131 (task 02): the offset-past-end teaching is pinned —
the RESOLVED identity, the true line count, the grep pointer."""
assert agent.OFFSET_PAST_END.format(
source="S", path="a.md", total_lines=10, offset=11
) == (
"S/a.md is 10 lines long — `offset` 11 is past the end. Use a "
"smaller offset (grep reports the exact line numbers)."
)
def test_read_range_args_plain_and_valid() -> None:
"""Phase 131 (task 02): the pure validator — both absent is a plain
read; valid ints pass through (either one-sided is ok: ``offset``
without ``limit`` reads to the end, ``limit`` without ``offset``
counts from the start — the caller normalizes)."""
assert agent._read_range_args(None, None) == (True, "", None, None)
assert agent._read_range_args(1, None) == (True, "", 1, None)
assert agent._read_range_args(None, 7) == (True, "", None, 7)
assert agent._read_range_args(3, 42) == (True, "", 3, 42)
@pytest.mark.parametrize(
("offset", "limit"),
[
(2.5, None), # a float
("3", None), # a string
(True, None), # a bool (an int subclass — the house guard excludes it)
(0, None), # zero
(-1, None), # a negative
(None, 2.5), # a float limit
(None, "10"), # a string limit
(None, False), # a bool limit
(None, 0), # a zero limit
(None, -3), # a negative limit
],
)
def test_read_range_args_bad_values_are_refused(
offset: Any, limit: Any
) -> None:
"""Phase 131 (task 02): a PRESENT bad value (not a whole int ≥ 1 —
the house ``isinstance(x, int) and not isinstance(x, bool)`` guard,
or ``< 1``) refuses with the values echoed; the range comes back
unusable (``None``/``None``)."""
ok, line, got_offset, got_limit = agent._read_range_args(offset, limit)
assert not ok
assert got_offset is None and got_limit is None
assert line == agent.BAD_RANGE_ARGS.format(offset=offset, limit=limit)
def test_bad_range_args_refused_before_any_lookup(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): the validation refusal fires in the flow —
BEFORE the dedupe and the DB resolve (a float ``offset`` with an
unknown path still gets the range refusal, not the no-document
line): counts in nothing, consumes a round, the tools stay
offered."""
def _boom(*_a: Any, **_k: Any) -> None:
raise AssertionError("no DB lookup for a bad range argument")
monkeypatch.setattr(agent, "find_document", _boom)
monkeypatch.setattr(agent, "all_documents", _boom)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/ghost.md", "offset": 2.5}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert holder.read_docs == [] and holder.tool_calls == 0
assert llm.requests[1][0][3]["content"] == agent.BAD_RANGE_ARGS.format(
offset=2.5, limit=None
)
assert llm.requests[1][1] == AGENT_TOOLS
def test_bad_range_args_beats_the_dedupe(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): a bad range argument on a document ALREADY
IN CONTEXT is the range refusal, not ``ALREADY_IN_CONTEXT`` — the
validation precedes the dedupe (a refusal either way: no count,
no re-append)."""
doc = _doc("S", "a.md", "A", "A-CONTENT")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/a.md"})],
[
ToolCallPiece(
id="c2", name="read", arguments={"path": "S/a.md", "offset": 0}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert holder.read_docs == [doc] and holder.tool_calls == 1
assert llm.requests[2][0][5]["content"] == agent.BAD_RANGE_ARGS.format(
offset=0, limit=None
)
def test_offset_past_end_refused_naming_line_count(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): ``offset`` = total+1 on a 10-line doc →
the pinned teaching (the true line count named, the grep pointer)
— a refusal: nothing counted, the doc NOT added to the context (a
corrected retry still proceeds)."""
doc = _lines_doc("S", "ten.md", 10)
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/ten.md", "offset": 11}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert holder.read_docs == [] and holder.tool_calls == 0
assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format(
source="S", path="ten.md", total_lines=10, offset=11
)
def test_offset_past_end_on_one_line_doc(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): a 1-line doc — ``offset=2`` is past the
end (the true count is 1); ``offset=1`` with an overshooting
``limit`` clamps to the single line."""
doc = _doc("S", "one.md", "One", "only")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
# (a) offset past the end of the 1-line doc.
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/one.md", "offset": 2}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format(
source="S", path="one.md", total_lines=1, offset=2
)
assert holder.read_docs == [] and holder.tool_calls == 0
# (b) the same doc read from line 1, limit overshoots — clamped.
holder2 = AgentHolder()
llm2 = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/one.md", "offset": 1, "limit": 5},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm2, holder2, _settings()))
assert llm2.requests[1][0][3]["content"] == (
"Document S/one.md:\ndate: 2024-06-15\nlines: 1-1 of 1\nonly"
)
assert holder2.read_docs == [doc] and holder2.tool_calls == 1
# --- the six-row dedupe matrix (one test per row) ---
def test_matrix_row1_plain_read_unread_doc_reads_full(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 1: plain ``read(path)``, doc NOT in ``read_docs`` →
the full read (truncate at the cap as today — the pre-phase-131
path, no ``lines:`` header line)."""
doc = _doc("S", "m1.md", "M1", "one\ntwo\nthree")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m1.md"})],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert holder.read_docs == [doc] and holder.tool_calls == 1
assert llm.requests[1][0][3]["content"] == (
"Document S/m1.md:\ndate: 2024-06-15\none\ntwo\nthree"
)
def test_matrix_row2_plain_read_truncated_doc_gets_reread_teaching(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 2 (the stuck scenario's first hop): a plain re-read
of a document whose first read TRUNCATED → the pinned
``REREAD_TRUNCATED`` teaching (the shown count from the matching
truncation tuple — the cap) — a refusal: no re-emit, no count,
no second DB lookup (the raw-arg fast path fires pre-resolve)."""
cap = 100
doc = _lines_doc("S", "m2.md", 20) # 150 chars > the cap
lookups: list[tuple[str, str]] = []
def _find(db: Any, source: str, path: str) -> Document | None:
lookups.append((source, path))
return doc if (source, path) == ("S", "m2.md") else None
monkeypatch.setattr(agent, "find_document", _find)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m2.md"})],
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/m2.md"})],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
# Round 1: the truncated plain read (the stuck trigger) — the
# pre-phase-131 truncation bytes, the holder tuple recorded.
assert llm.requests[1][0][3]["content"] == (
"Document S/m2.md:\ndate: 2024-06-15\n"
+ doc.content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))
)
assert holder.read_truncations == [("S/m2.md", cap, len(doc.content))]
# Round 2: the plain re-read → the teaching (the shown count = the
# cap from the matching tuple).
assert llm.requests[2][0][5]["content"] == agent.REREAD_TRUNCATED.format(
shown=cap, source="S", path="m2.md"
)
# A refusal: the counters stay at the first read's, no re-append,
# no second lookup.
assert holder.read_docs == [doc]
assert holder.tool_calls == 1
assert lookups == [("S", "m2.md")]
def test_matrix_row3_plain_read_full_doc_stays_already_in_context(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 3: a plain re-read of a document read IN FULL (never
truncated) → ``ALREADY_IN_CONTEXT`` — byte-identical to
pre-phase-131 (the raw-arg fast path, no DB lookup)."""
doc = _doc("S", "m3.md", "M3", "FULL-TEXT") # fits under any cap
lookups: list[tuple[str, str]] = []
def _find(db: Any, source: str, path: str) -> Document | None:
lookups.append((source, path))
return doc if (source, path) == ("S", "m3.md") else None
monkeypatch.setattr(agent, "find_document", _find)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m3.md"})],
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/m3.md"})],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
assert holder.read_docs == [doc] and holder.tool_calls == 1
assert lookups == [("S", "m3.md")] # the re-read deduped pre-lookup
def test_matrix_row4_range_read_truncated_doc_proceeds(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 4 (the stuck scenario's FIX): after a TRUNCATED plain
read, a range read of the SAME document PROCEEDS — the ``lines:``
header + exactly the named lines. The doc was appended ONCE (the
known set prevents the re-append); the range read counts as a
successful call; the slice fits, so no new truncation tuple."""
cap = 100
doc = _lines_doc("S", "m4.md", 20) # 150 chars > the cap
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m4.md"})],
[
ToolCallPiece(
id="c2",
name="read",
arguments={"path": "S/m4.md", "offset": 5, "limit": 3},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
# Round 1: the truncated plain read.
assert llm.requests[1][0][3]["content"].endswith(
"\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))
)
# Round 2: the range read PROCEEDED — lines 5-7 of 20, exactly.
assert llm.requests[2][0][5]["content"] == (
"Document S/m4.md:\ndate: 2024-06-15\nlines: 5-7 of 20\n"
"line 5\nline 6\nline 7"
)
assert holder.read_docs == [doc] # appended exactly once
assert holder.tool_calls == 2 # both reads are successful calls
assert holder.read_truncations == [("S/m4.md", cap, len(doc.content))]
def test_matrix_row5_range_read_full_doc_stays_already_in_context(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 5: a range read of a document read IN FULL (never
truncated) → ``ALREADY_IN_CONTEXT`` — the range is a SUBSET of the
text already in the prompt (the matrix needs the resolved pair, so
the check runs after the resolve)."""
doc = _lines_doc("S", "m5.md", 5)
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m5.md"})],
[
ToolCallPiece(
id="c2",
name="read",
arguments={"path": "S/m5.md", "offset": 2, "limit": 2},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
assert holder.read_docs == [doc] and holder.tool_calls == 1
assert holder.read_truncations == []
def test_matrix_row6_range_read_unread_doc_proceeds(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Matrix row 6: a range read of a NOT-yet-read document PROCEEDS —
the ``lines:`` header + the slice, the doc appended, the call
counted (a first read, plain or range, behaves like a first read)."""
doc = _lines_doc("S", "m6.md", 8)
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/m6.md", "offset": 3, "limit": 2},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/m6.md:\ndate: 2024-06-15\nlines: 3-4 of 8\nline 3\nline 4"
)
assert holder.read_docs == [doc] and holder.tool_calls == 1
# --- the slice math + the result shape ---
def test_range_read_offset_one_no_limit_reads_whole_doc(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): ``offset=1`` without ``limit`` reads to the
end of the document — the true extent ``1-N of N``, the whole
content (the slice, not the plain-read shape)."""
doc = _doc("S", "r1.md", "R1", "a\nb\nc")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/r1.md", "offset": 1}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/r1.md:\ndate: 2024-06-15\nlines: 1-3 of 3\na\nb\nc"
)
assert holder.read_docs == [doc] and holder.tool_calls == 1
def test_range_read_offset_at_total_reads_last_line(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): ``offset`` = the last line number →
exactly the last line (``N-N of N``)."""
doc = _doc("S", "r2.md", "R2", "a\nb\nc")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/r2.md", "offset": 3, "limit": 5},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/r2.md:\ndate: 2024-06-15\nlines: 3-3 of 3\nc"
)
def test_range_read_limit_overshoot_clamps_to_total(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): an overshooting ``limit`` clamps ``end`` to
the last line (reads to the end, never past it)."""
doc = _doc("S", "r3.md", "R3", "a\nb\nc")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/r3.md", "offset": 2, "limit": 10},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/r3.md:\ndate: 2024-06-15\nlines: 2-3 of 3\nb\nc"
)
def test_range_read_limit_without_offset_counts_from_start(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): ``limit`` WITHOUT ``offset`` is a line count
from the start of the document (offset = 1 — the helper's
documented one-sided semantics)."""
doc = _doc("S", "r4.md", "R4", "a\nb\nc")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/r4.md", "limit": 2}
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/r4.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb"
)
def test_range_read_image_doc_carries_marker_after_lines_line(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): a range read of a standalone-IMAGE document
carries the :data:`agent.IMAGE_DOC_MARKER` line exactly where the
plain read puts it — after the header lines (here, after
``lines:``)."""
doc = _doc("S", "img.png", "Img", "IMG-DESC-OF-LINE-1\nline-2-text")
doc.is_image = True
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/img.png", "offset": 2, "limit": 1},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/img.png:\ndate: 2024-06-15\nlines: 2-2 of 2\n"
f"{agent.IMAGE_DOC_MARKER}\n"
"line-2-text"
)
assert holder.read_docs == [doc] and holder.tool_calls == 1
def test_range_read_slice_over_cap_truncates_with_slice_length_total(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): the cap applies to the SLICE — an over-cap
slice is cut at the cap + the shared marker + the pinned notice
(``{total}`` = the SLICE's length), and the holder tuple's
``chars_total`` is the slice's length, NOT the document's (the
loop still yields one ToolResultPiece for it)."""
cap = 20
lines = [f"L{i:02d}" + "x" * 30 for i in range(1, 11)] # 10 lines × 33 chars
content = "\n".join(lines)
doc = _doc("S", "cap.md", "Cap", content)
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/cap.md", "offset": 5, "limit": 6},
)
],
[StreamPiece("content", "ans")],
)
out = asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
# The slice: lines 5-10 (6 × 33 chars + 5 newlines = 203 > cap).
slice_text = "\n".join(lines[4:10])
assert len(slice_text) == 203
assert len(content) == 339 # the document is much bigger
assert llm.requests[1][0][3]["content"] == (
"Document S/cap.md:\ndate: 2024-06-15\nlines: 5-10 of 10\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
# The tuple: the argument, the cap kept, the SLICE's length (203,
# not the document's 339).
assert holder.read_truncations == [("S/cap.md", cap, 203)]
assert holder.read_docs == [doc] and holder.tool_calls == 1
# The loop yields the ToolResultPiece with the slice's total.
pieces = [p for p in out if isinstance(p, ToolResultPiece)]
assert len(pieces) == 1
assert pieces[0].argument == "S/cap.md"
assert pieces[0].chars_shown == cap
assert pieces[0].chars_total == 203
def test_range_read_truncated_then_plain_reread_teaching_then_range_proceeds(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): a TRUNCATED range read (offset=1, no limit
— the whole 339-char doc under a 20-char cap) records the slice
tuple; a plain re-read then gets ``REREAD_TRUNCATED`` with the
shown count from THAT (range) tuple; a subsequent range read still
PROCEEDS (truncated-then-range, regardless of which read
truncated)."""
cap = 20
lines = [f"L{i:02d}" + "x" * 30 for i in range(1, 11)]
content = "\n".join(lines)
doc = _doc("S", "t.md", "T", content)
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="read", arguments={"path": "S/t.md", "offset": 1}
)
],
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/t.md"})],
[
ToolCallPiece(
id="c3",
name="read",
arguments={"path": "S/t.md", "offset": 5, "limit": 2},
)
],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
# Round 1: the truncated range read (the whole doc, offset=1) —
# the slice is the whole doc here (339 chars), the tuple's total
# is the slice's length.
assert llm.requests[1][0][3]["content"].startswith(
"Document S/t.md:\ndate: 2024-06-15\nlines: 1-10 of 10\n"
)
assert llm.requests[1][0][3]["content"].endswith(
"\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=339)
)
# Round 2: the plain re-read → the teaching, shown from the (range)
# truncation tuple.
assert llm.requests[2][0][5]["content"] == agent.REREAD_TRUNCATED.format(
shown=cap, source="S", path="t.md"
)
# Round 3: the range read of the truncated doc PROCEEDS — lines 5-6
# (66 + 1 = 67 chars > the cap → truncated again; the tuple's total
# is the slice's length, the document's is irrelevant).
slice_text = "\n".join(lines[4:6])
assert llm.requests[3][0][7]["content"] == (
"Document S/t.md:\ndate: 2024-06-15\nlines: 5-6 of 10\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
assert holder.read_truncations == [("S/t.md", cap, 339), ("S/t.md", cap, 67)]
assert holder.read_docs == [doc] # appended exactly once, still
assert holder.tool_calls == 2 # the refusal counted nothing
def test_plain_read_after_range_read_is_already_in_context(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 131 (task 02): a range read appends the doc to
``read_docs`` ONCE — so a later PLAIN read of the range-read
(never-truncated) document is ``ALREADY_IN_CONTEXT`` (the model has
committed to this doc; the ``lines:`` line told it what it holds)."""
doc = _doc("S", "p.md", "P", "a\nb\nc")
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1",
name="read",
arguments={"path": "S/p.md", "offset": 1, "limit": 2},
)
],
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/p.md"})],
[StreamPiece("content", "ans")],
)
asyncio.run(_run(llm, holder, _settings()))
assert llm.requests[1][0][3]["content"] == (
"Document S/p.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb"
)
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
assert holder.read_docs == [doc] and holder.tool_calls == 1
def test_unknown_tool_name_refused(monkeypatch: pytest.MonkeyPatch) -> None:
holder = AgentHolder()
llm = ScriptedLLM(
+40
View File
@@ -127,6 +127,46 @@ def test_tool_event_shape_is_type_name_argument_only() -> None:
assert dumped["type"] == "tool" # default — call sites never spell it out
def test_tool_frame_range_read_carries_offset_and_limit() -> None:
"""Phase 131 (task 04): a range ``read``'s frame carries the line
range the model requested — the wire's assertion surface for the
owner's grep → read(offset/limit) dance (the story E2E pins the
REAL ints the mock parsed from the grep result)."""
frame = sse_event(
ChatToolEvent(
name="read", argument="S/p.md", offset=1500, limit=30
).model_dump()
)
assert frame == (
'data: {"type": "tool", "name": "read", "argument": "S/p.md", '
'"offset": 1500, "limit": 30}\n\n'
)
assert _payload(frame) == {
"type": "tool",
"name": "read",
"argument": "S/p.md",
"offset": 1500,
"limit": 30,
}
def test_tool_event_range_fields_absent_not_null_when_omitted() -> None:
"""Phase 131 (task 04): the ``offset`` / ``limit`` keys are ABSENT
(never ``null``) when omitted — a plain ``read`` (no range) and a
``grep`` (no range parameters at all) both serialize to the
byte-identical pre-phase shape (the omission pattern of
``ChatDoneEvent.answer_image``). A bad range VALUE never reaches
the frame at all: ``app.api.chat`` applies the whole-line-number
guard before building the event (the E2E suite pins the frame the
endpoint emits)."""
for dumped in (
ChatToolEvent(name="read", argument="S/p.md").model_dump(),
ChatToolEvent(name="read", argument="S/p.md", offset=None, limit=None).model_dump(),
ChatToolEvent(name="grep", argument="trap").model_dump(),
):
assert set(dumped.keys()) == {"type", "name", "argument"}, dumped
def test_retry_frame_serializes_exactly() -> None:
"""Phase 67: the ``retry`` frame is exactly ``{type: "retry",
attempt: int, max_attempts: int}`` — a transient status the client's