phase: 131_read_line_ranges
All 44 progress lines are pure dots, final line at 100%, exit 0. Everything is verified. Final report: **Phase 131 — final verification pass (all 4 tasks already complete in working tree)** - **Verified implemented:** `read(offset, limit)` schema + teaching descriptions; rewritten truncation notice; full 6-row dedupe matrix + range validation + `lines: a-b of N` header + slice cap in `_execute_tool`; wire-level `offset`/`limit` on `ChatToolEvent` (absent keys dropped → non-range frames byte-identical); mock-LLM `read the lines` flow; handbook repro (wording + line numbers derived from the real PDF at test time) - **Tests:** `uv run pytest --no-cov` → 3097 passed, exit 0; `uv run pytest --cov=app` → **99% on app/** (>90%); `uv run pytest tests/e2e/test_read_line_ranges.py -v --no-cov` → 2/2 in isolation (47.6 s); `tests/integration/test_read_line_ranges_handbook.py` → 2/2 (3.5 s); 10-file targeted E2E batch pinning pre-phase surfaces (grep/ls/plain-read/ALREADY_IN_CONTEXT/truncation) → 31/31; `ruff check .` clean; `pyright` 0 errors - **Completion criteria:** all 5 green — (1) exact range lines + `lines:` header, plain reads byte-identical (existing pins pass unchanged) ✅ (2) truncated-then-range succeeds end-to-end on the real handbook; other re-reads get teaching refusals ✅ (3) find-traps answered from real PDF text, no hardcoded wording ✅ (4) coverage + isolated E2E ✅ (5) no behavior change in completed phases ✅ - **Docs:** README + AGENTS reviewed in full. Fixed 3 inconsistencies: README missing `BOR_HISTORY_MAX_TURNS`/`BOR_HISTORY_MAX_CHARS` rows (pre-existing phase-74 gap — env-tunable, unit-pinned, undocumented); `.env.example` missing the same two vars (added); `.env.example` stale `BOR_READ_MAX_CHARS` comment (still said "grep-pointer notice (phase 95)" — updated to the phase-131 workflow notice). AGENTS.md verified consistent — no changes needed. All 67 `.env.example` vars documented, all defaults match `app/config.py`, pages/auth tables match live routes - **Note:** PLAN.md §7.1 still states the retired 46rem chat column (superseded by phase 100, which AGENTS.md rule 5 records correctly) — PLAN.md is off-limits for this pass, flagged only - **Next pending phase:** `132_image_upload_feedback`
This commit is contained in:
+42
-1
@@ -303,6 +303,20 @@ def get_llm() -> LLMClient:
|
||||
return _llm
|
||||
|
||||
|
||||
def _whole_line_number(value: Any) -> bool:
|
||||
"""A ``read`` range argument's whole-line-number guard (phase 131,
|
||||
task 04).
|
||||
|
||||
The same validation the agent's ``_read_range_args`` applies
|
||||
server-side (``app/rag/agent.py``): an ``int`` (bools excluded —
|
||||
the house guard) and ``>= 1``. A value that fails is a REFUSED
|
||||
call (the ``BAD_RANGE_ARGS`` teaching), and its ``tool`` frame
|
||||
carries no range — the refusal surfaces as the round's result, not
|
||||
as the call's arguments.
|
||||
"""
|
||||
return isinstance(value, int) and not isinstance(value, bool) and value >= 1
|
||||
|
||||
|
||||
def sse_event(payload: dict[str, Any]) -> str:
|
||||
"""Serialize one SSE frame: ``data: <json>\\n\\n`` (PLAN §4)."""
|
||||
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
|
||||
@@ -905,8 +919,35 @@ async def chat(
|
||||
"pattern" if piece.name == "grep" else "path"
|
||||
)
|
||||
argument = argument if isinstance(argument, str) else None
|
||||
# Phase 131 (task 04): a range ``read``'s line
|
||||
# range rides the SAME frame — the additive
|
||||
# ``offset`` / ``limit`` fields (the wire's
|
||||
# assertion surface for the owner's
|
||||
# grep → read(offset/limit) dance). Only a
|
||||
# well-formed range (``_whole_line_number``,
|
||||
# the agent's own guard) lands on the frame;
|
||||
# absent → omitted on serialization, so every
|
||||
# non-range frame stays byte-identical to
|
||||
# pre-phase (the ChatToolEvent serializer).
|
||||
offset = (
|
||||
piece.arguments.get("offset")
|
||||
if piece.name == "read"
|
||||
else None
|
||||
)
|
||||
offset = offset if _whole_line_number(offset) else None
|
||||
limit = (
|
||||
piece.arguments.get("limit")
|
||||
if piece.name == "read"
|
||||
else None
|
||||
)
|
||||
limit = limit if _whole_line_number(limit) else None
|
||||
yield sse_event(
|
||||
ChatToolEvent(name=piece.name, argument=argument).model_dump()
|
||||
ChatToolEvent(
|
||||
name=piece.name,
|
||||
argument=argument,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
).model_dump()
|
||||
)
|
||||
continue
|
||||
if isinstance(piece, RetryPiece):
|
||||
|
||||
+345
-70
@@ -94,10 +94,28 @@ task 04):
|
||||
permission 2026-09-10, ``TODO.md`` L5): the first ``read_max_chars``
|
||||
chars plus the shared :data:`TRUNCATION_MARKER` and the pinned
|
||||
:data:`READ_TRUNCATION_NOTICE` (the rest is NOT in the model's
|
||||
context; ``grep`` — which searches the whole document — is the
|
||||
follow-up), with the truncation recorded on the holder so the loop
|
||||
context; the notice teaches the phase-131 workflow — ``grep``
|
||||
reports line numbers, then ``read`` that part with ``offset`` /
|
||||
``limit``), with the truncation recorded on the holder so the loop
|
||||
yields a :class:`app.rag.llm.ToolResultPiece` (task 02 → the SSE
|
||||
``tool_result`` frame + UI marker). **A6 re-revised contract
|
||||
``tool_result`` frame + UI marker). Phase 131 (task 02): ``read``
|
||||
gains the OPTIONAL line range — ``offset`` (1-based first line) +
|
||||
``limit`` (line count; ``offset`` alone reads to the end, ``limit``
|
||||
alone counts from line 1 — :func:`_read_range_args` validates
|
||||
whole ints ≥ 1, a bad value is the :data:`BAD_RANGE_ARGS` refusal
|
||||
with the values echoed): a range read returns EXACTLY the named
|
||||
lines under a third ``lines: {offset}-{end} of {total_lines}``
|
||||
header line (the true extent — an overshooting ``limit`` clamps to
|
||||
the last line), the cap applies to the SLICE (the holder tuple's
|
||||
``chars_total`` is then the slice's length, not the document's),
|
||||
and an ``offset`` past the end is the :data:`OFFSET_PAST_END`
|
||||
teaching. The dedupe matrix opens the ONE previously-dead case — a
|
||||
range read of a document whose earlier read was TRUNCATED (the
|
||||
stuck scenario: the range the model names is exactly the part it
|
||||
does NOT have); a PLAIN re-read of a truncated document gets the
|
||||
:data:`REREAD_TRUNCATED` teaching instead of the (wrong) full-text
|
||||
claim, and every other re-read of a fully-read document stays
|
||||
:data:`ALREADY_IN_CONTEXT`. **A6 re-revised contract
|
||||
(phase 118, owner directive 2026-09-15):** the retrieval
|
||||
``<documents>`` path seeds SUMMARIES only — a suggested document's
|
||||
full text never enters the prompt on the retrieval path; full text
|
||||
@@ -145,7 +163,15 @@ task 04):
|
||||
:data:`ALREADY_IN_CONTEXT` (phase 72, task 05 gate iteration:
|
||||
the line names the correct action — answer from the text already
|
||||
in the prompt, do not call read again — so a fired refusal ends
|
||||
the loop instead of inviting a repeat); an unknown document (a ``read`` or scoped ``grep`` whose
|
||||
the loop instead of inviting a repeat); a ``read`` whose optional
|
||||
``offset``/``limit`` is not a whole line number ≥ 1 (phase 131) →
|
||||
:data:`BAD_RANGE_ARGS` (the bad values echoed); a range ``read``
|
||||
whose ``offset`` is past the document's end (phase 131) →
|
||||
:data:`OFFSET_PAST_END` (the true line count named); a PLAIN
|
||||
re-read of a document whose earlier read was TRUNCATED (phase 131)
|
||||
→ :data:`REREAD_TRUNCATED` (the grep → read(offset/limit) teaching
|
||||
— the full text is NOT in the prompt, unlike the line above); an
|
||||
unknown document (a ``read`` or scoped ``grep`` whose
|
||||
combined ``source/path`` matches nothing — a bare source name, which
|
||||
can never be a document, included) → ``"No document at '…' — check
|
||||
the ls output."`` with the argument echoed as passed (the model sees
|
||||
@@ -313,12 +339,15 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"document you have already read — its full text is "
|
||||
"already in your prompt. Very large documents are "
|
||||
"truncated: you receive the first part plus a TRUNCATED "
|
||||
"notice naming how many more characters exist — the "
|
||||
"notice is authoritative, the document did NOT end "
|
||||
"where it stopped. Follow it and use `grep` (pattern) "
|
||||
"to locate the rest — it searches the whole document. "
|
||||
"Call one tool at a time — wait for this result before "
|
||||
"your next call."
|
||||
"notice naming how many more characters exist. To read "
|
||||
"the rest: use `grep` (pattern) to find where your "
|
||||
"text is — it reports line numbers — then call `read` "
|
||||
"again on the same document with `offset` (1-based "
|
||||
"first line) and `limit` (number of lines) to read "
|
||||
"exactly that part. Do not re-read a document you have "
|
||||
"already read in full — its full text is already in "
|
||||
"your prompt. Call one tool at a time — wait for this "
|
||||
"result before your next call."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -335,7 +364,27 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"document you have already read — its full "
|
||||
"text is already in your prompt."
|
||||
),
|
||||
}
|
||||
},
|
||||
"offset": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"The first line to read, 1-based. Omit it to "
|
||||
"read from the beginning of the document. "
|
||||
"Use it to read a specific part of a large "
|
||||
"document — grep reports the line numbers "
|
||||
"of what you found."
|
||||
),
|
||||
},
|
||||
"limit": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"The maximum number of lines to read, "
|
||||
"starting at offset. Omit it to read as "
|
||||
"many lines as fit under the character "
|
||||
"cap. Keep it small — you are reading a "
|
||||
"part, not the document."
|
||||
),
|
||||
},
|
||||
},
|
||||
"required": ["path"],
|
||||
},
|
||||
@@ -349,16 +398,17 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"Search the indexed documents for an exact string "
|
||||
"(case-insensitive) and return up to 20 matching lines "
|
||||
"as `source/path:line: text` — a locator, not a "
|
||||
"context-adder: read the winner with `read`. The "
|
||||
"pattern is a plain substring, NEVER a regex — if a "
|
||||
"pattern with regex syntax (like '.*' or '\\.') comes "
|
||||
"back with no matches, retry with the plain text you "
|
||||
"expect to see. For a normal search pass ONLY `pattern` "
|
||||
"— it searches every document and that is how you "
|
||||
"search the knowledge base; never pass a source name "
|
||||
"as `path` (a source name is not a document). Call one "
|
||||
"tool at a time — wait for this result before your "
|
||||
"next call."
|
||||
"context-adder: read the winner with `read` — for a "
|
||||
"large document, use the reported line number as "
|
||||
"`offset`. The pattern is a plain substring, NEVER a "
|
||||
"regex — if a pattern with regex syntax (like '.*' "
|
||||
"or '\\.') comes back with no matches, retry with "
|
||||
"the plain text you expect to see. For a normal "
|
||||
"search pass ONLY `pattern` — it searches every "
|
||||
"document and that is how you search the knowledge "
|
||||
"base; never pass a source name as `path` (a source "
|
||||
"name is not a document). Call one tool at a time — "
|
||||
"wait for this result before your next call."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -491,23 +541,82 @@ ALREADY_SHOWING = (
|
||||
"Answer from what you have."
|
||||
)
|
||||
|
||||
#: The read-truncation notice (phase 95, task 01) — appended, after the
|
||||
#: shared :data:`TRUNCATION_MARKER`, to a ``read`` result whose document
|
||||
#: is LONGER than ``settings.read_max_chars`` (``BOR_READ_MAX_CHARS``,
|
||||
#: default 128 000 chars ≈ 32k tokens). Two format fields: ``{total}``
|
||||
#: (the document's true character count) and ``{shown}`` (the cap — how
|
||||
#: many characters actually reached the model). The copy is pinned: it
|
||||
#: tells the model the read was truncated, that the rest is NOT in its
|
||||
#: context (the document did not end where it stopped), and names
|
||||
#: ``grep`` — the phase-70 harness-aligned locator that searches the
|
||||
#: WHOLE document — as the tool to find what it was looking for. A read
|
||||
#: at or under the cap appends nothing (byte-identical to the
|
||||
#: pre-phase-95 result).
|
||||
#: The read-truncation notice (phase 95, task 01; REWRITTEN phase 131,
|
||||
#: task 01) — appended, after the shared :data:`TRUNCATION_MARKER`, to
|
||||
#: a ``read`` result whose document is LONGER than
|
||||
#: ``settings.read_max_chars`` (``BOR_READ_MAX_CHARS``, default
|
||||
#: 128 000 chars ≈ 32k tokens). Two format fields: ``{total}`` (the
|
||||
#: document's true character count) and ``{shown}`` (the cap — how many
|
||||
#: characters actually reached the model). The copy is pinned: it tells
|
||||
#: the model the read was truncated and that the rest is NOT in its
|
||||
#: context (the document did not end where it stopped), then teaches
|
||||
#: the phase-131 workflow — the old dead-end ("use grep to locate the
|
||||
#: rest", with no way to act on the line number) is gone: ``grep``
|
||||
#: reports LINE NUMBERS, and the model reads the part it needs with
|
||||
#: ``read``'s ``offset`` (1-based first line) + ``limit`` (number of
|
||||
#: lines). A read at or under the cap appends nothing (byte-identical
|
||||
#: to the pre-phase-95 result).
|
||||
READ_TRUNCATION_NOTICE = (
|
||||
"TRUNCATED — this document is {total} characters; only the first "
|
||||
"{shown} are in your context. The rest is NOT shown. Use grep "
|
||||
"(pattern) to locate what you need — grep searches the whole "
|
||||
"document."
|
||||
"(pattern) to find what you need — it reports line numbers — "
|
||||
"then read that part with `offset` (1-based first line) and "
|
||||
"`limit` (number of lines)."
|
||||
)
|
||||
|
||||
#: The ``read`` range-argument validation refusal (phase 131, task
|
||||
#: 02): ``offset`` / ``limit``, when present, must each be a whole
|
||||
#: line number ≥ 1 — an ``int`` (bools excluded — the house
|
||||
#: ``isinstance(x, int) and not isinstance(x, bool)`` guard) and not
|
||||
#: ``< 1``. The bad values are echoed (``{offset!r}`` /
|
||||
#: ``{limit!r}`` — an absent value prints as ``None``), so the model
|
||||
#: sees exactly what it sent. Absent/``None`` for BOTH is fine (a
|
||||
#: plain read); ``offset`` without ``limit`` reads to the END of the
|
||||
#: document; ``limit`` without ``offset`` is treated as ``offset=1``
|
||||
#: (a line count from the start) — the two one-sided semantics
|
||||
#: :func:`_read_range_args` validates and the read branch applies.
|
||||
#: Still a refusal: counts in nothing, consumes a round (the phase-72
|
||||
#: style — no silent argument repair), and it fires BEFORE the dedupe
|
||||
#: and the DB resolve (a bad range argument is refused regardless of
|
||||
#: what is in context).
|
||||
BAD_RANGE_ARGS = (
|
||||
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
|
||||
"(you passed offset={offset!r}, limit={limit!r})."
|
||||
)
|
||||
|
||||
#: The plain re-read of a TRUNCATED document (phase 131, task 02 — the
|
||||
#: dedupe matrix's middle row): the model already holds the first
|
||||
#: ``{shown}`` characters (the cap from the matching
|
||||
#: ``holder.read_truncations`` tuple — the most recent truncation of
|
||||
#: the pair, plain or range) and its first read of the document hit
|
||||
#: the cap, so the rest is NOT in its context. The line points at the
|
||||
#: working workflow — ``grep`` reports line numbers, then ``read``
|
||||
#: that part with ``offset`` / ``limit`` — instead of the
|
||||
#: pre-phase-131 dead end (the plain :data:`ALREADY_IN_CONTEXT` line,
|
||||
#: which claimed the FULL text was in the prompt). ``{source}`` /
|
||||
#: ``{path}`` are the argument's split pair; ``{shown}`` is a plain
|
||||
#: integer. A refusal: no re-emit, ``tool_calls`` NOT incremented,
|
||||
#: consumes a round, fires on the raw-arg fast path (no DB lookup).
|
||||
REREAD_TRUNCATED = (
|
||||
"You already have the first {shown} characters of {source}/{path} "
|
||||
"in your context. To read the rest, find where your text is with "
|
||||
"grep (it reports line numbers), then call read with `offset` "
|
||||
"(1-based first line) and `limit` (lines)."
|
||||
)
|
||||
|
||||
#: The range-``read`` offset past the end (phase 131, task 02): the
|
||||
#: document is ``{total_lines}`` lines long (``len(content.
|
||||
#: splitlines())``) and the ``offset`` names a line beyond it.
|
||||
#: ``{source}`` / ``{path}`` are the RESOLVED identity; ``{total_lines}``
|
||||
#: and ``{offset}`` are plain integers (no ``!r``). The pointer — grep
|
||||
#: reports the exact line numbers — hands the model back to the
|
||||
#: locator that got it here. A refusal: counts in nothing, consumes a
|
||||
#: round; the document is NOT added to the context, so a corrected
|
||||
#: retry (a smaller offset) still proceeds.
|
||||
OFFSET_PAST_END = (
|
||||
"{source}/{path} is {total_lines} lines long — `offset` {offset} "
|
||||
"is past the end. Use a smaller offset (grep reports the exact "
|
||||
"line numbers)."
|
||||
)
|
||||
|
||||
#: The image-document marker (phase 122, task 05): the line prefixed to
|
||||
@@ -1208,6 +1317,48 @@ def _no_document_refusal(db: Session, arg: str) -> str:
|
||||
return f"No document at '{arg}' — check the ls output."
|
||||
|
||||
|
||||
def _read_range_args(
|
||||
offset: Any, limit: Any
|
||||
) -> tuple[bool, str, int | None, int | None]:
|
||||
"""Validate ``read``'s optional line range (phase 131, task 02).
|
||||
|
||||
Returns ``(ok, error_line, offset, limit)``:
|
||||
|
||||
* both absent/``None`` → ``(True, "", None, None)`` — a plain read
|
||||
(no range): from the beginning, capped as pre-phase-131.
|
||||
* either PRESENT but not a whole int ≥ 1 (not an ``int`` — bools
|
||||
excluded — or ``< 1``) → ``(False, BAD_RANGE_ARGS.format(...),
|
||||
None, None)`` — the values echoed with ``!r`` (an absent one
|
||||
prints ``None``). A refusal: counts in nothing, consumes a
|
||||
round; it fires before the dedupe and the DB resolve.
|
||||
* ok → ``(True, "", offset, limit)`` — either may be ``None``:
|
||||
``offset`` without ``limit`` reads to the END of the document,
|
||||
``limit`` without ``offset`` is treated as ``offset=1`` (a line
|
||||
count from the start) — the caller applies both one-sided
|
||||
semantics.
|
||||
|
||||
Pure (no I/O) — unit tests drive it directly.
|
||||
"""
|
||||
|
||||
def _whole_line_number(value: Any) -> bool:
|
||||
return (
|
||||
isinstance(value, int) and not isinstance(value, bool) and value >= 1
|
||||
)
|
||||
|
||||
if offset is None and limit is None:
|
||||
return True, "", None, None
|
||||
if (offset is not None and not _whole_line_number(offset)) or (
|
||||
limit is not None and not _whole_line_number(limit)
|
||||
):
|
||||
return (
|
||||
False,
|
||||
BAD_RANGE_ARGS.format(offset=offset, limit=limit),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
return True, "", offset, limit
|
||||
|
||||
|
||||
def grep_document(content: str, pattern: str) -> list[tuple[int, str]]:
|
||||
"""Every line of *content* that contains *pattern*, in file order.
|
||||
|
||||
@@ -1243,7 +1394,9 @@ class AgentHolder:
|
||||
chars_total)`` tuple per TRUNCATED ``read`` — the combined
|
||||
``source/path`` the model passed, the cap kept
|
||||
(``settings.read_max_chars``), and the document's true length — in
|
||||
execution order. Recorded on truncation only; a read at or under the
|
||||
execution order. For a TRUNCATED RANGE read (phase 131, task 02)
|
||||
``chars_total`` is the SLICE's length (the requested size of the
|
||||
part that was read), not the document's. Recorded on truncation only; a read at or under the
|
||||
cap appends nothing. A truncated read is still a **successful**
|
||||
call: ``tool_calls`` increments as today and ``read_docs`` appends
|
||||
as today — this list only carries the truncation signal the agent
|
||||
@@ -1285,7 +1438,19 @@ def _execute_tool(
|
||||
(the rest of the document is NOT in the model's context), and a
|
||||
``(argument, cap, total)`` tuple is appended to
|
||||
``holder.read_truncations`` (the signal the agent loop turns into a
|
||||
:class:`app.rag.llm.ToolResultPiece`). Document targets are combined ``source/path``
|
||||
:class:`app.rag.llm.ToolResultPiece`). Phase 131 (task 02): a
|
||||
``read`` may carry the optional line range ``offset`` (1-based
|
||||
first line) + ``limit`` (line count) — validated by
|
||||
:func:`_read_range_args` (a bad value → :data:`BAD_RANGE_ARGS`,
|
||||
before the dedupe and the resolve), the dedupe matrix (a range
|
||||
read of a TRUNCATED document proceeds — the stuck-scenario fix —
|
||||
a plain re-read of a truncated one gets :data:`REREAD_TRUNCATED`,
|
||||
every other re-read of a fully-read one stays
|
||||
:data:`ALREADY_IN_CONTEXT`), an ``offset`` past the end →
|
||||
:data:`OFFSET_PAST_END`, the slice shaped under a ``lines:
|
||||
{offset}-{end} of {total_lines}`` header line and capped like the
|
||||
whole document (the tuple's ``chars_total`` is then the slice's
|
||||
length). Document targets are combined ``source/path``
|
||||
strings, resolved by :func:`_resolve_path` (the canonical identity,
|
||||
phase 70).
|
||||
"""
|
||||
@@ -1350,18 +1515,57 @@ def _execute_tool(
|
||||
arg = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
if not arg:
|
||||
return MISSING_READ_ARGS
|
||||
# Phase 131 (task 02): the OPTIONAL line range — validated
|
||||
# after the path check and BEFORE the dedupe: a bad range
|
||||
# argument is a refusal regardless of what is in context
|
||||
# (counts in nothing, consumes a round — the phase-72 style;
|
||||
# no silent argument repair, no DB lookup).
|
||||
ok, range_error, offset, limit = _read_range_args(
|
||||
call.arguments.get("offset"), call.arguments.get("limit")
|
||||
)
|
||||
if not ok:
|
||||
return range_error
|
||||
# A one-sided range: ``limit`` without ``offset`` counts from
|
||||
# the start of the document (first line 1 — the helper's
|
||||
# documented semantics); ``offset`` without ``limit`` reads to
|
||||
# the end (the slice math below).
|
||||
ranged = offset is not None or limit is not None
|
||||
first = offset if offset is not None else 1
|
||||
# Phase 118 (A6): the dedupe set is ``holder.read_docs`` ONLY —
|
||||
# the ``seed_docs`` are SUMMARY blocks in the prompt, not full
|
||||
# text, so a FIRST read of a suggested document adds its full
|
||||
# text through the path below; only a document ALREADY READ is
|
||||
# refused.
|
||||
known = {(doc.source, doc.path) for doc in holder.read_docs}
|
||||
# The dedupe check needs no DB: the split pair of a combined
|
||||
# identity that is already in full-text context is in `known`
|
||||
# as-is (the resolve below would find the same document).
|
||||
if "/" in arg:
|
||||
# Phase 131 (task 02): the TRUNCATED set, keyed on the
|
||||
# RESOLVED (source, path) pair — the holder tuples carry the
|
||||
# raw argument the model passed, and the same first-slash
|
||||
# split the resolve below performs recovers the pair. A pair
|
||||
# keeps the MOST RECENT truncation's shown count (a plain
|
||||
# truncation followed by a truncated range read → the range
|
||||
# read's).
|
||||
truncated_shown: dict[tuple[str, str], int] = {}
|
||||
for argument, shown, _total in holder.read_truncations:
|
||||
a = argument.strip()
|
||||
if "/" in a:
|
||||
t_source, _, t_path = a.partition("/")
|
||||
truncated_shown[(t_source, t_path)] = shown
|
||||
# The raw-arg fast path handles the "plain read + already in
|
||||
# context" rows with no DB lookup (the pre-phase-131 check —
|
||||
# the split pair of a combined identity that is in
|
||||
# full-text context is in `known` as-is, the resolve below
|
||||
# would find the same document — now split on the truncated
|
||||
# set): truncated earlier → the :data:`REREAD_TRUNCATED`
|
||||
# teaching (the rest is NOT in the prompt — point at the
|
||||
# grep → read(offset/limit) workflow); never truncated →
|
||||
# :data:`ALREADY_IN_CONTEXT` (byte-identical to pre-phase-131).
|
||||
if not ranged and "/" in arg:
|
||||
src, _, p = arg.partition("/")
|
||||
if (src, p) in known:
|
||||
if (src, p) in truncated_shown:
|
||||
return REREAD_TRUNCATED.format(
|
||||
shown=truncated_shown[(src, p)], source=src, path=p
|
||||
)
|
||||
return ALREADY_IN_CONTEXT
|
||||
doc, _source, _path = _resolve_path(db, arg)
|
||||
if doc is None:
|
||||
@@ -1370,7 +1574,40 @@ def _execute_tool(
|
||||
# a path-like argument that matches an indexed document's
|
||||
# path gets the "did you mean …?" teaching (phase 72).
|
||||
return _no_document_refusal(db, arg)
|
||||
holder.read_docs.append(doc)
|
||||
# Range read + already in context (phase 131, task 02 — the
|
||||
# dedupe matrix's range rows; the truncated-set check needs
|
||||
# the RESOLVED pair, so it runs here, after the resolve, where
|
||||
# ``doc`` is known): the truncated-then-range case PROCEEDS
|
||||
# (the stuck-scenario fix — the range the model names is
|
||||
# exactly the part it does NOT have); a range of a fully-read
|
||||
# document is :data:`ALREADY_IN_CONTEXT` (the range is a
|
||||
# subset of the text already in the prompt).
|
||||
if (
|
||||
ranged
|
||||
and (doc.source, doc.path) in known
|
||||
and (doc.source, doc.path) not in truncated_shown
|
||||
):
|
||||
return ALREADY_IN_CONTEXT
|
||||
lines = doc.content.splitlines()
|
||||
total_lines = len(lines)
|
||||
if ranged and first > total_lines:
|
||||
# The offset names a line past the end — teaching refusal:
|
||||
# the true line count named, grep reports the exact line
|
||||
# numbers (the locator that got the model here). Counts in
|
||||
# nothing, consumes a round; the document is NOT added to
|
||||
# the context, so a corrected retry still proceeds.
|
||||
return OFFSET_PAST_END.format(
|
||||
source=doc.source,
|
||||
path=doc.path,
|
||||
total_lines=total_lines,
|
||||
offset=first,
|
||||
)
|
||||
# The checks above refused every duplicate that must be
|
||||
# refused — append at most ONCE: a first read (plain or range)
|
||||
# appends; the range read of an earlier TRUNCATED read of the
|
||||
# same document (the row that proceeded) does not re-append.
|
||||
if (doc.source, doc.path) not in known:
|
||||
holder.read_docs.append(doc)
|
||||
holder.tool_calls += 1
|
||||
# Phase 122 (task 05): an image document's content IS the vision
|
||||
# description — the marker line (a third line between the
|
||||
@@ -1379,43 +1616,81 @@ def _execute_tool(
|
||||
# stays byte-identical to pre-122.
|
||||
marker = f"{IMAGE_DOC_MARKER}\n" if doc.is_image else ""
|
||||
cap = settings.read_max_chars
|
||||
if len(doc.content) > cap:
|
||||
# Phase 95 (owner permission 2026-09-10, ``TODO.md`` L5): the
|
||||
# read cap — cut at the cap and tell the model the truth: the
|
||||
# shared marker + the pinned notice name how much more exists
|
||||
# and point at ``grep`` (which searches the WHOLE document).
|
||||
# A truncated read is still a successful call (both counters
|
||||
# above already bumped); the holder tuple is the truncation
|
||||
# signal only, so the loop can yield a ToolResultPiece.
|
||||
# ``raw_path`` (a str here — ``arg`` is non-empty only when it
|
||||
# was) is the exact argument the matching SSE ``tool`` frame
|
||||
# carries, so the UI can match the two (task 02); the cast
|
||||
# keeps pyright honest about the holder tuple's first element.
|
||||
holder.read_truncations.append(
|
||||
(cast("str", raw_path), cap, len(doc.content))
|
||||
)
|
||||
# Phase 106 (D5): the date rides every document the model
|
||||
# sees — the ``read`` result's SECOND line; the FIRST line
|
||||
# stays ``Document {source}/{path}:`` BYTE-IDENTICAL (the
|
||||
# E2E mock's ``_READ_RESULT_PREFIX`` header contract).
|
||||
if not ranged:
|
||||
# Plain read — the pre-phase-131 bytes, untouched.
|
||||
if len(doc.content) > cap:
|
||||
# Phase 95 (owner permission 2026-09-10, ``TODO.md``
|
||||
# L5): the read cap — cut at the cap and tell the model
|
||||
# the truth: the shared marker + the pinned notice name
|
||||
# how much more exists and teach the grep →
|
||||
# read(offset/limit) workflow (phase 131 — the notice
|
||||
# itself names the follow-up call). A truncated read is
|
||||
# still a successful call (both counters above already
|
||||
# bumped); the holder tuple is the truncation signal
|
||||
# only, so the loop can yield a ToolResultPiece.
|
||||
# ``raw_path`` (a str here — ``arg`` is non-empty only
|
||||
# when it was) is the exact argument the matching SSE
|
||||
# ``tool`` frame carries, so the UI can match the two
|
||||
# (task 02); the cast keeps pyright honest about the
|
||||
# holder tuple's first element.
|
||||
holder.read_truncations.append(
|
||||
(cast("str", raw_path), cap, len(doc.content))
|
||||
)
|
||||
# Phase 106 (D5): the date rides every document the
|
||||
# model sees — the ``read`` result's SECOND line; the
|
||||
# FIRST line stays ``Document {source}/{path}:``
|
||||
# BYTE-IDENTICAL (the E2E mock's ``_READ_RESULT_PREFIX``
|
||||
# header contract).
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{marker}"
|
||||
f"{doc.content[:cap]}\n"
|
||||
f"{TRUNCATION_MARKER}\n"
|
||||
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))}"
|
||||
)
|
||||
# At or under the cap: the pre-phase-95 result plus the
|
||||
# phase-106 D5 date line (first line byte-identical — the
|
||||
# mock's header contract; no truncation marker, no notice,
|
||||
# no holder entry, no ToolResultPiece) — and, phase 122,
|
||||
# the image-document marker line for image docs only.
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{marker}"
|
||||
f"{doc.content[:cap]}\n"
|
||||
f"{doc.content}"
|
||||
)
|
||||
# Range read (phase 131, task 02): EXACTLY the named lines —
|
||||
# the TRUE extent on the third line (``end`` clamps to the
|
||||
# last line: an overshooting ``limit`` reads to the end, never
|
||||
# past it; ``offset`` without ``limit`` reads to the end), the
|
||||
# image-doc marker (when one) exactly where the plain read
|
||||
# puts it (after the header lines), the cap applied to the
|
||||
# SLICE.
|
||||
end = total_lines if limit is None else min(first + limit - 1, total_lines)
|
||||
slice_text = "\n".join(lines[first - 1 : end])
|
||||
if len(slice_text) > cap:
|
||||
# The cap on the slice: the same marker + the pinned notice
|
||||
# (``{total}`` = the slice's length — how much of THIS read
|
||||
# was cut); the holder tuple's ``chars_total`` is the
|
||||
# slice's length, NOT the document's (documented on
|
||||
# :class:`AgentHolder.read_truncations`).
|
||||
holder.read_truncations.append(
|
||||
(cast("str", raw_path), cap, len(slice_text))
|
||||
)
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"lines: {first}-{end} of {total_lines}\n"
|
||||
f"{marker}{slice_text[:cap]}\n"
|
||||
f"{TRUNCATION_MARKER}\n"
|
||||
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))}"
|
||||
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))}"
|
||||
)
|
||||
# At or under the cap: the pre-phase-95 result plus the
|
||||
# phase-106 D5 date line (first line byte-identical — the
|
||||
# mock's header contract; no truncation marker, no notice, no
|
||||
# holder entry, no ToolResultPiece) — and, phase 122, the
|
||||
# image-document marker line for image docs only.
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{marker}"
|
||||
f"{doc.content}"
|
||||
f"lines: {first}-{end} of {total_lines}\n"
|
||||
f"{marker}{slice_text}"
|
||||
)
|
||||
if call.name == "show_image":
|
||||
# Phase 124 (task 02; LOCKED A3/A5): the 4th agent tool — the
|
||||
|
||||
@@ -197,11 +197,39 @@ class ChatToolEvent(BaseModel):
|
||||
``done`` shapes are unchanged — the read document is reflected in
|
||||
``done.sources`` instead (a grep adds no source: it is a locator,
|
||||
locked A5).
|
||||
|
||||
Phase 131 (task 04): ``offset`` / ``limit`` — the LINE RANGE a
|
||||
``read`` call requested (the optional 1-based first line + line
|
||||
count, PLAN §6's phase-131 revision). Additive in the
|
||||
:class:`ChatDoneEvent.answer_image` style: absent (``None``) on
|
||||
every non-range call — ``ls`` / ``grep`` / plain ``read``, and
|
||||
every pre-phase frame — where BOTH keys are DROPPED on
|
||||
serialization (ABSENT, never ``null``), so those frames stay
|
||||
byte-identical to pre-phase (the story E2E suites pin them as
|
||||
exact dicts). Only a valid range read carries them (a bad range
|
||||
value is a refused call — the refusal surfaces as the round's
|
||||
result, and the frame carries no range). The client ignores the
|
||||
extra fields (PLAN §4); they exist so the wire — and the phase-131
|
||||
story E2E — can assert the range the model requested.
|
||||
"""
|
||||
|
||||
type: Literal["tool"] = "tool"
|
||||
name: str # "ls" | "read" | "grep" (whatever AGENT_TOOLS names)
|
||||
argument: str | None = None # the single string argument passed, or null
|
||||
offset: int | None = None # read only: the 1-based first line requested
|
||||
limit: int | None = None # read only: the line count requested
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
data = handler(self)
|
||||
# Absent range → ABSENT keys (never null): the non-range frame
|
||||
# stays byte-identical to pre-phase-131 (the omission pattern
|
||||
# of SourceRef.image_url / ChatDoneEvent.answer_image).
|
||||
if self.offset is None:
|
||||
data.pop("offset", None)
|
||||
if self.limit is None:
|
||||
data.pop("limit", None)
|
||||
return data
|
||||
|
||||
|
||||
class ChatToolResultEvent(BaseModel):
|
||||
|
||||
Reference in New Issue
Block a user