phase: 131_read_line_ranges
All 44 progress lines are pure dots, final line at 100%, exit 0. Everything is verified. Final report: **Phase 131 — final verification pass (all 4 tasks already complete in working tree)** - **Verified implemented:** `read(offset, limit)` schema + teaching descriptions; rewritten truncation notice; full 6-row dedupe matrix + range validation + `lines: a-b of N` header + slice cap in `_execute_tool`; wire-level `offset`/`limit` on `ChatToolEvent` (absent keys dropped → non-range frames byte-identical); mock-LLM `read the lines` flow; handbook repro (wording + line numbers derived from the real PDF at test time) - **Tests:** `uv run pytest --no-cov` → 3097 passed, exit 0; `uv run pytest --cov=app` → **99% on app/** (>90%); `uv run pytest tests/e2e/test_read_line_ranges.py -v --no-cov` → 2/2 in isolation (47.6 s); `tests/integration/test_read_line_ranges_handbook.py` → 2/2 (3.5 s); 10-file targeted E2E batch pinning pre-phase surfaces (grep/ls/plain-read/ALREADY_IN_CONTEXT/truncation) → 31/31; `ruff check .` clean; `pyright` 0 errors - **Completion criteria:** all 5 green — (1) exact range lines + `lines:` header, plain reads byte-identical (existing pins pass unchanged) ✅ (2) truncated-then-range succeeds end-to-end on the real handbook; other re-reads get teaching refusals ✅ (3) find-traps answered from real PDF text, no hardcoded wording ✅ (4) coverage + isolated E2E ✅ (5) no behavior change in completed phases ✅ - **Docs:** README + AGENTS reviewed in full. Fixed 3 inconsistencies: README missing `BOR_HISTORY_MAX_TURNS`/`BOR_HISTORY_MAX_CHARS` rows (pre-existing phase-74 gap — env-tunable, unit-pinned, undocumented); `.env.example` missing the same two vars (added); `.env.example` stale `BOR_READ_MAX_CHARS` comment (still said "grep-pointer notice (phase 95)" — updated to the phase-131 workflow notice). AGENTS.md verified consistent — no changes needed. All 67 `.env.example` vars documented, all defaults match `app/config.py`, pages/auth tables match live routes - **Note:** PLAN.md §7.1 still states the retired 46rem chat column (superseded by phase 100, which AGENTS.md rule 5 records correctly) — PLAN.md is off-limits for this pass, flagged only - **Next pending phase:** `132_image_upload_feedback`
This commit is contained in:
+755
-28
@@ -17,7 +17,18 @@ unknown-source refusal that counts nothing, the NOT-A-FOLDER teaching
|
||||
with the parent's subfolders, the 50-file cap + grep-pointer note),
|
||||
``read`` on the canonical combined form (split at the
|
||||
FIRST slash, full content, the bare-source-name refusal, the
|
||||
already-in-context dedupe, missing-args refusals), the phase-68 ``grep``
|
||||
already-in-context dedupe, missing-args refusals), the phase-131
|
||||
(task 02) range-read execution (the optional ``offset``/``limit``:
|
||||
:func:`agent._read_range_args` validation — whole ints ≥ 1, bools
|
||||
excluded — with the pinned :data:`agent.BAD_RANGE_ARGS` echo, the
|
||||
six-row dedupe matrix (a plain re-read of a TRUNCATED doc gets the
|
||||
pinned :data:`agent.REREAD_TRUNCATED` teaching; the truncated-then-
|
||||
range row PROCEEDS — the stuck-scenario fix —; every other re-read of
|
||||
a fully-read doc stays ``ALREADY_IN_CONTEXT``),
|
||||
:data:`agent.OFFSET_PAST_END` (the true line count named), the slice
|
||||
math + the ``lines: a-b of N`` header shape (plain reads byte-
|
||||
identical to pre-phase-131), the cap on the slice with the slice-
|
||||
length tuple ``chars_total``), the phase-68 ``grep``
|
||||
contract under its new name (the locked A5 pins: fixed substring,
|
||||
case-insensitive, 20-cap in catalog order, 200-char lines, locator-only
|
||||
— ``read_docs`` untouched, no-match lines counted), the round cap
|
||||
@@ -227,9 +238,12 @@ def test_agent_tools_names_and_parameters() -> None:
|
||||
# the prompt yet); read adds one of them (or any other document)
|
||||
# to the context; only an already-read document is refused. The
|
||||
# combined-identity handoff to the `ls` output / the <documents>
|
||||
# blocks stays; the truncation-notice paragraph (phase 95) and
|
||||
# the one-call-at-a-time discipline clause (task 05) survive the
|
||||
# rewrite byte-identical.
|
||||
# blocks stays; the one-call-at-a-time discipline clause (task
|
||||
# 05) survives byte-identical. Phase 131 (task 01): the old
|
||||
# truncation dead-end paragraph ("Follow it and use `grep`
|
||||
# (pattern) to locate the rest") is replaced by the working
|
||||
# workflow — grep reports line numbers, then read the part with
|
||||
# `offset` (1-based first line) + `limit` (number of lines).
|
||||
assert read["description"] == (
|
||||
"The <documents> section shows the SUMMARIES of the "
|
||||
"top-ranked documents — their full texts are NOT in "
|
||||
@@ -240,17 +254,20 @@ def test_agent_tools_names_and_parameters() -> None:
|
||||
"document you have already read — its full text is "
|
||||
"already in your prompt. Very large documents are "
|
||||
"truncated: you receive the first part plus a TRUNCATED "
|
||||
"notice naming how many more characters exist — the "
|
||||
"notice is authoritative, the document did NOT end "
|
||||
"where it stopped. Follow it and use `grep` (pattern) "
|
||||
"to locate the rest — it searches the whole document. "
|
||||
"notice naming how many more characters exist. To read "
|
||||
"the rest: use `grep` (pattern) to find where your text "
|
||||
"is — it reports line numbers — then call `read` again "
|
||||
"on the same document with `offset` (1-based first line) "
|
||||
"and `limit` (number of lines) to read exactly that "
|
||||
"part. Do not re-read a document you have already read "
|
||||
"in full — its full text is already in your prompt. "
|
||||
"Call one tool at a time — wait for this result before "
|
||||
"your next call."
|
||||
)
|
||||
# The byte-preserved contracts inside the rewrite, pinned as
|
||||
# substrings: the combined source/path identity, the
|
||||
# truncation-notice paragraph, the one-call-at-a-time
|
||||
# discipline sentence.
|
||||
# phase-131 grep → read(offset, limit) workflow paragraph, the
|
||||
# one-call-at-a-time discipline sentence.
|
||||
assert (
|
||||
"by its combined `source/path` string, exactly as shown in "
|
||||
"the `ls` output or the <documents> blocks"
|
||||
@@ -258,10 +275,11 @@ def test_agent_tools_names_and_parameters() -> None:
|
||||
assert (
|
||||
"Very large documents are truncated: you receive the first "
|
||||
"part plus a TRUNCATED notice naming how many more "
|
||||
"characters exist — the notice is authoritative, the "
|
||||
"document did NOT end where it stopped. Follow it and use "
|
||||
"`grep` (pattern) to locate the rest — it searches the "
|
||||
"whole document."
|
||||
"characters exist. To read the rest: use `grep` (pattern) "
|
||||
"to find where your text is — it reports line numbers — "
|
||||
"then call `read` again on the same document with "
|
||||
"`offset` (1-based first line) and `limit` (number of "
|
||||
"lines) to read exactly that part."
|
||||
) in read["description"]
|
||||
assert (
|
||||
"Call one tool at a time — wait for this result before "
|
||||
@@ -269,9 +287,30 @@ def test_agent_tools_names_and_parameters() -> None:
|
||||
) in read["description"]
|
||||
read_params = read["parameters"]
|
||||
assert read_params["type"] == "object"
|
||||
# Phase 131 (task 01, LOCKED A2): offset/limit are OPTIONAL —
|
||||
# ``required`` stays ``["path"]``.
|
||||
assert read_params["required"] == ["path"]
|
||||
assert set(read_params["properties"]) == {"path"}
|
||||
assert set(read_params["properties"]) == {"path", "offset", "limit"}
|
||||
assert read_params["properties"]["path"]["type"] == "string"
|
||||
# The line-range parameters (phase 131, task 01, LOCKED A2):
|
||||
# BOTH integer, 1-based (``offset`` = first line, ``limit`` =
|
||||
# line count) — the pi-harness ``read`` shape the model was
|
||||
# trained on. The descriptions carry the grep → read(offset,
|
||||
# limit) teaching (pinned copy).
|
||||
assert read_params["properties"]["offset"]["type"] == "integer"
|
||||
assert read_params["properties"]["limit"]["type"] == "integer"
|
||||
assert read_params["properties"]["offset"]["description"] == (
|
||||
"The first line to read, 1-based. Omit it to read from the "
|
||||
"beginning of the document. Use it to read a specific part "
|
||||
"of a large document — grep reports the line numbers of "
|
||||
"what you found."
|
||||
)
|
||||
assert read_params["properties"]["limit"]["description"] == (
|
||||
"The maximum number of lines to read, starting at offset. "
|
||||
"Omit it to read as many lines as fit under the character "
|
||||
"cap. Keep it small — you are reading a part, not the "
|
||||
"document."
|
||||
)
|
||||
# The combined source/path string is the canonical document identity
|
||||
# (phase 70) — the description pins it with a worked example; the
|
||||
# bare-path contract is stated up front (phase 72, task 02). Phase
|
||||
@@ -303,13 +342,14 @@ def test_agent_tools_names_and_parameters() -> None:
|
||||
"Search the indexed documents for an exact string "
|
||||
"(case-insensitive) and return up to 20 matching lines "
|
||||
"as `source/path:line: text` — a locator, not a "
|
||||
"context-adder: read the winner with `read`. The "
|
||||
"pattern is a plain substring, NEVER a regex — if a "
|
||||
"pattern with regex syntax (like '.*' or '\\.') comes "
|
||||
"back with no matches, retry with the plain text you "
|
||||
"expect to see. For a normal search pass ONLY `pattern` "
|
||||
"— it searches every document and that is how you "
|
||||
"search the knowledge base; never pass a source name "
|
||||
"context-adder: read the winner with `read` — for a "
|
||||
"large document, use the reported line number as "
|
||||
"`offset`. The pattern is a plain substring, NEVER a "
|
||||
"regex — if a pattern with regex syntax (like '.*' or "
|
||||
"'\\.') comes back with no matches, retry with the plain "
|
||||
"text you expect to see. For a normal search pass ONLY "
|
||||
"`pattern` — it searches every document and that is how "
|
||||
"you search the knowledge base; never pass a source name "
|
||||
"as `path` (a source name is not a document). Call one "
|
||||
"tool at a time — wait for this result before your "
|
||||
"next call."
|
||||
@@ -1956,16 +1996,27 @@ def test_read_at_cap_plus_one_truncates_with_marker_and_notice(
|
||||
|
||||
|
||||
def test_read_truncation_notice_is_pinned() -> None:
|
||||
"""Phase 95: the notice copy is pinned — it names the true length
|
||||
(``{total}``), the cap kept (``{shown}``), states the rest is NOT
|
||||
shown (the document did not end where it stopped), and points at
|
||||
``grep`` (which searches the whole document)."""
|
||||
"""Phase 95 (REWRITTEN phase 131, task 01): the notice copy is
|
||||
pinned — it names the true length (``{total}``), the cap kept
|
||||
(``{shown}``), states the rest is NOT shown (the document did
|
||||
not end where it stopped), and teaches the working workflow —
|
||||
the old dead-end is gone: ``grep`` reports line numbers, then
|
||||
read that part with ``offset`` / ``limit`` (BOTH parameters
|
||||
named in the notice itself)."""
|
||||
assert READ_TRUNCATION_NOTICE.format(shown=100, total=250) == (
|
||||
"TRUNCATED — this document is 250 characters; only the first "
|
||||
"100 are in your context. The rest is NOT shown. Use grep "
|
||||
"(pattern) to locate what you need — grep searches the whole "
|
||||
"document."
|
||||
"(pattern) to find what you need — it reports line numbers — "
|
||||
"then read that part with `offset` (1-based first line) and "
|
||||
"`limit` (number of lines)."
|
||||
)
|
||||
# Both line-range parameters are named in the notice — the model
|
||||
# stuck at the truncation sees the exact follow-up call.
|
||||
assert "`offset`" in READ_TRUNCATION_NOTICE
|
||||
assert "`limit`" in READ_TRUNCATION_NOTICE
|
||||
# The old dead-end sentence (no way to act on the line number)
|
||||
# is gone.
|
||||
assert "to locate what you need" not in READ_TRUNCATION_NOTICE
|
||||
|
||||
|
||||
def test_run_agent_yields_tool_result_after_tool_frame_before_next_round(
|
||||
@@ -2081,6 +2132,682 @@ def test_read_truncation_paths_with_suggested_seed(
|
||||
assert holder2.tool_calls == 0 and holder2.read_docs == []
|
||||
|
||||
|
||||
# ---------- phase 131 (task 02): the range-read execution (offset/limit) ----------
|
||||
|
||||
|
||||
def _lines_doc(source: str, path: str, n_lines: int, prefix: str = "line ") -> Document:
|
||||
"""A fixture document of EXACTLY *n_lines* lines (``prefix{i}``)."""
|
||||
content = "\n".join(f"{prefix}{i}" for i in range(1, n_lines + 1))
|
||||
return _doc(source, path, path, content)
|
||||
|
||||
|
||||
def test_bad_range_args_is_pinned() -> None:
|
||||
"""Phase 131 (task 02): the range-argument validation refusal is
|
||||
pinned — the contract (whole line numbers ≥ 1) with the bad values
|
||||
echoed (``!r`` — an absent value prints as ``None``)."""
|
||||
assert agent.BAD_RANGE_ARGS.format(offset=2.5, limit=None) == (
|
||||
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
|
||||
"(you passed offset=2.5, limit=None)."
|
||||
)
|
||||
assert agent.BAD_RANGE_ARGS.format(offset=None, limit="10") == (
|
||||
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
|
||||
"(you passed offset=None, limit='10')."
|
||||
)
|
||||
assert agent.BAD_RANGE_ARGS.format(offset=True, limit=None) == (
|
||||
"`offset` and `limit` are whole line numbers — pass integers ≥ 1 "
|
||||
"(you passed offset=True, limit=None)."
|
||||
)
|
||||
|
||||
|
||||
def test_reread_truncated_is_pinned() -> None:
|
||||
"""Phase 131 (task 02): the plain re-read teaching for a TRUNCATED
|
||||
document is pinned — the shown count, the resolved identity, and
|
||||
the working workflow (grep reports line numbers → read that part
|
||||
with ``offset`` + ``limit``)."""
|
||||
assert agent.REREAD_TRUNCATED.format(shown=100, source="S", path="big.md") == (
|
||||
"You already have the first 100 characters of S/big.md in your "
|
||||
"context. To read the rest, find where your text is with grep "
|
||||
"(it reports line numbers), then call read with `offset` "
|
||||
"(1-based first line) and `limit` (lines)."
|
||||
)
|
||||
|
||||
|
||||
def test_offset_past_end_is_pinned() -> None:
|
||||
"""Phase 131 (task 02): the offset-past-end teaching is pinned —
|
||||
the RESOLVED identity, the true line count, the grep pointer."""
|
||||
assert agent.OFFSET_PAST_END.format(
|
||||
source="S", path="a.md", total_lines=10, offset=11
|
||||
) == (
|
||||
"S/a.md is 10 lines long — `offset` 11 is past the end. Use a "
|
||||
"smaller offset (grep reports the exact line numbers)."
|
||||
)
|
||||
|
||||
|
||||
def test_read_range_args_plain_and_valid() -> None:
|
||||
"""Phase 131 (task 02): the pure validator — both absent is a plain
|
||||
read; valid ints pass through (either one-sided is ok: ``offset``
|
||||
without ``limit`` reads to the end, ``limit`` without ``offset``
|
||||
counts from the start — the caller normalizes)."""
|
||||
assert agent._read_range_args(None, None) == (True, "", None, None)
|
||||
assert agent._read_range_args(1, None) == (True, "", 1, None)
|
||||
assert agent._read_range_args(None, 7) == (True, "", None, 7)
|
||||
assert agent._read_range_args(3, 42) == (True, "", 3, 42)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("offset", "limit"),
|
||||
[
|
||||
(2.5, None), # a float
|
||||
("3", None), # a string
|
||||
(True, None), # a bool (an int subclass — the house guard excludes it)
|
||||
(0, None), # zero
|
||||
(-1, None), # a negative
|
||||
(None, 2.5), # a float limit
|
||||
(None, "10"), # a string limit
|
||||
(None, False), # a bool limit
|
||||
(None, 0), # a zero limit
|
||||
(None, -3), # a negative limit
|
||||
],
|
||||
)
|
||||
def test_read_range_args_bad_values_are_refused(
|
||||
offset: Any, limit: Any
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a PRESENT bad value (not a whole int ≥ 1 —
|
||||
the house ``isinstance(x, int) and not isinstance(x, bool)`` guard,
|
||||
or ``< 1``) refuses with the values echoed; the range comes back
|
||||
unusable (``None``/``None``)."""
|
||||
ok, line, got_offset, got_limit = agent._read_range_args(offset, limit)
|
||||
assert not ok
|
||||
assert got_offset is None and got_limit is None
|
||||
assert line == agent.BAD_RANGE_ARGS.format(offset=offset, limit=limit)
|
||||
|
||||
|
||||
def test_bad_range_args_refused_before_any_lookup(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): the validation refusal fires in the flow —
|
||||
BEFORE the dedupe and the DB resolve (a float ``offset`` with an
|
||||
unknown path still gets the range refusal, not the no-document
|
||||
line): counts in nothing, consumes a round, the tools stay
|
||||
offered."""
|
||||
|
||||
def _boom(*_a: Any, **_k: Any) -> None:
|
||||
raise AssertionError("no DB lookup for a bad range argument")
|
||||
|
||||
monkeypatch.setattr(agent, "find_document", _boom)
|
||||
monkeypatch.setattr(agent, "all_documents", _boom)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/ghost.md", "offset": 2.5}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert holder.read_docs == [] and holder.tool_calls == 0
|
||||
assert llm.requests[1][0][3]["content"] == agent.BAD_RANGE_ARGS.format(
|
||||
offset=2.5, limit=None
|
||||
)
|
||||
assert llm.requests[1][1] == AGENT_TOOLS
|
||||
|
||||
|
||||
def test_bad_range_args_beats_the_dedupe(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a bad range argument on a document ALREADY
|
||||
IN CONTEXT is the range refusal, not ``ALREADY_IN_CONTEXT`` — the
|
||||
validation precedes the dedupe (a refusal either way: no count,
|
||||
no re-append)."""
|
||||
doc = _doc("S", "a.md", "A", "A-CONTENT")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/a.md"})],
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c2", name="read", arguments={"path": "S/a.md", "offset": 0}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
assert llm.requests[2][0][5]["content"] == agent.BAD_RANGE_ARGS.format(
|
||||
offset=0, limit=None
|
||||
)
|
||||
|
||||
|
||||
def test_offset_past_end_refused_naming_line_count(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): ``offset`` = total+1 on a 10-line doc →
|
||||
the pinned teaching (the true line count named, the grep pointer)
|
||||
— a refusal: nothing counted, the doc NOT added to the context (a
|
||||
corrected retry still proceeds)."""
|
||||
doc = _lines_doc("S", "ten.md", 10)
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/ten.md", "offset": 11}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert holder.read_docs == [] and holder.tool_calls == 0
|
||||
assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format(
|
||||
source="S", path="ten.md", total_lines=10, offset=11
|
||||
)
|
||||
|
||||
|
||||
def test_offset_past_end_on_one_line_doc(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a 1-line doc — ``offset=2`` is past the
|
||||
end (the true count is 1); ``offset=1`` with an overshooting
|
||||
``limit`` clamps to the single line."""
|
||||
doc = _doc("S", "one.md", "One", "only")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
# (a) offset past the end of the 1-line doc.
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/one.md", "offset": 2}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == agent.OFFSET_PAST_END.format(
|
||||
source="S", path="one.md", total_lines=1, offset=2
|
||||
)
|
||||
assert holder.read_docs == [] and holder.tool_calls == 0
|
||||
# (b) the same doc read from line 1, limit overshoots — clamped.
|
||||
holder2 = AgentHolder()
|
||||
llm2 = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/one.md", "offset": 1, "limit": 5},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm2, holder2, _settings()))
|
||||
assert llm2.requests[1][0][3]["content"] == (
|
||||
"Document S/one.md:\ndate: 2024-06-15\nlines: 1-1 of 1\nonly"
|
||||
)
|
||||
assert holder2.read_docs == [doc] and holder2.tool_calls == 1
|
||||
|
||||
|
||||
# --- the six-row dedupe matrix (one test per row) ---
|
||||
|
||||
|
||||
def test_matrix_row1_plain_read_unread_doc_reads_full(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 1: plain ``read(path)``, doc NOT in ``read_docs`` →
|
||||
the full read (truncate at the cap as today — the pre-phase-131
|
||||
path, no ``lines:`` header line)."""
|
||||
doc = _doc("S", "m1.md", "M1", "one\ntwo\nthree")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m1.md"})],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/m1.md:\ndate: 2024-06-15\none\ntwo\nthree"
|
||||
)
|
||||
|
||||
|
||||
def test_matrix_row2_plain_read_truncated_doc_gets_reread_teaching(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 2 (the stuck scenario's first hop): a plain re-read
|
||||
of a document whose first read TRUNCATED → the pinned
|
||||
``REREAD_TRUNCATED`` teaching (the shown count from the matching
|
||||
truncation tuple — the cap) — a refusal: no re-emit, no count,
|
||||
no second DB lookup (the raw-arg fast path fires pre-resolve)."""
|
||||
cap = 100
|
||||
doc = _lines_doc("S", "m2.md", 20) # 150 chars > the cap
|
||||
lookups: list[tuple[str, str]] = []
|
||||
|
||||
def _find(db: Any, source: str, path: str) -> Document | None:
|
||||
lookups.append((source, path))
|
||||
return doc if (source, path) == ("S", "m2.md") else None
|
||||
|
||||
monkeypatch.setattr(agent, "find_document", _find)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m2.md"})],
|
||||
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/m2.md"})],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
|
||||
# Round 1: the truncated plain read (the stuck trigger) — the
|
||||
# pre-phase-131 truncation bytes, the holder tuple recorded.
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/m2.md:\ndate: 2024-06-15\n"
|
||||
+ doc.content[:cap]
|
||||
+ "\n"
|
||||
+ TRUNCATION_MARKER
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))
|
||||
)
|
||||
assert holder.read_truncations == [("S/m2.md", cap, len(doc.content))]
|
||||
# Round 2: the plain re-read → the teaching (the shown count = the
|
||||
# cap from the matching tuple).
|
||||
assert llm.requests[2][0][5]["content"] == agent.REREAD_TRUNCATED.format(
|
||||
shown=cap, source="S", path="m2.md"
|
||||
)
|
||||
# A refusal: the counters stay at the first read's, no re-append,
|
||||
# no second lookup.
|
||||
assert holder.read_docs == [doc]
|
||||
assert holder.tool_calls == 1
|
||||
assert lookups == [("S", "m2.md")]
|
||||
|
||||
|
||||
def test_matrix_row3_plain_read_full_doc_stays_already_in_context(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 3: a plain re-read of a document read IN FULL (never
|
||||
truncated) → ``ALREADY_IN_CONTEXT`` — byte-identical to
|
||||
pre-phase-131 (the raw-arg fast path, no DB lookup)."""
|
||||
doc = _doc("S", "m3.md", "M3", "FULL-TEXT") # fits under any cap
|
||||
lookups: list[tuple[str, str]] = []
|
||||
|
||||
def _find(db: Any, source: str, path: str) -> Document | None:
|
||||
lookups.append((source, path))
|
||||
return doc if (source, path) == ("S", "m3.md") else None
|
||||
|
||||
monkeypatch.setattr(agent, "find_document", _find)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m3.md"})],
|
||||
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/m3.md"})],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
assert lookups == [("S", "m3.md")] # the re-read deduped pre-lookup
|
||||
|
||||
|
||||
def test_matrix_row4_range_read_truncated_doc_proceeds(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 4 (the stuck scenario's FIX): after a TRUNCATED plain
|
||||
read, a range read of the SAME document PROCEEDS — the ``lines:``
|
||||
header + exactly the named lines. The doc was appended ONCE (the
|
||||
known set prevents the re-append); the range read counts as a
|
||||
successful call; the slice fits, so no new truncation tuple."""
|
||||
cap = 100
|
||||
doc = _lines_doc("S", "m4.md", 20) # 150 chars > the cap
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m4.md"})],
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c2",
|
||||
name="read",
|
||||
arguments={"path": "S/m4.md", "offset": 5, "limit": 3},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
|
||||
# Round 1: the truncated plain read.
|
||||
assert llm.requests[1][0][3]["content"].endswith(
|
||||
"\n"
|
||||
+ TRUNCATION_MARKER
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))
|
||||
)
|
||||
# Round 2: the range read PROCEEDED — lines 5-7 of 20, exactly.
|
||||
assert llm.requests[2][0][5]["content"] == (
|
||||
"Document S/m4.md:\ndate: 2024-06-15\nlines: 5-7 of 20\n"
|
||||
"line 5\nline 6\nline 7"
|
||||
)
|
||||
assert holder.read_docs == [doc] # appended exactly once
|
||||
assert holder.tool_calls == 2 # both reads are successful calls
|
||||
assert holder.read_truncations == [("S/m4.md", cap, len(doc.content))]
|
||||
|
||||
|
||||
def test_matrix_row5_range_read_full_doc_stays_already_in_context(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 5: a range read of a document read IN FULL (never
|
||||
truncated) → ``ALREADY_IN_CONTEXT`` — the range is a SUBSET of the
|
||||
text already in the prompt (the matrix needs the resolved pair, so
|
||||
the check runs after the resolve)."""
|
||||
doc = _lines_doc("S", "m5.md", 5)
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/m5.md"})],
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c2",
|
||||
name="read",
|
||||
arguments={"path": "S/m5.md", "offset": 2, "limit": 2},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
assert holder.read_truncations == []
|
||||
|
||||
|
||||
def test_matrix_row6_range_read_unread_doc_proceeds(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Matrix row 6: a range read of a NOT-yet-read document PROCEEDS —
|
||||
the ``lines:`` header + the slice, the doc appended, the call
|
||||
counted (a first read, plain or range, behaves like a first read)."""
|
||||
doc = _lines_doc("S", "m6.md", 8)
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/m6.md", "offset": 3, "limit": 2},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/m6.md:\ndate: 2024-06-15\nlines: 3-4 of 8\nline 3\nline 4"
|
||||
)
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
|
||||
|
||||
# --- the slice math + the result shape ---
|
||||
|
||||
|
||||
def test_range_read_offset_one_no_limit_reads_whole_doc(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): ``offset=1`` without ``limit`` reads to the
|
||||
end of the document — the true extent ``1-N of N``, the whole
|
||||
content (the slice, not the plain-read shape)."""
|
||||
doc = _doc("S", "r1.md", "R1", "a\nb\nc")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/r1.md", "offset": 1}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/r1.md:\ndate: 2024-06-15\nlines: 1-3 of 3\na\nb\nc"
|
||||
)
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
|
||||
|
||||
def test_range_read_offset_at_total_reads_last_line(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): ``offset`` = the last line number →
|
||||
exactly the last line (``N-N of N``)."""
|
||||
doc = _doc("S", "r2.md", "R2", "a\nb\nc")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/r2.md", "offset": 3, "limit": 5},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/r2.md:\ndate: 2024-06-15\nlines: 3-3 of 3\nc"
|
||||
)
|
||||
|
||||
|
||||
def test_range_read_limit_overshoot_clamps_to_total(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): an overshooting ``limit`` clamps ``end`` to
|
||||
the last line (reads to the end, never past it)."""
|
||||
doc = _doc("S", "r3.md", "R3", "a\nb\nc")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/r3.md", "offset": 2, "limit": 10},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/r3.md:\ndate: 2024-06-15\nlines: 2-3 of 3\nb\nc"
|
||||
)
|
||||
|
||||
|
||||
def test_range_read_limit_without_offset_counts_from_start(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): ``limit`` WITHOUT ``offset`` is a line count
|
||||
from the start of the document (offset = 1 — the helper's
|
||||
documented one-sided semantics)."""
|
||||
doc = _doc("S", "r4.md", "R4", "a\nb\nc")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/r4.md", "limit": 2}
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/r4.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb"
|
||||
)
|
||||
|
||||
|
||||
def test_range_read_image_doc_carries_marker_after_lines_line(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a range read of a standalone-IMAGE document
|
||||
carries the :data:`agent.IMAGE_DOC_MARKER` line exactly where the
|
||||
plain read puts it — after the header lines (here, after
|
||||
``lines:``)."""
|
||||
doc = _doc("S", "img.png", "Img", "IMG-DESC-OF-LINE-1\nline-2-text")
|
||||
doc.is_image = True
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/img.png", "offset": 2, "limit": 1},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/img.png:\ndate: 2024-06-15\nlines: 2-2 of 2\n"
|
||||
f"{agent.IMAGE_DOC_MARKER}\n"
|
||||
"line-2-text"
|
||||
)
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
|
||||
|
||||
def test_range_read_slice_over_cap_truncates_with_slice_length_total(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): the cap applies to the SLICE — an over-cap
|
||||
slice is cut at the cap + the shared marker + the pinned notice
|
||||
(``{total}`` = the SLICE's length), and the holder tuple's
|
||||
``chars_total`` is the slice's length, NOT the document's (the
|
||||
loop still yields one ToolResultPiece for it)."""
|
||||
cap = 20
|
||||
lines = [f"L{i:02d}" + "x" * 30 for i in range(1, 11)] # 10 lines × 33 chars
|
||||
content = "\n".join(lines)
|
||||
doc = _doc("S", "cap.md", "Cap", content)
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/cap.md", "offset": 5, "limit": 6},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
out = asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
|
||||
# The slice: lines 5-10 (6 × 33 chars + 5 newlines = 203 > cap).
|
||||
slice_text = "\n".join(lines[4:10])
|
||||
assert len(slice_text) == 203
|
||||
assert len(content) == 339 # the document is much bigger
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/cap.md:\ndate: 2024-06-15\nlines: 5-10 of 10\n"
|
||||
+ slice_text[:cap]
|
||||
+ "\n"
|
||||
+ TRUNCATION_MARKER
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
|
||||
)
|
||||
# The tuple: the argument, the cap kept, the SLICE's length (203,
|
||||
# not the document's 339).
|
||||
assert holder.read_truncations == [("S/cap.md", cap, 203)]
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
# The loop yields the ToolResultPiece with the slice's total.
|
||||
pieces = [p for p in out if isinstance(p, ToolResultPiece)]
|
||||
assert len(pieces) == 1
|
||||
assert pieces[0].argument == "S/cap.md"
|
||||
assert pieces[0].chars_shown == cap
|
||||
assert pieces[0].chars_total == 203
|
||||
|
||||
|
||||
def test_range_read_truncated_then_plain_reread_teaching_then_range_proceeds(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a TRUNCATED range read (offset=1, no limit
|
||||
— the whole 339-char doc under a 20-char cap) records the slice
|
||||
tuple; a plain re-read then gets ``REREAD_TRUNCATED`` with the
|
||||
shown count from THAT (range) tuple; a subsequent range read still
|
||||
PROCEEDS (truncated-then-range, regardless of which read
|
||||
truncated)."""
|
||||
cap = 20
|
||||
lines = [f"L{i:02d}" + "x" * 30 for i in range(1, 11)]
|
||||
content = "\n".join(lines)
|
||||
doc = _doc("S", "t.md", "T", content)
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1", name="read", arguments={"path": "S/t.md", "offset": 1}
|
||||
)
|
||||
],
|
||||
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/t.md"})],
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c3",
|
||||
name="read",
|
||||
arguments={"path": "S/t.md", "offset": 5, "limit": 2},
|
||||
)
|
||||
],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings(read_max_chars=cap)))
|
||||
# Round 1: the truncated range read (the whole doc, offset=1) —
|
||||
# the slice is the whole doc here (339 chars), the tuple's total
|
||||
# is the slice's length.
|
||||
assert llm.requests[1][0][3]["content"].startswith(
|
||||
"Document S/t.md:\ndate: 2024-06-15\nlines: 1-10 of 10\n"
|
||||
)
|
||||
assert llm.requests[1][0][3]["content"].endswith(
|
||||
"\n"
|
||||
+ TRUNCATION_MARKER
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=339)
|
||||
)
|
||||
# Round 2: the plain re-read → the teaching, shown from the (range)
|
||||
# truncation tuple.
|
||||
assert llm.requests[2][0][5]["content"] == agent.REREAD_TRUNCATED.format(
|
||||
shown=cap, source="S", path="t.md"
|
||||
)
|
||||
# Round 3: the range read of the truncated doc PROCEEDS — lines 5-6
|
||||
# (66 + 1 = 67 chars > the cap → truncated again; the tuple's total
|
||||
# is the slice's length, the document's is irrelevant).
|
||||
slice_text = "\n".join(lines[4:6])
|
||||
assert llm.requests[3][0][7]["content"] == (
|
||||
"Document S/t.md:\ndate: 2024-06-15\nlines: 5-6 of 10\n"
|
||||
+ slice_text[:cap]
|
||||
+ "\n"
|
||||
+ TRUNCATION_MARKER
|
||||
+ "\n"
|
||||
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
|
||||
)
|
||||
assert holder.read_truncations == [("S/t.md", cap, 339), ("S/t.md", cap, 67)]
|
||||
assert holder.read_docs == [doc] # appended exactly once, still
|
||||
assert holder.tool_calls == 2 # the refusal counted nothing
|
||||
|
||||
|
||||
def test_plain_read_after_range_read_is_already_in_context(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 131 (task 02): a range read appends the doc to
|
||||
``read_docs`` ONCE — so a later PLAIN read of the range-read
|
||||
(never-truncated) document is ``ALREADY_IN_CONTEXT`` (the model has
|
||||
committed to this doc; the ``lines:`` line told it what it holds)."""
|
||||
doc = _doc("S", "p.md", "P", "a\nb\nc")
|
||||
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
[
|
||||
ToolCallPiece(
|
||||
id="c1",
|
||||
name="read",
|
||||
arguments={"path": "S/p.md", "offset": 1, "limit": 2},
|
||||
)
|
||||
],
|
||||
[ToolCallPiece(id="c2", name="read", arguments={"path": "S/p.md"})],
|
||||
[StreamPiece("content", "ans")],
|
||||
)
|
||||
asyncio.run(_run(llm, holder, _settings()))
|
||||
assert llm.requests[1][0][3]["content"] == (
|
||||
"Document S/p.md:\ndate: 2024-06-15\nlines: 1-2 of 3\na\nb"
|
||||
)
|
||||
assert llm.requests[2][0][5]["content"] == agent.ALREADY_IN_CONTEXT
|
||||
assert holder.read_docs == [doc] and holder.tool_calls == 1
|
||||
|
||||
|
||||
def test_unknown_tool_name_refused(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
holder = AgentHolder()
|
||||
llm = ScriptedLLM(
|
||||
|
||||
Reference in New Issue
Block a user