phase: 141_read_range_dedupe_fix
All gates green and verified. Final report: **Phase 141 final verification pass — complete** - **Verified (tasks 01–04 already in `complete/`):** `AgentHolder.read_coverage` span recording (all 4 read outcomes, refusals record nothing); coverage dedupe matrix (range read of uncovered lines proceeds, fully-covered re-read → `ALREADY_IN_CONTEXT` with a now-true claim, partial plain re-read → `REREAD_PARTIAL` naming merged spans, `REREAD_TRUNCATED` retired, fast path retired per task 02); `TOOLS_SECTION` re-read sentence re-cut to coverage-true form (prompt lock re-pinned); owner-scenario integration repro (pure + truncated variants) and mock `read two ranges` flow + dedicated E2E — all match the task specs exactly; no code defects found. - **Tests/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL 99% (5498 stmts, `app/rag/agent.py` 99%) — run twice, both green; `uv run pytest tests/e2e/test_read_range_dedupe.py -v --no-cov` → 2 passed in isolation; `uv run ruff check . && uv run pyright` → All checks passed / 0 errors. - **Completion criteria:** (1) two different ranges both reach the model, re-covered re-read refused truthfully — ✅ (integration + E2E); (2) partial plain re-read gets the `offset`/`limit` teaching, false claim gone both directions — ✅; (3) plain-read + phase-131 result shapes byte-identical, existing pins pass unchanged — ✅; (4) all gate commands green, >90% — ✅; (5) no behavior change in completed phases (grep/ls, truncation notice, `read_docs`/`read_truncations` signals) — ✅ (whole project suite green). - **Documentation:** README.md — fixed pre-existing phase-140 gap: `BOR_PDF_PICTORIAL` / `BOR_PDF_PICTORIAL_MIN_CHARS` were in `.env.example`/Settings but undocumented (README claims every `.env.example` var is covered); added the pictorial-PDF-pass bullet to the PDF section + two config-reference rows. AGENTS.md — verified consistent (rules, quick-ref commands, phase refs all check out) — no changes needed. - **Notable:** overview's "fast path survives for holder-state rows" note is superseded by task 02's authoritative decision (every dedupe row resolves — "full" needs the true line count); implementation follows task 02. No phase files touched; changes left in the working tree for the harness commit. - **Next pending phase:** `142_kb_overview_freshness`.
This commit is contained in:
+270
-111
@@ -109,13 +109,21 @@ task 04):
|
||||
the last line), the cap applies to the SLICE (the holder tuple's
|
||||
``chars_total`` is then the slice's length, not the document's),
|
||||
and an ``offset`` past the end is the :data:`OFFSET_PAST_END`
|
||||
teaching. The dedupe matrix opens the ONE previously-dead case — a
|
||||
range read of a document whose earlier read was TRUNCATED (the
|
||||
stuck scenario: the range the model names is exactly the part it
|
||||
does NOT have); a PLAIN re-read of a truncated document gets the
|
||||
:data:`REREAD_TRUNCATED` teaching instead of the (wrong) full-text
|
||||
claim, and every other re-read of a fully-read document stays
|
||||
:data:`ALREADY_IN_CONTEXT`. **A6 re-revised contract
|
||||
teaching. The dedupe is COVERAGE-based (phase 141, task 02 —
|
||||
:data:`AgentHolder.read_coverage` records the line span each
|
||||
successful read actually put in the model's context, and the merged
|
||||
spans decide): a range read PROCEEDS unless the requested lines are
|
||||
FULLY covered by earlier reads (the owner's stuck-scenario fix —
|
||||
two DIFFERENT ranges of a document both reach the model, and the
|
||||
phase-131 truncated-then-range row is covered by the state — a
|
||||
truncated first read covers only its cut span); a re-read of
|
||||
FULLY-covered lines (a read-in-full document, or an earlier span)
|
||||
gets :data:`ALREADY_IN_CONTEXT` — the claim is then TRUE (the named
|
||||
lines ARE in the prompt); a PLAIN re-read of a PARTIALLY-covered
|
||||
document gets the :data:`REREAD_PARTIAL` teaching (the covered
|
||||
span(s) named in line form, read the rest with ``offset``/
|
||||
``limit``) instead of the (wrong) full-text claim. **A6
|
||||
re-revised contract
|
||||
(phase 118, owner directive 2026-09-15):** the retrieval
|
||||
``<documents>`` path seeds SUMMARIES only — a suggested document's
|
||||
full text never enters the prompt on the retrieval path; full text
|
||||
@@ -157,20 +165,28 @@ task 04):
|
||||
:data:`NOT_A_FOLDER`, the drill-down teaching with the argument
|
||||
echoed and the deepest existing ancestor's direct subfolders
|
||||
listed, so the model self-corrects in the next round; a document
|
||||
already READ into full-text context (phase 118: the suggested
|
||||
seeds are summary blocks in the prompt, not full text — only an
|
||||
already-read document is refused) →
|
||||
READ into FULL-text context — a re-read whose lines are FULLY
|
||||
covered by earlier reads (phase 141: the COVERAGE state, not the
|
||||
pair set, decides — the raw-arg fast path is retired, every dedupe
|
||||
row resolves the document first; phase 118: the suggested seeds
|
||||
are summary blocks in the prompt, not full text, so a first read
|
||||
of a suggested document still proceeds) →
|
||||
:data:`ALREADY_IN_CONTEXT` (phase 72, task 05 gate iteration:
|
||||
the line names the correct action — answer from the text already
|
||||
in the prompt, do not call read again — so a fired refusal ends
|
||||
the loop instead of inviting a repeat); a ``read`` whose optional
|
||||
``offset``/``limit`` is not a whole line number ≥ 1 (phase 131) →
|
||||
:data:`BAD_RANGE_ARGS` (the bad values echoed); a range ``read``
|
||||
whose ``offset`` is past the document's end (phase 131) →
|
||||
:data:`OFFSET_PAST_END` (the true line count named); a PLAIN
|
||||
re-read of a document whose earlier read was TRUNCATED (phase 131)
|
||||
→ :data:`REREAD_TRUNCATED` (the grep → read(offset/limit) teaching
|
||||
— the full text is NOT in the prompt, unlike the line above); an
|
||||
the loop instead of inviting a repeat; phase 141: the claim is
|
||||
then true — the named lines ARE in the prompt); a ``read`` whose
|
||||
optional ``offset``/``limit`` is not a whole line number ≥ 1
|
||||
(phase 131) → :data:`BAD_RANGE_ARGS` (the bad values echoed; fires
|
||||
FIRST — before any dedupe or resolve); a range ``read`` whose
|
||||
``offset`` is past the document's end (phase 131) →
|
||||
:data:`OFFSET_PAST_END` (the true line count named; fires after
|
||||
the resolve, before the coverage matrix — an offset past the end
|
||||
is refused regardless of coverage); a PLAIN re-read of a document
|
||||
only PARTIALLY covered by earlier reads (a truncated read, or a
|
||||
range read — phase 141) → :data:`REREAD_PARTIAL` (the covered
|
||||
span(s) named, the grep → read(offset/limit) teaching — the full
|
||||
text is NOT in the prompt, unlike the line above); an
|
||||
unknown document (a ``read`` or scoped ``grep`` whose
|
||||
combined ``source/path`` matches nothing — a bare source name, which
|
||||
can never be a document, included) → ``"No document at '…' — check
|
||||
@@ -584,24 +600,33 @@ BAD_RANGE_ARGS = (
|
||||
"(you passed offset={offset!r}, limit={limit!r})."
|
||||
)
|
||||
|
||||
#: The plain re-read of a TRUNCATED document (phase 131, task 02 — the
|
||||
#: dedupe matrix's middle row): the model already holds the first
|
||||
#: ``{shown}`` characters (the cap from the matching
|
||||
#: ``holder.read_truncations`` tuple — the most recent truncation of
|
||||
#: the pair, plain or range) and its first read of the document hit
|
||||
#: the cap, so the rest is NOT in its context. The line points at the
|
||||
#: working workflow — ``grep`` reports line numbers, then ``read``
|
||||
#: that part with ``offset`` / ``limit`` — instead of the
|
||||
#: pre-phase-131 dead end (the plain :data:`ALREADY_IN_CONTEXT` line,
|
||||
#: which claimed the FULL text was in the prompt). ``{source}`` /
|
||||
#: ``{path}`` are the argument's split pair; ``{shown}`` is a plain
|
||||
#: integer. A refusal: no re-emit, ``tool_calls`` NOT incremented,
|
||||
#: consumes a round, fires on the raw-arg fast path (no DB lookup).
|
||||
REREAD_TRUNCATED = (
|
||||
"You already have the first {shown} characters of {source}/{path} "
|
||||
"in your context. To read the rest, find where your text is with "
|
||||
"grep (it reports line numbers), then call read with `offset` "
|
||||
"(1-based first line) and `limit` (lines)."
|
||||
#: The plain re-read of a PARTIALLY-covered document (phase 141,
|
||||
#: task 02 — the dedupe matrix's middle row, generalizing and
|
||||
#: RETIRING the phase-131 ``REREAD_TRUNCATED`` line, whose char-based
|
||||
#: ``{shown}`` copy is replaced by the line-based truth): earlier
|
||||
#: reads put SOME — not all — of the document's lines in the
|
||||
#: model's context (a truncated read covers its cut span, a range
|
||||
#: read covers its own span — :data:`AgentHolder.read_coverage` is
|
||||
#: the state this row decides on), so the rest is NOT in its
|
||||
#: context. The line names the covered span(s) in line form
|
||||
#: (``{covered}`` — the merged spans rendered by
|
||||
#: :func:`format_covered_spans`, comma-joined ``a–b`` pairs, a single
|
||||
#: line bare ``a``) and points at the working workflow — ``grep``
|
||||
#: reports line numbers, then ``read`` that part with ``offset`` /
|
||||
#: ``limit`` — instead of the dead end (the plain
|
||||
#: :data:`ALREADY_IN_CONTEXT` line, which claimed the FULL text was
|
||||
#: in the prompt — false for a partial document). ``{source}`` /
|
||||
#: ``{path}`` are the RESOLVED identity; ``{total}`` is the
|
||||
#: document's true line count (``len(content.splitlines())``) — both
|
||||
#: plain, no ``!r``. A refusal: no re-emit, ``tool_calls`` NOT
|
||||
#: incremented, consumes a round; fires after the resolve (the true
|
||||
#: line count only the resolve knows — the raw-arg fast path is
|
||||
#: retired).
|
||||
REREAD_PARTIAL = (
|
||||
"You have lines {covered} of {source}/{path} in your context "
|
||||
"({total} lines total). To read the rest, find where your text "
|
||||
"is with grep (it reports line numbers), then call read with "
|
||||
"`offset` (1-based first line) and `limit` (number of lines)."
|
||||
)
|
||||
|
||||
#: The range-``read`` offset past the end (phase 131, task 02): the
|
||||
@@ -1359,6 +1384,55 @@ def _read_range_args(
|
||||
return True, "", offset, limit
|
||||
|
||||
|
||||
def covered_spans(
|
||||
records: Sequence[tuple[str, str, int, int]], source: str, path: str
|
||||
) -> list[tuple[int, int]]:
|
||||
"""Merged coverage spans for ONE document (phase 141, task 02).
|
||||
|
||||
Selects the ``read_coverage`` records keyed by the RESOLVED
|
||||
``(source, path)`` pair (records for other documents are ignored)
|
||||
and returns their inclusive line spans, MERGED: overlapping spans
|
||||
collapse into their union (a span contained in another is
|
||||
absorbed) and ADJACENT spans join (``(1, 10) + (11, 20)`` →
|
||||
``(1, 20)`` — adjacency counts as continuous coverage), sorted by
|
||||
first line. Pure over holder state — no DB.
|
||||
"""
|
||||
merged: list[list[int]] = []
|
||||
for first, last in sorted(
|
||||
(a, b) for s, p, a, b in records if s == source and p == path
|
||||
):
|
||||
if merged and first <= merged[-1][1] + 1:
|
||||
merged[-1][1] = max(merged[-1][1], last)
|
||||
else:
|
||||
merged.append([first, last])
|
||||
return [(first, last) for first, last in merged]
|
||||
|
||||
|
||||
def fully_covered(
|
||||
spans: Sequence[tuple[int, int]], first: int, last: int
|
||||
) -> bool:
|
||||
"""Whether the requested ``(first, last)`` span sits inside the
|
||||
merged coverage union (phase 141, task 02) — per line, inclusive:
|
||||
one merged span must start at or before ``first`` and end at or
|
||||
after ``last``. Pure — no DB.
|
||||
"""
|
||||
return any(
|
||||
span_first <= first and last <= span_last for span_first, span_last in spans
|
||||
)
|
||||
|
||||
|
||||
def format_covered_spans(spans: Sequence[tuple[int, int]]) -> str:
|
||||
"""Render merged coverage spans for the :data:`REREAD_PARTIAL`
|
||||
``{covered}`` field (phase 141, task 02): comma-joined ``a–b``
|
||||
pairs (the en dash — one span → the bare ``a–b``; a single line →
|
||||
the bare ``a``), in span order. Pure — no DB.
|
||||
"""
|
||||
return ", ".join(
|
||||
f"{first}" if first == last else f"{first}–{last}"
|
||||
for first, last in spans
|
||||
)
|
||||
|
||||
|
||||
def grep_document(content: str, pattern: str) -> list[tuple[int, str]]:
|
||||
"""Every line of *content* that contains *pattern*, in file order.
|
||||
|
||||
@@ -1408,6 +1482,29 @@ class AgentHolder:
|
||||
nothing). Read by ``app.api.chat`` after the stream for the done
|
||||
frame's ``answer_image`` field (task 04) — the ``read_docs``
|
||||
precedent (an API-layer read-after-stream signal).
|
||||
``read_coverage`` (phase 141, task 01): one ``(source, path,
|
||||
first_line, last_line)`` record per covered span — INCLUSIVE line
|
||||
numbers over the document's ``content.splitlines()``, in read
|
||||
order, keyed by the RESOLVED ``(source, path)`` pair (the resolved
|
||||
document's own identity — the same pair ``read_docs`` keys on; by
|
||||
the :func:`_resolve_path` first-slash contract the raw argument's
|
||||
split and the resolved pair agree). One span per SUCCESSFUL
|
||||
``read`` — a refused read records NOTHING (the refusal style of
|
||||
every refusal: it counts in nothing): a plain read at or under the
|
||||
cap covers ``(1, total_lines)``; a truncated plain read covers
|
||||
line 1 through the count from ``splitlines()`` on the ACTUAL cut
|
||||
string, ``content[:cap]`` (the lines the model actually saw; the
|
||||
char cap is never re-derived into lines arithmetically); a range
|
||||
read covers ``(first, end)`` — the TRUE extent the ``lines:
|
||||
{first}-{end} of {total}`` header reports (an overshooting
|
||||
``limit`` is already clamped into ``end``); a truncated range
|
||||
read covers the same cut count into the slice — ``first - 1 +
|
||||
len(slice[:cap].splitlines())``, clamped to ``end``. The
|
||||
reworked dedupe matrix (phase 141, task 02) decides on these
|
||||
spans; ``read_docs`` / ``read_truncations`` keep
|
||||
their EXACT current meaning (API-layer signal, SSE matching,
|
||||
per-turn log) — this list is NEW state beside them, not a
|
||||
replacement.
|
||||
"""
|
||||
|
||||
read_docs: list[Document] = field(default_factory=list)
|
||||
@@ -1415,6 +1512,7 @@ class AgentHolder:
|
||||
scaffold_stripped: int = 0
|
||||
read_truncations: list[tuple[str, int, int]] = field(default_factory=list)
|
||||
shown_images: list[Document] = field(default_factory=list)
|
||||
read_coverage: list[tuple[str, str, int, int]] = field(default_factory=list)
|
||||
|
||||
|
||||
def _execute_tool(
|
||||
@@ -1442,15 +1540,30 @@ def _execute_tool(
|
||||
``read`` may carry the optional line range ``offset`` (1-based
|
||||
first line) + ``limit`` (line count) — validated by
|
||||
:func:`_read_range_args` (a bad value → :data:`BAD_RANGE_ARGS`,
|
||||
before the dedupe and the resolve), the dedupe matrix (a range
|
||||
read of a TRUNCATED document proceeds — the stuck-scenario fix —
|
||||
a plain re-read of a truncated one gets :data:`REREAD_TRUNCATED`,
|
||||
every other re-read of a fully-read one stays
|
||||
:data:`ALREADY_IN_CONTEXT`), an ``offset`` past the end →
|
||||
:data:`OFFSET_PAST_END`, the slice shaped under a ``lines:
|
||||
FIRST — before any dedupe or resolve), then the COVERAGE dedupe
|
||||
matrix (phase 141, task 02 — the raw-arg fast path is RETIRED:
|
||||
whether a span is "full" needs the document's TRUE line count,
|
||||
which only the resolve knows, so EVERY dedupe row resolves the
|
||||
document first — one cheap indexed lookup; the pre-phase-141 bug
|
||||
a false refusal dead-ended a whole stuck turn, and that cost is
|
||||
what forces the trade): a range read PROCEEDS unless the
|
||||
requested lines are FULLY covered by earlier reads (the owner's
|
||||
stuck-scenario fix — includes the phase-131 truncated-then-range
|
||||
row, a truncated first read covering only its cut span), a
|
||||
re-read of fully-covered lines gets :data:`ALREADY_IN_CONTEXT`
|
||||
(the claim is then true), a plain re-read of a PARTIALLY-covered
|
||||
one gets :data:`REREAD_PARTIAL` (the covered span(s) named), an
|
||||
``offset`` past the end → :data:`OFFSET_PAST_END` (after the
|
||||
resolve, before the matrix), the slice shaped under a ``lines:
|
||||
{offset}-{end} of {total_lines}`` header line and capped like the
|
||||
whole document (the tuple's ``chars_total`` is then the slice's
|
||||
length). Document targets are combined ``source/path``
|
||||
length). Phase 141 (task 01): every SUCCESSFUL read also appends
|
||||
its covered span — the RESOLVED pair + the inclusive line extent
|
||||
of what actually reached the model (per the four outcomes above) —
|
||||
to ``holder.read_coverage`` (the NEW state the reworked dedupe
|
||||
matrix, task 02, decides on); a refused read records nothing, and
|
||||
``read_docs`` / ``read_truncations`` keep their exact current
|
||||
meaning. Document targets are combined ``source/path``
|
||||
strings, resolved by :func:`_resolve_path` (the canonical identity,
|
||||
phase 70).
|
||||
"""
|
||||
@@ -1531,42 +1644,25 @@ def _execute_tool(
|
||||
# the end (the slice math below).
|
||||
ranged = offset is not None or limit is not None
|
||||
first = offset if offset is not None else 1
|
||||
# Phase 118 (A6): the dedupe set is ``holder.read_docs`` ONLY —
|
||||
# the ``seed_docs`` are SUMMARY blocks in the prompt, not full
|
||||
# text, so a FIRST read of a suggested document adds its full
|
||||
# text through the path below; only a document ALREADY READ is
|
||||
# refused.
|
||||
# Phase 118 (A6): ``holder.read_docs`` stays the read-once
|
||||
# bookkeeping for the API layer (``done.sources`` / the
|
||||
# per-turn log's ``sources=``) — the DEDUPE no longer reads
|
||||
# it: phase 141 (task 02) decides from the COVERAGE state
|
||||
# (below). The ``seed_docs`` are SUMMARY blocks in the prompt,
|
||||
# not full text, so a FIRST read of a suggested document still
|
||||
# adds its full text through the path below.
|
||||
known = {(doc.source, doc.path) for doc in holder.read_docs}
|
||||
# Phase 131 (task 02): the TRUNCATED set, keyed on the
|
||||
# RESOLVED (source, path) pair — the holder tuples carry the
|
||||
# raw argument the model passed, and the same first-slash
|
||||
# split the resolve below performs recovers the pair. A pair
|
||||
# keeps the MOST RECENT truncation's shown count (a plain
|
||||
# truncation followed by a truncated range read → the range
|
||||
# read's).
|
||||
truncated_shown: dict[tuple[str, str], int] = {}
|
||||
for argument, shown, _total in holder.read_truncations:
|
||||
a = argument.strip()
|
||||
if "/" in a:
|
||||
t_source, _, t_path = a.partition("/")
|
||||
truncated_shown[(t_source, t_path)] = shown
|
||||
# The raw-arg fast path handles the "plain read + already in
|
||||
# context" rows with no DB lookup (the pre-phase-131 check —
|
||||
# the split pair of a combined identity that is in
|
||||
# full-text context is in `known` as-is, the resolve below
|
||||
# would find the same document — now split on the truncated
|
||||
# set): truncated earlier → the :data:`REREAD_TRUNCATED`
|
||||
# teaching (the rest is NOT in the prompt — point at the
|
||||
# grep → read(offset/limit) workflow); never truncated →
|
||||
# :data:`ALREADY_IN_CONTEXT` (byte-identical to pre-phase-131).
|
||||
if not ranged and "/" in arg:
|
||||
src, _, p = arg.partition("/")
|
||||
if (src, p) in known:
|
||||
if (src, p) in truncated_shown:
|
||||
return REREAD_TRUNCATED.format(
|
||||
shown=truncated_shown[(src, p)], source=src, path=p
|
||||
)
|
||||
return ALREADY_IN_CONTEXT
|
||||
# Phase 141 (task 02): the raw-arg fast path is RETIRED —
|
||||
# whether a span is "full" needs the document's TRUE line
|
||||
# count, which only the resolve knows, so EVERY dedupe row
|
||||
# resolves the document first (one cheap indexed lookup; the
|
||||
# fast path's only saving was that one query, and the
|
||||
# pre-phase-141 bug — a false refusal dead-ending a whole
|
||||
# stuck turn — is what forces the trade). A fully-covered
|
||||
# plain re-read — the only row the old fast path handled —
|
||||
# still refuses with the byte-identical
|
||||
# :data:`ALREADY_IN_CONTEXT`; its implementation just goes
|
||||
# through the resolve.
|
||||
doc, _source, _path = _resolve_path(db, arg)
|
||||
if doc is None:
|
||||
# Echo the argument as passed — the model sees its own form
|
||||
@@ -1574,38 +1670,61 @@ def _execute_tool(
|
||||
# a path-like argument that matches an indexed document's
|
||||
# path gets the "did you mean …?" teaching (phase 72).
|
||||
return _no_document_refusal(db, arg)
|
||||
# Range read + already in context (phase 131, task 02 — the
|
||||
# dedupe matrix's range rows; the truncated-set check needs
|
||||
# the RESOLVED pair, so it runs here, after the resolve, where
|
||||
# ``doc`` is known): the truncated-then-range case PROCEEDS
|
||||
# (the stuck-scenario fix — the range the model names is
|
||||
# exactly the part it does NOT have); a range of a fully-read
|
||||
# document is :data:`ALREADY_IN_CONTEXT` (the range is a
|
||||
# subset of the text already in the prompt).
|
||||
if (
|
||||
ranged
|
||||
and (doc.source, doc.path) in known
|
||||
and (doc.source, doc.path) not in truncated_shown
|
||||
):
|
||||
return ALREADY_IN_CONTEXT
|
||||
lines = doc.content.splitlines()
|
||||
total_lines = len(lines)
|
||||
# The range read's TRUE extent (``end`` clamps an overshooting
|
||||
# ``limit`` to the last line; ``offset`` without ``limit``
|
||||
# reads to the end) — computed before the matrix (the range
|
||||
# coverage row needs it) and reused by the slice shaping
|
||||
# below.
|
||||
end = total_lines if limit is None else min(first + limit - 1, total_lines)
|
||||
if ranged and first > total_lines:
|
||||
# The offset names a line past the end — teaching refusal:
|
||||
# the true line count named, grep reports the exact line
|
||||
# numbers (the locator that got the model here). Counts in
|
||||
# nothing, consumes a round; the document is NOT added to
|
||||
# the context, so a corrected retry still proceeds.
|
||||
# The offset names a line past the end — teaching refusal
|
||||
# (fires after the resolve, BEFORE the coverage matrix —
|
||||
# an offset past the end is refused regardless of
|
||||
# coverage): the true line count named, grep reports the
|
||||
# exact line numbers (the locator that got the model
|
||||
# here). Counts in nothing, consumes a round; the document
|
||||
# is NOT added to the context, so a corrected retry still
|
||||
# proceeds.
|
||||
return OFFSET_PAST_END.format(
|
||||
source=doc.source,
|
||||
path=doc.path,
|
||||
total_lines=total_lines,
|
||||
offset=first,
|
||||
)
|
||||
# The COVERAGE dedupe matrix (phase 141, task 02 — the
|
||||
# rework of the phase-131 pair-set dedupe): the merged spans
|
||||
# for this RESOLVED document decide.
|
||||
# plain, full document covered → :data:`ALREADY_IN_CONTEXT`
|
||||
# plain, partial → :data:`REREAD_PARTIAL`
|
||||
# (the covered span(s) named — the REREAD_TRUNCATED role,
|
||||
# generalized: line-based truth)
|
||||
# range, requested span fully → :data:`ALREADY_IN_CONTEXT`
|
||||
# covered (the named lines ARE in the prompt — the claim is
|
||||
# true)
|
||||
# range, not fully covered → PROCEEDS (the owner's
|
||||
# stuck-scenario fix — includes the phase-131
|
||||
# truncated-then-range row: a truncated first read covers
|
||||
# only its cut span)
|
||||
spans = covered_spans(holder.read_coverage, doc.source, doc.path)
|
||||
if not ranged:
|
||||
if fully_covered(spans, 1, total_lines):
|
||||
return ALREADY_IN_CONTEXT
|
||||
if spans:
|
||||
return REREAD_PARTIAL.format(
|
||||
covered=format_covered_spans(spans),
|
||||
source=doc.source,
|
||||
path=doc.path,
|
||||
total=total_lines,
|
||||
)
|
||||
elif fully_covered(spans, first, end):
|
||||
return ALREADY_IN_CONTEXT
|
||||
# The checks above refused every duplicate that must be
|
||||
# refused — append at most ONCE: a first read (plain or range)
|
||||
# appends; the range read of an earlier TRUNCATED read of the
|
||||
# same document (the row that proceeded) does not re-append.
|
||||
# appends; a PROCEEDING re-read of an earlier partial read of
|
||||
# the same document (the owner's second range) does not
|
||||
# re-append.
|
||||
if (doc.source, doc.path) not in known:
|
||||
holder.read_docs.append(doc)
|
||||
holder.tool_calls += 1
|
||||
@@ -1636,6 +1755,18 @@ def _execute_tool(
|
||||
holder.read_truncations.append(
|
||||
(cast("str", raw_path), cap, len(doc.content))
|
||||
)
|
||||
# Phase 141 (task 01): coverage tracking — the lines
|
||||
# this cut actually put in the model's context: from
|
||||
# line 1 to the last line visible in the ACTUAL cut
|
||||
# string (``splitlines()`` on ``content[:cap]`` — the
|
||||
# counted-from-truth rule; a cap that cuts mid-line
|
||||
# ends the span at that line's visible prefix; the
|
||||
# char cap is never re-derived into lines
|
||||
# arithmetically). Task 02's dedupe matrix decides on
|
||||
# these spans — task 01 only records.
|
||||
holder.read_coverage.append(
|
||||
(doc.source, doc.path, 1, len(doc.content[:cap].splitlines()))
|
||||
)
|
||||
# Phase 106 (D5): the date rides every document the
|
||||
# model sees — the ``read`` result's SECOND line; the
|
||||
# FIRST line stays ``Document {source}/{path}:``
|
||||
@@ -1652,8 +1783,12 @@ def _execute_tool(
|
||||
# At or under the cap: the pre-phase-95 result plus the
|
||||
# phase-106 D5 date line (first line byte-identical — the
|
||||
# mock's header contract; no truncation marker, no notice,
|
||||
# no holder entry, no ToolResultPiece) — and, phase 122,
|
||||
# the image-document marker line for image docs only.
|
||||
# no TRUNCATION holder entry, no ToolResultPiece) — and,
|
||||
# phase 122, the image-document marker line for image
|
||||
# docs only.
|
||||
# Phase 141 (task 01): a plain read at or under the cap
|
||||
# covers the WHOLE document — lines 1..total_lines.
|
||||
holder.read_coverage.append((doc.source, doc.path, 1, total_lines))
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
@@ -1661,13 +1796,13 @@ def _execute_tool(
|
||||
f"{doc.content}"
|
||||
)
|
||||
# Range read (phase 131, task 02): EXACTLY the named lines —
|
||||
# the TRUE extent on the third line (``end`` clamps to the
|
||||
# last line: an overshooting ``limit`` reads to the end, never
|
||||
# past it; ``offset`` without ``limit`` reads to the end), the
|
||||
# the TRUE extent on the third line (``end`` — computed
|
||||
# before the matrix above: it clamps to the last line, an
|
||||
# overshooting ``limit`` reads to the end, never past it;
|
||||
# ``offset`` without ``limit`` reads to the end), the
|
||||
# image-doc marker (when one) exactly where the plain read
|
||||
# puts it (after the header lines), the cap applied to the
|
||||
# SLICE.
|
||||
end = total_lines if limit is None else min(first + limit - 1, total_lines)
|
||||
slice_text = "\n".join(lines[first - 1 : end])
|
||||
if len(slice_text) > cap:
|
||||
# The cap on the slice: the same marker + the pinned notice
|
||||
@@ -1678,6 +1813,19 @@ def _execute_tool(
|
||||
holder.read_truncations.append(
|
||||
(cast("str", raw_path), cap, len(slice_text))
|
||||
)
|
||||
# Phase 141 (task 01): the truncated range read covers
|
||||
# from ``first`` to the last line visible in the cut
|
||||
# SLICE — the same counted-from-truth rule as the plain
|
||||
# read above (``splitlines()`` on the actual cut string),
|
||||
# clamped to ``end``.
|
||||
holder.read_coverage.append(
|
||||
(
|
||||
doc.source,
|
||||
doc.path,
|
||||
first,
|
||||
min(end, first - 1 + len(slice_text[:cap].splitlines())),
|
||||
)
|
||||
)
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
@@ -1686,6 +1834,11 @@ def _execute_tool(
|
||||
f"{TRUNCATION_MARKER}\n"
|
||||
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))}"
|
||||
)
|
||||
# Phase 141 (task 01): the range read covers exactly the TRUE
|
||||
# extent the ``lines:`` header reports (``end`` already
|
||||
# clamped an overshooting ``limit`` — the span never goes
|
||||
# past the document's last line).
|
||||
holder.read_coverage.append((doc.source, doc.path, first, end))
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
@@ -1820,7 +1973,9 @@ async def run_agent(
|
||||
``tool`` frame and BEFORE the next model round (the API layer turns it
|
||||
into an SSE ``tool_result`` frame, task 02). After the loop finishes,
|
||||
*holder* carries the read documents, the executed tool-call count
|
||||
(re-lists included), and the ``read_truncations`` signal list.
|
||||
(re-lists included), the ``read_truncations`` signal list, and the
|
||||
``read_coverage`` spans (phase 141, task 01 — the lines each
|
||||
successful read actually put in the model's context).
|
||||
|
||||
History (phase 74, TODO L4): *history* is the client's prior turns
|
||||
already mapped to model messages by
|
||||
@@ -1875,11 +2030,15 @@ async def run_agent(
|
||||
``seed_docs`` are the suggested documents whose SUMMARY blocks the
|
||||
caller put in the *system_prompt* (phase 118: the retrieval seeds
|
||||
summaries, never full texts); reading one of them ADDS its full
|
||||
text to the context through the ordinary ``read`` path — only a
|
||||
document ALREADY READ is rejected with :data:`ALREADY_IN_CONTEXT`
|
||||
(the phase-72 teaching line — answer from the text already in the
|
||||
prompt); the rejection counts in nothing, but it still consumes a
|
||||
round.
|
||||
text to the context through the ordinary ``read`` path — a re-read
|
||||
whose lines are FULLY covered by earlier reads is rejected with
|
||||
:data:`ALREADY_IN_CONTEXT` (the phase-72 teaching line — answer
|
||||
from the text already in the prompt; phase 141: the coverage
|
||||
state decides, so the claim is then true, and a re-read of a
|
||||
PARTIALLY-covered document gets :data:`REREAD_PARTIAL` instead —
|
||||
the covered span(s) named, read the rest with ``offset``/
|
||||
``limit``); the rejection counts in nothing, but it still consumes
|
||||
a round.
|
||||
|
||||
DB sessions (SEC-14-04): *db_factory* is a callable that returns a
|
||||
new :class:`sqlalchemy.orm.Session` (e.g. ``lambda: SessionLocal()``).
|
||||
|
||||
Reference in New Issue
Block a user