fix(agent): teach the document-identity contract on ls/read/grep refusals — end the post-harness tool-loop rambling

Phase 72 (72_teaching_refusals) — completed under the 2026-09-04 controlled
methodology (owner directive: stop clearing/re-importing the homelab KB per
iteration; measure tool-calling accuracy on a controlled fixture KB, target
>90%).

Real-model gate verdicts (live, configured chat model 'lite', fixture KB):
- Controlled fixture battery (the new methodology's pass condition —
  contract accuracy >= 90%): PASS, 4 consecutive runs:
  gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/11 executed (73%) contract 11/11 (100%) 2026-09-04 (wall 43.4s)
  gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/13 executed (62%) contract 12/13 (92%) 2026-09-04 (wall 50.6s)
  gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 7/11 executed (64%) contract 11/11 (100%) 2026-09-04 (wall 46.8s)
  gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 9/15 executed (60%) contract 14/15 (93%) 2026-09-04 (wall 54.8s)
- Locked derived battery (phase-72 task 05, executed >= 90% bar, run
  unchanged on the same fixture KB):
  gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 5/15 executed (33%) contract 12/15 (80%) 2026-09-04 (wall 47.7s)
  The teaching works — every bare-path trap self-corrects in exactly one
  round, zero cap hits, zero repeat loops, 10/10 answered. The locked
  executed bar is blocked by ALREADY_IN_CONTEXT dedupe refusals on the
  corrected re-reads (the trap question seeds its target, so the correct
  combined-form read is refused for redundancy) — a copy-invariant model
  behavior (five copy variants, 0/15 re-reads flipped, 2026-09-03 -> 04)
  and an app-semantics decision for the owner (TOOL_CALLING_TESTING.md
  sections 5 and 7), not a copy lever.

Copy changes this phase owns (unit pins updated to follow):
- app/rag/agent.py: ls teaching refusals (path-like scope -> document-path
  line; unknown source -> no-source line with the source-name
  parenthetical), read/grep 'did you mean source/path?' teaching
  (find_path_candidates: exact or suffix path match, catalog order, cap 3),
  ALREADY_IN_CONTEXT naming the correct action (answer from the text
  already in the prompt), read tool description front-loaded with the
  do-not-read rule (the 2026-09-04 controlled telemetry: the re-read is
  the only remaining refusal class; contract accuracy 92-100% across runs)
- app/rag/prompts.py: TOOLS_SECTION states the document-identity contract
  up front (ls path = source name; read/grep = combined source/path
  including the source name; do-not-read for <documents> documents placed
  next to the read teaching; one-call-per-reply and never-repeat rules)
- tests: refusal pins (unit + integration), new dedicated E2E suite
  tests/e2e/test_tool_path_teaching.py (mock misuse flow, green in
  isolation), regression suites green in isolation (harness_aligned_tools,
  agent_document_tools, agent_unlimited_tools, search_tool, chat_rag).

Gates: uv run pytest green (1501); coverage TOTAL 99% (>90%); ruff +
pyright clean. Carries the still-uncommitted phase-71 todo/ -> complete/
move and both phases' .agent/reports/ (AGENTS.md 8).

app/rag/prompts.py

@@ -25,13 +25,20 @@ roughly what the KB contains before retrieval. With an empty row the
 prompt is byte-identical to the pre-phase text.
 
 Agent tools (phase 37; phase 70: the copy teaches the harness-aligned
-``ls`` / ``read`` / ``grep`` shapes): the **HIGH** prompt only carries a
-``<tools>`` section after the ``<documents>`` body — the grounded turn
-may extend its context through the three server-side tools (round-
-capped, see :mod:`app.rag.agent`; the cap is the bound and this section
-does not re-state it, phase 45). The LOW/deflection prompt never
-carries it (phase 71: the LOW prompt's only addition is the
-plain-text line below — it still has no ``<tools>`` section).
+``ls`` / ``read`` / ``grep`` shapes; phase 72: the copy states the
+document-identity contract up front — ``ls``'s ``path`` is a *source
+name*, not a directory or file path, and ``read``/``grep`` take the
+combined ``source/path`` string *including the source name* (a bare
+document path will not resolve) — the same two things the phase-72
+teaching refusals in :mod:`app.rag.agent` re-state after the fact, so
+the model carries the contract before it calls a tool): the **HIGH**
+prompt only carries a ``<tools>`` section after the ``<documents>``
+body — the grounded turn may extend its context through the three
+server-side tools (round-capped, see :mod:`app.rag.agent`; the cap is
+the bound and this section does not re-state it, phase 45). The
+LOW/deflection prompt never carries it (phase 71: the LOW prompt's
+only addition is the plain-text line below — it still has no
+``<tools>`` section).
 
 Deflection plain-text line (phase 71, owner-permitted 2026-09-03):
 the otherwise-locked ``LOW`` prompt gains exactly one instruction
@@ -89,28 +96,69 @@ _KB_INTRO = (
 #: The ``<tools>`` instructions section — **HIGH prompt only** (phase 37,
 #: task 03; phase 70: the copy is rewritten for the harness-aligned
 #: ``ls`` / ``read`` / ``grep`` shapes, names/args exactly as the
-#: ``AGENT_TOOLS`` schemas in :mod:`app.rag.agent`): a grounded turn may
-#: extend its context through the three server-side tools (round cap:
-#: ``BOR_AGENT_MAX_ROUNDS`` — the cap is the bound and this section does
-#: not re-state it, phase 45). Appended after the mode body
-#: (``<documents>``), so the instructions are the last thing the model
-#: reads. The LOW/deflection prompt never carries it — a deflection has
-#: no grounded context to extend (phase 71: the LOW prompt's only
-#: addition is the plain-text line in :func:`build_deflect_prompt`).
-#: The E2E mock keys off the ``<tools>`` marker's *presence*, not this
-#: wording.
+#: ``AGENT_TOOLS`` schemas in :mod:`app.rag.agent`; phase 72: the copy
+#: states the document-identity contract UP FRONT — ``ls``'s optional
+#: ``path`` is a *source name* (not a directory or file path) and
+#: ``read``/``grep`` take the combined ``source/path`` string *including
+#: the source name* (a bare document path will not resolve) — the same
+#: two things the phase-72 teaching refusals re-state after the fact):
+#: a grounded turn may extend its context through the three server-side
+#: tools (round cap: ``BOR_AGENT_MAX_ROUNDS`` — the cap is the bound and
+#: this section does not re-state it, phase 45). Appended after the mode
+#: body (``<documents>``), so the instructions are the last thing the
+#: model reads. The LOW/deflection prompt never carries it — a
+#: deflection has no grounded context to extend (phase 71: the LOW
+#: prompt's only addition is the plain-text line in
+#: :func:`build_deflect_prompt`). The E2E mock keys off the ``<tools>``
+#: marker's *presence*, not this wording. Task 05 (the live gate's
+#: iteration loop) keeps the baseline and carries the discipline rules
+#: the live telemetry motivated, refined across the task-05 re-runs of
+#: 2026-09-03/04 (run 1: 7/16 executed, 44% — the ``lite`` model obeyed
+#: the user's "open it / read it" and ``read`` seed-context documents,
+#: then repeated the refused call; run 2: 8/18, 44% — the repeat is
+#: gone, but a grep turn batched two calls per reply and the harness
+#: runs only the first of a batch): the do-not-read rule names the
+#: user-command scenario (a document already in the ``<documents>``
+#: section: do not call ``read``, answer from the text already in the
+#: prompt — an anchor on the concrete ``<document path="...">`` markup
+#: was tried and REVERTED: it primed the model to latch the seed
+#: documents' paths as ``ls`` scopes, regressing the incident turn);
+#: the one-call rule names the consequence (a batched second call is
+#: discarded — runs only the first); the never-repeat rule says why
+#: (the refusal already told you the correct form); the ``grep`` clause
+#: leads with "for a normal search pass only ``pattern``" (the gate's
+#: live runs showed the model scoping ``grep`` with an ``ls``-style
+#: source name — the incident shape, but on grep). The behavioral
+#: contract lives in the ``AGENT_TOOLS`` descriptions as well (the most
+#: local text at call time): ``read`` must not be called for a
+#: ``<documents>`` document at all; ``grep`` with only ``pattern``
+#: searches the whole knowledge base, and a source name is not a
+#: document.
 TOOLS_SECTION: str = (
     "<tools>\n"
     "You may extend your context with three tools. `ls` lists the "
-    "indexed documents as `source: X | path: Y | title: Z` lines "
-    "(pass a source name as `path` to list one source's documents; "
-    "omit it to list every document). `grep` locates an exact string "
-    "(case-insensitive) in the indexed documents and returns up to 20 "
-    "matching `source/path:line: text` lines — a locator, not a "
-    "context-adder: read the winner with `read`. `read` pulls in one "
-    "document by its combined `source/path` string, exactly as shown in "
-    "the `ls` output, adding its full content to your context. Answer "
-    "as soon as you have what you need.\n"
+    "indexed documents as `source: X | path: Y | title: Z` lines; its "
+    "optional `path` argument is a source name (e.g. 'homelab'), not a "
+    "directory or file path — omit it to list every document. `read` "
+    "pulls in one document by its combined `source/path` string, "
+    "exactly as shown in the `ls` output — including the source name — "
+    "adding its full content to your context. Do not call `read` for a "
+    "document already shown in the <documents> section, even when the "
+    "user asks you to open or read it — its full text is already in "
+    "your prompt; answer directly from it. For `read`, a bare document "
+    "path (without the source name) will not resolve. `grep` locates an "
+    "exact string (case-insensitive) in the indexed documents and "
+    "returns up to 20 matching `source/path:line: text` lines — a "
+    "locator, not a context-adder: read the winner with `read`; for a "
+    "normal search pass only `pattern` — its optional `path` argument "
+    "limits the search to one document you already know, by the same "
+    "combined `source/path` string; never a source name — a bare "
+    "document path (without the source name) will not resolve there "
+    "either. Make exactly one tool call per reply — a reply carrying "
+    "two tool calls runs only the first, the second is discarded — and "
+    "wait for the result before the next call. Never repeat a call that "
+    "was refused or already succeeded — the refusal already told you "
+    "the correct form. Answer as soon as you have what you need.\n"
     "</tools>"
 )
 
@@ -204,14 +252,33 @@ def build_high_prompt(
 ) -> str:
     """Grounded turn: locked persona (+ steering, + KB overview) + full
 texts of the top documents + the ``<tools>`` instructions (phase 37;
-the phase-70 copy teaches the ``ls`` / ``read`` / ``grep`` shapes).
+phase 70: the harness-aligned ``ls`` / ``read`` / ``grep`` shapes;
+phase 72: the copy states the document-identity contract — the
+source-name ``ls`` scope, the combined ``source/path`` identity for
+``read``/``grep`` — up front).
 
     Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
     → ``<documents>`` → ``<tools>``; empty steering/overview omit their
     section. ``<tools>`` is always present in the HIGH prompt (the round
     cap — not the prompt — decides whether the tools are actually
     offered to the model, see :mod:`app.rag.agent`).
+
+    Gate-iteration note (task 05, 2026-09-03/04): an in-context reminder
+    LEADING this section (the document texts are already context — do
+    not ``read`` one the user asked to open) was tried and REVERTED:
+    it never flipped the seed-doc reads (15/15 across gate runs 1-5)
+    and correlated with the incident-turn regression (the model latched
+    the seed documents' paths as ``ls`` scopes — cap reached on the
+    "list the files in this directory" turn) whenever the copy named
+    the ``<document>`` blocks explicitly.
     """
+    # 2026-09-04 (controlled tool-calling fast loop): the do-not-read
+    # rule for seed documents lives in TOOLS_SECTION and the ``read``
+    # tool descriptions (the copy levers that stuck — see the gate's
+    # telemetry in TOOL_CALLING_TESTING.md). A per-block instruction
+    # attribute at the ``source``/``path`` copy site was TRIED and
+    # REVERTED the same day (no improvement across runs; the block stays
+    # exactly the document identity + full text).
     blocks = [
         f'<document source="{doc.source}" path="{doc.path}" title="{doc.title}">\n'
         f"{doc.content}\n"