brain-of-reese/.agents/phases/todo/182_verbatim_show_text/00_phase.md

Phase 182 — Verbatim file display: a show_text tool + bor locator fences the server resolves

Story: n/a (owner chat 2026-10-14 — "show me my pi.dev config" / "show me the quadlet for gufo" answers are re-typed from context and silently add/remove characters, so the answer text cannot be copy-pasted; the owner has to open the source bubble and ignore the written code block) Context: app/rag/agent.py — the harness-aligned tool surface AGENT_TOOLS (L332: ls / read / grep) plus the 4th, DISPLAY tool SHOW_IMAGE_TOOL (L501) composed by agent_tools(settings) (L538), the shared identity resolver _resolve_path (L1357), the line-range validator _read_range_args (L1474) and the OFFSET_PAST_END (L671) / BAD_RANGE_ARGS (L628) teachings, the refusal block L571–L690 (MISSING_SHOW_ARGS L571, NOT_AN_IMAGE_DOC L581, ALREADY_SHOWING L585), AgentHolder (L1600–L1680: shown_images, read_coverage, read_truncations), _execute_tool (L1681, the show_image branch L2023–L2068 is the shape this phase mirrors); app/rag/prompts.py — TOOLS_SECTION (L293) and the appended-only-when-offered SHOW_IMAGE_CLAUSE (L379) + build_high_prompt(show_image=…) (L759); app/api/chat.py — the prompt/tool call site show_image=settings.answer_images (L529), the per-turn visible-content counter content_chars (L934), the done-frame builder (L1320–L1365, answer_ref/answer_image L1343–L1355), the pinned per-turn log line (L1272–L1291); app/schemas.py — SourceRef (L169), ChatToolResultEvent (L279), ChatDoneEvent.answer_image + its omission serializer (L305–L380), ChatMessageRecord.answer_image + _drop_image_when_absent (L1463, L1478); app/config.py — read_max_chars (L296) + its validator (L686), answer_images: bool = False (L539), log_large_response_bytes (L76); app/core/settings_catalog.py — read_max_chars (L267) / answer_images (L296) rows (the completeness pin in tests/unit/test_settings_catalog.py makes a new Settings field a build failure until it has a row); frontend/assets/markdown.js — renderMarkdown(md) (L43), fence-protection pass (L44–L48: fences → tables → escape+inline → blockquotes → paragraphs → restore, escape-first, no CDN); frontend/assets/app.js — the per-delta whole-bubble re-render renderMarkdown(acc) (L4322), the done handler (L4325–L4460), appendAnswerImage (L2598), the restore renderer (L2890–L2960), copyShareLinkWithFallback (L3250, the http-origin clipboard degradation), rememberBrainTurn save point (L4390–L4420); frontend/assets/shared.js — addAnswerImage (L573) + the record renderer (L619–L660); frontend/index.html L1696–L1701 and frontend/shared.html L197–L199 (classic brand.js + markdown.js, then the module script); frontend/assets/styles.css — .answer-image (L1028–L1059), the bounded-scroll-panel and .cell-clip conventions (PLAN §7.1, phases 156/167); tests/e2e/mock_llm.py — the scripted-trigger registry (L1162–L1520, SHOW_IMAGE_TRIGGER L2606, _SHOW_CONFIRM_MARKER, the "verified: no existing E2E question or fixture file contains the phrase" discipline); tests/e2e/test_answer_images.py (the two-app ON/OFF pattern), tests/unit/test_frontend_*.py (the JS/CSS text-pin convention).

Objective

When the owner asks "show me the quadlet for gufo", the exact bytes of the indexed file reach the chat answer — typed by the server, never by the model — in a panel with a path + line-range caption, a VERBATIM badge, line numbers, a working Copy button and a link to the source. The model only names the location (show_text(path, offset?, limit?), or an inline ```bor <path> <range> locator fence where it wants the block to sit); the server resolves the slice from Postgres and the client places it. This kills the class of bug where the answer's code block silently differs from the file, and it removes the user-facing line-number problem entirely: the numbers shown are the server's resolved extent, not the model's claim.

Design (shared by all tasks — the executor reads this, not the chat)

• D1 — the two mechanisms, one authority (A1). (a) a 5th agent tool show_text(path, offset?, limit?), the display sibling of show_image; (b) an inline locator fence the model may write in its answer — ```bor + the combined source/path + an optional range — so a resolved block can sit anywhere in the prose. Both are resolution requests, never content requests: the only party that supplies bytes is the server. show_text is the authority for teaching (a refusal teaches the model to self-correct before the user sees anything); the fence is placement sugar and ALSO resolves standalone (a fence with no prior show_text call still renders, because resolution is a plain (source, path) lookup, not a replay of a tool call).
• D2 — server resolves, client places (A4/A7). The client cannot resolve (no new route, A4), so: app/api/chat.py accumulates the turn's clean visible answer text (next to the existing content_chars counter, L934 — the post-scaffold-filter text only, never thinking), and at done-build time passes it through app.rag.verbatim.resolve_answer_blocks(db, answer, holder.shown_texts, settings); the result rides the done frame as the ADDITIVE answer_text array. markdown.js turns a bor fence into <div class="verbatim" data-bor-locator="<canonical>"> and drops the model's fence body; app.js/shared.js hydrate each placeholder by matching data-bor-locator against the record's answer_text[].locator (canonical-form match → unique path-only match → unresolved). A block the tool registered that the answer never locates renders APPENDED, at the answer_image position (below the answer, above the source clusters).
• D3 — the model's bytes are NEVER painted (A7, the phase's core contract). Inside a bor fence the model's body is discarded at render time — while streaming the placeholder reads Retrieving the exact text… (role="status", polite by definition), and a fence the model has not finished typing shows that SAME pending state rather than raw backticks, at done it becomes the resolved panel or, when it cannot be resolved, the honest state I could not verify this against the knowledge base — open <source> (persona: radically honest, never a silent substitution and never the model's possibly-corrupt text). The E2E proves it with a mock that deliberately types a CORRUPTED body and asserts the panel carries the DB bytes. renderMarkdown gains a SECOND, OPTIONAL argument ({ verbatim: true }); default OFF keeps every existing call site — the document viewer (document.js L494), the thinking blocks, the user bubbles — byte-identical, and a document that legitimately contains a ```bor fence still renders its own text.
• D4 — line ranges are the server's (A2). offset (1-based) / limit are OPTIONAL and validated by the existing _read_range_args (L1474): omitted → the WHOLE document (the taught default for "show me X"); offset alone → to the end; limit overshoot → CLAMPED to the last line; offset past the last line → the OFFSET_PAST_END-style refusal naming the document's true line count (counts in nothing, consumes one round, invisible to the user). Slicing uses content.splitlines() with the SAME 1-based indexing as read (L1836) and grep, so every line number in the app agrees. The char cap (settings.show_text_max_chars) cuts the SLICE at a line boundary, appends the shared TRUNCATION_MARKER (app/rag/retriever.py) and shrinks end_line to the last fully-shown line — the caption never lies.
• D5 — what the model gets back (A3). The tool result is a confirmation + the truth + an instruction, then the exact slice: Text shown to the user — these lines are displayed verbatim; describe them, do NOT retype them. / Document {source}/{path}: / date: … / {origin lines} / lines: {first}-{end} of {total_lines} / the slice. The Document … + date: + lines: header contract is the one read established (the E2E mock's _READ_RESULT_PREFIX pin keeps parsing it). Returning the text is deliberate (A21: LLM calls are free) — the model's commentary can then never disagree with the panel.
• D6 — new module, no import cycle (D2). app/rag/verbatim.py owns VerbatimBlock (frozen dataclass: source, path, title, start_line, end_line, total_lines, text, truncated, locator), the pure half (canonical_locator, slice_lines, fences_in, cap_slice) and one DB function (document_by_identity(db, combined), a bare select(Document).where(source==, path==) — the same (source, path) identity contract as agent._resolve_path, WITHOUT its did-you-mean suggestions). agent.py imports the pure half, so verbatim.py must never import agent.py (documented at the module top — protocol D2). Path traversal is a non-issue by construction: a traversal string is simply a non-existent row (404-shaped refusal, no filesystem access anywhere).
• D7 — canonical locator (the matching key). source/path for a whole document, source/path#12-40 for a range. The tolerant input grammar (both the fence parser and the tool's path argument) accepts source/path, source/path 12-40, source/path:12-40, source/path#12-40, source/path#L12-L40 and source/path 12 (single line); anything unparseable is not a bor locator at all and renders as an ordinary code block (the body is then the model's own, exactly as today, with a Copy button and no VERBATIM badge). Resolution normalizes to the canonical form, so the client's match is a string compare.
• D8 — caps, kill switch, observability (A8). answer_text: bool = True (BOR_ANSWER_TEXT, the master switch — false = the tool is not offered, the prompt clause is absent, resolution does not run, no answer_text key: every frame and every byte-identity contract is exactly pre-phase), show_text_max_chars: int = 12_000 (per block; 0 = inert, negative fails startup naming the field — the agent_max_rounds/read_max_chars pattern), show_text_max_blocks: int = 5 (per answer, in answer order; 0 = inert, negative fails startup). Worst-case frame growth 5 × 12 000 = 60 KB against the 256 KB budget, which the phase-167 byte-budget line (api … bytes=…, grep ' :: api ') makes visible. The §9 per-turn line gains ONE additive trailing field verbatim_blocks=N (the blocks actually sent, tool-registered + fence-resolved, deduped) — tests/unit/test_chat_gate.py's pinned line is updated in the same task, never silently.
• D9 — resolution scope (the honesty posture). Resolution runs on GROUNDED turns only (the agent ran). A bor fence hallucinated into a DEFLECTED answer has no registered blocks and the gate said "I haven't done anything like that" — it renders the unresolved state rather than leaking KB text through a turn that just disclaimed it. done.sources / related semantics are NOT changed (no regression to the citation surface): the panel itself carries the path, the line range, a Copy button and a link to the document viewer (the git origin link stays the viewer's job, phase 145's origin_for_source — the panel does not duplicate it) — the citation affordance sits at the point of use.
• NOT touched: show_image / SHOW_IMAGE_TOOL / answer_image (image display is out of scope, A6 — verified, not modified), the ls/read/grep contracts, the SSE event-type LIST (A15 stays at seven types — this phase rides done, no 8th event), documents/chunks schema (no migration: saved_chats.messages is raw JSONB, the phase-50/160 precedent), document.js (the viewer renders documents, not answers), app/core/response_budget.py, scripts/*, frontend/assets/router.js.

Locked decisions (owner-confirmed 2026-10-14, from the phase-authoring interview)

• A1 — BOTH mechanisms in this phase: the show_text tool AND the inline ```bor locator fence. (The owner chose "A + B in one phase" over the tool alone.)
• A2 — whole-file default + server clamp: offset/limit optional; omitted = the whole document; overshoot clamps; past-the-end refuses to the model with the true total; the caption always shows the SERVER's resolved extent. No expect-anchor snapping (considered and deferred — the clamp plus the model's own grep/read line numbers cover it).
• A3 — the tool returns the verbatim slice plus the "do not retype" instruction (not a terse confirmation, not the read_coverage-conditional variant).
• A4 — the bytes ride INLINE in done.answer_text and persist in the bor.chat.v1 brain record: no new bytes route, no new SSE event type, no new anonymous surface (a shared page renders from the share's own record, the same trust model as the phase-123 chat image).
• A5 — Copy buttons on EVERY fenced code block in an answer (live, restored and shared), but the VERBATIM badge + caption belong ONLY to a server-resolved block. The clipboard goes through the phase-55 degradation (a non-secure origin gets the inline select-to-copy fallback) — never a bare navigator.clipboard assumption.
• A6 — images are out of scope: show_image keeps images; show_text on an is_image document is REFUSED with a pointer to show_image. The phase's E2E asserts the answer figure is unchanged.
• A7 — the model's fence body never reaches the DOM (see D3). Unresolvable = honest, visible, non-copyable-as-truth state.
• A8 — settings: BOR_ANSWER_TEXT default true (master switch), BOR_SHOW_TEXT_MAX_CHARS default 12000, BOR_SHOW_TEXT_MAX_BLOCKS default 5; 0 = inert, negative = loud startup failure.
• A9 — additive wire only: answer_text is dropped when empty (absent, never null) — every frame without blocks, every pre-phase record and the whole default-off configuration stay byte-identical. No PLAN §2/§4 anchor changes beyond recording these revisions.

Dependencies

• 124_answer_images (complete) — the display-tool precedent this phase mirrors end to end (SHOW_IMAGE_TOOL → holder.shown_images → done.answer_image → appendAnswerImage → persistence → shared page).
• 131_read_line_ranges / 141_read_range_dedupe_fix (complete) — offset/limit + _read_range_args + the lines: N-M of T header contract and splitlines() indexing.
• 145_git_source_links (complete) — the document viewer's origin_for_source link, which the panel links TO rather than duplicating.
• 167_api_payload_diet (complete) — the byte-budget line that will show the done-frame growth.
• 180_markdown_inline_links_images (todo, already authored) — independent of this phase; both touch markdown.js and styles.css, so run it FIRST (or rebase) to avoid a renderer conflict.

Tasks

1. 01_settings_and_kill_switch.md — the three Settings fields + validators + the three CATALOG rows + .env.example (+ their unit/integration pins). Inert: nothing reads them yet.
2. 02_verbatim_resolver_module.md — app/rag/verbatim.py: the locator grammar, the canonical normalizer, splitlines() slicing, the line-boundary cap and resolve_answer_blocks (+ the resolver unit suite).
3. 03_show_text_tool.md — SHOW_TEXT_TOOL, the agent_tools gate, the _execute_tool branch, the refusal set, AgentHolder.shown_texts (+ agent unit suite).
4. 04_prompt_teaching_clause.md — SHOW_TEXT_CLAUSE inside <tools>, build_high_prompt(show_text=…), the app/api/chat.py call site (+ byte-identity pins when off).
5. 05_done_frame_and_persistence.md — the VerbatimBlockRef schema, ChatDoneEvent.answer_text, ChatMessageRecord.answer_text, the visible-text accumulator + resolution + verbatim_blocks=N in the per-turn line (+ integration/frame tests).
6. 06_markdown_locator_and_panel_css.md — renderMarkdown(md, opts)'s opt-in verbatim pass → .verbatim[data-bor-locator] (body dropped; ordinary fences keep rendering exactly as today), and the styles.css panel/copy-button CSS contract. Inert until task 07 opts in and wraps <pre> nodes at the DOM level.
7. 07_hydration_and_copy_buttons.md — frontend/assets/verbatim.js (+ index.html/shared.html script tags): hydrate-at-done, restore, shared page, error/settled paths, rememberBrainTurn persistence, copy buttons on every answer code block.
8. 08_e2e_and_docs_pass.md — the mock triggers (show_text call, corrupt-body fence, fence-only, off-app), tests/e2e/test_verbatim_show_text.py in isolation, the regression set, the full gates, the README/AGENTS full consistency pass, the atomic commit.

Testing & Quality

• Unit: tests/unit/test_config.py (the three defaults; env overrides; 0 accepted as inert; each negative value fails validation naming the field); tests/unit/test_settings_catalog.py (the completeness pin green with the three new rows); tests/unit/test_verbatim_resolver.py (NEW — the locator grammar incl. every tolerant form + rejects, canonical normalization, whole-file/offset/limit/clamp/past-end, the line-boundary cap + TRUNCATION_MARKER + shrunk end_line, fences_in ordering + dedupe, tool-then-fence dedupe, max_blocks cut, an is_image doc refused, an unknown path unresolved, a fenced bor line inside an ordinary fence, an unclosed fence, an empty answer); tests/unit/test_agent.py (the show_text branch: success shape + headers, shown_texts, refusal set — missing args, non-string, unknown path did-you-mean, image doc, bad range args, offset past end, block cap, duplicate identical show; every refusal counts in nothing and executes nothing; agent_tools offers 4 vs 5 tools by flag); tests/unit/test_prompts.py (the clause is INSIDE <tools> before </tools>, the off-build is byte-identical to pre-phase, TOOLS_SECTION itself unchanged); tests/unit/test_chat_gate.py (the per-turn line's new verbatim_blocks=N, and 0 on a turn with none); tests/unit/test_frontend_verbatim_markdown.py + tests/unit/test_frontend_verbatim_blocks.py + a CSS pin module (the test_frontend_* text-pin convention: the verbatim option, the dropped body, data-bor-locator, the .verbatim/.code-block/.verbatim-copy markers, the new <script> tags in index.html + shared.html (plain relative src — no cache-busting token in the path, the ?v= rewrite is the middleware's job, phase 160 A7), shared.js parity, no inline onclick — CSP default-src 'self').
• Integration: tests/integration/test_admin_settings_api.py / test_admin_settings_live.py (all three rows: bool + two ints, correct groups/labels, live apply without a restart, reset-to-env); the chat API suite — a grounded turn with a show_text call and/or a fence yields done.answer_text with the SERVER's text and extent; a turn with none has NO answer_text key (byte-identity); answer_text=false yields the pre-phase frame byte-identically; the saved-chat round trip preserves answer_text (extra="forbid" accepts it — the phase-113 lesson) and GET /api/chats/{id} returns it.
• E2E (AGENTS rule 4, task 08): tests/e2e/test_verbatim_show_text.py, run in isolation — the panel's text equals the fixture file's bytes (the mock's corrupted fence body never appears anywhere in the DOM), the caption reads source/path · lines 12–40 of N from the SERVER's extent, VERBATIM badge + line numbers + Copy button copy EXACTLY the panel's bytes (clipboard read with granted permissions, or the fallback field), the panel's open link targets /document?source=…&path=…, an unresolvable locator renders the honest state with no model bytes, restored-after-reload and the anonymous shared page render the same panel, every ordinary code block has a Copy button and no badge, and the show_image figure still renders (A6 regression).
• Coverage: >90% on app/ (uv run pytest --cov=app --cov-report=term-missing).
• Lint/types: uv run ruff check . && uv run pyright.

Documentation & Consistency

• The final pass reviews README.md and AGENTS.md in full against the project as it stands after this phase — fixing every inconsistency found, including pre-existing ones not caused by this phase.
• Tasks update README.md/AGENTS.md immediately when their own changes are user-visible (setup, commands, configuration, features, conventions). This phase's user-visible additions: the show_text capability + the bor locator syntax in the feature list, the three BOR_* settings in the env table, and (in AGENTS.md rule 5's UI list) the verbatim-panel contract "an answer's code block is either verbatim-and-badged or plainly the model's — never a silently-substituted hybrid".

Completion Criteria

• ask "show me the quadlet for <x>" on a running app (real or mock LLM) renders a panel whose text is byte-identical to documents.content's resolved slice, with a caption whose numbers came from the server, and a Copy button whose clipboard content equals that slice exactly.
• uv run python -c "from app.config import get_settings as g; s=g(); print(s.answer_text, s.show_text_max_chars, s.show_text_max_blocks)" prints True 12000 5; each negative value refuses startup naming its field.
• GET /api/admin/settings lists answer_text, show_text_max_chars, show_text_max_blocks; PUT/DELETE round-trip per the overlay contract.
• A grounded turn's done frame carries answer_text[] with source,path,title,start_line,end_line,total_lines,text,truncated,locator; a turn without verbatim content carries NO key; with BOR_ANSWER_TEXT=false the frame is byte-identical to pre-phase and only FOUR tools are offered.
• The per-turn log line ends with verbatim_blocks=N (grep it on a shown turn and on a plain turn).
• uv run pytest green, app/ coverage >90%, ruff + pyright clean.
• uv run pytest tests/e2e/test_verbatim_show_text.py -v --no-cov green in isolation; test_answer_images.py, test_harness_aligned_tools.py, test_agent_document_tools.py, test_markdown_tables.py (and the phase-180 suite if it landed first) green in isolation.
• README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
• no behavior change in completed phases (image answers, read/grep/ls, deflected byte-identity, the document viewer, saved-chat byte-identity).
• One atomic --no-gpg-sign Conventional Commit (e.g. feat(agent): display file contents verbatim with a show_text tool and server-resolved locator fences); the phase dir moves to .agents/phases/complete/.