brain-of-reese/.agents/phases/todo/95_read_truncation_cap/00_phase.md

Phase 95 — read cap: bounded reads, honest truncation, visible to LLM and user

Source: TODO.md L5 — "Brain of reese can sometimes feed huge documents into the LLM's context - sometimes too large for the LLM to handle. If the LLM calls read on a huge document there should be a sensible cap on the amount it can read at once. My LLMs all have a minimum cap of 128,000 tokens of context, so spec for that. The LLM should be informed the read was truncated, and should be offered a grep or search or find tool (whichever matches most closely to existing harnesses) to search the document for what it was looking for. There should be a visual indicator that the read was trnucated so the user knows what's going on." Story: n/a (owner TODO item — agent tooling on 37_agent_document_tools / 70_harness_aligned_tools; UI on the phase-37 tool lines). Context: read today returns the whole document (app/rag/agent.py _execute_tool read branch: f"Document {doc.source}/{doc.path}:\n{doc.content}") — fine for notes, context-obliterating for the huge files the owner describes. The house already has every primitive this phase needs: TRUNCATION_MARKER ([…truncated…], app/rag/retriever.py), the char-capped one-shot precedent (BOR_SUMMARY_MAX_CHARS, app/rag/summarizer.py), the piece family (StreamPiece / ToolCallPiece / RetryPiece in app/rag/llm.py, consumed by the app/api/chat.py _pump loop as SSE frames), and grep — the exact "search the document" tool the TODO asks for (already harness-aligned, phase 70). The UI renders one .tool-call line per executed call ("📄 Reading …", frontend/assets/app.js ~L898ff; the shared view renders the same from saved records, frontend/assets/shared.js ~L148); saved chats persist the turn's tools array (ToolCall in app/schemas.py, built client-side from the SSE tool frames into toolAcc, restored by app.js ~L1442 and rendered by shared.js).

Objective

A huge read can no longer flood the context: the result is capped (default 128 000 chars ≈ 32k tokens — spec'd against the owner's 128k-token minimum context), the LLM is told the read was truncated and pointed at grep for the rest, and the user sees a "(truncated — showing N of M chars)" marker on the Reading line — live, in saved chats, and on shared pages.

Owner-permitted decisions recorded here (PLAN.md is being redone by the owner)

• A7 scope clarification (owner permission 2026-09-10, TODO.md L5): A7's "never truncated" contract (the retrieved top-2 <documents> context — owner: "this should never happen", phase 24) is unchanged. The cap applies to the read tool path only, per the owner's explicit request in the TODO — the two paths are distinct (retrieval seeds vs. agent-requested additions). This reverses the "this should never happen" ruling for tool reads only; recorded here because PLAN.md is being redone.
• A15 extension (same permission): one new optional SSE event type tool_result (emitted only for truncated reads) — the event-type list of A15 grows from six to seven; existing frames and clients are untouched (a tool_result frame is additive; unknown types are ignored).
• Tool choice: the "grep or search or find tool (whichever matches most closely to existing harnesses)" is grep — it already exists, searches a single document when scoped, and is the phase-70 harness-aligned name. No new tool is added.

Design (shared by all tasks — the executor reads this, not the chat)

The cap (task 01)

• Setting: BOR_READ_MAX_CHARS — read_max_chars: int = 128_000 in app/config.py. Spec rationale (pinned in the docstring): 128 000 chars ≈ 32 000 tokens at the ~4-chars/token estimate the house already uses (app/rag/llm.py embed batching notes ~3 chars/token for code-dense text, 4 for prose) — a quarter of the 128k-token minimum context the owner names, leaving ~96k for the system prompt, the top-2 <documents>, the tool rounds, and the 32 768-token answer cap (max_output_tokens). Char-based (no tokenizer in the repo — the BOR_SUMMARY_MAX_CHARS precedent), env-tunable in both directions.
• The read branch: len(content) > cap → body = content[:cap] + TRUNCATION_MARKER + the pinned notice line: TRUNCATED — this document is {total} characters; only the first {shown} are in your context. The rest is NOT shown. Use grep (pattern) to locate what you need — grep searches the whole document. (constant READ_TRUNCATION_NOTICE with {shown}/{total} format fields, next to the other refusal constants). At exactly the cap: no marker (the document fit).
• Signaling the UI: AgentHolder gains read_truncations: list[tuple[str, int, int]] (argument, chars_shown, chars_total); the read branch appends on truncation. run_agent, after executing each round's calls, yields one new piece per new entry — ToolResultPiece (new, in app/rag/llm.py with the piece family): name, argument, truncated: bool, chars_shown: int, chars_total: int.

SSE + UI + persistence (task 02)

• SSE: app/api/chat.py _pump gains the branch (mirror the ToolCallPiece branch): ChatToolResultEvent (new, app/schemas.py next to ChatToolEvent) → {"type": "tool_result", "name": "read", "argument": "src/path", "truncated": true, "chars_shown": N, "chars_total": M}. Emitted after the matching tool frame (the call is already shown; the marker lands a beat later — the phase-37/48 "calling tool" timing is untouched). The module docstring's SSE contract paragraph + the A15-extension note are updated.
• Live UI: app.js — on a tool_result frame, find the newest .tool-call line whose text is the Reading line for that argument and append <span class="truncated-note"> (truncated — showing {N} of {M} chars)</span> (createElement + textContent — the house "this file never builds HTML" rule; no innerHTML). styles.css: .tool-call .truncated-note { color: var(--ink_soft); } — theme-neutral, no new hue (phase-92 invariant; it must gray out automatically under a monochrome theme — see phase 93).
• Saved + shared: ToolCall (app/schemas.py) gains truncated: bool = False, chars_shown: int | None = None (ge=0), chars_total: int | None = None (ge=0) (phase-83 bounds philosophy: small additive fields, no migration — saved JSON validates). app.js toolAcc: the tool_result frame stamps the matching entry. The save payload carries it; the restore path (app.js ~L1442) and shared.js render the same marker from the stored record, so a saved/shared chat shows the truncation pixel-identically (the phase-50 restore contract).

E2E (task 03)

tests/e2e/test_read_truncation_cap.py: the app under test boots with BOR_READ_MAX_CHARS=1500 (the test_import_extensions_env.py env-override pattern); a local-dir source seeds one ~3 000-char document; a scripted mock-LLM turn reads it; asserts — the SSE stream carries the tool_result frame with the right counts; the Reading line shows the marker; the mock's echo proves the LLM context carried […truncated…] + the grep pointer; the saved chat's tools record carries truncated: true + counts; the shared page renders the same marker.

Tasks

1. 01_read_cap.md — BOR_READ_MAX_CHARS + the truncated read result + ToolResultPiece + tool description + prompt teaching
2. 02_sse_and_ui.md — the tool_result SSE event + the live/saved/shared "(truncated …)" marker
3. 03_e2e_truncated_read.md — the dedicated story suite

Testing & Quality

• Unit/integration: cap boundary (at cap / cap+1); the marker + notice text pinned; a non-truncated read byte-identical to today's result; ToolResultPiece emission order (after the tool frame, before the next round); holder accounting (truncations don't touch tool_calls — a truncated read is still a successful call); the SSE frame in the chat integration suite; the ToolCall schema round-trip (old saved chats without the fields still validate — the phase-50 backward-compat rule); read-already-in-context / refusal paths untouched.
• Coverage: >90% on new/modified code (uv run pytest --cov=app --cov-report=term-missing).
• E2E: uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov in isolation; the existing suites (test_agent_document_tools.py, test_chat_history.py, test_share_chat.py, test_big_read_progress.py) stay green in isolation.
• Lint/types: uv run ruff check . && uv run pyright.

Completion Criteria

• a document longer than BOR_READ_MAX_CHARS returns first-cap-chars + […truncated…] + the pinned grep-pointer notice to the LLM (unit-pinned)
• the user sees "(truncated — showing N of M chars)" on the Reading line — live, in a saved chat, and on the shared page
• a document at/under the cap is read byte-identically to today (no marker, no frame)
• the top-2 <documents> retrieval path is untouched (A7's never-truncated contract holds for it)
• test suite green, coverage >90%, ruff + pyright clean
• no behavior change in completed phases; one atomic Conventional Commit, --no-gpg-sign (e.g. feat(agent): cap read at 128k chars with honest truncation and a visible UI marker)