add PLAN.md

chore(agent): phase 93-95 roadmap from TODO.md — theme completion, ls tree drill-down, read truncation cap
Convert the three unchecked TODO.md items into an executable phase roadmap (Protocol B, appended after phase 92): - 93_theme_semantic_completion (TODO L3): the ok/err/accent state families become Theme-tab-controlled (B3 revised, owner permission 2026-09-10) + surface panels behind every page head - 94_ls_tree_drilldown (TODO L4): ls becomes a source -> folder -> file tree with sync-time lite-model folder summaries; controlled tool-calling battery as the accuracy/performance gate - 95_read_truncation_cap (TODO L5): read capped at BOR_READ_MAX_CHARS (128k chars ~= 32k tokens, spec'd on the 128k-token minimum context), LLM-visible truncation notice pointing at grep, new tool_result SSE event (A15 extension) + the visible UI marker Owner decisions (B3 / A7 scope / A15) are recorded in the phase files; .agents/PLAN.md is being redone separately per the owner.
2026-09-10 13:08:55 -04:00 · 2026-09-10 11:37:25 -04:00 · 2026-09-10 09:28:58 -04:00
24 changed files with 963 additions and 0 deletions
@@ -0,0 +1,372 @@
 # Brain of Reese — Master Plan (minimal)
 > **Status:** Minimal re-land (2026-09-10). Phases 01–92 shipped;
 > `todo/` holds 93–95 (see §12). Section numbers and anchor IDs match the
 > previous full plan (in git history, commit `dac4a3e`) so the codebase's
 > `PLAN §…` / anchor comments stay valid — consult that revision or the
 > phase records in `.agents/phases/complete/` for full detail and the
 > complete owner-permission revision log.
 >
 > **Rule:** Every agent reads this file first. `LOCKED` anchors in §2 are
 > settled — a change requires explicit owner permission, recorded as a
 > dated revision note under the table. Never a silent deviation
 > (AGENTS.md rule 3).
 >
 > **This is deliberately minimal:** it captures the tech stack, the
 > architecture an agent needs to work, and the locked decisions. It does
 > NOT recap the shipped work — the git log, the README, and the completed
 > phase directories are the history.
 ---
 ## 1. Mission
 A **knowledge-base chatbot** ("Brain of Reese") over the owner's homelab
 documentation — git repos, local directories, and uploaded archives
 registered on the admin Sources page — embedded into Postgres + pgvector.
 Product feel: chippy, upbeat, and **radically honest** — if retrieval
 surfaces nothing relevant it says *"I haven't done anything like that"*
 and offers alternatives instead of hallucinating. Self-hosted LLMs only.
 **Deliberately out of scope:** binary/non-text ingestion; real-time file
 watching (the import script / Sync button is the refresh loop); PR
 tooling for docs push (the owner opens the PR); multi-admin/per-user
 accounts (one admin + hand-out tokens is the model).
 ---
 ## 2. Tech Stack & Architectural Anchors (LOCKED)
 | # | Component | Decision |
 |---|-----------|----------|
 | A1 | Runtime | Python 3.12+, `uv` for all package management (API + scripts + tests, one venv) |
 | A2 | Web framework | FastAPI + Pydantic v2 + Uvicorn (async, SSE-friendly) |
 | A3 | Database | **PostgreSQL 17 + pgvector** (local build of official `postgres:17`, `db/Containerfile`), cosine (`<=>`) search; one system for relational + vectors |
 | A4 | Orchestration | `compose.yaml`, `podman compose up -d` (dev: `db` only; `--profile prod` adds the app container) |
 | A5 | LLM backend | OpenAI-compatible self-hosted endpoint `https://aipi.reeseapps.com/v1` via the `openai` async client: **`turbo`** (chat, streams `reasoning_content` thinking), **`embed`** (embeddings), **`lite`** (one-shot: document summaries, KB overview; phase 94 adds sync-time folder summaries) |
 | A6 | Embedding dim | **768** (verified against the live endpoint); `chunks.embedding` is fixed at table creation — a dim mismatch must **fail loudly**, never silently re-embed |
 | A7 | Retrieval→context | **Hybrid:** cosine top-100 ∪ Postgres FTS top-30 (OR tsquery, `ts_rank`), fused with **RRF (k=60)** → parent docs ranked by best fused chunk → **full text of top-N=2 documents, never truncated** on the retrieval path (owner 2026-08-24: "this should never happen"; the agent `read`-tool cap is a separate owner-permitted path — phase 95) |
 | A8 | Honesty gate | Deflect (LOW mode) **only when** best cosine < `BOR_RELEVANCE_THRESHOLD` (0.62) **and** zero FTS hits; LOW prompt carries weak-hit *titles only* + the `DEFLECT_MODE` marker (the E2E mock keys on its presence) + the plain-text no-tools line |
 | A9 | Content scope | Default `md, markdown, txt, yaml, yml, json, py` + Podman quadlet family + `j2`; `BOR_IMPORT_EXTENSIONS` may name **any** well-formed extension or narrow the set; hidden (dot) paths + exclusion list + per-source `ignore_paths` (raw prefixes, no globs) always apply |
 | A10 | State & auth | `POST /api/chat` is **stateless** (client-provided `history` only, budget-trimmed — nothing stored per conversation). Auth = single admin, signed `bor_session` cookie (Starlette `SessionMiddleware` + itsdangerous; no server-side session store); admin-issued SHA-256-hashed access tokens are the only other identity; **only** shared chats stay anonymous. Fail-loud at boot while `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` are empty |
 | A11 | Frontend | Vanilla HTML/CSS/JS in git, **no CDN** — everything served by FastAPI `StaticFiles`; system font stack; the navbar views are views of ONE shell document (`frontend/index.html` + `router.js` deep-links, phase 76) |
 | A12 | Aux services | **None** (no Valkey/queue/SeaweedFS): sync runs in-process, the login rate limit is in-memory per-process, sessions are the signed cookie. Restart clearing in-memory state is accepted |
 | A13 | Migrations | Alembic + SQLAlchemy 2.0 (sync) + psycopg 3; **every migration ships a tested downgrade** (A13 — reversible) |
 | A14 | Debugging | `debugpy` imported **only when `DEBUGPY=1`** (`app/core/debugging.py`); never imported otherwise (unit-tested) |
 | A15 | Chat transport | **SSE** from `POST /api/chat`; event types `thinking`, `delta`, `tool`, `retry`, `done`, `error` (+ `tool_result`, owner-permitted extension — phase 95, not yet landed); no proxy buffering, no client caching on the stream |
 | A16 | Testing | Per phase: unit + integration (pytest) with **>90% coverage on `app/`** + one dedicated **Playwright E2E file**, run **in isolation** (`--no-cov`); E2E uses a deterministic mock LLM by default (`E2E_REAL_LLM=1` opts into live aipi) |
 | A17 | Git | Conventional Commits, **always `--no-gpg-sign`**, one atomic commit per completed phase |
 | A18 | Docs push | Save-a-answer-as-doc: server-side draft (`doc_drafts`, long body never in a URL — unguessable `uuid4` token is the URL credential) pushed to a **generic git remote** (`BOR_DOCS_REPO`, URL or local path; no `gh`) on a dedicated branch, `--ff-only`, re-cut per push; **no PR tooling**; inert (hidden + 409) while the repo is unset |
 | A19 | Deploys & caching | HTML pages `Cache-Control: no-cache` (no etag/last-modified); `/assets/*` versioned (`?v=<token>`) + `immutable, max-age=1y`; **the token is the deploy** (git `HEAD` short SHA; content-hash fallback) — a deploy is a commit the browser sees without a hard refresh |
 | A20 | Security headers | Every response: `Content-Security-Policy: default-src 'self'` (+ the theme's inline-`<style>` hash on themed pages), `frame-ancestors 'none'`, `X-Content-Type-Options: nosniff`; login rate-limited 10 attempts / 15 min (in-memory, A12) |
 **UI/theming anchors (B):**
 | # | Decision |
 |---|----------|
 | B1 | Theme `ui_settings` columns: **NULL/empty = "use the default"** — env value for the 3 strings, built-in palette for colors (colors have no env fallback) |
 | B3 | **Revised (owner 2026-09-10, phase 93):** the semantic state families (`--ok-*`, `--err-*`, `--accent-*`) **join** the storable palette — 17 variables total, all `NULL = built-in` (the built-in theme stays byte-identical). The 2026-09-09 lock that they are "not identity" is lifted |
 | B4 | **Byte-identical contract:** with no `ui_settings` row (or a fully default theme) the served HTML is byte-identical to the built-in default — no `#bor-theme` tag, caching rewrites are rewrite-only |
 | B5 | Admin-only nav views (Sources, Git sources, Tuning, History, Tokens, Theme) are hidden from non-admins; a monochrome theme must keep every state **text** label ("text + color, never color alone") |
 **Key recent revisions (full log in the previous plan revision / phase
 records):** A10 — single-admin cookie auth (16), saved chats + shares +
 staleness (50/51/53), client history budgets (74), access tokens (79).
 A15 — `thinking` (17), `tool` (37), unlimited tool rounds under
 `BOR_AGENT_MAX_ROUNDS` (45), `retry` (67), harness-aligned
 `ls`/`read`/`grep` surface (70). A7 — the **retrieval** path's
 never-truncated contract is unchanged; the phase-95 `read`-tool cap is
 the only exception, owner-permitted 2026-09-10.
 ---
 ## 3. High-Level Architecture
 ```
 Browser (SPA shell, no CDN)
  │  HTTP pages · SSE chat · JSON API
  ▼
 FastAPI app (app/main.py)
  ├── middleware: SessionMiddleware → caching (A19) → SecurityHeaders (A20)
  ├── /api/*  chat, docs, git-sources, steering, sync, chats,
  │           doc-drafts, tokens, ui-settings, auth, health, suggestions
  ├── RAG pipeline (app/rag/)  embed → retrieve → gate → agent → turbo
  └── StaticFiles (frontend/)  — shell routes registered before the mount
  │            SQL (psycopg)                     OpenAI-compat
  ▼            ▼                                 ▼
 Postgres 17 + pgvector              aipi.reeseapps.com/v1
 (db service, podman compose)        (self-hosted: turbo / embed / lite)
 Sources registry (Postgres `git_sources`, admin-managed; DB rows win,
 BOR_GIT_SOURCES is the empty-table git-only fallback):
  git repos → clone/pull via scripts/git_sync.py (the ONLY git-invocation
              site; stdlib subprocess)   local dirs → walked directly
  archives  → unpacked under BOR_UPLOAD_DIR (zip-bomb guarded)
 ```
 | Component | Lives in |
 |-----------|----------|
 | App + middlewares + auth/tokens/rate-limit/theming/caching | `app/main.py`, `app/core/` |
 | RAG (retriever, prompts, agent, llm, chunker, importer, summarizer, overview, scaffolding, suggestions, …) | `app/rag/` |
 | API routers | `app/api/` |
 | Import / sync tooling | `scripts/import_docs.py`, `app/api/sync.py`, `scripts/git_sync.py`, `scripts/eval_retrieval.py`, `scripts/llm_probe.py` |
 | Frontend (shell + standalone pages) | `frontend/` (`index.html` shell; `document.html`, `shared.html`, `login.html`, `doc-edit.html`) |
 | Migrations (0001–0015, all reversible) | `alembic/versions/` |
 ---
 ## 4. Chat Turn & SSE Contract (§3/§4 in older comments)
 ```
 POST /api/chat {message, history?}   (auth: require_user)
  → embed(question)                                   [retried, phase 67 / A15 ext.]
  → cosine top-100 ∪ FTS top-30 → RRF fuse (k=60)     [A7]
  ├─ HIGH (cosine ≥ 0.62 OR fts_hits > 0):
  │    persona + <knowledge_base> + <tuning> + full top-2 <documents>
  │    + <tools> → agent loop (ls/read/grep; round-capped
  │    BOR_AGENT_MAX_ROUNDS=10, 0 = tools off) → turbo streamed
  └─ LOW (A8): DEFLECT_MODE prompt (weak titles only, no tools,
       byte-identical direct-stream path) → turbo streamed
  → query_log row + per-turn log line (§9)
 ```
 SSE frames (`data: <json>\n\n`): `thinking` (before first delta) →
 `tool` (grounded turns, one per model call) → `retry` (pre-first-frame
 restarts) → `delta` (answer tokens) → `done` `{deflected, sources[],
 suggestions[]}` (terminal). Failure: `error` (terminal — no `done`, no
 `query_log` row; a pre-stream DB outage is a plain 503 JSON).
 `BOR_STREAM_THINKING=0` suppresses `thinking` frames server-side (chars
 still counted). LLM retries (phase 67, A15 extension): `BOR_LLM_RETRIES`=3 × flat
 `BOR_LLM_RETRY_DELAY`=5 s, **only before a request has streamed its
 first output frame**.
 Retrieval details (A7/A8) and the persona/`<tools>` prompt contract: see
 the previous plan revision §6 or `app/rag/retriever.py` /
 `app/rag/prompts.py` / `app/rag/agent.py` — the module docstrings carry
 the full contracts. **Persona text changes through the plan, not in code
 (phase 03 convention); the `DEFLECT_MODE` marker and the mock-LLM markers
 (`SUMMARY_MODE`, `KB_OVERVIEW_MODE`, …) may not change without updating
 `tests/e2e/mock_llm.py`.**
 ---
 ## 5. Data Model & Chunking (§5 in older comments)
 Tables (full column detail: `app/models.py` — it is the living doc;
 migrations 0001–0015, each with a tested downgrade):
 | Table | Purpose |
 |-------|---------|
 | `documents` | one row per imported file — full content, `(source, path)` unique, sha256 `content_hash`, `lite` `summary` (non-markdown) |
 | `chunks` | retrieval units; `embedding VECTOR(768)`; `position −1` = the embedded summary chunk; stored `tsvector` (GIN) for FTS |
 | `query_log` | every question: top score, FTS hits, deflection, sources, latency (threshold-tuning record) |
 | `steering_notes` | owner tuning notes → `<tuning>` section of every turn (char-budgeted) |
 | `kb_overview` | single row `id=1`: `lite`-generated KB outline → `<knowledge_base>` section (regenerated on KB change) |
 | `git_sources` | source registry: `kind` `git`\|`local`, `url`/`path`, `ignore_paths` JSONB (phase 89) |
 | `saved_chats` | owner-saved conversations; `messages` = the raw `bor.chat.v1` JSONB; `share_token` (NULL = private, `uuid4` → `/shared/<token>`); `sources_version` (stale when < current) |
 | `sources_meta` | single row `id=1`: the KB generation counter — bumped once per KB-changing sync |
 | `doc_drafts` | save-as-doc drafts; `token` (uuid4) is the URL credential; `draft` → `pushed` (branch + sha) |
 | `api_tokens` | access tokens; only the SHA-256 of the full `bor_…` string is stored; `revoked_at` = dead |
 | `ui_settings` | single row `id=1`: Theme tab persistence (3 strings + 9 identity colors; NULL = default, B1 — grows to 17 vars in phase 93) |
 Single-row tables use `id = 1` (the `kb_overview` precedent).
 **Chunking** (format-aware, `app/rag/chunker.py`): markdown on
 `##`/`###` headings (paragraph sub-split > `BOR_CHUNK_TARGET_CHARS`=2000,
 200 overlap); YAML/JSON on top-level keys; Python on top-level defs via
 `ast`; txt on paragraphs; quadlet/j2 plain text. Every format honors the
 **1200-char hard cap** (aipi ~1024-token request limit).
 **Import workflow** (script and UI Sync share the importer): sha256
 delta (unchanged files skip), two-phase upsert (one transaction per
 file), `--prune` drops deleted/ignored files, non-markdown files get
 `lite` summaries (best-effort/fail-soft), and a KB-changing run
 regenerates the `kb_overview` + bumps `sources_meta.version` exactly
 once. A failed source aborts the run — no partial junk.
 ---
 ## 6. Retrieval & Persona
 (See §4's flow and the anchor rows A7/A8. The locked persona, section
 order — `<relevance>` → `<knowledge_base>` → `<tuning>` → mode body —
 and the `<tools>` teaching live in `app/rag/prompts.py`; the agent loop,
 teaching refusals, and scaffolding filter in `app/rag/agent.py` /
 `app/rag/scaffolding.py`. Empty prompt sections omit themselves — a
 no-notes/no-overview prompt is byte-identical to the pre-steering text.)
 Tool surface (harness-aligned, phase 70): **`ls`** (lists indexed docs
 `source: X | path: Y | title: Z`; `path` = a source name — becomes a
 drill-down tree in phase 94), **`read`** (combined `source/path`
 including the source name; appends the full document — gets a
 truncation cap in phase 95), **`grep`** (case-insensitive **fixed
 substring**, ≤20 `source/path:line: text` matches, optional one-doc
 scope; a locator that adds no source/context — never a regex).
 Rejected calls get deterministic teaching refusals and consume a round;
 `holder.tool_calls` counts executed calls only.
 ---
 ## 7. UI/UX Standards
 - **Layout (§7.1):** sticky 64px header + `<main>` + footer; container
  `max-width: 72rem` centered; chat is a **centered 46rem column** (2×
  = 92rem at ≥1500px desktops — deliberate, not a bug); Sources uses a
  **full-width responsive table** — no skinny single-column lists
  (lists/tables/grids ≥80–90% of container width); document viewer is a
  near-fullscreen same-page modal; mobile ≤640px: hamburger nav,
  ≥44px touch targets, safe-area composer.
 - **Accessibility (§7.2, WCAG 2.1 AA):** semantic landmarks on every
  view, labeled controls (icon buttons get `aria-label`),
  `aria-live="polite"` stream, `role="status"`/`role="alert"`,
  3px `:focus-visible`, `prefers-reduced-motion` respected, text pairs
  ≥4.5:1 (state is **text + color, never color alone**).
 - **Theming (B1–B5):** built-in dark palette in
  `app/core/theming.py::BUILTIN_COLORS` (the `:root`-drift unit test
  parses built-ins from `styles.css` — no second copy); the admin Theme
  tab persists the palette in `ui_settings`, injected pre-paint as
  `<style id="bor-theme">` with a matching CSP hash (A20); **B4
  byte-identical** when nothing is set.
 - **No CDN (§7.3):** zero external `<script>`/`<link>` (integration test
  on the index page); markdown rendering is a small local
  escape-first function; esbuild minify is build-time only.
 - **Never-stale feedback (§7.4):** every control state — idle /
  thinking / calling tool / streaming / retrying / done (answer or
  deflected) / error / KB-offline / stopped — has a defined UI, every
  failure path re-enables its controls, a 300 s pre-token guard
  (`TURN_TIMEOUT_MS`) turns a hung stream into the error state (counts
  only visible time — phase 73), and **no auto-follow during a turn**
  (viewport moves only on user intent, phase 42). Pinned by unit tests
  on the `app.js` state machine + the story E2E suites.
 - Frontend house rules: `app.js` and siblings **never build HTML
  strings** (createElement + textContent); asset paths carry
  `?v=<token>` (A19).
 ---
 ## 8. Debugging
 `DEBUGPY` unset/`0` → `debugpy` never imported (unit-tested, A14).
 `DEBUGPY=1` → non-blocking listener on `0.0.0.0:${DEBUGPY_PORT:-5678}`;
 IDE attaches on demand. Wired at `app/main.py` module import
 (`app/core/debugging.py`), so `uvicorn app.main:app`, `python -m
 scripts.…`, and tests all honor it.
 ---
 ## 9. Observability
 - Logs: single-line `timestamp LEVEL logger :: message` on stdout, INFO
  default (`BOR_LOG_LEVEL`), third-party loggers capped at WARNING.
 - **Per-chat-turn line (required — AGENTS.md rule 10):**
  ```
  question=… embed_ms=… top_score=… fts_hits=… summary_hits=… tuning=N
  kb_chars=N history_msgs=N threshold=… deflected=… sources=…
  thinking_chars=… tool_calls=N total_ms=… retries=N scaffold_stripped=N
  ```
  (`sources=` = retrieval + agent-read docs, deduped; `tool_calls=`
  executed only; a cancelled turn writes neither line nor `query_log`
  row.)
 - Importer: per-file `added|updated|unchanged|pruned` + a greppable
  cron-safe summary line (`import: summary files=… added=… … formats=…`).
 - `query_log` is the durable tuning/gap-finding record.
 ---
 ## 10. Testing & Quality Gates (A16 — non-negotiable)
 | Gate | Command |
 |------|---------|
 | Unit + integration | `uv run pytest` |
 | Coverage **>90%** on `app/` | `uv run pytest --cov=app --cov-report=term-missing` |
 | Story E2E, **in isolation** | `uv run pytest tests/e2e/test_<story>.py -v --no-cov` (DB up: `podman compose up -d db`) |
 | Lint + types | `uv run ruff check . && uv run pyright` |
 E2E determinism: `tests/e2e/mock_llm.py` is a deterministic
 OpenAI-compatible mock (genuine L2-normalized token-overlap embeddings,
 so the cosine gate behaves like production); `tests/e2e/slow_llm.py` is
 the slow/dead variant; `E2E_REAL_LLM=1` opts into live aipi. Story E2E
 fixtures truncate/re-import per module — suites must run in isolation.
 Real-model tool-calling verification uses the controlled methodology in
 `TOOL_CALLING_TESTING.md` + `scripts/agent_realmodel_check.py`
 (fixture KB: `tests/fixtures/test_kb.dump.sql`; the 1,000-doc live-replica
 snapshot restores via the `restore-test-db` skill).
 ---
 ## 11. Import & Update Workflow
 ```bash
 uv run python -m scripts.import_docs                # sync sources + index delta
 uv run python -m scripts.import_docs --prune        # also drop deleted/ignored
 uv run python -m scripts.import_docs --source ~/X   # extra directories
 # or one click: the admin Sources page "Sync sources" (in-process, 409 while running)
 uv run python -m scripts.eval_retrieval "<question>"   # rank hybrid results (tuning)
 uv run python -m scripts.llm_probe                     # models + embedding dim sanity
 ```
 The loop for git sources is *commit → re-run*. Uploads:
 `POST /api/git-sources/upload` unpacks + registers (202); the scan is
 deferred to Sync (phase 90). Source removal prunes on the next sync
 (phase 69).
 ---
 ## 12. Current State & Roadmap
 - **Shipped: phases 01–92** (`.agents/phases/complete/` — read-only
  history; the shipped-features recap of the previous plan revision and
  the README cover it). Migrations through 0015.
 - **Next up (`todo/`, in order — already authored via
  `phase-authoring`):**
  1. `93_theme_semantic_completion` — 8 semantic state colors join the
     Theme-tab palette (17 total, B3 revised) + surface panels behind
     every page head.
  2. `94_ls_tree_drilldown` — `ls` becomes a filesystem-style drill-down
     tree with sync-time `lite` folder summaries (`folder_summaries`
     table, migration 0017) + real-model battery.
  3. `95_read_truncation_cap` — capped `read` (default 128 000 chars
     ≈ 32k tokens, `BOR_READ_MAX_CHARS`), honest truncation notice to
     the LLM + visible "(truncated — showing N of M chars)" marker
     (new optional `tool_result` SSE frame — A15 extended to seven
     event types).
 - **Next free phase number: 96.**
 - **Post-v1 hooks (deliberately not built):** HNSW index at scale;
  inotify auto-import; more providers (the OpenAI-compatible client is
  the seam); multi-user accounts (the `require_user` split is the seam);
  index-backed `grep`.
 **Per-phase completion** (AGENTS.md rules 8/9): unit + integration
 green, coverage >90%, the phase's Playwright E2E green in isolation, UI
 verified against §7, one atomic `--no-gpg-sign` Conventional Commit,
 phase dir moved to `complete/`.
 ---
 ## 13. House Conventions (quick catch-up)
 - **Env:** every setting is a `BOR_`-prefixed env var (or gitignored
  `.env`; see `.env.example`); `get_settings()` is `lru_cache`d.
  Kill switches follow the `agent_max_rounds` pattern: `0` disables the
  feature, a negative value fails startup **loudly** (validator names
  the field).
 - **Fail loud, never half-configured:** missing admin secrets, bad
  extension lists, dim mismatches, unresolvable docs branches — all
  refuse to start or refuse the request with a named reason.
 - **Shared marker:** `TRUNCATION_MARKER = "[…truncated…]"`
  (`app/rag/retriever.py`) is the only overflow marker; char budgets are
  the pattern for anything sent to `lite`.
 - **Byte-identical contracts are load-bearing:** deflected turns
  (A8), empty prompt sections, the B4 theme no-op, and the phase-33/54
  rewrite-only caching all have tests that assert byte-identity.
 - **git is invoked only in `scripts/git_sync.py`** (A11 — stdlib
  subprocess); everything else talks Postgres.
 - **One story → one phase → one dedicated Playwright file** (AGENTS.md
  rules 4/9); `.agents/` is tracked; only `.agents/phase-sessions/` and
  `.agents/pipeline.log` are gitignored.
 - **Full-history references:** the previous full plan (all revision
  notes, complete API table, column-level data model) is in git —
  `git show dac4a3e:.agents/PLAN.md`; per-phase decisions are in
  `.agents/phases/complete/*/00_phase.md`; owner test methodologies in
  `TOOL_CALLING_TESTING.md`; skills for model testing/KB restore in
  `.agents/skills/`.
@@ -0,0 +1,62 @@
 # Phase 93 — Theme completion: semantic state colors become tab-controlled + readable page heads
 **Source:** `TODO.md` L3 — "I created a black/white/gray theme for brain of reese and found multiple cases of color still in the UI which tells me the customization is not complete. Screenshots are in the theme_fixes/ folder. Note the green text "Theme saved", the red "Revoked" tag, the red "Stale" tag, The green "Local" tag, The yellow "Listing documents" and "Reading" tool calls. Also the header and description of each page needs a background - the grid makes it hard to read."
 **Story:** n/a (owner TODO item — theme-customization completion on `91_admin_theme_tab` / `92_theme_save_and_coverage`).
 **Context:** The admin Theme tab (phases 91/92) persists the **9 identity variables** (`bg, surface, ink, ink_soft, line, grid_line, brand, brand_soft, brand_ink`) in the single-row `ui_settings` table (`app/models.py` `UiSettings`), resolves them in `app/core/theming.py` (`BUILTIN_COLORS` / `COLOR_FIELDS` / `effective_settings`), serves them via `GET/PUT /api/ui-settings` (`app/api/ui_settings.py`), injects them pre-paint as `<style id="bor-theme">:root{…}</style>` with a matching CSP hash (`theme_style_tag` / `theme_csp_hash` / `app/core/caching.py`), and edits them live in `frontend/assets/theme.js` (the `FIELDS` array drives pickers, live preview, PUT body, contrast warnings). Phase 92 removed every hardcoded color literal, but the **semantic state families are raw `:root` variables the tab cannot reach** — `--ok-bg/--ok-ink` (the green "Theme saved." result text, the green `LOCAL` badge `.git-source-kind.is-local`), `--err-bg/--err-ink/--err-line` (the rose `Stale` / `Revoked` pills `.stale-pill`), `--accent-bg/--accent-ink/--accent-line` (the amber agent tool-call lines `.tool-call`), built-in values at `frontend/assets/styles.css` L29–36. Separately, every page's `h1` + description (`.page-head`) sits directly on the background grid texture and is hard to read (all 8 screenshots in `theme_fixes/`).
 ## Objective
 Make theme customization **complete**: a monochrome (black/white/gray) theme can be saved with zero residual color, because the three semantic state families become Theme-tab-controlled (17 palette variables total) with `NULL = built-in` (the default theme renders byte-identical to today); and every page's header + description block gets a solid surface background so the grid never fights the heading text. State stays honest: every state element keeps its **text** label (the "text + color, never color alone" house rule), so a grayscale theme conveys state by words, not hues.
 ## Owner-permitted decision recorded here (PLAN.md is being redone by the owner)
 - **B3 revised (owner permission 2026-09-10, TODO.md L3):** the semantic families `--ok-*`, `--err-*`, `--accent-*` become **storable and Theme-tab-controlled** (the 2026-09-09 lock that they "are NOT identity … not configurable from the tab" is lifted). `NULL = built-in`, so the built-in theme's look and all existing AA ratios are unchanged; only the tab's reachable surface grows.
 - Header panel design (owner-permitted by the same TODO line — "needs a background"): a solid `var(--surface)` panel behind each page head, not a full-bleed band (ASSUMPTION in task 03).
 ## Dependencies
 - `92_theme_save_and_coverage` (complete) — the whole phase builds on it: the `ui_settings` row + resolver + admin API, the pre-paint injection + CSP hash in `app/core/caching.py`, the `#view-theme` form (`frontend/index.html`) + editor (`frontend/assets/theme.js`), and the E2E suites `tests/e2e/test_admin_theme_tab.py` / `test_theme_save_and_coverage.py` / `test_configurable_brand.py` (all stay green — a no-op theme must remain byte-identical).
 ## Design (shared by all tasks — the executor reads this, not the chat)
 ### 17-variable palette (tasks 01, 02)
 The 9 identity variables are untouched. **8 semantic variables** join the storable palette, column names mirroring the CSS variables:
 | Column / field | CSS var | Built-in (from `styles.css` `:root`) | UI role |
 |---|---|---|---|
 | `ok_bg` | `--ok-bg` | `#10241b` | success pill/text background ("Theme saved.") |
 | `ok_ink` | `--ok-ink` | `#6ee7a8` | success text |
 | `err_bg` | `--err-bg` | `#2d0a0a` | Stale/Revoked pill background |
 | `err_ink` | `--err-ink` | `#fca5a5` | Stale/Revoked pill text |
 | `err_line` | `--err-line` | `#ef4444` | Stale/Revoked pill border (decorative — no contrast duty) |
 | `accent_bg` | `--accent-bg` | `#2b2110` | deflection banner / tool-line background |
 | `accent_ink` | `--accent-ink` | `#fbbf24` | tool-call line text ("Listing…", "Reading…") |
 | `accent_line` | `--accent-line` | `#f59e0b` | tool-line left border / deflection border (decorative) |
 Server plumbing mirrors the phase-92 `grid_line` pattern exactly: migration `0016` (after `0015_grid_line.py`), 8 nullable `String(7)` columns, `BUILTIN_COLORS` extended (the `:root`-drift test in `tests/unit/test_theming.py` parses the built-ins from `styles.css` — no new hardcoded palette copy), `COLOR_FIELDS` order = the 9 identity vars **then** the 8 semantic vars (structural first, brand middle, state last). Everything downstream is `COLOR_FIELDS`-driven and picks the 8 up with **zero logic change**: `effective_settings`, `theme_style_tag` (the pre-paint tag gains 8 declarations when any var is non-default; the byte-identical no-op contract holds — the tag is still `""` only when all 17 equal their built-ins), `theme_csp_hash` (recomputed per theme, runtime), the `app/core/caching.py` injection, and the `COLOR_FIELDS`-driven loops in `app/api/ui_settings.py`.
 ### Contrast pairs 5 → 8 (task 02)
 The tab's client-side AA warnings (`theme.js` `PAIRS`, L184) gain three ink-on-bg pairs: `ok_ink` on `ok_bg`, `err_ink` on `err_bg`, `accent_ink` on `accent_bg`. The two `_line` vars stay excluded (decorative borders, no contrast duty — same rule as `--line`/`--grid-line`). The authoritative table lives in `app/core/theming.py`'s docstring — extend it there and in `theme.js`'s pair list together.
 ### Page-head panel (task 03)
 `.page-head` (the shell's standard frame: `h1` + description — `styles.css` L1536 ff.) gets `background: var(--surface)`, `padding`, `border: 1px solid var(--line)` and the house card radius, on EVERY page: the seven shell views (`#view-chat`, `#view-tuning`, `#view-rag`, `#view-git-sources`, `#view-history`, `#view-tokens`, `#view-theme` — one shared rule covers them) plus the standalone pages' header blocks (`frontend/login.html`, `frontend/document.html`, `frontend/shared.html`, `frontend/doc-edit.html` — audit each; apply the same panel treatment to whatever class carries their h1 + lede, including the shared-page head that reuses the `.page-head h1` size per the `styles.css` L3189 note). The panel must not break: the `#view-history .page-head` flex row (title left, stale pill right — `styles.css` L2550), the mobile wrap of `.page-head-row` (L4221), or any existing header/nav/responsive E2E.
 ### Monochrome E2E (task 04)
 One dedicated story suite: as admin, `PUT /api/ui-settings` an all-gray 17-color theme (every channel R=G=B; pairs still ≥ 4.5:1), then assert per element that the **computed** color is grayscale and the **text** state label is still present: "Theme saved." result (`#theme-result`), a `Stale` pill (History — seed an out-of-generation saved chat, the phase-53 pattern in `tests/e2e/test_stale_saved_chats.py`), a `Revoked` pill (Tokens — create + revoke via the UI), the `LOCAL` badge (Git sources — register a local-directory source, the `tests/e2e/test_local_directory_sources.py` pattern), and a `.tool-call` line (Chat — a mock-LLM turn that executes a tool, the `tests/e2e/test_agent_document_tools.py` pattern). Plus: every page head's computed `background-color` is non-transparent, and a fresh `page.goto` of each page paints the gray palette pre-paint (the `#bor-theme` tag carries all 17 when non-default).
 ## Tasks
 1. `01_semantic_columns.md` — migration 0016 + `UiSettings` columns + `BUILTIN_COLORS`/`effective_settings` + schema + API validation
 2. `02_theme_tab_state_section.md` — `#view-theme` form fields + `theme.js` `FIELDS`/`PAIRS` (5→8 pairs) + live preview/Reset coverage
 3. `03_page_head_background.md` — `.page-head` (and standalone-page) surface panel across every page
 4. `04_e2e_monochrome_theme.md` — dedicated Playwright suite: full-gray theme → zero residual color, state text intact, heads readable
 ## Testing & Quality
 - Unit/integration: resolver round-trip for all 17 (NULL=clear, built-in→NULL normalization, 422 on bad hex names the field); pre-paint tag + CSP hash with the 17 vars (non-default subset, no-op byte-identical); drift test covers the 17; `PUT/GET /api/ui-settings` integration for the 8 new fields; existing theming suites (`tests/unit/test_theming.py`, `test_ui_settings.py`, `test_caching.py`, `tests/integration/test_ui_settings_api.py`, `test_security_headers.py`) stay green.
 - Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
 - E2E: `uv run pytest tests/e2e/test_theme_semantic_completion.py -v --no-cov` in isolation; the existing theme suites (`test_admin_theme_tab.py`, `test_theme_save_and_coverage.py`, `test_configurable_brand.py`, `test_dark_tech_theme.py`, `test_header_consistency.py`, `test_nav_*`, responsive) stay green in isolation.
 - Lint/types: `uv run ruff check . && uv run pyright`.
 ## Completion Criteria
 - [ ] An all-gray 17-color theme saved from the tab renders with **no colored element anywhere** (verified by the new E2E's computed-style assertions) while Stale/Revoked/Local/Tool state text remains visible
 - [ ] Default (no row / all NULL) deployment serves byte-identical HTML (no `#bor-theme` tag) and the built-in theme's 8 semantic look is unchanged
 - [ ] Every page head (7 shell views + 4 standalone pages) has a solid surface background over the grid
 - [ ] test suite green, coverage >90%, ruff + pyright clean, dedicated E2E green in isolation
 - [ ] no behavior change in completed phases (phases 01–92 suites green)
 - [ ] one atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(theme): make semantic state colors tab-controlled and give page heads a surface panel`)
@@ -0,0 +1,32 @@
 # Task 01 — Semantic color columns: migration, resolver, schema, API
 **Phase:** `93_theme_semantic_completion` · **Source:** `TODO.md:3` — "I created a black/white/gray theme for brain of reese and found multiple cases of color still in the UI which tells me the customization is not complete. … Note the green text "Theme saved", the red "Revoked" tag, the red "Stale" tag, The green "Local" tag, The yellow "Listing documents" and "Reading" tool calls. …" (this task makes the 8 semantic values storable + resolvable — the server half)
 **Story:** n/a (owner TODO item).
 ## Objective
 Persist and resolve the 8 semantic palette values (`ok_bg, ok_ink, err_bg, err_ink, err_line, accent_bg, accent_ink, accent_line`) with the exact same plumbing as the 9 identity vars, so `effective_settings` returns all 17 and the pre-paint tag / CSP hash pick them up with zero further change.
 ## Work
 1. `alembic/versions/0016_ui_settings_semantic.py` — new migration (down revision `0015_grid_line`, the current head): add the 8 nullable `String(7)` columns to `ui_settings` (no server defaults — the row is created only by the PUT upsert, house rule); **tested downgrade** drops them (A13).
 2. `app/models.py` — `UiSettings`: the 8 `Mapped[str | None] = mapped_column(String(7), nullable=True)` columns after `brand_ink`, with the B1 `NULL = the built-in` docstring extended to the semantic family (citing the B3 revision, owner permission 2026-09-10, `TODO.md` L3 — PLAN.md is being redone by the owner, the decision is recorded in `00_phase.md`).
 3. `app/core/theming.py` — extend `BUILTIN_COLORS` with the 8 built-ins **parsed from / equal to** the existing `:root` values in `frontend/assets/styles.css` (`--ok-bg #10241b`, `--ok-ink #6ee7a8`, `--err-bg #2d0a0a`, `--err-ink #fca5a5`, `--err-line #ef4444`, `--accent-bg #2b2110`, `--accent-ink #fbbf24`, `--accent-line #f59e0b` — no new hardcoded copy; the drift test is the guard); `COLOR_FIELDS` = the 9 identity vars then the 8 semantic vars (identity, brand, then state); `effective_settings` resolves 20 values (3 strings + 17 colors) with the same column-by-column DB-over-built-in merge.
   - Update the module docstring: the 9-variable identity table stays; add a short "semantic families" table (the 8 vars, roles, and the note that they are now storable — B3 revised) and extend the "five contrast pairs" paragraph to the **eight** pairs (see `02_theme_tab_state_section.md`).
 4. `app/schemas.py` — `UiSettingsIn`: the 8 `str | None = None` fields; the output model (`UiSettingsOut` or its current name in the same file): same 8, documented as effective values.
 5. `app/api/ui_settings.py` — the color loops are `COLOR_FIELDS`-driven (phase 92): verify `_validate_colors` / `_validate_strings` and the GET builder need no per-field edits; if any explicit field list remains, extend it with the 8. Keep the 422 fixed-detail style (a bad hex names the field) and the built-in→NULL normalization (a semantic color equal to its built-in stores NULL).
 6. `frontend/assets/styles.css` — **no change expected**: the 8 vars already exist in `:root` with the built-in values (L29–36). If the drift test (below) surfaces a mismatch between the `:root` literals and step 3's table, fix the table, not the CSS.
 - ASSUMPTION: `NULL = built-in` for the semantic family (B1 rule, no env fallback for colors) — the default deployment is byte-identical and the owner's existing monochrome theme (a `ui_settings` row) keeps its saved identity values; the 8 new columns are NULL there until the owner re-saves with grays (the tab's live values after this phase are the built-ins, which the owner then edits).
 - ASSUMPTION: `COLOR_FIELDS` order = identity (9) then semantic (8) — the pre-paint tag's byte layout changes for non-default themes (the CSP hash is runtime-computed, so nothing static breaks); the no-op tag stays `""`.
 ## Testing & Quality
 - Unit: `tests/unit/test_theming.py` — the `:root`-drift assertion now covers all 17 (it parses `styles.css`, so it must pass unmodified or with the 8 added to its expected set); `theme_style_tag` with one non-default semantic var → tag contains exactly the 17 declarations in `COLOR_FIELDS` order; all-17-default → `""` (byte-identical contract); `theme_csp_hash` matches the tag. `tests/unit/test_ui_settings.py` — built-in→NULL normalization + 422 naming for the new fields.
 - Integration: `tests/integration/test_ui_settings_api.py` — PUT with the 8 fields (hex lowercased, empty→NULL, built-in→NULL) and GET returning effective values; admin-only gate unchanged (token user 403).
 - `tests/integration/test_security_headers.py` — the CSP `style-src` hash still matches the injected tag (it computes it at runtime).
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] `uv run alembic upgrade head` and `uv run alembic downgrade -1` both green (tested downgrade)
 - [ ] `effective_settings` returns 20 values; `PUT/GET /api/ui-settings` round-trips the 8 semantic fields
 - [ ] no-op theme still serves byte-identical HTML (no `#bor-theme` tag)
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (phases 01–92 suites green)
@@ -0,0 +1,28 @@
 # Task 02 — Theme tab: the "State colors" section (8 pickers, 5→8 contrast pairs)
 **Phase:** `93_theme_semantic_completion` · **Source:** `TODO.md:3` — "…multiple cases of color still in the UI which tells me the customization is not complete. Note the green text "Theme saved", the red "Revoked" tag, the red "Stale" tag, The green "Local" tag, The yellow "Listing documents" and "Reading" tool calls. …" (this task exposes the 8 semantic values in the admin Theme tab)
 **Story:** n/a (owner TODO item).
 ## Objective
 The Theme tab edits all 17 palette variables: a new "State colors" fieldset with the 8 semantic pickers, live-preview and Save/Reset coverage, and the client-side AA warnings extended from five to eight pairs.
 ## Work
 1. `frontend/index.html` — `#view-theme` (the shell view, ~L972): add a "State colors" fieldset after the palette fieldset with 8 `<input type="color">` fields, E2E-stable ids following the house pattern: `theme-ok-bg`, `theme-ok-ink`, `theme-err-bg`, `theme-err-ink`, `theme-err-line`, `theme-accent-bg`, `theme-accent-ink`, `theme-accent-line` (labels name the role: "Success text (--ok-ink)", "Error pill background (--err-bg)", …). Static defaults = the built-ins (the house contract: the drift/E2E pattern asserts static values against `styles.css` `:root`). Also update the palette fieldset's "five pairs" copy to reflect the eight (the exact legend text is the executor's call — keep the WCAG 2.1 AA (4.5:1) wording).
 2. `frontend/assets/theme.js` — `FIELDS` (L128ff): append the 8 entries `{ field, id, kind: "color" }` in the same fieldset order. Everything that iterates `FIELDS` then covers them automatically: live preview (`documentElement.style.setProperty`, L265), `collectBody` (the PUT body), `clearPreview`, and the `applyServedTheme` / served-theme sync from phase 92 (the `#bor-theme` tag content builder at ~L306 maps the fields — verify it uses `FIELDS` and gains the 8 declarations in `COLOR_FIELDS` order).
 3. `frontend/assets/theme.js` — `PAIRS` (L184): add the three ink-on-bg pairs `[ok_ink, ok_bg]`, `[err_ink, err_bg]`, `[accent_ink, accent_bg]` (the two `_line` vars stay excluded — decorative borders have no contrast duty, same rule as `--line`/`--grid-line`). Update the "five pairs" comments (L85, L154) to eight.
 4. `app/core/theming.py` docstring — the authoritative pair table (done in task 01 if not already): five → eight pairs; `theme.js`'s `PAIRS` and this table must never diverge (note the mirror relationship in both).
 - ASSUMPTION: fieldset/label wording ("State colors"; Success / Error / Notice roles for ok / err / accent) — the TODO names the *elements*, not the labels; the executor keeps the house label style ("Role (--var-name)").
 - ASSUMPTION: the 3 new warning pairs are ink-on-bg only (matching how the existing 5 pairs are chosen — "the pairs the layout actually pairs"); no new warning for the decorative `_line` vars.
 ## Testing & Quality
 - Unit (house style — frontend files are read as text, e.g. `tests/unit/test_big_read_progress.py`): extend or add a theming-frontend test asserting `theme.js`'s `FIELDS` lists all 12 + 8 ids in order and `PAIRS` has exactly 8 entries; `index.html` `#view-theme` contains all 20 color inputs with the E2E-stable ids and built-in static values.
 - Unit: `tests/unit/test_theming.py` — the pair table in the docstring matches `theme.js` `PAIRS` (if the house already pins this mirror, extend it; otherwise add the assertion here).
 - E2E coverage lands in task 04 (the monochrome suite drives the new pickers through Save/Reset).
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] the tab shows 20 color fields; typing a gray in any of the 8 previews live (computed style on `<html>`), persists on Save (PUT body carries all 20), and returns to the built-in on Reset
 - [ ] a failing pair among the 8 is listed in `#theme-contrast` (`role=alert`) as `--ok-ink on --ok-bg: x.x:1 — needs 4.5:1` in the house format
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (phase 91/92 E2E suites green)
@@ -0,0 +1,36 @@
 # Task 03 — Page heads get a surface panel (grid no longer fights the headings)
 **Phase:** `93_theme_semantic_completion` · **Source:** `TODO.md:3` — "…Also the header and description of each page needs a background - the grid makes it hard to read." (all 8 `theme_fixes/` screenshots show the `h1` + description sitting directly on the grid texture: Knowledge base, Git sources, Global Tuning, Access tokens, Theme, History)
 **Story:** n/a (owner TODO item).
 ## Objective
 Every page's header + description block reads cleanly over the background grid: one shared `.page-head` surface panel rule covering all shell views, plus the equivalent treatment for standalone pages whose h1/lede sits directly on the grid.
 ## Work
 1. `frontend/assets/styles.css` — the `.page-head` rule (L1536ff): add the panel — `background: var(--surface)`, `padding: 1rem 1.25rem` (tune to the existing card rhythm), `border: 1px solid var(--line)`, `border-radius` matching the house card radius. This single rule covers every shell view that uses the class — `#view-tuning` (L303), `#view-rag` (L365), `#view-git-sources` (L499), `#view-history` (L739), `#view-tokens` (L842), `#view-theme` (L998) in `frontend/index.html` (and `#view-chat` if it carries a `.page-head` — audit).
 2. Layout safety (verified, not assumed):
   - `#view-history .page-head` is a **flex row** (title left, stale pill right — L2550ff): the panel must wrap the whole row without breaking the flex alignment (padding on the flex container, not on the children).
   - Mobile: `.page-head-row { flex-wrap: wrap; }` (L4221) must keep working inside the panel; no horizontal overflow at 360px (the E2E responsive suites are the gate).
   - The panel must not introduce a new color literal — `var(--surface)` / `var(--line)` only (phase-92 invariant: zero hardcoded color literals outside `:root`).
 3. Standalone pages — audit each and apply the same panel treatment ONLY where the h1/lede sits directly on the grid:
   - `frontend/doc-edit.html` — already `.page-head` (L36): covered by step 1, verify.
   - `frontend/login.html` — `h1#login-title` (L121): if the title/lede are already inside a card, leave untouched; otherwise give the header block the same panel.
   - `frontend/document.html` — `h1#doc-title` (L118): same rule.
   - `frontend/shared.html` — `h1#shared-title` (L143): same rule (its head reuses the `.page-head h1` size per the L3189 note — if it carries the class it is covered by step 1).
 4. `tests/unit/` — house-style CSS text test (the `test_background_no_motion.py` pattern of parsing `styles.css` rules): assert the `.page-head` rule declares a non-transparent `background` built from a `var(--…)` (no literal), plus padding/border/radius.
 - ASSUMPTION: solid `var(--surface)` panel (not translucent, not full-bleed band, no blur — the phase-08 perf anchor forbids blur) — the TODO only says "needs a background"; surface is the house card color and is itself tab-controlled (a monochrome theme grays it automatically).
 - ASSUMPTION: standalone pages whose heading is already inside a card need no change (the TODO targets grid-exposed text; the executor documents any deliberate skip in the commit message).
 ## Testing & Quality
 - Unit: the CSS text test above.
 - Integration: none expected (pure CSS) — the API suites stay green.
 - E2E (gate, run in isolation): `test_header_consistency.py`, `test_nav_consistency.py`, `test_sticky_navbar.py`, `test_responsive_polish.py`, `test_history_page_width.py`, `test_mobile_hamburger_nav.py` — all must stay green; the new panel's assertions land in task 04's suite.
 - Coverage: **>90%** on `app/` (unchanged code — keep the gate green).
 ## Completion Criteria
 - [ ] every shell view's h1 + description sits on a solid surface panel (visual: `theme_fixes/` screenshots 2, 3, 4, 6, 8 conditions gone)
 - [ ] the History flex-row head and the 360px wrap are unbroken
 - [ ] zero new color literals outside `:root`
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work
@@ -0,0 +1,35 @@
 # Task 04 — E2E: a complete monochrome theme (the dedicated story suite)
 **Phase:** `93_theme_semantic_completion` · **Source:** `TODO.md:3` — the whole item (this task is the story gate: a saved black/white/gray theme leaves **no** residual color anywhere and every page head reads cleanly)
 **Story:** n/a (owner TODO item — one Playwright file per story, run in isolation, A16).
 ## Objective
 `tests/e2e/test_theme_semantic_completion.py` proves the TODO is done: with an all-gray 17-color theme saved from the admin tab, every state element the owner screenshot computes gray, state text stays present, every page head has a non-transparent background, and a fresh load paints the gray palette pre-paint.
 ## Work
 1. `tests/e2e/test_theme_semantic_completion.py` (new, isolated — `tests/e2e/conftest.py` fixtures `app_server` + `mock_llm`; admin login via `tests/e2e/auth_helpers.py`):
   - **Save the gray theme:** `PUT /api/ui-settings` with all 17 colors on a grayscale ramp where every channel is R=G=B and all **eight** contrast pairs pass ≥ 4.5:1 (e.g. `bg #111111, surface #1e1e1e, ink #f2f2f2, ink_soft #b3b3b3, line #3a3a3a, grid_line #2b2b2b, brand #9a9a9a, brand_soft #2c2c2c, brand_ink #d4d4d4, ok_bg #161616, ok_ink #e0e0e0, err_bg #191919, err_ink #e6e6e6, err_line #6a6a6a, accent_bg #1c1c1c, accent_ink #dedede, accent_line #787878` — the executor computes/pins exact values that pass; the strings stay default). Drive it THROUGH the tab UI (set the 8 new pickers + save) for at least one round-trip — the form wiring is the point — plus a direct-PUT path for the rest.
   - **Grayscale helper:** a computed-style check asserting `r == g == b` (the "still color" detector the TODO names).
   - **State elements (computed gray + text intact):**
     - `#theme-result` "Theme saved." (Theme page, after save — the green text from screenshot 7)
     - `.stale-pill` "Stale" (History — seed an out-of-generation saved chat, the `tests/e2e/test_stale_saved_chats.py` pattern)
     - the Revoked pill (Tokens — generate + revoke a token through the UI, the `tests/e2e/test_api_tokens.py` pattern)
     - `.git-source-kind.is-local` "Local" (Git sources — register a local-directory source, the `tests/e2e/test_local_directory_sources.py` pattern; screenshot 3)
     - a `.tool-call` line (Chat — a `mock_llm` turn that executes one tool call; `ls` needs no KB, the `tests/e2e/test_agent_document_tools.py` pattern; screenshots 1: the yellow "Listing documents" / "Reading")
   - **Page heads:** on each of the seven shell views (`/`, `/tuning.html`, `/sources.html`, `/git-sources.html`, `/history.html`, `/tokens.html`, `/theme.html`) + the login page: the h1/lede block's computed `background-color` is non-transparent, and its ink-on-panel ratio ≥ 4.5 (screenshots 2, 3, 4, 6, 8).
   - **Pre-paint + persistence:** a fresh `page.goto` on a themed page finds the `#bor-theme` tag whose content carries all 17 variables (the non-default case) and the body's computed background equals the saved `bg` on first paint; after `Reset to defaults` the tag is gone from the live document and a fresh load serves byte-identical default HTML (no tag).
 2. Run it in isolation and keep the existing suites green in isolation: `test_admin_theme_tab.py`, `test_theme_save_and_coverage.py`, `test_configurable_brand.py`, `test_dark_tech_theme.py`, `test_header_consistency.py`, `test_security_headers.py` (the CSP hash must cover the 17-var tag).
 - ASSUMPTION: the mock-LLM tool turn uses `ls` (zero KB dependency) — the `.tool-call` line is the same DOM element whether the tool is `ls`/`read`/`grep`, so one line suffices for the TODO's "Listing documents" / "Reading" cases.
 - ASSUMPTION: the login/document/shared standalone pages are checked for head background only where task 03 gave them a panel (the executor mirrors task 03's audit outcomes here).
 ## Testing & Quality
 - This IS the E2E task: `uv run pytest tests/e2e/test_theme_semantic_completion.py -v --no-cov` green in isolation (db up: `podman compose up -d db`).
 - Regression gate: the suites named in Work item 2 stay green in isolation; full unit + integration green; coverage **>90%** on `app/`.
 ## Completion Criteria
 - [ ] the suite is green in isolation and encodes every screenshot in `theme_fixes/` as a computed-style assertion
 - [ ] a grayscale theme ⇒ zero colored pixels in the asserted elements; state text ("Stale", "Revoked", "Local", "Theme saved.") still visible
 - [ ] all page heads have a non-transparent, AA-readable background
 - [ ] no-op/Reset deployment byte-identical (no `#bor-theme` tag)
 - [ ] full test suite green, coverage >90%
@@ -0,0 +1,60 @@
 # Phase 94 — `ls` becomes a drill-down tree with sync-time folder summaries
 **Source:** `TODO.md` L4 — "Brain of reese can end up indexing thousands of files. When the llm calls `list` it can end up filling the context entirely with just the document lists. I think list should work like a filesystem tree where it shows only the current "folder" (or at the top level, the list of synced projects) and then allow the LLM to drill down into the tree structure as it needs. We should add a level of "smart" to this by summarizing what each folder contains (at sync time) so when the LLM calls list it gets a summary of what's in each directory. Run tests against this new architecture and ensure accuracy and performance aren't too badly impacted."
 **Story:** n/a (owner TODO item — agent tooling on `37_agent_document_tools` / `45_agent_unlimited_tools` / `68_search_tool` / `70_harness_aligned_tools`).
 **Context:** Today `ls` (the harness-aligned surface, phase 70 — `app/rag/agent.py` `AGENT_TOOLS`) lists **every** indexed document as `source: X | path: Y | title: Z` lines (or one source's documents when `path` is a source name) — with a 1,340-document KB that is the entire context in one call. `read`/`grep` take the combined `source/path` identity; `grep` is the whole-KB locator. The one-shot `lite` machinery exists: document summaries (`app/rag/summarizer.py`, `SUMMARY_MODE`) and the KB overview (`app/rag/overview.py`, `KB_OVERVIEW_MODE`) — both change-gated at import, best-effort/fail-soft, with a marker the deterministic E2E mock (`tests/e2e/mock_llm.py`) keys on. Sync paths: `scripts/import_docs.py` (`_run()`, the overview hook) and `POST /api/sync` (`app/api/sync.py`, step 5). The controlled accuracy/performance methodology is `TOOL_CALLING_TESTING.md` + `uv run python -m scripts.agent_realmodel_check --restore --mode fixture` (the 8-document, 2-source, **folder-structured** fixture KB in `tests/fixtures/test_kb.dump.sql`: `deployments/{ansible,ci,quadlet}` + `homelab/{backups,containers,networking}`).
 ## Objective
 Rebuild `ls` as a filesystem-style tree the LLM drills through: top level lists the synced projects (sources) with a per-source summary; `ls` of a source lists its folders (each with a sync-time summary) + top-level files; `ls` of a folder lists its subfolders + files. Folder summaries are generated at sync time by the `lite` model (change-gated, fail-soft). The controlled tool-calling battery then proves accuracy and performance are not badly impacted.
 ## Dependencies
 - `93_theme_semantic_completion` (todo) — queue order only; no code dependency (different subsystems).
 ## Owner-permitted decision recorded here (PLAN.md is being redone by the owner)
 - **Tool-surface revision (owner permission 2026-09-10, TODO.md L4):** the `ls` RESULT format and `path` semantics change (tree drill-down); the `ls` NAME and the `read`/`grep` contract (combined `source/path`) are untouched. This extends the phase-70 harness-aligned surface per the owner's explicit request; the prompt teaching (`TOOLS_SECTION`) is rewritten to match in the same change.
 ## Design (shared by all tasks — the executor reads this, not the chat)
 ### Tree shape (task 03)
 Document paths are already tree-like (`deployments/ansible/lab-inventory.md`); a **folder** is a path prefix. Three `ls` levels, one argument:
 | Call | Result |
 |---|---|
 | `ls()` (no path) | every registered source (registry order, `list_source_names`): `name — N documents` + an indented summary line when one is stored (folders with no docs still list as `0 documents`) |
 | `ls(path=<source>)` | the source's **root folder**: each subfolder `  <folder>/ — N documents[: summary]` (count = documents with `path == folder` or `path startswith folder + "/"`), then the root's own file lines `source: X | path: Y | title: Z` |
 | `ls(path=<source>/<folder>[/…])` | that folder's subfolders (same shape, paths relative to the source) + the folder's own file lines |
 - **File-line cap:** a folder's own files list at most **50** lines (path order — catalog order), then one deterministic note: `…and {N−50} more documents in this folder — use grep (pattern) to find a specific one.` (folds the old whole-KB flood into one bounded level; `grep` stays the locator — no new tool).
 - **Summaries:** shown for the SOURCE (top level) and for each SUBFOLDER, when stored; absent → the count line only (no placeholder).
 - **Refusals (adapted from the phase-70/72 teaching lines, same style — one line, argument echoed, counts in nothing, consumes a round):** unknown first segment → the existing no-source refusal; a `source/…` argument whose folder matches no indexed prefix → a NOT-A-FOLDER line teaching the drill-down (echo the argument, list the parent folder's subfolders so the model can self-correct in the next round). `ls` of a registered source with no documents keeps the `0 documents:` behavior.
 - **`holder.tool_calls`** increments on every successful listing (top/root/folder); refusals count in nothing (unchanged house rule).
 ### Folder summaries (tasks 01, 02)
 - **Storage:** new table `folder_summaries` — PK `(source, folder_path)` (column types mirror `Document.source` / `Document.path`), `summary` `Text`, `updated_at` `timestamptz`. `folder_path = ""` is the SOURCE ROOT (the top-level source summary); a folder is any other path prefix.
 - **Generation** (`app/rag/folder_summaries.py`, mirroring `app/rag/overview.py`): marker `FOLDER_SUMMARY_MODE` (the E2E mock keys on it — `tests/e2e/mock_llm.py` gains the canned branch); input per folder = its **recursive subtree** — every document whose path equals the folder or starts with `folder + "/"` (the same set the `ls` count shows; `folder_path ""` = the whole source), each as `path` + `title` + first summary line — capped like the overview; `lite` one-shot via `LLMClient.chat`; the instruction asks for a **1–3 sentence** plain-text, grounded description of what the folder's documents cover (shorter than a document summary — there can be hundreds of folders); non-empty validation, else `LLMError` → **fail-soft** (log, keep the previous summary — an old outline is better than none, the phase-31 rule).
 - **Gate:** regenerate only when the import changed the KB (the overview's change gate: added/updated > 0) or the table is empty; `--limit` debug runs skip (mirror the overview); a `lite` failure never fails the sync (log + continue — the overview's `overview=failed` status token is NOT extended: folder summaries are auxiliary, the KB is the product).
 - **Scope rules:** summarize folders with **≥ 2 documents** (a single-document folder is fully described by its one file line); after a changed sync, DELETE rows for folders that no longer have ≥ 2 documents (pruned/renamed folders — a 3→1 doc folder's summary goes stale and is dropped); rows for untouched folders persist (an unchanged folder's summary is still true).
 ### Accuracy & performance gate (task 05)
 `TOOL_CALLING_TESTING.md`'s controlled loop: `uv run python -m scripts.agent_realmodel_check --restore --mode fixture` — the 10-turn fixture battery against the live model through the real grounded path. The verdict line + per-turn wall times + the comparison against the recorded phase-70/72 baseline go into `TOOL_CALLING_TESTING.md` (dated section). Accuracy (contract/verdict) is the gate; wall time is reported. If a turn regresses, iterate on the ls copy levers (output format / `TOOLS_SECTION` teaching) with the micro-loop (`--turns 3`) per the methodology, then re-run the full battery.
 ## Tasks
 1. `01_folder_summary_model.md` — migration 0017 + `FolderSummary` model + `app/rag/folder_summaries.py` generator (marker, fail-soft, prune) + mock-LLM branch
 2. `02_sync_wiring.md` — change-gated generation in both sync paths (`scripts/import_docs.py`, `app/api/sync.py`)
 3. `03_ls_tree.md` — the three-level `ls` rewrite (`app/rag/agent.py`) + `AGENT_TOOLS` description + `TOOLS_SECTION` teaching
 4. `04_e2e_drilldown.md` — dedicated Playwright suite: scripted drill-down turns, tree shapes, cap + note
 5. `05_realmodel_battery.md` — controlled 10-turn battery vs baseline, verdict recorded in `TOOL_CALLING_TESTING.md`
 ## Testing & Quality
 - Unit/integration: folder extraction + grouping; summary prompt building + capped input + fail-soft + prune; generator unit (mock LLM); sync wiring integration (both paths, change-gated, fail-soft, same-transaction convention); `ls` tree output for every level/edge (empty source, unknown source, unknown folder, root vs nested, single-doc folder, 50+ file cap + note); `holder.tool_calls` accounting; `read`/`grep` untouched.
 - Coverage: **>90%** on new/modified code (`uv run pytest --cov=app --cov-report=term-missing`).
 - E2E: `uv run pytest tests/e2e/test_ls_tree_drilldown.py -v --no-cov` in isolation; the existing agent-tool suites (`test_agent_document_tools.py`, `test_agent_unlimited_tools.py`, `test_harness_aligned_tools.py`, `test_search_tool.py`, `test_grep_regex_teaching.py`, `test_response_to_docs.py`) stay green in isolation.
 - Lint/types: `uv run ruff check . && uv run pyright`.
 ## Completion Criteria
 - [ ] `ls()` lists sources with summaries; `ls(source)` / `ls(source/folder)` drill down; file lines capped at 50 + grep pointer
 - [ ] folder summaries exist in the DB after a changed sync and are visible in `ls` output; a `lite` outage leaves old summaries intact and the sync green
 - [ ] `read`/`grep` behavior byte-identical for their contracts (existing suites green)
 - [ ] the 10-turn fixture battery verdict is recorded in `TOOL_CALLING_TESTING.md` and accuracy is at or above the phase-70/72 baseline (or an owner-acknowledged tradeoff is written down)
 - [ ] test suite green, coverage >90%, ruff + pyright clean
 - [ ] no behavior change in completed phases; one atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(agent): make ls a drill-down tree with sync-time folder summaries`)
@@ -0,0 +1,34 @@
 # Task 01 — Folder summary storage + generator (the "smart" half)
 **Phase:** `94_ls_tree_drilldown` · **Source:** `TODO.md:4` — "We should add a level of "smart" to this by summarizing what each folder contains (at sync time) so when the LLM calls list it gets a summary of what's in each directory."
 **Story:** n/a (owner TODO item).
 ## Objective
 Store per-folder plain-text summaries and generate them with the `lite` model — a new `folder_summaries` table, a `FOLDER_SUMMARY_MODE` one-shot generator mirroring the KB-overview contract (change-gated, fail-soft, E2E-mockable), and the prune rule for stale folders.
 ## Work
 1. `alembic/versions/0017_folder_summaries.py` (down revision `0016_ui_settings_semantic`, the head after phase 93) — new table `folder_summaries`: PK `(source, folder_path)` (String lengths mirroring `Document.source` / `Document.path` in `app/models.py`), `summary` `Text` NOT NULL, `updated_at` `timestamptz` (server default now, house style); **tested downgrade** drops it (A13).
 2. `app/models.py` — `FolderSummary` model with a docstring citing the phase-94 design (`00_phase.md`): `folder_path = ""` is the source root; rows exist only for folders with ≥ 2 documents.
 3. `app/rag/folder_summaries.py` (new module — mirror `app/rag/overview.py`'s shape and house docstring style):
   - `FOLDER_SUMMARY_MODE = "FOLDER_SUMMARY_MODE"` — system-prompt marker the deterministic E2E mock keys on (same convention as `SUMMARY_MODE` / `KB_OVERVIEW_MODE`).
   - `FOLDER_SUMMARY_INSTRUCTION` — the locked `lite` instruction: a **1–3 sentence** plain-text summary of what the folder's documents cover (natural language, no markdown, strictly grounded in the listed titles/paths/summary lines).
   - `folder_of(path) -> str` — the directory prefix before the last `/` (`""` for root-level files); module-level so unit tests can drive it.
   - `group_by_folder(rows) -> dict[(source, folder_path), list[docs]]` — module-level; ONE concept end to end (document it in the module docstring): a folder row's documents are its **recursive subtree** — every document whose path equals the folder or starts with `folder + "/"` — exactly the set the `ls` count rule in `00_phase.md` counts. Candidate rows per source: `folder_path ""` (the source root — ALL of the source's documents, this is the top-level source summary) + every distinct folder prefix of an indexed path. A doc under `a/b/` therefore contributes to BOTH the `a` row and the `a/b` row (and the `""` row) — that is intended: each level's listing shows its own accurate subtree summary.
   - `summarize_folder(source, folder_path, docs, llm) -> str` — build the prompt from the folder's documents (`path`, `title`, first summary line each), cap the input (reuse the overview's cap constant or its own — the executor picks, default ≤ 8000 chars) with the shared `TRUNCATION_MARKER` on overflow, one `LLMClient.chat` call, non-empty validation (else `LLMError` — the client already rejects empty content; re-assert defensively, the summarizer's rule).
   - `generate_folder_summaries(db, llm, *, skip: bool = False) -> dict` — the orchestrator: `skip` (the `--limit` case) is a no-op; group; for each folder with **≥ 2 documents** call `summarize_folder` and UPSERT (fail-soft per folder: catch `LLMError`, log, keep the previous row); DELETE rows whose folder no longer has ≥ 2 documents; return a small stats dict (generated/failed/pruned) for logging. Only flush — the CALLER commits (the phase-53 `bump_sources_version` convention: the sync path owns the transaction).
 4. `tests/e2e/mock_llm.py` — a canned `FOLDER_SUMMARY_MODE` branch (the mock keys on the marker in the system prompt): return a deterministic one-liner, e.g. `"Fixture folder summary for <folder path>."` (the exact template is the executor's call — it must name the folder so E2E can assert on it).
 - ASSUMPTION: 1–3 sentence summaries (the TODO says "summarizing what each folder contains" — folders are skims, not reads; the document summary's 3–6 sentences would blow up a 20-folder listing).
 - ASSUMPTION: ≥ 2 documents per summarized folder (a single-doc folder's one file line IS its summary — no `lite` burn).
 - ASSUMPTION: fail-soft = log + keep previous (the phase-31 "an old outline is better than none" rule); a `lite` outage never fails the sync (no new status token — the KB overview's `overview=failed` token stays overview-specific).
 - ASSUMPTION: subtree (recursive) grouping — the summary scope equals the `ls` count scope at every level (one mental model for the LLM: the number next to a folder is the number of documents its summary describes); the ≥ 2 rule and the prune rule both apply to the recursive count.
 ## Testing & Quality
 - Unit (`tests/unit/`, new `test_folder_summaries.py`): `folder_of` (root, one level, deep); `group_by_folder` (multi-source, nested — a doc under `a/b/` present in the `a`, `a/b`, and `""` groups; recursive-count < 2 folders excluded); prompt building (cap + `TRUNCATION_MARKER` on overflow, ordering stable); `summarize_folder` non-empty validation; `generate_folder_summaries` with a fake `LLMClient` — happy path upsert, per-folder fail-soft (one folder raises → others land, previous kept, stats right), prune (a folder dropping below 2 docs loses its row; an untouched folder's row survives), `skip=True` no-op.
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] migration up/down green; `FolderSummary` + generator importable and unit-tested
 - [ ] the E2E mock returns the canned folder summary when the marker is present
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (the agent/sync suites green)
@@ -0,0 +1,28 @@
 # Task 02 — Sync wiring: folder summaries regenerate change-gated, both paths
 **Phase:** `94_ls_tree_drilldown` · **Source:** `TODO.md:4` — "…summarizing what each folder contains (at sync time)…" (this task is the *at sync time* part)
 **Story:** n/a (owner TODO item).
 ## Objective
 Both sync paths regenerate folder summaries in the same transaction as the KB changes, change-gated like the KB overview, best-effort/fail-soft, with `--limit` debug runs skipping — mirroring the phase-31/53 overview + version-bump conventions exactly.
 ## Work
 1. `scripts/import_docs.py` — in `_run()` next to the existing KB-overview regeneration (the change-gated block after `import_sources`): call `generate_folder_summaries(session, llm, skip=<the --limit flag>)` under the same gate the overview uses (added/updated > 0) **or** when the `folder_summaries` table is empty (first run after migration 0017 — the `_overview_row_exists` pattern, extended to a table-empty check). Same event loop, same `LLMClient` instance (the phase-31 convention). Log the stats dict (generated/failed/pruned) on the run's summary line — the line-extension house rule (PLAN §9 ample logging).
 2. `app/api/sync.py` — in the in-process sync task, at step 5 (next to the overview regeneration in the docstring's step list): the identical call + gate (`--limit` has no equivalent here; the sync button always runs the full walk). A `lite` failure in folder summaries must NOT flip the sync to failed (the overview's fail-soft is the template) and must NOT block the `bump_sources_version` (the bump stays change-gated on the KB, not on the summaries).
 3. Transaction convention: `generate_folder_summaries` only **flushes** (its docstring says so) — each sync path commits in its own transaction, exactly like `bump_sources_version` (phase 53) and the overview write, so a failed sync never half-writes summaries.
 4. Docstrings: update `scripts/import_docs.py`'s module docstring (the "refresh the stored KB overview" paragraph) and `app/api/sync.py`'s step list to name the folder-summary step.
 - ASSUMPTION: the sync-button path (no `--limit` concept) always regenerates on a changed KB; the table-empty first-run trigger applies to both paths.
 - ASSUMPTION: no new sync-status surface (the `GET /api/sync/status` shape is untouched — folder summaries are auxiliary; failures are log-only).
 ## Testing & Quality
 - Integration (extend the overview pattern — `tests/integration/test_import_docs_overview.py` is the template): a changed import (fake `LLMClient` keyed on `FOLDER_SUMMARY_MODE`) upserts the expected rows in the same transaction; an unchanged re-import burns **zero** `lite` calls; a folder whose subtree dropped below 2 docs is pruned; one folder raising `LLMError` keeps the previous row, lands the others, and leaves the run's exit code / sync status green; `--limit` (script path) skips generation; a fresh DB (table empty) generates on an unchanged walk.
 - The existing `tests/integration/test_sync_api.py` + `test_import_docs_git.py` stay green (status shape unchanged).
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] after `uv run python -m scripts.import_docs` (changed KB) and after `POST /api/sync` (changed KB), `folder_summaries` holds one row per ≥ 2-doc folder subtree, visible in the run logs
 - [ ] unchanged re-syncs make zero `FOLDER_SUMMARY_MODE` calls (asserted by the fake client)
 - [ ] a `lite` outage leaves old summaries intact and both sync paths report success
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work
@@ -0,0 +1,56 @@
 # Task 03 — The three-level `ls` rewrite (+ tool description + prompt teaching)
 **Phase:** `94_ls_tree_drilldown` · **Source:** `TODO.md:4` — "I think list should work like a filesystem tree where it shows only the current "folder" (or at the top level, the list of synced projects) and then allow the LLM to drill down into the tree structure as it needs. … so when the LLM calls list it gets a summary of what's in each directory."
 **Story:** n/a (owner TODO item).
 ## Objective
 `ls` lists one tree level per call — sources at the top, subfolders + files below — with the stored summaries shown next to each folder, the file lines capped, and the prompt/tool-description teaching updated so the model drills instead of flooding.
 ## Work
 1. `app/rag/agent.py` — new module-level helpers (monkeypatchable, house style):
   - `ls_top(db) -> list[tuple[str, int, str | None]]` — registered sources in `list_source_names` order (registry truth, the phase-70/72 invariant — a source with 0 docs still lists), each with its recursive document count and its stored summary (`folder_summaries` row for `(source, "")` — `None` when absent).
   - `ls_folder(db, source, folder) -> tuple[list[tuple[str, int, str | None]], list[tuple[str, str, str]], int]` — for one folder: (a) its direct subfolders in path order, each `(sub_path_relative_to_source, recursive_count, summary_or_None)`; (b) its direct file lines `(source, path, title)` in path order (catalog order — the same order `GET /api/docs` serves); (c) the TOTAL direct-file count (pre-cap, for the note). Subfolder/file membership is pure SQL prefix logic on `Document.path` (the existence rule below).
   - Folder existence: folder `F` under registered source `S` exists ⟺ `F == ""` OR some indexed path `p` of `S` satisfies `p.startswith(F + "/")` (a document's OWN path is never a folder — nothing starts with `path + "/"`).
 2. `app/rag/agent.py` — rewrite the `ls` branch of `_execute_tool` (the `list_catalog`-based block):
   - **No path** → the top-level listing, pinned template:
     ```
     {N} sources:
     {source} — {n} documents
       {summary}
     ```
     one block per source (registry order); the indented summary line only when stored; no blank line between blocks.
   - **Path = registered source** (no `/`) → its root folder, pinned template:
     ```
     {identity} — {n_files} documents, {n_folders} folders:
       {sub}/ — {m} documents: {summary}
     source: {S} | path: {p} | title: {t}
     …and {n_files − 50} more documents in this folder — use grep (pattern) to find a specific one.
     ```
     `identity` = the source name; subfolder lines 2-space-indented, path order, `: {summary}` only when stored; file lines EXACTLY the existing `source: X | path: Y | title: Z` format (the canonical `read`/`grep` identity — unchanged); the cap note only when `n_files > 50` (the cap is a pinned module constant `LS_MAX_FILE_LINES = 50`). A registered source with no documents: the header line alone (`… — 0 documents, 0 folders:`) — the old `0 documents:` behavior preserved in spirit.
   - **Path = `source/folder…`** (contains `/`) → split at the first `/`; unknown source → the existing `NO_SOURCE_NOT_A_DIRECTORY` refusal (teaching parenthetical intact); unknown folder → the NEW `NOT_A_FOLDER` refusal line, phase-72 teaching style (one line, argument echoed, parent's subfolders listed so the model self-corrects next round, e.g. `'homelab/netwoking' is not a folder — homelab has: backups/ containers/ networking/`); otherwise the same template as the root level with `identity = source + "/" + folder`.
   - The old `LS_PATH_NOT_A_SOURCE` refusal (a `/` in the scope was always wrong) is **deleted** — a `/` now names a folder; update the unit tests that pin it (see Testing).
   - `holder.tool_calls += 1` on every successful listing (top/root/folder); refusals count in nothing (unchanged).
 3. `app/rag/agent.py` — `AGENT_TOOLS` `ls` entry: rewrite the function description to the tree contract (pinned copy):
   > "List the knowledge base as a tree, one level at a time. With no path: the synced sources — each with its document count and a summary of its contents. With a source name (no '/'): that source's top-level folders and files. With a `source/folder` path: that folder's subfolders and files. Folder lines carry a summary of what the folder contains. File lines are `source: X | path: Y | title: Z` — use the combined `source/path` with `read` and `grep`. Call one tool at a time — wait for this result before your next call."
   and the `path` parameter description: "Optional — a source name (e.g. 'homelab') to list its top level, or a `source/folder` path to drill down (e.g. 'homelab/active'). Omit it to list every source."
 4. `app/rag/prompts.py` — `TOOLS_SECTION`: rewrite the `ls` teaching paragraph to the drill-down contract (top level = sources; folders list subfolders + files only — never the whole KB in one call; summaries tell what's in a folder before drilling; `grep` stays the locator for finding one document without listing). `read`/`grep` teaching untouched.
 5. `app/rag/agent.py` module docstring — the phase-70 tool-surface paragraph: add the phase-94 revision note (owner permission 2026-09-10, `TODO.md` L4 — the `ls` RESULT format + `path` semantics changed; the tool NAME and the `read`/`grep` contract are untouched; PLAN.md is being redone by the owner, the decision is recorded in `00_phase.md`).
 - ASSUMPTION: `LS_MAX_FILE_LINES = 50` pinned module constant (no env var — the TODO asks for a shape change, not a knob; the constant lives next to `SEARCH_MAX_MATCHES`).
 - ASSUMPTION: the exact punctuation of the three pinned templates above is the contract (unit tests pin it byte-for-byte, the house pattern); "documents"/"folders" stay unpluralized (stable for the model, like today's `0 documents:` line).
 - ASSUMPTION: the NOT-A-FOLDER refusal lists the PARENT folder's direct subfolders (bounded — the parent's own listing, so no new flood path).
 ## Testing & Quality
 - Unit (`tests/unit/test_agent.py` — extend; it pins the tool surface and refusal copy today): `ls_top` (registry order, 0-doc source, summary present/absent); `ls_folder` (subfolder recursion + counts, direct-file membership, path order, the file-path-is-not-a-folder rule); the three result templates byte-for-byte (top / root / nested, empty folder, summary absent); the 50-line cap + note (51 files → 50 lines + `…and 1 more…`); refusals (unknown source → `NO_SOURCE_NOT_A_DIRECTORY` prefix intact; unknown folder → `NOT_A_FOLDER` with the parent's subfolders; the deleted `LS_PATH_NOT_A_SOURCE` is gone); `holder.tool_calls` accounting (success vs refusal); `read`/`grep` branches byte-identical for their contracts.
 - Unit: `AGENT_TOOLS` `ls` description + `path` parameter + `TOOLS_SECTION` contain the tree contract (the house string-pin tests updated, not deleted).
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] `ls()` / `ls(source)` / `ls(source/folder)` render the three pinned templates against a multi-folder DB (unit-pinned)
 - [ ] a 500-file folder costs the model 50 file lines + one note, never 500
 - [ ] the `read`/`grep` contracts and their suites are byte-identical (no change)
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (deflected turns, the round cap, the log line all untouched)
@@ -0,0 +1,33 @@
 # Task 04 — E2E: the scripted drill-down (the dedicated story suite)
 **Phase:** `94_ls_tree_drilldown` · **Source:** `TODO.md:4` — the whole item (this task is the story gate: the LLM drills source → folder → file through the UI, summaries visible, cap held)
 **Story:** n/a (owner TODO item — one Playwright file per story, run in isolation, A16).
 ## Objective
 `tests/e2e/test_ls_tree_drilldown.py` proves the new architecture end to end: a folder-structured KB synced under the deterministic mock LLM, scripted agent turns that drill `ls()` → `ls(source)` → `ls(source/folder)` → `read(source/file)`, the drill-down tool lines rendering in the UI, the folder summary the mock generated appearing in the model's context, and the 50-line cap holding on a wide folder.
 ## Work
 1. `tests/e2e/test_ls_tree_drilldown.py` (new, isolated — `tests/e2e/conftest.py` fixtures `app_server` + `mock_llm`; admin login via `tests/e2e/auth_helpers.py`):
   - **Seed:** a temp local directory tree — two sources (two temp roots), each with nested folders, e.g. `alpha/{one,two}/…md` and `beta/{gamma}/…md` (the `tests/e2e/test_local_directory_sources.py` pattern for registering a local source + Sync); the mock's canned `FOLDER_SUMMARY_MODE` branch (task 01) makes sync store a deterministic summary per ≥ 2-doc folder, so the test can assert on exact summary text.
   - **Scripted turns** (extend `tests/e2e/mock_llm.py`'s scripted-tool-turn machinery — the `tests/e2e/test_agent_document_tools.py` pattern — with a drill-down script):
     1. "What sources are indexed?" → the mock calls `ls()` → assert the `.tool-call` "🔎 Listing documents" line renders, and the mock's grounded answer quotes the top-level shape (a `— N documents` line per source; the canned source-root summary text for a source with ≥ 2 docs — the mock echoes what it received, the house scripted-turn way of asserting on tool results).
     2. "What's in source `alpha`?" → the mock calls `ls("alpha")` → assert the folder lines (`one/ — …`, `two/ — …` + their summaries) and any root file lines, via the mock's echo.
     3. "List `alpha/two`" → the mock calls `ls("alpha/two")` → assert the file lines in the exact `source: X | path: Y | title: Z` format.
     4. "Read the file" → the mock calls `read("alpha/two/<file>.md")` → the "📄 Reading …" line renders and the final answer cites the document (grounded-turn contract, the phase-37 assertion pattern).
   - **Cap:** one source with a folder holding 51 tiny files (51 one-line `.md` files in the temp tree) → the mock's `ls` of that folder echoes exactly 50 file lines + the `…and 1 more documents…` note.
   - **Refusal visible to the model:** one turn where the mock calls `ls("alpha/nope")` → the next round the mock (scripted) recovers with `ls("alpha")` — the phase-72 self-correction contract now carries the tree's NOT-A-FOLDER teaching.
 2. Run in isolation: `uv run pytest tests/e2e/test_ls_tree_drilldown.py -v --no-cov` (db up: `podman compose up -d db`).
 3. Regression gate, in isolation: `test_agent_document_tools.py`, `test_agent_unlimited_tools.py`, `test_harness_aligned_tools.py`, `test_search_tool.py`, `test_grep_regex_teaching.py`, `test_sync_button.py`, `test_local_directory_sources.py`.
 - ASSUMPTION: assertions on tool RESULTS go through the deterministic mock echoing the received result into its final answer (the mock is the only lens the E2E has on the LLM's context — the house pattern from the agent-tool suites); the DOM assertions cover the tool lines + the answer.
 - ASSUMPTION: the drill-down mock script is a NEW script in `mock_llm.py` (keyed like the existing ones), not a change to the existing scripts — the old suites stay green unmodified.
 ## Testing & Quality
 - This IS the E2E task; additionally the mock's new branch/script needs a unit-level smoke (the mock is a real OpenAI-compatible server — the conftest boots it; no extra unit needed beyond the E2E itself, per the existing agent suites' practice).
 - Coverage: **>90%** on `app/` (this task adds test code only — keep the gate green).
 ## Completion Criteria
 - [ ] the suite is green in isolation and encodes the full drill-down: sources → folders (with summaries) → files → grounded read
 - [ ] the 51-file folder costs the model 50 lines + the note (asserted via the mock echo)
 - [ ] the NOT-A-FOLDER teaching is visible to the model and a scripted recovery works
 - [ ] full test suite green, coverage >90%
@@ -0,0 +1,33 @@
 # Task 05 — Accuracy & performance: the controlled tool-calling battery vs baseline
 **Phase:** `94_ls_tree_drilldown` · **Source:** `TODO.md:4` — "Run tests against this new architecture and ensure accuracy and performance aren't too badly impacted."
 **Story:** n/a (owner TODO item).
 ## Objective
 Run the controlled 10-question fixture battery (the `TOOL_CALLING_TESTING.md` methodology — real configured chat model, restored fixture KB, the exact mirror of the grounded path) against the new `ls` architecture, iterate on the copy levers if any turn regresses, and record the verdict + timings + baseline comparison in `TOOL_CALLING_TESTING.md`.
 ## Work
 1. Fixture snapshot freshness: if `tests/fixtures/test_kb.dump.sql` predates migration 0017 (it does — it has no `folder_summaries` rows), refresh the snapshot: restore, re-import the fixture directory through the real pipeline (real embeddings — the methodology's ~1–2 s rebuild), let the sync-time folder summaries generate against the live `lite` endpoint, and re-dump (one transaction snapshot, the house "hand-written, unguessable, fixed-size KB snapshotted to a SQL dump" rule). The dump's `sources_meta`/`kb_overview` rows follow the same refresh.
 2. The loop:
   ```bash
   podman compose up -d db
   uv run python -m scripts.agent_realmodel_check --restore --mode fixture
   ```
   (full 10 turns; the micro-loop `--turns 3` is the iteration tool when a variant looks off).
 3. Review the 10 questions' "unambiguously correct tool behavior" against the tree: where the tree changes the correct answer (e.g. a "list everything" expectation becomes "drill into the right folder"), update the battery question's expected-behavior note in `TOOL_CALLING_TESTING.md` — the question itself stays; the correct-behavior definition moves with the owner-permitted surface change. If a question's intent no longer maps, flag it in the recorded section (do NOT silently rewrite a question).
 4. Iterate only on copy levers (the `ls` output templates / `AGENT_TOOLS` description / `TOOLS_SECTION`) if a turn regresses — micro-loop first (`--turns 3`), full battery for the verdict. Any template change here re-pins the task-03 unit tests in the same commit.
 5. Record a dated section in `TOOL_CALLING_TESTING.md` (the house format — per-turn lines with wall seconds + the verdict line + total wall): the run against the new architecture, the per-turn verdicts, the comparison against the last recorded baseline (phase 70/72 numbers in the file), and the conclusion. Accuracy (contract/verdict) is the gate per the TODO; wall time is reported as "performance."
 - ASSUMPTION: "not too badly impacted" = **accuracy at or above baseline** (no contract/verdict regression on the 10-question battery) and **wall time within ~20% of the baseline total** (the drill-down can add rounds — the tree is a few extra `ls` hops; a modest wall increase is expected and reportable, a big one is not). A violation ⇒ iterate the copy levers; if still in violation, STOP and write the tradeoff into the recorded section for the owner (the TODO leaves the threshold to judgment — the executor must not silently accept).
 - ASSUMPTION: the battery runs against the LIVE aipi endpoint (the methodology's premise — `E2E_REAL_LLM` is the E2E opt-in; `agent_realmodel_check` is the real-model tool by definition). If the endpoint is unreachable, the task is blocked — report it, don't fake the verdict.
 ## Testing & Quality
 - No new app code is expected from this task (docs + fixture dump + possibly the task-03 copy levers, re-pinned). The full suite + coverage >90% stays green; ruff + pyright clean.
 - The recorded section in `TOOL_CALLING_TESTING.md` is the artifact — it must be reproducible from the file alone (commands, timings, verdicts, baseline diff).
 ## Completion Criteria
 - [ ] `TOOL_CALLING_TESTING.md` carries the dated phase-94 section: 10/10 per-turn lines, verdict, total wall, baseline comparison, conclusion
 - [ ] accuracy ≥ baseline (or the recorded owner-acknowledgment exists)
 - [ ] wall time within the 20% band (or the recorded tradeoff exists)
 - [ ] fixture dump refreshed (carries `folder_summaries` rows) and `--restore` still ~0.03 s hot
 - [ ] full test suite green, coverage >90%
@@ -0,0 +1,49 @@
 # Phase 95 — `read` cap: bounded reads, honest truncation, visible to LLM and user
 **Source:** `TODO.md` L5 — "Brain of reese can sometimes feed huge documents into the LLM's context - sometimes too large for the LLM to handle. If the LLM calls read on a huge document there should be a sensible cap on the amount it can read at once. My LLMs all have a minimum cap of 128,000 tokens of context, so spec for that. The LLM should be informed the read was truncated, and should be offered a grep or search or find tool (whichever matches most closely to existing harnesses) to search the document for what it was looking for. There should be a visual indicator that the read was trnucated so the user knows what's going on."
 **Story:** n/a (owner TODO item — agent tooling on `37_agent_document_tools` / `70_harness_aligned_tools`; UI on the phase-37 tool lines).
 **Context:** `read` today returns the **whole** document (`app/rag/agent.py` `_execute_tool` read branch: `f"Document {doc.source}/{doc.path}:\n{doc.content}"`) — fine for notes, context-obliterating for the huge files the owner describes. The house already has every primitive this phase needs: `TRUNCATION_MARKER` (`[…truncated…]`, `app/rag/retriever.py`), the char-capped one-shot precedent (`BOR_SUMMARY_MAX_CHARS`, `app/rag/summarizer.py`), the piece family (`StreamPiece` / `ToolCallPiece` / `RetryPiece` in `app/rag/llm.py`, consumed by the `app/api/chat.py` `_pump` loop as SSE frames), and `grep` — the exact "search the document" tool the TODO asks for (already harness-aligned, phase 70). The UI renders one `.tool-call` line per executed call ("📄 Reading …", `frontend/assets/app.js` ~L898ff; the shared view renders the same from saved records, `frontend/assets/shared.js` ~L148); saved chats persist the turn's `tools` array (`ToolCall` in `app/schemas.py`, built client-side from the SSE `tool` frames into `toolAcc`, restored by `app.js` ~L1442 and rendered by `shared.js`).
 ## Objective
 A huge `read` can no longer flood the context: the result is capped (default 128 000 chars ≈ 32k tokens — spec'd against the owner's 128k-token minimum context), the LLM is told the read was truncated and pointed at `grep` for the rest, and the user sees a "(truncated — showing N of M chars)" marker on the Reading line — live, in saved chats, and on shared pages.
 ## Owner-permitted decisions recorded here (PLAN.md is being redone by the owner)
 - **A7 scope clarification (owner permission 2026-09-10, TODO.md L5):** A7's "never truncated" contract (the retrieved **top-2 `<documents>`** context — owner: "this should never happen", phase 24) is **unchanged**. The cap applies to the **`read` tool path only**, per the owner's explicit request in the TODO — the two paths are distinct (retrieval seeds vs. agent-requested additions). This reverses the "this should never happen" ruling **for tool reads only**; recorded here because PLAN.md is being redone.
 - **A15 extension (same permission):** one new **optional** SSE event type `tool_result` (emitted only for truncated reads) — the event-type list of A15 grows from six to seven; existing frames and clients are untouched (a `tool_result` frame is additive; unknown types are ignored).
 - **Tool choice:** the "grep or search or find tool (whichever matches most closely to existing harnesses)" is **`grep`** — it already exists, searches a single document when scoped, and is the phase-70 harness-aligned name. No new tool is added.
 ## Design (shared by all tasks — the executor reads this, not the chat)
 ### The cap (task 01)
 - **Setting:** `BOR_READ_MAX_CHARS` — `read_max_chars: int = 128_000` in `app/config.py`. Spec rationale (pinned in the docstring): 128 000 chars ≈ **32 000 tokens** at the ~4-chars/token estimate the house already uses (`app/rag/llm.py` embed batching notes ~3 chars/token for code-dense text, 4 for prose) — **a quarter of the 128k-token minimum context** the owner names, leaving ~96k for the system prompt, the top-2 `<documents>`, the tool rounds, and the 32 768-token answer cap (`max_output_tokens`). Char-based (no tokenizer in the repo — the `BOR_SUMMARY_MAX_CHARS` precedent), env-tunable in both directions.
 - **The read branch:** `len(content) > cap` → body = `content[:cap]` + `TRUNCATION_MARKER` + the pinned notice line:
  `TRUNCATED — this document is {total} characters; only the first {shown} are in your context. The rest is NOT shown. Use grep (pattern) to locate what you need — grep searches the whole document.`
  (constant `READ_TRUNCATION_NOTICE` with `{shown}`/`{total}` format fields, next to the other refusal constants). At exactly the cap: no marker (the document fit).
 - **Signaling the UI:** `AgentHolder` gains `read_truncations: list[tuple[str, int, int]]` (argument, chars_shown, chars_total); the read branch appends on truncation. `run_agent`, after executing each round's calls, yields one new piece per new entry — `ToolResultPiece` (new, in `app/rag/llm.py` with the piece family): `name`, `argument`, `truncated: bool`, `chars_shown: int`, `chars_total: int`.
 ### SSE + UI + persistence (task 02)
 - **SSE:** `app/api/chat.py` `_pump` gains the branch (mirror the `ToolCallPiece` branch): `ChatToolResultEvent` (new, `app/schemas.py` next to `ChatToolEvent`) → `{"type": "tool_result", "name": "read", "argument": "src/path", "truncated": true, "chars_shown": N, "chars_total": M}`. Emitted **after** the matching `tool` frame (the call is already shown; the marker lands a beat later — the phase-37/48 "calling tool" timing is untouched). The module docstring's SSE contract paragraph + the A15-extension note are updated.
 - **Live UI:** `app.js` — on a `tool_result` frame, find the newest `.tool-call` line whose text is the Reading line for that argument and append `<span class="truncated-note"> (truncated — showing {N} of {M} chars)</span>` (createElement + textContent — the house "this file never builds HTML" rule; no innerHTML). `styles.css`: `.tool-call .truncated-note { color: var(--ink_soft); }` — theme-neutral, **no new hue** (phase-92 invariant; it must gray out automatically under a monochrome theme — see phase 93).
 - **Saved + shared:** `ToolCall` (`app/schemas.py`) gains `truncated: bool = False`, `chars_shown: int | None = None (ge=0)`, `chars_total: int | None = None (ge=0)` (phase-83 bounds philosophy: small additive fields, no migration — saved JSON validates). `app.js` `toolAcc`: the `tool_result` frame stamps the matching entry. The save payload carries it; the restore path (`app.js` ~L1442) and `shared.js` render the same marker from the stored record, so a saved/shared chat shows the truncation **pixel-identically** (the phase-50 restore contract).
 ### E2E (task 03)
 `tests/e2e/test_read_truncation_cap.py`: the app under test boots with `BOR_READ_MAX_CHARS=1500` (the `test_import_extensions_env.py` env-override pattern); a local-dir source seeds one ~3 000-char document; a scripted mock-LLM turn `read`s it; asserts — the SSE stream carries the `tool_result` frame with the right counts; the Reading line shows the marker; the mock's echo proves the LLM context carried `[…truncated…]` + the grep pointer; the saved chat's `tools` record carries `truncated: true` + counts; the shared page renders the same marker.
 ## Tasks
 1. `01_read_cap.md` — `BOR_READ_MAX_CHARS` + the truncated read result + `ToolResultPiece` + tool description + prompt teaching
 2. `02_sse_and_ui.md` — the `tool_result` SSE event + the live/saved/shared "(truncated …)" marker
 3. `03_e2e_truncated_read.md` — the dedicated story suite
 ## Testing & Quality
 - Unit/integration: cap boundary (at cap / cap+1); the marker + notice text pinned; a non-truncated read byte-identical to today's result; `ToolResultPiece` emission order (after the tool frame, before the next round); `holder` accounting (truncations don't touch `tool_calls` — a truncated read is still a successful call); the SSE frame in the chat integration suite; the `ToolCall` schema round-trip (old saved chats without the fields still validate — the phase-50 backward-compat rule); `read`-already-in-context / refusal paths untouched.
 - Coverage: **>90%** on new/modified code (`uv run pytest --cov=app --cov-report=term-missing`).
 - E2E: `uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov` in isolation; the existing suites (`test_agent_document_tools.py`, `test_chat_history.py`, `test_share_chat.py`, `test_big_read_progress.py`) stay green in isolation.
 - Lint/types: `uv run ruff check . && uv run pyright`.
 ## Completion Criteria
 - [ ] a document longer than `BOR_READ_MAX_CHARS` returns first-cap-chars + `[…truncated…]` + the pinned grep-pointer notice to the LLM (unit-pinned)
 - [ ] the user sees "(truncated — showing N of M chars)" on the Reading line — live, in a saved chat, and on the shared page
 - [ ] a document at/under the cap is read byte-identically to today (no marker, no frame)
 - [ ] the top-2 `<documents>` retrieval path is untouched (A7's never-truncated contract holds for it)
 - [ ] test suite green, coverage >90%, ruff + pyright clean
 - [ ] no behavior change in completed phases; one atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(agent): cap read at 128k chars with honest truncation and a visible UI marker`)
@@ -0,0 +1,37 @@
 # Task 01 — The read cap: setting, truncated result, the piece, the teaching
 **Phase:** `95_read_truncation_cap` · **Source:** `TODO.md:5` — "If the LLM calls read on a huge document there should be a sensible cap on the amount it can read at once. My LLMs all have a minimum cap of 128,000 tokens of context, so spec for that. The LLM should be informed the read was truncated, and should be offered a grep or search or find tool (whichever matches most closely to existing harnesses) to search the document for what it was looking for."
 **Story:** n/a (owner TODO item).
 ## Objective
 `read` returns at most `BOR_READ_MAX_CHARS` characters (default 128 000 ≈ 32k tokens, spec'd in the docstring against the 128k-token minimum context); an over-cap read carries `[…truncated…]` + the pinned grep-pointer notice; the truncation is signaled to the API layer via a new `ToolResultPiece`; the tool description and prompt teach the contract.
 ## Work
 1. `app/config.py` — `read_max_chars: int = 128_000` (env `BOR_READ_MAX_CHARS`), docstring with the pinned spec rationale: 128 000 chars ≈ 32 000 tokens at the ~4-chars/token house estimate (`app/rag/llm.py` embed-batching note: ~3 chars/token code-dense, 4 prose) — a quarter of the 128k-token minimum context, leaving ~96k for the system prompt + the top-2 `<documents>` + tool rounds + the 32 768-token answer cap. Char-based per the `BOR_SUMMARY_MAX_CHARS` precedent (no tokenizer in the repo).
 2. `app/rag/agent.py`:
   - `READ_TRUNCATION_NOTICE` constant (next to the other refusal constants) with `{shown}`/`{total}` format fields, pinned copy:
     `TRUNCATED — this document is {total} characters; only the first {shown} are in your context. The rest is NOT shown. Use grep (pattern) to locate what you need — grep searches the whole document.`
   - The read branch of `_execute_tool`: when `len(doc.content) > settings.read_max_chars`, the result body is `content[:cap]` + `\n` + `TRUNCATION_MARKER` (`app/rag/retriever.py` — import it, don't retype) + `\n` + the formatted notice; otherwise the result is **byte-identical to today** (`f"Document {doc.source}/{doc.path}:\n{doc.content}"`). On truncation, append `(argument, cap, total)` to the holder (see step 3).
   - `AgentHolder` (the dataclass at ~L680): new field `read_truncations: list[tuple[str, int, int]] = field(default_factory=list)` — docstring: recorded per truncated `read`; a truncated read is still a **successful** call (`tool_calls` increments as today, `read_docs` appends as today).
   - `run_agent`: after executing a round's calls (where `_execute_tool` results are appended to the messages), yield one `ToolResultPiece` per NEW `holder.read_truncations` entry (snapshot `len` before the round's executions; iterate the new tail) — the piece lands AFTER the round's `tool` frame(s) and BEFORE the next model round.
 3. `app/rag/llm.py` — `ToolResultPiece` (with the piece family, next to `ToolCallPiece`): fields `name: str`, `argument: str | None`, `truncated: bool`, `chars_shown: int`, `chars_total: int`; docstring citing the A15 extension (owner permission 2026-09-10, `TODO.md` L5 — recorded in `00_phase.md`; PLAN.md is being redone by the owner).
 4. `app/rag/agent.py` — `AGENT_TOOLS` `read` description: append the truncation sentence (pinned copy): "Very large documents are truncated: you receive the first part plus a TRUNCATED notice naming how many more characters exist — the notice is authoritative, the document did NOT end where it stopped. Follow it and use `grep` (pattern) to locate the rest — it searches the whole document."
 5. `app/rag/prompts.py` — `TOOLS_SECTION`: one added line to the `read` teaching (short, the house tone): the cap + the notice + the grep follow-up. `ls`/`grep` teaching untouched (phase 94 owns `ls`).
 6. `app/rag/agent.py` module docstring — the tool-surface paragraph: the phase-95 revision note (the A7 scope clarification: retrieval top-2 stays never-truncated; the read-tool path is capped per the owner's explicit request).
 - ASSUMPTION: 128 000 chars default (≈ 32k tokens), env `BOR_READ_MAX_CHARS` — the TODO says "sensible cap … spec for that"; the quarter-budget of the 128k minimum is the spec (confirmed in the roadmap).
 - ASSUMPTION: `grep` is the offered tool (it already exists and is harness-aligned — no new tool); the notice names it explicitly so the model doesn't guess.
 - ASSUMPTION: the marker goes AFTER the body content (the model reads the first part cleanly, then the notice) — matching how the summarizer appends `TRUNCATION_MARKER` at its cut.
 ## Testing & Quality
 - Unit (`tests/unit/test_agent.py`): boundary — a document of exactly `cap` chars → no marker, byte-identical to today's result; `cap + 1` → marker + notice with the right `{total}`/`{shown}`; the notice text pinned; `holder.read_truncations` content; `tool_calls`/`read_docs` accounting unchanged for a truncated read; `run_agent` yields the `ToolResultPiece` after the tool frame and before the next round (fake LLM stream), exactly one per truncated read, zero for a short read; the `read` refusal paths (`ALREADY_IN_CONTEXT`, `_no_document_refusal`) untouched.
 - Unit: `AGENT_TOOLS` read description + `TOOLS_SECTION` pin the new copy (house string tests).
 - Integration: `tests/integration/test_chat_api.py` — the `ToolResultPiece` flows through (the SSE half is task 02; here the agent loop's yield order against the real prompt path).
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] `read` of a > 128 000-char document returns first 128 000 chars + `[…truncated…]` + the pinned notice (unit-pinned)
 - [ ] a ≤ cap document is read byte-identically to today
 - [ ] `ToolResultPiece` exists, is yielded in order, and carries (argument, shown, total)
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (the retrieval `<documents>` path and the deflected path byte-identical)
@@ -0,0 +1,35 @@
 # Task 02 — The `tool_result` SSE event + the visible marker (live, saved, shared)
 **Phase:** `95_read_truncation_cap` · **Source:** `TODO.md:5` — "There should be a visual indicator that the read was trnucated so the user knows what's going on." (plus the A15 extension + saved/shared persistence)
 **Story:** n/a (owner TODO item).
 ## Objective
 The truncation the LLM is told about is also told to the USER: a `tool_result` SSE frame (emitted only for truncated reads) and a "(truncated — showing N of M chars)" marker on the Reading line — live during the stream, in saved chats, and on shared pages.
 ## Work
 1. `app/schemas.py` — `ChatToolResultEvent` next to `ChatToolEvent`: `type: str = "tool_result"`, `name: str`, `argument: str | None`, `truncated: bool = True`, `chars_shown: int (ge=0)`, `chars_total: int (ge=0)`; docstring citing the A15 extension (owner permission 2026-09-10 — the event-type list grows six → seven; additive, existing frames untouched).
 2. `app/api/chat.py` — the `_pump` piece loop: a new `isinstance(piece, ToolResultPiece)` branch (mirror the `ToolCallPiece` branch) → `yield sse_event(ChatToolResultEvent(name=piece.name, argument=piece.argument, truncated=piece.truncated, chars_shown=piece.chars_shown, chars_total=piece.chars_total).model_dump())`. Update the module docstring's SSE contract paragraph (the event list + the "emitted after the matching `tool` frame" timing + the A15-extension note).
 3. `frontend/assets/app.js`:
   - the SSE `tool_result` handler: find the NEWEST `.tool-call` line in the turn's scratchpad whose text is the Reading line for the frame's argument (the `📄 Reading {argument}` line the `tool` frame created) and append a marker: `document.createElement("span")` + `className = "truncated-note"` + `textContent = " (truncated — showing " + N + " of " + M + " chars)"` (the house "this file never builds HTML" rule — createElement/textContent only, never innerHTML).
   - `toolAcc` (the saved-session tools array, built from the `tool` frames): on the `tool_result` frame, stamp the matching entry (same argument, newest) with `truncated: true` + `chars_shown` + `chars_total` — the save payload then carries it with zero other change.
   - the phase-14 LOCAL restore path (~L1442, `for (const t of m.tools)`): a stored `t.truncated` renders the same marker next to the restored Reading line.
 4. `frontend/assets/shared.js` — the shared view's tool-line render (~L148, the "read → the Reading line" mapping): the same marker from the stored record.
 5. `frontend/assets/styles.css` — `.tool-call .truncated-note { color: var(--ink_soft); }` — theme-neutral, no new hue (the phase-92 zero-literal invariant; under phase 93's monochrome theme it grays automatically).
 6. `app/schemas.py` — `ToolCall` (the saved-session record, ~L443): add `truncated: bool = False`, `chars_shown: int | None = Field(default=None, ge=0)`, `chars_total: int | None = Field(default=None, ge=0)`; docstring note — pre-phase-95 saved chats (no fields) validate unchanged (the phase-50 backward-compat rule; no migration — `ChatMessage.tools` is JSON).
 - ASSUMPTION: marker copy pinned exactly as ` (truncated — showing {N} of {M} chars)` — plain integers, no thousands separators (the assertion target for unit + E2E).
 - ASSUMPTION: the marker is a CSSOM/DOM append to the existing line (no new line, no re-render) — the phase-37/48 tool-line lifecycle (one line per call, "Stop" label contract) is untouched.
 - ASSUMPTION: `tool_result` frames for non-truncated reads are NOT emitted (one frame = one noteworthy event; the live marker only appears when truncation actually happened).
 ## Testing & Quality
 - Integration (`tests/integration/test_chat_api.py`): a grounded turn with a truncated read (test DB + a long document + the cap lowered via settings override) streams `tool` → `tool_result` (asserted in order, right counts) → `delta…`; a non-truncated read streams NO `tool_result`; the deflected path streams no tool frames at all (A8 unchanged).
 - Unit (house frontend-text style): `app.js` contains the `tool_result` handler + the exact marker template + the `toolAcc` stamp; `shared.js` the same render; `styles.css` the `.truncated-note` rule uses only a `var(--…)` color.
 - Unit (`tests/unit/` schemas): `ToolCall` round-trips with and without the new fields (old-shape JSON validates — the backward-compat assertion); `ChatToolResultEvent` shape.
 - Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
 ## Completion Criteria
 - [ ] the SSE stream of a truncated read carries exactly one `tool_result` frame with the right counts, after its `tool` frame
 - [ ] the Reading line shows the marker live; a saved chat re-renders it; the shared page shows it
 - [ ] old saved chats (no `truncated` field) load and render unchanged
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (the six existing event types byte-identical)
@@ -0,0 +1,33 @@
 # Task 03 — E2E: the truncated read (the dedicated story suite)
 **Phase:** `95_read_truncation_cap` · **Source:** `TODO.md:5` — the whole item (this task is the story gate: the LLM is told about the truncation, the user sees the marker — live, saved, shared)
 **Story:** n/a (owner TODO item — one Playwright file per story, run in isolation, A16).
 ## Objective
 `tests/e2e/test_read_truncation_cap.py` proves the contract end to end under the deterministic mock LLM: a document over the (lowered) cap is read truncated, the `tool_result` frame lands, the Reading line carries the marker, the LLM's context carried `[…truncated…]` + the grep pointer, and the marker survives save → shared.
 ## Work
 1. `tests/e2e/test_read_truncation_cap.py` (new, isolated — `tests/e2e/conftest.py` fixtures `app_server` + `mock_llm`; admin login via `tests/e2e/auth_helpers.py`):
   - **Lowered cap:** the app under test boots with `BOR_READ_MAX_CHARS=1500` (the env-override-for-the-app-under-test pattern from `tests/e2e/test_import_extensions_env.py`).
   - **Seed:** one local-directory source containing one ~3 000-character `.md` document (deterministic content — e.g. a repeated but varied paragraph block; the exact text is the executor's, pinned in the test so the counts are known), registered + synced (the `tests/e2e/test_local_directory_sources.py` pattern).
   - **Scripted turn** (extend `tests/e2e/mock_llm.py` with a new script, the `tests/e2e/test_agent_document_tools.py` pattern): the mock calls `read("<source>/<file>.md")` and then answers by **echoing** what its tool result contained (the house way of asserting on the LLM's context).
   - **Assertions:**
     1. the SSE stream (captured over the websocket/SSE read the chat suites use) carries `{"type": "tool", "name": "read", …}` THEN exactly one `{"type": "tool_result", "name": "read", "truncated": true, "chars_shown": 1500, "chars_total": <known>}` THEN the answer's `delta`/`done` frames;
     2. the Reading line in the scratchpad shows ` (truncated — showing 1500 of <known> chars)` (the pinned copy, task 02);
     3. the mock's echoed answer proves the LLM context carried `[…truncated…]` AND the `TRUNCATED — … Use grep …` notice (assert both substrings in the rendered answer);
     4. **save → shared:** save the chat (the phase-50 save flow) → the stored message's `tools` record has `truncated: true` + the counts (assert via the saved-chats API) → open the shared page (`/shared/<token>`) → the Reading line there shows the same marker;
     5. **control:** a second turn reading a SHORT document (under 1500 chars) → NO `tool_result` frame, NO marker, the answer's echo shows no `[…truncated…]`.
 2. Run in isolation: `uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov` (db up: `podman compose up -d db`).
 3. Regression gate, in isolation: `test_agent_document_tools.py`, `test_chat_history.py`, `test_share_chat.py`, `test_big_read_progress.py`, `test_stop_generation.py`.
 - ASSUMPTION: the SSE capture reuses whatever stream-reading helper the existing chat suites use (the `test_chat_rag.py` / `test_stop_generation.py` pattern) — no new harness machinery.
 - ASSUMPTION: `chars_total` is asserted against the pinned seed document's exact length (the test computes `len(content)` from the same string it writes).
 ## Testing & Quality
 - This IS the E2E task.
 - Coverage: **>90%** on `app/` (this task adds test code only — keep the gate green).
 ## Completion Criteria
 - [ ] the suite is green in isolation and encodes: frame order, live marker, LLM-visible notice (via the echo), save + shared fidelity, and the no-truncation control
 - [ ] full test suite green, coverage >90%
 - [ ] no behavior change in completed work (the five gated suites green)