phase: 96_oneshot_resilience

All checks complete. Final report:

**Phase 96 (one-shot resilience) — final verification pass, all green** (all 4 task files already in `complete/`; verified the working-tree implementation against the design)

- `LLMClient.chat()` empty-content retry (D1–D3) via `_chat_once` + `_EmptyContentError` (carries `finish_reason`), under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` — verified in diff
- `missing_folder_summaries()` + `generate_folder_summaries(only_missing=…)` — verified; `folder_summary_table_empty` deleted, both sync gates switched to the gap probe
- `.env.example` comments updated (chat-turn stream + one-shot summary calls)

**Test / lint / coverage results**
- `uv run pytest --cov=app --cov-report=term-missing` → **1988 passed**, coverage **99%** (gate >90%)
- `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` → **2 passed** (isolated)
- Regressions, each isolated: `test_ls_tree_drilldown` 3 passed, `test_sync_button` 3 passed, `test_local_directory_sources` 3 passed, `test_llm_retry` 4 passed
- `uv run ruff check . && uv run pyright` → clean (0 errors)

**Completion criteria:** retry-then-recover unit-pinned ✓ · exhaustion + `BOR_LLM_RETRIES=0` byte-identical ✓ · streaming path untouched ✓ · gap-fill both sync paths, other rows byte-identical incl. `updated_at` ✓ · no-gap zero-burn ✓ · phase E2E green ✓ · regression E2Es green ✓ · full suite + >90% + lint/types ✓ · no completed-phase behavior change (full suite green) ✓. Commit left to the harness per executor rules (working tree, 16 files).

**Deviations:** none. **Next pending phase:** `97_kb_tree_catalog`.

.agents/phases/complete/96_oneshot_resilience/00_phase.md

@@ -0,0 +1,90 @@
+# Phase 96 — One-shot LLM resilience: retry empty replies, self-heal missing folder summaries
+
+**Source:** 2026-09-11 incident (owner chat) — the then-`lite` model intermittently returned empty completions: its internal monologue (the wire's `reasoning_content` field) consumed the entire `max_tokens=2048` budget, so the reply arrived as `content=""` + `finish_reason="length"`. Measured 2 empties in 90 folder-summary calls on the `deploy` source sync (≈2–7% per call in follow-up probes), losing two stored rows: `deploy/Deployments` (283 docs) and `deploy/Deployments/stackexpected/website/app` (7 docs). The app handled it exactly as designed (`LLMClient.chat` refuses to store a silent summary; `generate_folder_summaries` is per-folder fail-soft) — but nothing retried, and nothing ever filled the hole: the folder-summary gate (`changed KB or empty table`) skips regeneration on unchanged re-syncs, so missing rows persist until a KB change. The owner replaced the `lite` model server-side (it no longer emits `reasoning_content` — verified live: 16/16 clean, no `reasoning_content` field); this phase removes the remaining app-side exposure so ANY future transient empty reply from ANY one-shot consumer cannot silently lose data.
+**Story:** n/a (incident-driven resilience — one Playwright file per phase, run in isolation, A16).
+**Context:**
+- `app/rag/llm.py` — `LLMClient.chat()` (~L335–378) is the one-shot surface: transport failure → `LLMError` (wrapped), choiceless reply → `LLMError`, `content` None/blank → `LLMError("…returned empty content — refusing to store a silent summary")`. No retry. Phase 67's retry machinery (`BOR_LLM_RETRIES`=3 / `BOR_LLM_RETRY_DELAY`=5 s, flat; `chat_stream_retried`, `RetryPiece`) covers the **streaming** chat-turn path only — phase 67's owner-locked A1 explicitly scoped it to the chat turn and left `chat()` out ("admin/import paths with their own fail-fast behavior").
+- `chat()` consumers (all fail-soft per call, all protected once `chat()` retries): `app/rag/summarizer.py` L113 (document summaries — failure leaves `documents.summary` NULL + a `summary_errors` count), `app/rag/overview.py` L182 (KB overview — failure keeps the old row; its `_overview_row_exists` gate already self-heals a missing row on the next unchanged walk), `app/rag/folder_summaries.py` L267 via `summarize_folder` → `generate_folder_summaries` (per-folder fail-soft; **no self-heal** — the gap this phase fills), `app/rag/llm.py` L641 `check_models` (the sync-time "ping" probe).
+- `app/config.py` — `llm_retries: int = 3`, `llm_retry_delay: float = 5` (phase 67; validators already enforce the house kill-switch pattern: 0 disables, negative fails startup).
+- The openai SDK's own `max_retries=2` (kept by `LLMClient`) already re-POSTs **wire-level** 500s — the gap is the **semantic** failure: the endpoint is healthy, answers, and says nothing.
+- `scripts/import_docs.py` `_run()` (~L335–360) and `app/api/sync.py` `_run_sync()` (L276–281) — the folder-summary gate, one copy per path: `changed = summary.added + summary.updated > 0`; script path `folders_due = changed or _folder_summaries_table_empty()` (`--limit` skips entirely); API path `if changed or folder_summary_table_empty(fs_db)`. `folder_summary_table_empty` (`app/rag/folder_summaries.py`) is the bounded `LIMIT 1` probe.
+- `app/rag/folder_summaries.py` — `group_by_folder` (the one recursive-subtree concept), `MIN_DOCS_PER_FOLDER = 2`, `generate_folder_summaries` (sorted candidate iteration, per-folder fail-soft upsert, then a prune pass deleting rows whose folder lost ≥ 2 docs; flushes only — the sync path owns the transaction).
+- `tests/unit/test_llm_client.py` — the fake-client harness (`_FakeCompletion`, `_make_chat_client`, `SimpleNamespace` completions); existing pins `test_chat_missing_content_raises_llm_error` / `test_chat_whitespace_only_content_raises_llm_error` (~L989–1000) match `"empty content"`.
+- `tests/unit/test_folder_summaries.py` — the duck-typed fake LLM with a `.calls` counter (the generator's unit pattern).
+- `tests/integration/test_sync_folder_summaries.py` + `tests/integration/test_import_docs_overview.py` — the change-gate integration pattern (fake `LLMClient` keyed on the mode marker; zero-call assertions on unchanged re-runs).
+- `tests/e2e/mock_llm.py` — `compose_answer`'s `FOLDER_SUMMARY_MODE` branch (L1504) returns the deterministic one-liner `Fixture folder summary for <folder>.` from the user message's `Folder: …` header (imported `FOLDER_HEADER_PREFIX` — the mock can never drift from it); phase 67's failure-injection machinery: module-level `_fail_posts` counters + `_bump_fail`, keyed by trigger phrase, **reset after the success they guard**.
+- `tests/e2e/test_ls_tree_drilldown.py` (phase 94) — the scripted `ls` drill-down turn pattern (the mock echoes the tool result into its grounded answer — the E2E's only lens on the LLM context) and the temp-local-source seed + `POST /api/sync` pattern; `tests/e2e/test_llm_retry.py` (phase 67) — the test-server env-override pattern (`BOR_LLM_RETRY_DELAY=0`).
+
+## Objective
+A transient empty one-shot LLM reply can no longer silently lose data: `LLMClient.chat()` retries an empty-content reply under the existing `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` policy (logged per attempt) before raising, and an **unchanged**-KB sync detects folder-summary gaps (candidate folders with no stored row) and fills **only** the missing rows — so a failed folder summary self-heals on the next sync instead of persisting until a KB change.
+
+## Dependencies
+- `67_llm_retry` (complete) — the retry policy knobs (`BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY`), the logging/kill-switch conventions, and the mock's failure-injection pattern.
+- `94_ls_tree_drilldown` (complete) — `folder_summaries`, `generate_folder_summaries`, the change-gate, the E2E drill-down + sync patterns.
+
+## Decisions recorded here (no PLAN.md change needed — no new technology, no new env vars, no new SSE events)
+- **D1 — knob reuse:** the one-shot path honors the **existing** `BOR_LLM_RETRIES` (default 3) / `BOR_LLM_RETRY_DELAY` (default 5 s) — no new settings. Phase 67's owner-locked A3 named these the house LLM-retry policy; extending them to `chat()` is a deliberate extension of that policy, not a deviation (the lock scoped *what phase 67 shipped*, it does not prohibit `chat()` from honoring the same knobs). `.env.example` comments are updated to say so. (If the owner prefers a separate one-shot knob, task 01 is a one-line change.)
+- **D2 — retry condition:** `chat()` retries **empty content only** (`content` None or whitespace — the exact incident failure class). A choiceless reply still raises immediately (no retry), and transport failures keep today's behavior (the SDK's `max_retries=2` already covers wire-level 500s; an app-level transport retry would double-stack the SDK policy and multiply the 300 s timeout).
+- **D3 — exhaustion contract:** after `1 + llm_retries` empty attempts, `chat()` raises the same `LLMError` type with an updated message naming the attempts (`…returned empty content on all {N} attempts — refusing to store a silent summary`); with `llm_retries=0` the message stays **byte-identical** to today's (the feature is fully off — phase 67 kill-switch convention). All consumers' fail-soft behavior is untouched (document summary → NULL + count, overview → old row, folder → skip + old row, probe → sync fails loudly as today).
+- **D4 — targeted gap-fill:** `generate_folder_summaries` gains `only_missing=False`. The changed-KB path is **unchanged** (full regeneration + prune). The unchanged-KB gate changes from "table empty?" to "any candidate row missing?" (subsumes the table-empty first-run case exactly); when a gap exists it runs `only_missing=True` — only the missing candidates are generated, existing rows stay byte-identical (text AND `updated_at`), and the prune pass still runs (a no-op on an unchanged KB, the invariant kept). `--limit` (script only) still skips generation entirely.
+- **Non-goals:** the streaming path (`chat_stream` / `chat_stream_retried`) is untouched (phase 67 territory); no `documents.summary`-NULL catch-up (the retry eliminates the failure class — expected per-doc loss p⁴; the importer's unchanged-skip would be a separate, larger change); no KB-overview catch-up (its `_overview_row_exists` gate already self-heals); no UI/SSE/sync-status surface change (folder stats stay log-only, phase 94 contract); no aipi/model-side change (owner already done).
+
+## Design (shared by all tasks — the executor reads this, not the chat)
+
+### The one-shot retry (task 01)
+`app/rag/llm.py` `LLMClient`:
+- Extract the single-attempt body of `chat()` into a private `_chat_once(messages, model) -> str` (the existing try/except transport wrap + choiceless check + empty-content check, verbatim behavior). `chat()` becomes: attempt 1 via `_chat_once`; on the **empty-content** `LLMError` only, while attempts remain: `logger.warning` (PLAN §9: model, `finish_reason` of the empty reply when available, `attempt n of N`) + `await asyncio.sleep(settings.llm_retry_delay)` (flat, phase 67) + retry via `_chat_once`. Any other `LLMError` propagates immediately.
+- `N = 1 + settings.llm_retries` total attempts. Exhaustion → the D3 message. `llm_retries=0` → one attempt, the byte-identical legacy message.
+- The empty-content check must capture the reply's `finish_reason` for the log line (the incident's signature was `finish_reason="length"` — it makes the warning greppable and self-explaining).
+- `chat()` docstring: the phase-30 "a silent empty summary must never be stored" contract stays, extended with the retry policy (D1/D2/D3). `.env.example`: the two phase-67 comments gain "(chat-turn stream and one-shot summary calls)".
+- `check_models` benefits automatically (no change).
+
+### The gap detector + targeted fill (task 02)
+`app/rag/folder_summaries.py`:
+- `missing_folder_summaries(db) -> list[tuple[str, str]]` — the candidate folders (the generator's exact candidate computation: one catalog query ordered by `(source, path)`, `group_by_folder`, subtrees with ≥ `MIN_DOCS_PER_FOLDER`) that have **no stored row**, sorted by `(source, folder_path)`. The gate probe and the fill both use this one function — one concept, as with `group_by_folder`.
+- `generate_folder_summaries(db, llm, *, skip=False, only_missing=False)` — `only_missing=True` restricts the sorted candidate iteration to the keys from `missing_folder_summaries(db)` (computed from the SAME catalog pass — no second full query inside the generator: derive the missing set from the candidates already grouped). Upsert, fail-soft, and the prune pass are otherwise unchanged; stats dict shape unchanged (the caller logs the mode).
+- `folder_summary_table_empty` is deleted in task 03 (both call sites replaced — the function becomes dead code; its unit tests move to `missing_folder_summaries`).
+
+### The sync gates (task 03)
+Both paths, identical shape (the phase-94 "one gate, two call sites" convention):
+- Changed KB → `generate_folder_summaries(session, llm)` — full regeneration (today's behavior, byte-identical).
+- Unchanged KB → `missing = missing_folder_summaries(session)`; non-empty → `generate_folder_summaries(session, llm, only_missing=True)` + a log line naming the gap-fill (script: the summary line token becomes `folder_summaries=<g>/<f>/<p> (gap-fill)`; API: `sync: folder_summaries gap-fill stats=…`); empty → today's skip log.
+- `--limit` (script) → unchanged full skip. Transaction convention unchanged (generator flushes, the path commits).
+
+### The E2E (task 04)
+`tests/e2e/mock_llm.py` — the `FOLDER_SUMMARY_MODE` branch gains the incident-shape injection, keyed by the folder LABEL (the seeded KB's folder names carry the trigger — deterministic, no env, no request-order state beyond the phase-67 per-key counter, reset after the success it guards):
+- label ends with `/e2e_empty_once` → the FIRST non-stream POST for that label returns the incident envelope: same OpenAI shape, `message.content=""`, `finish_reason="length"`; later POSTs return the normal `Fixture folder summary for <folder>.` line.
+- label ends with `/e2e_empty_always` → every non-stream POST for that label returns the empty envelope.
+`tests/e2e/test_oneshot_llm_retry.py` (new; `app_server` + `mock_llm` conftest fixtures; admin login via `tests/e2e/auth_helpers.py`; sync via `POST /api/sync` per `test_sync_button.py`; the test server boots with `BOR_LLM_RETRY_DELAY=0` per the phase-67 pattern; default `BOR_LLM_RETRIES=3`):
+1. Seed a temp local-dir source with three ≥ 2-doc folders: `e2e_empty_once/`, `e2e_empty_always/`, `normal/`.
+2. Sync #1 → a scripted `ls <source>` drill-down turn (the phase-94 echo pattern) asserts: the `e2e_empty_once/` line carries `: Fixture folder summary for <src>/e2e_empty_once.` (the retry recovered the row — without task 01 it would be absent), the `normal/` line does too, and the `e2e_empty_always/` line has NO `: …` suffix (exhausted → fail-soft → absent) while the sync still reports success (the folder-stats failure never flips the run, phase 94).
+3. Delete the `normal` stored row directly (simulating a historical failure) → Sync #2 (KB unchanged) → the `normal` row is back with its deterministic text; the `e2e_empty_once` row's `updated_at` is **unchanged** (targeted fill — no full regeneration); `e2e_empty_always` remains absent (the gap-fill attempts it, exhausts, stays absent, sync still succeeds).
+
+## Tasks
+1. `01_chat_retry.md` — `chat()` empty-content retry under `BOR_LLM_RETRIES`/`BOR_LLM_RETRY_DELAY` + logging + unit pins
+2. `02_gapfill_generator.md` — `missing_folder_summaries` + `generate_folder_summaries(only_missing=…)` + unit pins
+3. `03_sync_gates.md` — the gap gate in `scripts/import_docs.py` + `app/api/sync.py` (replacing the table-empty probe) + integration pins
+4. `04_e2e_oneshot_retry.md` — the mock's incident-shape injection + `tests/e2e/test_oneshot_llm_retry.py` + regressions + commit
+
+## Testing & Quality
+- Unit: `tests/unit/test_llm_client.py` (retry matrix: empty-then-success = 2 attempts + 1 sleep; all-empty = `1 + retries` attempts + the exhaustion message; first-success = 1 attempt, 0 sleeps, byte-identical return; choiceless reply = no retry, immediate error; transport error = no app-level retry; `llm_retries=0` = legacy byte-identical message; sleep value honored — record `asyncio.sleep`); `tests/unit/test_folder_summaries.py` (`missing_folder_summaries` shapes: fresh table → all candidates, full table → empty, one row deleted → that folder; `only_missing=True` → only missing keys generated, existing rows byte-identical incl. `updated_at`, zero LLM calls when there is no gap, prune still runs; `only_missing=False` unchanged).
+- Integration: `tests/integration/test_sync_folder_summaries.py` + the `test_import_docs_overview.py` pattern (both sync paths: unchanged + gap → targeted fill of exactly the missing rows; unchanged + no gap → zero `FOLDER_SUMMARY_MODE` calls; changed → full regeneration as today; `--limit` skips).
+- E2E (mandatory, A16): `tests/e2e/test_oneshot_llm_retry.py`, in isolation, deterministic mock, `BOR_LLM_RETRY_DELAY=0`.
+- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
+- Lint/types: `uv run ruff check . && uv run pyright`.
+
+## Completion Criteria
+- [ ] a one-shot `chat()` call whose first reply is empty content (None or whitespace) is retried up to `BOR_LLM_RETRIES` times with `BOR_LLM_RETRY_DELAY` between attempts, one `WARNING` per retry (model + finish_reason + attempt count), and succeeds when a later attempt returns content (unit-pinned)
+- [ ] an all-empty `chat()` raises `LLMError` naming the attempts after exactly `1 + BOR_LLM_RETRIES` attempts; `BOR_LLM_RETRIES=0` reproduces today's single-attempt behavior and message byte-identically
+- [ ] the streaming path is untouched: `test_llm_retry.py` (phase 67) green in isolation
+- [ ] an unchanged-KB sync with a missing folder-summary row regenerates exactly that row (both `scripts/import_docs.py` and `POST /api/sync`), leaves every other row byte-identical (incl. `updated_at`), and logs the gap-fill (integration-pinned)
+- [ ] an unchanged-KB sync with no gap burns zero `lite` folder-summary calls (the phase-94 zero-burn invariant holds)
+- [ ] `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` green in isolation: the `e2e_empty_once` folder summary is recovered by the retry (visible in the `ls` drill-down), `e2e_empty_always` stays absent with the sync green, and the deleted-row gap self-heals on the next unchanged sync
+- [ ] regression E2E green in isolation: `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py`
+- [ ] `uv run pytest` green, coverage >90%, `uv run ruff check . && uv run pyright` clean
+- [ ] no behavior change in completed phases; one atomic Conventional Commit, `--no-gpg-sign`
+
+## Commit
+```bash
+git add -A .agents/ app/ scripts/ tests/ && git commit --no-gpg-sign -m "fix(rag): retry empty one-shot LLM replies and self-heal missing folder summaries on unchanged syncs"
+```

.agents/phases/complete/96_oneshot_resilience/01_chat_retry.md

@@ -0,0 +1,38 @@
+# Task 01 — `chat()` retries an empty one-shot reply under the house retry policy
+
+**Phase:** `96_oneshot_resilience` · **Story:** n/a (incident-driven resilience — phase 67's owner-locked A1 left the one-shot path out of scope; this task closes that gap per owner request 2026-09-11).
+
+## Objective
+`LLMClient.chat()` (the one-shot surface used by document summaries, the KB overview, folder summaries, and the sync probe) retries an **empty-content** reply up to `BOR_LLM_RETRIES` times with `BOR_LLM_RETRY_DELAY` between attempts, logs each retry, and only then raises — so a transient "the model answered but said nothing" reply (the 2026-09-11 incident: `content=""`, `finish_reason="length"`, the whole budget spent in `reasoning_content`) no longer loses a summary on its first attempt.
+
+## Work
+1. `app/rag/llm.py` — `LLMClient`:
+   - Extract `chat()`'s single-attempt body verbatim into a private `_chat_once(self, messages, model) -> str`: the existing `try/except` transport wrap (→ `LLMError` with the sanitized base URL), the choiceless check (`…returned no choices`), and the empty-content check (`content` None or `not content.strip()`). `_chat_once` must also surface the empty reply's `finish_reason` to its caller for the log line — e.g. raise a small internal signal or return it alongside; keep the public error messages exactly as they are today for the no-retry cases.
+   - Rewrite `chat()` around it: `N = 1 + self.settings.llm_retries` total attempts. Attempt 1 → on the **empty-content** failure only: while attempts remain, `logger.warning(…)` with the model name, the empty reply's `finish_reason`, and `attempt {n} of {N}` (PLAN §9 ample logging — this line is the greppable record of the incident class), then `await asyncio.sleep(self.settings.llm_retry_delay)` (flat delay — the phase-67 convention), then the next attempt. Every other `LLMError` (transport, no-choices) propagates immediately — **no** app-level retry (the openai SDK's own `max_retries=2` already re-POSTs wire-level failures; an app-level transport retry would stack on top of it).
+   - Exhaustion: raise `LLMError` with the updated message `f"…returned empty content on all {N} attempts — refusing to store a silent summary"` (same base-URL sanitization as today). When `llm_retries == 0`, raise the **current** message verbatim (`…returned empty content — refusing to store a silent summary`) — the kill-switch must be byte-identical to pre-phase-96 behavior (house byte-identical convention).
+   - Update `chat()`'s docstring: the phase-30 contract ("a silent empty summary must never be stored") stands; add the retry policy (D1/D2/D3 of `00_phase.md`): which failures retry (empty content only), the knobs, the flat delay, the exhaustion message.
+   - `check_models` (the "ping" probe) needs no change — it benefits automatically.
+2. `.env.example` — the `BOR_LLM_RETRIES` / `BOR_LLM_RETRY_DELAY` comments gain that they now cover "the chat-turn stream (phase 67) and one-shot summary calls (phase 96)". No new settings, no `app/config.py` change (the validators for both knobs already exist from phase 67).
+
+- ASSUMPTION: the retry loop lives inside `chat()` itself (one place, every consumer protected) — NOT in each caller; the generators' per-folder fail-soft semantics are untouched (they still catch the post-exhaustion `LLMError` exactly as today).
+- ASSUMPTION: `finish_reason` availability — the empty reply is already parsed (`resp.choices[0]`); reading `.finish_reason` off it is free. If a provider omits it, log `finish_reason=None` (the line must never crash on the diagnostic path).
+
+## Testing & Quality
+- Unit — `tests/unit/test_llm_client.py` (extend the existing `_FakeCompletion`/`_make_chat_client` harness; the fake completions object can be scripted to yield a sequence of replies per `create()` call):
+  - empty-then-success: 2 attempts, exactly 1 sleep of `llm_retry_delay`, returns the second reply's trimmed content; the `WARNING` fired once (caplog).
+  - all-empty with `llm_retries=3` (default): 4 attempts, 3 sleeps, `LLMError` matching `all 4 attempts`; with a custom `llm_retries=1`: 2 attempts, message names 2.
+  - first-attempt success: exactly 1 `create()` call, **zero** sleeps, return value byte-identical to today's behavior (the happy path is untouched).
+  - choiceless reply: 1 attempt, no retry, the existing `no choices` error.
+  - transport failure: 1 attempt, no retry, the existing wrapped error.
+  - `llm_retries=0`: empty reply → 1 attempt, zero sleeps, the **legacy** message byte-identical (assert the exact string).
+  - sleep value: record `asyncio.sleep` calls (monkeypatch) — flat `llm_retry_delay` each time, never a growing backoff.
+  - update the two existing pins (`test_chat_missing_content_raises_llm_error`, `test_chat_whitespace_only_content_raises_llm_error`, ~L989–1000) to the new contract — either pin the exhaustion path with a low-retry setting or set `llm_retries=0` there to keep the legacy-message assertion; both `None` and whitespace-only content must still refuse.
+- Coverage: **>90%** on the modified `app/rag/llm.py` lines (`uv run pytest --cov=app --cov-report=term-missing`).
+- No integration/E2E in this task (tasks 03/04 cover the paths end to end).
+
+## Completion Criteria
+- [ ] `uv run pytest tests/unit/test_llm_client.py -v` green with the new pins above
+- [ ] full suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
+- [ ] the streaming path (`chat_stream`, `chat_stream_retried`) is byte-identical — no diff outside `chat()`/`_chat_once` in `app/rag/llm.py`
+- [ ] `.env.example` comments updated; no new settings
+- [ ] no behavior change in completed work (phase 67's retry suite green)

.agents/phases/complete/96_oneshot_resilience/02_gapfill_generator.md

@@ -0,0 +1,36 @@
+# Task 02 — Gap detector + targeted fill in the folder-summary generator
+
+**Phase:** `96_oneshot_resilience` · **Story:** n/a.
+
+## Objective
+Give `app/rag/folder_summaries.py` the two primitives the sync gate (task 03) and the self-heal need: `missing_folder_summaries(db)` — the candidate folders whose stored row is absent (one concept, the generator's exact candidate computation) — and `generate_folder_summaries(…, only_missing=False)`, which restricts the upsert pass to those missing keys so an unchanged-KB gap-fill never re-burns `lite` for a folder that already has a good summary.
+
+## Work
+1. `app/rag/folder_summaries.py`:
+   - `missing_folder_summaries(db) -> list[tuple[str, str]]`:
+     - Run the generator's catalog query verbatim (`select(Document.source, Document.path, Document.title, Document.summary).order_by(Document.source, Document.path)`), group via `group_by_folder`, keep the candidates (subtree ≥ `MIN_DOCS_PER_FOLDER`) — the SAME computation `generate_folder_summaries` does, so the gap can never disagree with what a regeneration would cover.
+     - Subtract the stored keys: one `select(FolderSummary.source, FolderSummary.folder_path)` (bounded, no count scan — the `folder_summary_table_empty` house pattern).
+     - Return `sorted(missing)` by `(source, folder_path)`. Empty list when nothing is missing (including an empty table over an empty KB — no candidates, no gap).
+   - `generate_folder_summaries(db, llm, *, skip=False, only_missing=False)`:
+     - When `only_missing=True`, after computing `groups`/`candidates` (the existing pass), intersect the iteration keys with `missing_folder_summaries(db)` computed from the SAME grouped rows (do not re-run the catalog query — derive the missing set from the candidates minus the one stored-key select). Upsert loop, per-folder fail-soft, and the prune pass are otherwise **unchanged**; the stats dict keeps its `{"generated", "failed", "pruned"}` shape (the caller logs the mode, not the generator).
+     - When `only_missing=False` (default) the function is byte-identical in behavior to today.
+     - Update the docstring: the new parameter, the "existing rows stay byte-identical (text AND `updated_at`) under `only_missing`" contract, and that the prune pass still runs in both modes (a no-op on an unchanged KB, the invariant kept).
+   - Do NOT delete `folder_summary_table_empty` here — task 03 replaces both call sites and removes it (its unit tests move there).
+
+- ASSUMPTION: "candidate" means exactly what `generate_folder_summaries` already means (the `00_phase.md`/phase-94 recursive-subtree rule) — a single-document folder is still not a candidate, so it can never appear as a "gap".
+- ASSUMPTION: `only_missing` is a **fill**, not a refresh — a stored row is never regenerated or re-stamped by an `only_missing` run, even a stale-looking one (staleness is the changed-KB regeneration's job, as today).
+
+## Testing & Quality
+- Unit — `tests/unit/test_folder_summaries.py` (extend the existing duck-typed fake-LLM + in-memory/SQLite-DB patterns used by that file):
+  - `missing_folder_summaries`: fresh table + populated KB → exactly the candidate set; fully-populated table → `[]`; delete one stored row → `[(that source, that folder)]`; a single-document folder present in the KB and absent from the table → NOT listed (not a candidate); a stored row for a folder that dropped below 2 docs → not listed as missing (it's stale, not missing — the prune pass owns it).
+  - `only_missing=True` with two missing candidates + two present ones: exactly 2 `llm.chat` calls (the missing keys, sorted order); the two present rows are byte-identical afterward (summary text **and** `updated_at` unchanged — capture both before/after); stats `generated=2, failed=0, pruned=0`.
+  - `only_missing=True` with no gap: **zero** `llm.chat` calls, no rows touched (the zero-burn invariant the gate relies on).
+  - `only_missing=True` + a folder that lost ≥ 2 docs since its row was written (seed a stale row manually): the prune pass still deletes it (`pruned=1`) while the genuine missing folders are filled.
+  - `only_missing=False` regression: existing generator tests stay green unchanged (full regeneration + prune).
+- Coverage: **>90%** on the new/modified `app/rag/folder_summaries.py` lines.
+
+## Completion Criteria
+- [ ] `uv run pytest tests/unit/test_folder_summaries.py -v` green with the pins above
+- [ ] `generate_folder_summaries` default path is behavior-identical (all pre-existing tests unmodified and green)
+- [ ] full suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
+- [ ] no behavior change in completed work (phase 94's generator contract intact)

.agents/phases/complete/96_oneshot_resilience/03_sync_gates.md

@@ -0,0 +1,43 @@
+# Task 03 — The gap gate in both sync paths (replace the table-empty probe)
+
+**Phase:** `96_oneshot_resilience` · **Story:** n/a.
+
+## Objective
+Wire task 02's primitives into the folder-summary gate in BOTH sync paths so an unchanged-KB sync self-heals missing rows (targeted fill) while a no-gap unchanged sync still burns zero `lite` calls and a changed-KB sync still regenerates everything exactly as today. This is the half of the fix that turns "a failed folder summary persists until a KB change" into "the next sync heals it".
+
+## Work
+1. `scripts/import_docs.py` — the `_run()` folder-summary block (the change-gated region next to the KB-overview regeneration):
+   - Keep the `--limit` full skip exactly as today (an incomplete debug walk must never regenerate or gap-fill).
+   - Changed KB (`summary.added + summary.updated > 0`) → `generate_folder_summaries(session, llm)` — full regeneration (unchanged).
+   - Unchanged KB → replace `folders_due = _folder_summaries_table_empty()` with `missing = missing_folder_summaries(session)` and `folders_due = bool(missing)`; when due, call `generate_folder_summaries(session, llm, only_missing=True)`. (This subsumes the table-empty first-run case exactly: empty table ⇒ every candidate is missing ⇒ `only_missing` over all candidates == a full generation.)
+   - The run's summary line token stays `folder_summaries=<g>/<f>/<p>`; when the run took the gap-fill path, append ` (gap-fill)` to the token (the PLAN §9 greppable-cron-safe line — the line-extension house rule). The `folder_stats is None` → `skipped` rendering is unchanged.
+   - Update the module docstring's folder-summary paragraph (the phase-94 text) to name the gap-fill trigger alongside the table-empty one.
+2. `app/api/sync.py` — `_run_sync()` (the `fs_db` block, ~L271–283):
+   - `changed = summary.added + summary.updated > 0`. `changed` → full regeneration (unchanged). Unchanged → `missing = missing_folder_summaries(fs_db)`; non-empty → `generate_folder_summaries(fs_db, llm, only_missing=True)` + `logger.info("sync: folder_summaries gap-fill stats=%s", …)`; empty → the existing `sync: folder_summaries skipped (KB unchanged)` log.
+   - Update the block's docstring comment (the phase-94 text) the same way.
+   - No `sync_status` surface change (folder stats stay log-only — the phase-94 contract).
+3. `app/rag/folder_summaries.py` — remove `folder_summary_table_empty` now that BOTH call sites use `missing_folder_summaries` (it becomes dead code). If anything else imports it (grep first), keep it and note why in the commit body; the expectation is exactly the two sync paths.
+4. Imports: `missing_folder_summaries` in both sync modules (replace the `folder_summary_table_empty` imports).
+
+- ASSUMPTION: the gate's "unchanged" decision is the importer's `added + updated` count exactly as today — a prune-only run (files deleted, none added/updated) is unchanged for the gate's purposes, so a prune that drops a folder below 2 docs is healed by the FULL regeneration on the next CHANGED run, not by a gap-fill (prune and gap-fill never race).
+- ASSUMPTION: transaction convention unchanged — the generator flushes, each path commits in its own short-lived session (phase-53), so a gap-fill never half-writes and never blocks the `bump_sources_version` (the bump stays change-gated on the KB, not the summaries).
+
+## Testing & Quality
+- Integration — extend `tests/integration/test_sync_folder_summaries.py` (the phase-94 suite) and mirror the `test_import_docs_overview.py` fake-`LLMClient`-keyed-on-`FOLDER_SUMMARY_MODE` pattern; the fake records its `FOLDER_SUMMARY_MODE` calls so zero-burn and targeted-fill are directly asserted:
+  - **Script path, unchanged + gap:** import a changed KB (rows land), delete one stored row directly, re-run with no KB change → exactly **one** new `FOLDER_SUMMARY_MODE` call (the deleted row's folder only), that row is back with the deterministic fake text, every other row byte-identical (summary + `updated_at`), summary line carries `folder_summaries=…(gap-fill)`.
+  - **Script path, unchanged + no gap:** re-run with no change and a complete table → **zero** `FOLDER_SUMMARY_MODE` calls, token `folder_summaries=skipped` (the phase-94 zero-burn invariant, unchanged).
+  - **Script path, changed:** a KB change still triggers FULL regeneration (call count == candidate count, all rows re-stamped) — today's behavior.
+  - **Script path, `--limit`:** still skips generation entirely (zero calls, `skipped`).
+  - **API path** (`tests/integration/test_sync_api.py` extension or the folder suite's API variant): the same three branches via `POST /api/sync` — unchanged+gap → targeted fill + the `gap-fill` log; unchanged+no-gap → zero calls; changed → full.
+  - **First-run subsumption:** a fresh table (no rows) + populated KB on an unchanged walk → the gap-fill path generates every candidate (equivalent to the old table-empty trigger) — assert the full candidate set lands.
+  - The existing `test_sync_api.py` + `test_import_docs_git.py` + the phase-94 suite stay green (status shape + full-regeneration behavior unchanged).
+- Coverage: **>90%** on the modified `app/` lines (`app/api/sync.py`; `scripts/` is outside the `app/` coverage gate but the integration tests exercise it).
+
+## Completion Criteria
+- [ ] unchanged-KB sync with one deleted row regenerates exactly that row on BOTH paths, leaves the rest byte-identical, and logs the gap-fill (integration-pinned)
+- [ ] unchanged-KB sync with a complete table burns zero folder-summary `lite` calls (both paths)
+- [ ] changed-KB sync regenerates all candidates (both paths) — byte-identical to today's behavior
+- [ ] `--limit` (script) skips generation entirely; `sync_status` shape unchanged (API)
+- [ ] `folder_summary_table_empty` removed (or retained with a documented reason); no dead imports
+- [ ] `uv run pytest` green, coverage >90%, `uv run ruff check . && uv run pyright` clean
+- [ ] no behavior change in completed work (phase 94 + phase 53 bump contract intact)

.agents/phases/complete/96_oneshot_resilience/04_e2e_oneshot_retry.md

@@ -0,0 +1,41 @@
+# Task 04 — E2E: the incident shape, retried and healed (the dedicated story suite)
+
+**Phase:** `96_oneshot_resilience` · **Source:** 2026-09-11 incident (owner chat — the whole item: the empty-reply failure class + the missing-row gap). **Story:** n/a (A16 — one Playwright file per phase, run in isolation).
+
+## Objective
+`tests/e2e/test_oneshot_llm_retry.py` proves the whole fix end to end against the deterministic mock: a folder whose FIRST one-shot reply arrives in the exact incident shape (`content=""`, `finish_reason="length"`) still ends up with its stored summary (the task-01 retry recovered it, visible in the `ls` drill-down), a folder whose replies are ALWAYS empty stays absent without failing the sync (task-01 exhaustion + the fail-soft contract), and a row deleted behind the app's back is self-healed by the next unchanged sync with the other rows untouched (tasks 02/03 targeted fill).
+
+## Work
+1. `tests/e2e/mock_llm.py` — the `compose_answer` `FOLDER_SUMMARY_MODE` branch (L1504) gains the incident-shape injection, keyed by the folder LABEL parsed from the `Folder: …` header (the existing parse — the seeded KB's folder names carry the trigger, so the injection is a pure function of the request, the house marker-flow convention):
+   - A module-level per-label counter (the phase-67 `_fail_posts`/`_bump_fail` pattern; the mock is single-conversation per E2E server — reset each label's counter after the success it guards, so a second sync re-drives the sequence deterministically).
+   - Label ends with `/e2e_empty_once` → the FIRST non-stream POST for that label returns the incident envelope: the mock's normal OpenAI chat-completion shape with `choices[0].message.content = ""` and `choices[0].finish_reason = "length"`; every later POST returns the normal `Fixture folder summary for {folder}.` line. (Inject at the NON-STREAM response path — `chat()` posts `stream=false`; the phase-67 `_llm_500` injection shows where the non-stream branch answers.)
+   - Label ends with `/e2e_empty_always` → EVERY non-stream POST for that label returns the empty envelope.
+   - No other mock branch changes; the trigger strings are this suite's own folder names, so no other E2E can hit them (they seed different trees).
+2. `tests/e2e/test_oneshot_llm_retry.py` (new — isolated: `tests/e2e/conftest.py` fixtures `app_server` + `mock_llm`; admin login via `tests/e2e/auth_helpers.py`; the test server boots with `BOR_LLM_RETRY_DELAY=0` per the phase-67 pattern in `test_llm_retry.py`, default `BOR_LLM_RETRIES=3`):
+   - **Seed:** one temp local-dir source (the `test_local_directory_sources.py` registration + Sync pattern) with three ≥ 2-doc folders — `e2e_empty_once/` (2 `.md` files), `e2e_empty_always/` (2), `normal/` (2) — small, distinct contents.
+   - **Sync #1** via `POST /api/sync` (the `test_sync_button.py` pattern; wait for completion) → then a scripted `ls <source>` drill-down turn (the phase-94 `test_ls_tree_drilldown.py` echo pattern — the mock's grounded answer carries the `ls` result, the E2E's only lens on the LLM context) asserting:
+     - the `e2e_empty_once/` line is `e2e_empty_once/ — 2 documents: Fixture folder summary for <src>/e2e_empty_once.` — the retry recovered the row (without task 01 this line has no `: …` suffix);
+     - the `normal/` line carries its summary;
+     - the `e2e_empty_always/` line is `e2e_empty_always/ — 2 documents` with **no** `: …` suffix (all 4 attempts empty → `LLMError` → per-folder fail-soft → no row);
+     - the sync reported success (the status endpoint / UI success state — a folder-summary exhaustion never flips the run, phase 94).
+   - **Gap-fill:** delete the `normal` stored row directly from the DB (the test process has DB access via the conftest engine — the same connection the app uses), with no KB change, capture `e2e_empty_once`'s row `updated_at` → **Sync #2** → assert:
+     - a second scripted `ls <source>` turn: `normal/` carries `: Fixture folder summary for <src>/normal.` again (healed); `e2e_empty_once/` still carries its summary;
+     - the DB: the `normal` row's summary is the deterministic text; the `e2e_empty_once` row's `updated_at` is **unchanged** (targeted fill — a full regeneration would have re-stamped it);
+     - `e2e_empty_always` is STILL absent (the gap-fill attempted it, exhausted, stayed absent) and the sync still succeeded.
+3. Run in isolation: `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` (db up: `podman compose up -d db`).
+4. Regression gate, in isolation: `test_ls_tree_drilldown.py` (phase 94 — the drill-down + folder-summary surface), `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py` (phase 67 — the streaming retry path must be untouched).
+
+- ASSUMPTION: assertions on what the LLM's context carried go through the deterministic mock echoing the `ls` result into its answer (the house pattern from the agent-tool suites); DB-level assertions (row presence, `updated_at`) use the test process's direct DB access — the E2E's two established lenses.
+- ASSUMPTION: the mock's per-label counters reset per sync (after the success they guard), so sync #2 re-drives `e2e_empty_once` from "first POST empty" — which is FINE here: the row already exists, so the gap-fill never calls it at all (the `updated_at` assertion proves that); `e2e_empty_always` exhausts again by design.
+- ASSUMPTION: `BOR_LLM_RETRY_DELAY=0` on the test server keeps the 4-attempt exhaustion paths instant (phase 67's fast-suite convention); default `BOR_LLM_RETRIES=3` means the `e2e_empty_always` folder costs 4 POSTs per sync that attempts it — the suite stays fast.
+
+## Testing & Quality
+- This IS the E2E task; the mock's new branch is covered by the suite itself (the mock is a real OpenAI-compatible server — no separate unit needed, per the existing agent/tool suites' practice).
+- Coverage: **>90%** on `app/` (this task adds test code only — keep the gate green).
+
+## Completion Criteria
+- [ ] `uv run pytest tests/e2e/test_oneshot_llm_retry.py -v --no-cov` green in isolation and encoding: incident-shaped empty reply → retried → summary visible in the `ls` drill-down; always-empty folder → absent with the sync green; deleted row → self-healed on the next unchanged sync with `updated_at` untouched on the other rows
+- [ ] the mock injection is label-keyed and resets deterministically (a re-run of the suite is green without manual state cleanup)
+- [ ] regression E2E green in isolation: `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_local_directory_sources.py`, `test_llm_retry.py`
+- [ ] full test suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
+- [ ] one atomic Conventional Commit, `--no-gpg-sign` (message in `00_phase.md`), phase dir moved to `.agents/phases/complete/`

.agents/phases/todo/97_kb_tree_catalog/00_phase.md

@@ -0,0 +1,104 @@
+# Phase 97 — The RAG view shows the KB as the drill-down tree the agent sees, with editable folder descriptions
+
+**Source:** Owner request (chat, 2026-09-11) — "The RAG page shows a list of all files uploaded by the user, but the agent sees a tree structure after the last phase [94]. I want the UI to show what the agent sees — a tree that the user can click through so the user can see descriptions of the directories. The user should be able to edit the descriptions of the directories just like they can edit the summaries of any file."
+**Story:** n/a (owner request — the RAG view on the `02_story_import_documents` catalog; the folder descriptions + tree concept on `94_ls_tree_drilldown`; the edit affordance pattern on `57_edit_document_summaries`).
+**Context:** The RAG (Knowledge base) view — `frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (shell view module, phase 76) — renders the flat all-documents table from `GET /api/docs` (`app/api/docs.py`: source | path | title | chunks | indexed, `#docs-tbody`), with KB-wide stat cards, the sync button/poll machinery, and the phase-16 anonymous gate. Phase 94 made the agent's `ls` a drill-down tree over stored sync-time folder descriptions: `folder_summaries` (migration 0017, PK `(source, folder_path)`, `folder_path ""` = source root, rows only for ≥ 2-doc folders), the shared folder concept (`app/rag/folder_summaries.folder_of`; existence rule: a folder exists ⟺ some indexed path starts with `folder + "/"`; counts are the recursive subtree), and the pure level functions `app/rag/agent.py` `ls_top` / `group_folder_listing` / `ls_folder`. Phase 57's edit affordance (admin `PATCH /api/documents/summary` + the `.doc-summary` Edit → inline textarea → Save/Cancel → live-region status in `frontend/assets/document.js`) is the interaction model to mirror. Migration head is `0017_folder_summaries`. The deterministic E2E mock already returns the canned `FOLDER_SUMMARY_MODE` one-liner naming the folder (phase 94).
+
+## Objective
+The admin's RAG (Knowledge base) view shows the catalog as the same drill-down tree the agent's `ls` walks — sources at the top, then per level the subfolders with their stored descriptions and the level's files — so the owner can click through the directories, read the descriptions the agent reads, and edit (or clear) any directory's description with the phase-57 inline edit affordance. A manually-edited description persists: the sync-time generator never overwrites and never prunes it.
+
+## Dependencies
+- `94_ls_tree_drilldown` (complete) — the `folder_summaries` table, the stored per-folder descriptions, and the shared folder concept (`folder_of`, the existence rule, the recursive counts, `group_folder_listing` for the cross-check).
+- `57_edit_document_summaries` (complete) — the edit affordance pattern (Edit → inline textarea → Save/Cancel → live-region status) and the admin-gated summary-PATCH precedent.
+- `96_oneshot_resilience` (todo) — queue order only (numeric); no code dependency (different subsystem).
+
+## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
+- **The tree replaces the flat all-documents table in the RAG view** (derived from the request: "I want the UI to show what the agent sees"). KB-wide stat cards stay; the per-level file table keeps the existing 5-column contract (`#docs-table` / `#docs-tbody`) so file rows are unchanged in shape; the flat `GET /api/docs` endpoint is untouched (API surface unchanged). Existing story suites whose catalog assertions assume the flat layout are updated IN THIS PHASE (tasks 07/08) — the asserted behavior (a document is listed, linkable, viewable) is preserved; only navigation gains a drill step.
+- **Manual edits persist across syncs** (derived from "just like they can edit the summaries of any file" — an edited file summary is never silently rewritten by a re-import): `folder_summaries` gains `manually_edited`; the phase-94 generator SKIPS (never overwrites) and never prunes a manual row. Clearing a description deletes the row — the next KB-changing sync regenerates an AI description for that folder (that IS the reset path; no separate regenerate button in v1).
+- **The UI tree is a superset of the `ls` top level:** sources list as registered sources in registry order (the `ls()` order, including a registered 0-document source — the phase-70/72 invariant) FOLLOWED by indexed sources that are not registered (path order) — ad-hoc `scripts/import_docs --source ~/X` imports and removed-but-not-yet-pruned sources. The catalog has never hidden an indexed document (the phase-02 contract); the agent's `ls` keeps showing registry sources only (phase 94) — unchanged.
+- **No re-embedding on edit** (contrast with phase 57): a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line) — the PATCH is a pure DB write, no LLM call.
+
+## Design (shared by all tasks — the executor reads this, not the chat)
+
+### The tree endpoint (task 02)
+`GET /api/docs/tree` (admin, like `/api/docs`) returns the FULL recursive tree in one fetch — the UI drills client-side, zero per-level fetches:
+
+```json
+{
+  "sources": [
+    {
+      "name": "alpha", "documents": 5, "summary": "… or null",
+      "children": [
+        { "kind": "folder", "path": "one", "documents": 2, "summary": "… or null",
+          "children": [
+            { "kind": "file", "path": "one/a.md", "title": "A", "chunks": 3,
+              "indexed_at": "2026-09-11T08:00:00+00:00" }
+          ]
+        },
+        { "kind": "file", "path": "root-note.md", "title": "Root note", "chunks": 1,
+          "indexed_at": "…" }
+      ]
+    }
+  ]
+}
+```
+
+- **Sources:** registered sources first in `list_source_names` (registry) order, then indexed-only sources (alphabetical) — the superset rule above. `documents` = the source's whole recursive count; `summary` = the stored `(source, "")` row or null. A registered 0-document source still lists (`0`, no children).
+- **Folder nodes:** `path` source-relative (never `""` — the source node IS the root); direct subfolders only, in path order (the phase-94 existence rule); `documents` = the recursive subtree count; `summary` = the stored row (AI OR manual) or null; `children` = its own subfolders + direct files, same shape.
+- **File nodes:** direct files only (`folder_of(path) == parent`), in path order (catalog order — the same order `GET /api/docs` serves); `path` source-relative; `title` / `chunks` (content + `is_summary` chunks — the same count `/api/docs` returns) / `indexed_at` verbatim.
+- **Pure builder** `build_kb_tree(names, doc_rows, summaries)` — module-level in `app/api/docs.py` (unit-testable without a DB): `names` = the registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order; `summaries` = `{(source, folder_path): summary}` over ALL stored rows. It reuses `app.rag.folder_summaries.folder_of` and the phase-94 existence rule — ONE concept end to end: the UI tree is the `ls` tree plus file metadata. **Cross-check property (unit-pinned):** for a single-source dataset the builder's level equals `app.rag.agent.group_folder_listing`'s output (same subfolder paths/counts/summaries, same file paths/titles in order) — "the UI shows what the agent sees" as a test, at the root and one nested level.
+- `GET /api/docs` stays untouched.
+
+### Edit persistence (task 01)
+- Migration `0018_folder_summary_manual_flag` (down `0017_folder_summaries` — confirm with `alembic heads`): `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; tested downgrade drops the column (A13, the `test_migration_0017.py` pattern).
+- `generate_folder_summaries` (`app/rag/folder_summaries.py`): fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into one `{(source, folder_path): row}` dict). Upsert loop: an existing row with `manually_edited` is SKIPPED (no `lite` call — no burn on owner text) and counted `stats["kept_manual"] += 1`. Prune loop: deletes only NON-manual rows (a manual row for a folder that dropped below 2 documents is kept — owner content persists until cleared). The generator's logger line gains `kept_manual=%d`. **The import summary-line token STAYS 3 fields** (`scripts/import_docs.py` `folder_summaries=<generated>/<failed>/<pruned>` — `tests/integration/test_import_docs_overview.py` pins it).
+
+### The edit endpoint (task 03)
+`PATCH /api/folders/summary` (admin) — body `{"source": str, "folder_path": str, "summary": str}`; `folder_path = ""` = the source root. Lives in `app/api/docs.py` next to the phase-57 document-summary PATCH (same `kb` router family).
+- **Source check:** registered (registry) OR has indexed documents → else 404 `{"detail": "source not found"}`.
+- **Folder check:** `""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some path starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem).
+- **Non-empty after strip** → upsert the row with `manually_edited = True` + fresh UTC `updated_at`. **Empty/whitespace** → delete the row if present (clear — the phase-57 analog; the reset path). Commit; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
+- No LLM/embedding call (the decision above); the docstring says so.
+
+### The RAG view (tasks 04/05)
+`frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (the house createElement/textContent contract — never innerHTML with document-derived data):
+- **New static skeleton** (in `index.html`, after the stat cards, before the existing file `.table-wrap`):
+  - `nav#kb-crumb[aria-label="Catalog location"][hidden]` — JS fills the segments.
+  - `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title` (the level's full source-relative path, e.g. `alpha/two`), `p#kb-level-summary`. (The level Edit button lands in task 05.)
+  - `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table` (visually-hidden caption, the `.docs-table` styling) — `thead`: **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled rows). ONE table for every level: at the top level the rows are the SOURCES themselves (the `ls()` equivalence — sources list exactly like folders: name, count, description).
+- **File table:** the existing `.table-wrap` / `#docs-table` / `#docs-tbody` is UNCHANGED in columns (Source | Path | Title | Chunks | Indexed) and in row rendering (`makeRow` — the path link still opens `openDocumentModal`, the no-JS href escape hatch intact). It now holds the CURRENT LEVEL's direct files; hidden when the level has none (at the top level it is always hidden — files are seen per source, as with `ls(source)`).
+- **Navigation state:** module-level `current = {source: string | null, folder: string | null}` (`{null, null}` = top level; `folder ""` = the source root). Source row click → `{source, ""}`; folder link click → `{source, folder}`; breadcrumb segment click → that ancestor. Breadcrumb: hidden at the top level; one link per ancestor (source, then folders), the last segment a `span` with `aria-current="page"`.
+- **Level rendering:** the level block shows the CURRENT level's stored description (source: the `(source, "")` row; folder: its row) and is hidden when none is stored (the `ls` rule — count only, no placeholder); the folders table = the direct subfolders (count + stored description + row Edit button in task 05); the file table = the direct files. `#folders-wrap` hidden when no subfolders.
+- **Stat cards:** unchanged KB-wide values — computed by walking the whole in-memory tree (documents, chunks sum, max `indexed_at`) — identical to today's flat walk.
+- **Empty state:** `#sources-empty` shows ONLY when the tree has zero sources (nothing registered, nothing indexed); a registered 0-document source renders its row (`0 documents`) instead.
+- **Never-stale reload (PLAN §7.4):** every re-fetch (boot, `bor:view-refresh`, sync-success settle, upload-success settle) re-renders the current level; if the current position no longer exists in the new tree (source unregistered/pruned, folder gone), the view RESETS to the top level — no stale breadcrumb, no stale block. The `loadSeq` race token (phase 79) carries over to the tree load.
+- **Unchanged:** the sync button/poll/label/banner/error-modal machinery, the anonymous gate (no `/api/docs*` fetch for anonymous — the gate branch never fetches the tree), the shell router wiring, the stat-card markup.
+- **Edit (task 05):** Edit buttons on `#kb-level` AND on every source/folder row's Description cell (a description can be CREATED where none is stored — a < 2-document folder or the generator's fail-soft miss; the button is always present — the view is admin-only already, the endpoint's `require_admin` is the API-level gate). The phase-57 interaction: swap the description text node for `textarea.kb-summary-editor` (prefilled via `value` — the XSS contract), Save / Cancel, `p.kb-summary-status[role="status"][aria-live="polite"]`. Save → `PATCH /api/folders/summary` `{"source", "folder_path", "summary"}` (folder_path `""` for the source root). 200 → re-render the text (textContent only) + status "Description updated." (empty save → the text goes away — level block hidden / cell emptied — + "Description cleared."); the in-memory tree node's summary updates in place (no re-fetch). Failure → neutral retry copy (the phase-55 convention), the editor stays open with the user's text. Cancel → restore the text node.
+- **`styles.css`:** `.kb-crumb` (+ link / current segment), `.kb-level` (+ title / summary typography), `.folder-link`, `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem — the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house dark-tech palette, AA contrast (text + color, never color alone), no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, ≥ 24px targets, no CDN.
+
+## Tasks
+1. `01_manual_flag.md` — migration 0018 + `FolderSummary.manually_edited` + the generator's skip/keep logic + stats/log
+2. `02_tree_api.md` — `GET /api/docs/tree` (pure builder + schemas + endpoint)
+3. `03_folder_summary_api.md` — `PATCH /api/folders/summary` (update / create / clear)
+4. `04_tree_ui.md` — the RAG view tree: level rendering, drill navigation, breadcrumb, level block, folder table, reload fallback
+5. `05_folder_edit_ui.md` — the edit affordance (level block + rows) + PATCH wiring + editor + status + styles + source pins
+6. `06_e2e_kb_tree.md` — the dedicated Playwright suite `tests/e2e/test_kb_tree.py`
+7. `07_catalog_suites_import_sync.md` — the existing import/sync/upload story suites onto the drill-down catalog
+8. `08_catalog_suites_viewer_nav.md` — the existing viewer/nav/UX story suites + the regression run
+
+## Testing & Quality
+- Unit: `build_kb_tree` (multi-source order, indexed-only sources, a 0-document registered source, nested counts, the existence rule, a file/folder name collision, ordering, summaries present/absent, file metadata verbatim) + the `group_folder_listing` cross-check; the generator's keep/keep-out (a manual row survives regeneration AND prune, stats, the logger line); migration 0018 up/down (A13).
+- Integration: the tree endpoint (shape, ordering, 403 anonymous, empty registry, indexed-only + orphan visibility per the superset rule, the stat-walk equivalence); the PATCH endpoint (update, create where no row exists, source root, clear + double-clear, 404 unknown source/folder/traversal, 403 non-admin) — the `tests/integration/test_docs_api.py` pattern; the `import_docs` line keeps its 3-field `folder_summaries=` token with a manual row present.
+- Frontend source pins (house style, the `tests/unit/test_save_chat_ui.py` pattern): `sources.js` (the `/api/docs/tree` fetch, the drill-state transitions, the PATCH path + body, the textContent contract, the status copies, the reset-to-top fallback, the three refresh wirings), `styles.css` (the new class names).
+- E2E (mandatory, A16): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` in isolation; the updated story suites green in isolation (the task 07/08 lists), including `test_ls_tree_drilldown.py`, `test_edit_summaries.py`, `test_import_documents.py`.
+- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
+- Lint/types: `uv run ruff check . && uv run pyright`.
+
+## Completion Criteria
+- [ ] The RAG view (admin) lists the sources with counts + descriptions; clicking through shows, per level, the subfolders (count + description) and the level's files (existing 5-column table, the document-modal link unchanged)
+- [ ] The level block shows the current directory's description; Edit → inline textarea → Save round-trips to `PATCH /api/folders/summary` (the new text renders; an empty save clears it — the row is deleted)
+- [ ] A manual description survives a KB-changing sync (the generator's `kept_manual`), and clearing it resets the folder to AI generation on the next sync
+- [ ] The agent's `ls` output is byte-identical for the same data (the phase-94 suites green) — the tree reads the same rows; the only schema change is the additive flag column
+- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
+- [ ] `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the updated story suites green in isolation
+- [ ] One atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(kb): show the catalog as the drill-down tree the agent sees, with editable folder descriptions`)

.agents/phases/todo/97_kb_tree_catalog/01_manual_flag.md

@@ -0,0 +1,28 @@
+# Task 01 — Manual-edit flag: migration 0018 + the generator keeps the owner's descriptions
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+A stored folder description the owner edited can never be overwritten (or pruned) by the sync-time generator — `folder_summaries.manually_edited` (migration 0018) plus the phase-94 generator's skip/keep logic.
+
+## Work
+1. `alembic/versions/0018_folder_summary_manual_flag.py` (down revision `0017_folder_summaries` — confirm with `uv run alembic heads`) — add `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; the **tested downgrade** drops the column (A13 — the `tests/integration/test_migration_0017.py` pattern: a new `tests/integration/test_migration_0018.py`).
+2. `app/models.py` — `FolderSummary.manually_edited: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))` + the docstring: set ONLY by `PATCH /api/folders/summary` (phase 97, task 03); the generator (below) skips a manual row on regeneration and never prunes it — an owner correction is never silently rewritten (the phase-97 `00_phase.md` decision).
+3. `app/rag/folder_summaries.py` — `generate_folder_summaries`:
+   - Fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into a single `{(source, folder_path): FolderSummary}` dict used by both loops).
+   - Upsert loop: an existing row with `manually_edited` → SKIP (no `llm.chat` call — no `lite` burn on owner text) and `stats["kept_manual"] += 1`.
+   - Prune loop: delete only rows whose `manually_edited` is False (a manual row for a folder that dropped below 2 documents is KEPT — owner content persists until cleared).
+   - The stats dict gains `"kept_manual": 0`; the logger line becomes `folder_summaries: generated=%d failed=%d pruned=%d kept_manual=%d`.
+   - **The import summary-line token STAYS 3 fields** — `scripts/import_docs.py`'s `folder_summaries=<generated>/<failed>/<pruned>` is unchanged (`tests/integration/test_import_docs_overview.py` pins it; `app/api/sync.py` just logs the dict).
+4. Docstrings: the module docstring's storage paragraph + the `generate_folder_summaries` docstring note the flag's two rules (skip on regenerate, keep on prune) — the house "module docstrings carry the contracts" rule.
+
+## Testing & Quality
+- Unit (`tests/unit/test_folder_summaries.py` extensions): a manual row survives a regeneration pass (skipped — the fake LLM is never called for it — `kept_manual` right, the row's text/stamp untouched); a manual row survives the prune (its folder drops below 2 documents); a non-manual row is still pruned and still regenerated; `skip=True` no-op unchanged; the logger line format (caplog, 4 fields).
+- Integration: `test_migration_0018.py` (up: column present, default false, existing rows backfilled false; down: column dropped); an `test_import_docs_overview.py`-pattern case with a manually-edited row inserted directly — the gate fires, the token stays `folder_summaries=<g>/<f>/<p>`, and the manual row's text is unchanged afterward.
+- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
+
+## Completion Criteria
+- [ ] migration up/down green; the flag round-trips through the model
+- [ ] the generator never overwrites or prunes a manual row; stats + log carry `kept_manual`; the import token is unchanged
+- [ ] full test suite green, coverage >90%
+- [ ] no behavior change in completed work (the phase-94 generator + sync suites green)

.agents/phases/todo/97_kb_tree_catalog/02_tree_api.md

@@ -0,0 +1,26 @@
+# Task 02 — `GET /api/docs/tree`: the full recursive tree the UI renders
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+One admin endpoint that returns the catalog as the recursive tree the agent's `ls` walks (with the file metadata the rows and stat cards need) — the RAG view's single fetch (the shape contract is in `00_phase.md`, section "The tree endpoint").
+
+## Work
+1. `app/schemas.py` — next to `DocList`: `KbTreeFile` (`path`, `title`, `chunks: int (ge=0)`, `indexed_at: str`), `KbTreeFolder` (`path`, `documents: int (ge=0)`, `summary: str | None`, `children` — recursive union of folder/file nodes), `KbTreeSource` (`name`, `documents: int (ge=0)`, `summary: str | None`, `children`), `KbTree` (`sources: list[KbTreeSource]`). The `00_phase.md` JSON shape is the contract (Pydantic v2 handles the recursive union; `kind` discriminators optional — the JSON keys `kind: "folder" | "file"` must appear as spec'd).
+2. `app/api/docs.py`:
+   - `build_kb_tree(names, doc_rows, summaries) -> list[KbTreeSource]` — PURE, module-level (unit-testable without a DB). Inputs: `names` = registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order (source, path); `summaries` = `{(source, folder_path): str}` over ALL stored rows. Behavior per `00_phase.md`: sources = `names` first (each always present, even 0 documents), then the distinct indexed sources not in `names` (alphabetical — the superset rule); a folder node exists iff some indexed path starts with `folder + "/"` (the phase-94 existence rule — reuse `app.rag.folder_summaries.folder_of`, never re-derive "folder"); `documents` = recursive subtree count; subfolders in path order, direct files (`folder_of(path) == parent`) in input (catalog) order; file nodes carry `title` / `chunks` / `indexed_at` verbatim; `summary` = the stored row (any row — AI or manual) or null.
+   - `GET /docs/tree` endpoint (`Depends(require_admin)`, `response_model=KbTree`) — compose: `names = list_source_names(db)` (import from `app.rag.agent` — `app/api/chat.py` already imports from that module, no cycle), `doc_rows` = the same outerjoin/grouped query `GET /docs` uses (id excluded is fine — the tree has no document ids), `summaries` = all `folder_summaries` rows for the listed sources; call the builder.
+   - Module docstring: add the new endpoint's contract line (the house docstring convention — one line per endpoint).
+3. `GET /api/docs` is UNCHANGED (same query, same response — its suites stay green untouched).
+
+## Testing & Quality
+- Unit (`tests/unit/` — new `test_kb_tree_builder.py` or extend the existing docs unit file): multi-source — registry order preserved + an indexed-only source appended alphabetically; a registered 0-document source (`documents: 0`, no children); a nested document counts into the source, every ancestor folder, and its own folder; the existence rule (a folder node only with a true descendant — a file path is never a folder); a file/folder NAME COLLISION (a document sharing a directory's name — both appear: the folder node via its descendants + the file node); subfolder path order + file catalog order; `summary` present/absent; file `chunks` / `indexed_at` / `title` verbatim; an indexed document under an UNLISTED source is impossible (every doc source is listed by construction — the superset rule) while the registry order still leads.
+- **The cross-check property (the "UI shows what the agent sees" pin):** for a single-source dataset, the builder's root level equals `app.rag.agent.group_folder_listing(source, "", (path, title) rows, {folder: summary})` — same subfolder `(path, count, summary)` triples in order AND the same file `(path, title)` pairs in order (uncapped — pass the full file list; the builder has no 50-line cap, the UI is for humans); repeat for one nested level.
+- Integration (`tests/integration/test_docs_api.py` extensions, the house pattern — real app + real DB): seed documents across two sources (one with nested folders, one flat) + `folder_summaries` rows → the shape/ordering/counts/summaries; 403 anonymous; empty registry + no documents → `{"sources": []}`; an indexed-only source (documents with a source absent from `git_sources`) lists after the registered ones; the stat-walk equivalence — summing the file nodes' `chunks` + their count equals what `GET /api/docs` returns for the same data (the stat-card values are unchanged by the redesign).
+- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
+
+## Completion Criteria
+- [ ] `GET /api/docs/tree` returns the `00_phase.md` shape (admin; 403 otherwise); `GET /api/docs` byte-identical in behavior
+- [ ] the pure builder is unit-pinned, including the `group_folder_listing` cross-check at two levels
+- [ ] full test suite green, coverage >90%, ruff + pyright clean
+- [ ] no behavior change in completed work (the phase-94 ls suites green)

.agents/phases/todo/97_kb_tree_catalog/03_folder_summary_api.md

@@ -0,0 +1,28 @@
+# Task 03 — `PATCH /api/folders/summary`: the admin folder-description editor endpoint
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The endpoint behind the view's edit affordance: update / create / clear a stored folder description, marking every non-empty save `manually_edited` (task 01's keep/keep-out rules apply from this point on).
+
+## Work
+1. `app/schemas.py` — `FolderSummaryUpdate` (`source: str`, `folder_path: str` — `""` = the source root, `summary: str`) + `FolderSummaryResult` (`source`, `folder_path`, `summary: str | None`), next to the phase-57 `SummaryUpdate` / `SummaryResult`.
+2. `app/api/docs.py` — `PATCH /folders/summary` on the same `kb` router, next to the phase-57 `PATCH /documents/summary` (the house docstring line for it):
+   - `Depends(require_admin)` — the catalog is admin-only (phase 16); the endpoint gate is the API-level defense in depth (the RAG view never renders for anonymous).
+   - **Source check:** the source is registered (`list_source_names(db)`) OR has indexed documents (`SELECT 1 FROM documents WHERE source = … LIMIT 1`) → else 404 `{"detail": "source not found"}`.
+   - **Folder check:** `folder_path == ""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some `Document.path` starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem access).
+   - **Non-empty after `strip()`** → upsert the `folder_summaries` row with `summary = stripped text`, `manually_edited = True`, fresh UTC `updated_at` (insert or update — a manual description can be created where no row exists: a < 2-document folder, or the generator's fail-soft miss).
+   - **Empty/whitespace** → `db.delete(row)` when a row exists (clear — the phase-57 analog; the row may be AI-written or manual, either way it is gone; the next KB-changing sync regenerates an AI row — the reset path). A clear with no row is a 200 no-op.
+   - `db.commit()`; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
+   - **No LLM/embedding call** — a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line); the docstring records the deliberate contrast with phase 57's `is_summary` re-embed.
+3. Router registration needs no change (same router object as `/docs` / `/documents/summary`).
+
+## Testing & Quality
+- Integration (`tests/integration/test_docs_api.py` extensions): update an existing AI-written row (text replaced, `manually_edited` true, `updated_at` advanced — a second save with a different text updates in place); create where no row exists (< 2-document folder); the source root (`folder_path: ""`) round-trips; clear (the row is deleted, the response `summary` null, a second clear is a 200 no-op); 404 unknown source / unknown folder / a traversal folder path / a folder under an unknown source; 403 anonymous AND a live access-token user who is not the admin (`require_admin` — the phase-79 token users exist); the round-trip through the task-02 endpoint — `GET /api/docs/tree` shows the new text after the save and null after the clear.
+- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
+
+## Completion Criteria
+- [ ] the endpoint contract is pinned (update / create / source-root / clear / double-clear / 404s / 403s)
+- [ ] every non-empty save sets `manually_edited` — the task-01 generator keep/keep-out applies from this point on
+- [ ] no LLM call on the path — source pin: the PATCH handler never constructs an `LLMClient` (grep pin, the house source-pin pattern)
+- [ ] full test suite green, coverage >90%, ruff + pyright clean

.agents/phases/todo/97_kb_tree_catalog/04_tree_ui.md

@@ -0,0 +1,37 @@
+# Task 04 — The RAG view renders the tree: drill navigation, breadcrumb, level block, folder table
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The Knowledge base view (admin) shows the catalog as the drill-down tree — sources at the top, then per level the subfolders (count + description) and the level's files — replacing the flat all-documents table; descriptions are read-only here (task 05 adds the edit affordance).
+
+## Work
+1. `frontend/index.html` `#view-rag` — static skeleton additions (the house no-JS-safe skeleton convention), after `#stat-cards`, before the existing file `.table-wrap`:
+   - `nav#kb-crumb[aria-label="Catalog location"][hidden]` — empty; the JS fills the segments (createElement).
+   - `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title`, `p#kb-level-summary` (the level Edit button lands in task 05 — do NOT add it here).
+   - `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table.kb-folders-table` (a `.visually-hidden` caption, the `.docs-table` styling) — `thead` **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled). ONE table for every level — at the top level the rows are the SOURCES themselves (the `ls()` equivalence: sources list exactly like folders — name, count, description).
+   - The existing `.table-wrap` / `#docs-table` / `#docs-tbody` block, `#sources-empty`, `#stat-cards`, the sync button/result/banner/gate: UNCHANGED.
+2. `frontend/assets/sources.js` — the catalog load becomes tree-based (the module header docstring gains the phase-97 section, the house convention):
+   - `loadDocs()` → `loadTree()`: `fetch("/api/docs/tree")` in the admin branch only (the anonymous gate branch NEVER fetches — the phase-16 soft rule, unchanged); keep the `loadSeq` race token (phase 79) — only the newest load may touch the DOM; store the parsed tree in a module-level `kbTree`; render the current level.
+   - `current = {source: string | null, folder: string | null}` — module state, initial `{null, null}` = top level (`folder ""` = the source root).
+   - `renderLevel()` from `current` + `kbTree`:
+     - **Top level:** breadcrumb hidden; level block hidden; folders table = the source rows (name in the Folder cell, recursive count, the `(source, "")` description text when stored); the file table HIDDEN (files are seen per source — `ls()` shows no files at the top).
+     - **Inside a source/folder:** breadcrumb segments (the source name, then the folder chain — each an `<a class="kb-crumb-link">`, the last segment a `<span aria-current="page">`); level block = the current level's stored description (title = the full source-relative path, e.g. `alpha/two`) — HIDDEN when none is stored (the `ls` rule: count only, no placeholder); folders table = the direct subfolders (`.folder-link` name, count, stored description text); file table = the direct files — `makeRow` UNCHANGED (the path link still `preventDefault`s + `openDocumentModal`; the no-JS href escape hatch intact); `#folders-wrap` hidden when the level has no subfolders, the file wrap hidden when it has no direct files.
+     - **Stat cards:** walk the WHOLE tree (document count, chunks sum, max `indexed_at`) — the values identical to today's flat walk (`fmtDate` reuse).
+     - **Empty state:** `#sources-empty` visible ONLY when `kbTree.sources` is empty (both wraps hidden); a registered 0-document source renders its row (`0`) instead — the semantic change is deliberate (the `ls` invariant), noted in the module docstring.
+   - **Navigation (client-side, no fetch, no URL change):** source/folder link click → set `current`, re-render; breadcrumb link click → that ancestor; the top-level breadcrumb (if ever rendered) → `{null, null}`.
+   - **Refresh wirings — the `loadDocs` call sites become `loadTree` (same points, same semantics):** the `bor:view-refresh` listener (admin branch), `applySyncSuccess` (the KB just changed), the upload-success branch of `startSyncPolling`. **Never-stale fallback (PLAN §7.4):** after a re-fetch, if `current.source` is no longer in the tree's sources, or `current.folder` no longer exists under it, reset `current = {null, null}` BEFORE rendering — no stale breadcrumb, no stale block.
+   - The sync button / poll / label / banner / error-modal / gate code is otherwise UNTOUCHED.
+3. `frontend/assets/styles.css` — `.kb-crumb` (+ `.kb-crumb-link`, the `aria-current` segment styling), `.kb-level` (+ title/summary typography), `.kb-folders-table` (reuse the `.docs-table` rules), `.folder-link` (row drill link — hover/focus affordance, ≥ 24px target) — house dark-tech palette, AA contrast, no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, no CDN.
+
+## Testing & Quality
+- Frontend source pins (house pattern — new `tests/unit/test_kb_tree_ui.py`, mirroring `tests/unit/test_sync_button.py`'s sources.js pins): the exact `/api/docs/tree` fetch path; the `#kb-crumb` / `#kb-level` / `#folders-tbody` / `#kb-level-title` ids referenced; the drill-state transitions (source click, folder click, breadcrumb up, top reset); the textContent contract (no `innerHTML` with document-derived data — the module's standing rule); the empty-state semantic (`sources` empty → `#sources-empty`); the reset-to-top fallback on a vanished location; `loadTree` wired at exactly the three refresh points (view-refresh, sync success, upload success) — and the anonymous branch still never fetches.
+- The existing sources.js / sync-button / router pins stay green (the untouched machinery).
+- Coverage: the pins are the unit gate; the `app/` gate is untouched (no backend in this task).
+
+## Completion Criteria
+- [ ] an admin RAG view lists the sources (count + description when stored); clicking through drills (the breadcrumb goes up); the level block shows the current directory's description; the level's files render as the existing 5-column rows with the modal link
+- [ ] the stat cards / empty state / sync machinery behave as today (the untouched pins green); anonymous still gets the gate with zero catalog fetches
+- [ ] a re-fetch after the current location vanished resets to the top level (pinned)
+- [ ] `uv run pytest` green (the pins); `uv run ruff check . && uv run pyright` clean
+- [ ] no backend change; the phase-94 + phase-57 suites green

.agents/phases/todo/97_kb_tree_catalog/05_folder_edit_ui.md

@@ -0,0 +1,28 @@
+# Task 05 — Folder description editor: the phase-57 affordance on the level block + the rows
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The owner edits (or clears) any directory's description with the exact interaction of the file-summary editor — Edit button → inline textarea (prefilled) → Save/Cancel → live-region status — wired to `PATCH /api/folders/summary` (task 03).
+
+## Work
+1. `frontend/index.html` — inside `#kb-level`: `button.kb-summary-edit#kb-level-edit` (type=button, the phase-57 label "Edit", ≥ 24px target) after `p#kb-level-summary`.
+2. `frontend/assets/sources.js` — ONE shared editor function (mirrors the phase-57 wiring in `frontend/assets/document.js`; this module never builds HTML from document-derived data — the editor parts are static createElement, the description text is textContent only):
+   - **Edit buttons:** on `#kb-level` (the static button) AND on every source/folder row's Description cell (created with the row in task 04's row builder — `.kb-summary-edit`). Always present — a description can be CREATED where none is stored (a < 2-document folder, or the generator's fail-soft miss; the editor opens prefilled with the empty string). No whoami gate in the view (the RAG view is admin-only already — the phase-16 gate; the endpoint's `require_admin` is the API-level gate).
+   - **On Edit:** swap the description text node (level block `p` content / row cell content) for: `textarea.kb-summary-editor` (current value via `.value` — the XSS contract), `button.kb-summary-save` "Save", `button.kb-summary-cancel` "Cancel", `p.kb-summary-status[role="status"][aria-live="polite"]`; focus the textarea.
+   - **Save** → `PATCH /api/folders/summary` with `{"source", "folder_path", "summary"}` — `folder_path` is `""` for the source root, the source-relative folder path otherwise (both known from `current` / the row data).
+     - 200 → re-render the description text (textContent only) — the level block AND/OR the row cell where the edit happened — + status "Description updated."; an empty save → the text goes away (level block hidden when empty / row cell emptied) + status "Description cleared."; update the in-memory `kbTree` node's `summary` in place (no re-fetch — the tree state stays coherent; the reload fallback is the safety net).
+     - Failure (non-2xx / network) → neutral retry copy (the phase-55 convention — e.g. "Couldn't save the description — try again."), the editor stays open with the user's text, the stored text is untouched.
+   - **Cancel** → restore the text node (and drop the editor + status).
+3. `frontend/assets/styles.css` — `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem, the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house palette, AA contrast, `:focus-visible` via the global rule, no CDN.
+4. Module header docstring: the phase-97 edit section (the house per-phase note convention).
+
+## Testing & Quality
+- Source pins (`tests/unit/test_kb_tree_ui.py` extensions): the exact PATCH path + body shape (`folder_path ""` for the source root); the editor element construction (textarea `.value` prefill, Save/Cancel, the live-region `role="status"`); the status copies "Description updated." / "Description cleared."; the in-place `kbTree` update on success; the failure path keeps the editor (pinned via the status-copy + open-editor assertions, the phase-55 pin style); textContent re-render (no `innerHTML` on the description); the `styles.css` class names (the five `.kb-summary-*` + `.kb-summary-edit` if distinct).
+- Coverage: the pins are the unit gate; no backend change.
+
+## Completion Criteria
+- [ ] Edit → Save round-trips: the new text renders in the row and/or level block, the server row is `manually_edited` (the task-03 integration pins cover the flag)
+- [ ] an empty save clears (the text is gone, "Description cleared."); Cancel restores; a failure keeps the editor + shows neutral copy
+- [ ] a description can be created where none was stored (the editor opens prefilled empty; the save creates the row — task-03 pin)
+- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean

.agents/phases/todo/97_kb_tree_catalog/06_e2e_kb_tree.md

@@ -0,0 +1,32 @@
+# Task 06 — The dedicated story suite: `tests/e2e/test_kb_tree.py`
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The phase's Playwright gate (A16 — one dedicated E2E file, run in isolation): the drill-down tree, the directory descriptions, and the edit/clear round-trip pinned against the real app + the deterministic mock.
+
+## Work
+`tests/e2e/test_kb_tree.py` — module docstring per the house pattern (story mapping, the run command, the fixture description, the mock notes):
+- **Fixture** (the phase-94 suite's pattern — a host temp-dir tree via `tmp_path_factory`, TWO local sources registered through the authenticated API, the real in-process `POST /api/sync` pipeline; the mock's existing `FOLDER_SUMMARY_MODE` branch stores the canned one-liner naming the folder, so every stored description is deterministic):
+  - `alpha/` — `root-note.md` at the source root, `one/` (2 docs), `two/` (2 docs);
+  - `beta/` — `gamma/` (2 docs).
+  - Total: 7 documents; alpha counts 5 (root + 2 + 2), beta counts 2.
+- **Tests** (Playwright Mapping Rule — each test is one distinct observable):
+  1. `test_top_level_lists_sources_with_descriptions` — the source rows: `alpha` (5) + `beta` (2) in registry order with the canned source-root descriptions; the top-level file table is HIDDEN (files are per-source — the `ls()` equivalence); the stat cards read 7 documents + the computed chunks total.
+  2. `test_drill_into_source` — click the `alpha` row → breadcrumb `alpha` (aria-current); the level block shows alpha's root description (the canned text); the folder rows `one` / `two` each with count 2 + their canned summaries; the file row `root-note.md` (title + chunks column) in the file table.
+  3. `test_drill_into_folder` — drill to `alpha` → `two` → breadcrumb `alpha` → `two`; the level block shows `two`'s description; the file rows `two-a` / `two-b` (titles, the Source column `alpha`); `#folders-wrap` hidden (no subfolders); the breadcrumb link on `alpha` goes back up to the source level.
+  4. `test_edit_folder_description` — on the `alpha` level, click Edit on the `one/` row (or the level block for `alpha` — the executor's pick, pinned in the test) → the textarea prefilled with the canned text → set the new text → Save → the new text renders in the row/block + status "Description updated."; a SQL assert (the `SessionLocal` house pattern) on `folder_summaries`: the row's `summary` is the new text AND `manually_edited` is true.
+  5. `test_clear_folder_description` — an empty save → the text is gone (row cell emptied / level block hidden) + status "Description cleared."; the SQL assert: no row for the folder.
+  6. `test_manual_description_survives_a_changed_sync` — edit a description (test-4 style); add a new file to the fixture dir; `POST /api/sync` (the mock regenerates the OTHER folders' summaries) → the edited description is UNCHANGED (the catalog re-fetch renders it; the SQL row keeps the manual text + the flag) while the untouched folders show the canned regenerated text (the `kept_manual` path, E2E-pinned).
+  7. `test_reload_falls_back_to_top_level` — drill to `alpha/two`; delete `two/`'s documents + chunks directly (the house DB pattern); re-show the RAG view (the nav link — the `bor:view-refresh` trigger) → the breadcrumb is hidden and the top level renders (the never-stale contract, PLAN §7.4).
+  8. `test_anonymous_sees_the_gate` — the RAG view for an anonymous visitor: the sign-in gate visible, no folders/file table, no Edit affordance, and no `/api/docs/tree` request (the phase-16 soft rule — assert via the response/locator state, the existing suites' pattern).
+- **MOCK note:** no mock changes needed (the `FOLDER_SUMMARY_MODE` branch is phase 94's). The suite is mock-only in the standard sense — `E2E_REAL_LLM=1` keeps the canned summaries unavailable, so the suite follows the phase-94 convention (its assertions key on the canned text; the module docstring states the requirement).
+
+## Testing & Quality
+- Run in isolation (DB up): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov`.
+- The suite must not modify app/frontend source to pass — it asserts on the shipped behavior of tasks 01–05.
+
+## Completion Criteria
+- [ ] the suite green in isolation; the test → observable mapping documented in the module docstring
+- [ ] the edit/clear/survive-sync/fallback observables all pinned (tests 4–7)
+- [ ] no changes to completed-phase code beyond what tasks 01–05 shipped

.agents/phases/todo/97_kb_tree_catalog/07_catalog_suites_import_sync.md

@@ -0,0 +1,35 @@
+# Task 07 — The existing import / sync / upload story suites onto the drill-down catalog
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The story suites that assert the catalog through the old flat table keep asserting the SAME behavior (a document is listed, linkable, viewable; the stat cards hold) — with a drill step where the asserted path lives under a folder or source. No product behavior changes; navigation only.
+
+## Work
+- **The recipe** (apply per suite — mechanical, assertion intent unchanged):
+  1. **Nested-path row assertions** (`#docs-tbody tr` has_text a path with `/`): before asserting, drill — click the source row (top level), then each folder link (`#folders-tbody .folder-link`) one per path segment, until the level containing the asserted file renders. A small local drill helper per suite (a few lines: `for seg in path_segments: page.locator(f"#folders-tbody .folder-link:has-text('{seg}')").click()`) is fine — the house style is per-suite helpers, no new shared module unless two suites need the identical one.
+  2. **Total row counts** (`#docs-tbody tr` to_have_count N over a whole KB): the flat total no longer exists in one tbody — re-assert per level (drill to each folder, assert that level's count; the sum is implied) or assert the stat cards (`#stat-docs` still holds the KB total — unchanged).
+  3. **`.first` visible waits** (the catalog-rendered signal): the top-level file table is now always hidden (files are per-source — the `ls()` equivalence) — drill into a source (and folder, if all its files are nested) BEFORE the wait; the source/folder row itself is a valid "the catalog rendered" signal where the intent is just that.
+  4. **Empty-state assertions:** unchanged semantics where the registry is empty (`#sources-empty`); where a source IS registered but the KB is cleared, `#docs-tbody` is still 0 rows (the file table hides; the source row renders `0` — adjust any `#sources-empty` expectation in that state if present).
+  5. Suites that seed by direct `import_sources` (no registry row) work unchanged in principle — their source appears as an indexed-only source row (the `00_phase.md` superset rule); the drill just clicks that row.
+- **Suites (the known set from a source scan — the executor runs each in isolation and fixes whatever is red; the list is a starting point, not a guarantee):**
+  - `test_import_documents.py` — EXPECTED_ROWS all nested under `homelab/…` + `deployments/…`: drill per folder for each row; the 13-stat + chunk/last-indexed cards; the hidden-junk absence (scoped after a drill); the empty-state case (registry empty — should already pass).
+  - `test_admin_auth.py` — the 13 row count (per-level re-assert or the stat cards).
+  - `test_sticky_navbar.py` — the TOTAL_DOCS count (same).
+  - `test_sync_upload_progress.py` — the N_FILES counts + the `docs/*.md` rows (drill `docs`).
+  - `test_upload_no_scan.py` — the `notes/…` rows + counts (drill `notes`; the `skipme.md` absence).
+  - `test_sync_button.py`, `test_sync_model_down.py` — the FIXTURE_DOC row (drill as needed).
+  - `test_import_extensions_env.py` — the SH_REL / MD_REL rows (drill).
+  - `test_quadlet_jinja_import.py` — the `homelab/quadlet/…` rows (drill).
+  - `test_retrieval_quality.py` — the `.hidden` absence (scoped after a drill).
+  - `test_archive_upload_sources.py` — the cleared-state row count (the empty-state semantic, point 4).
+  - anything else that turns red in the regression run (the full E2E list — the AGENTS.md rule-9 gate).
+- **Do NOT change the asserted product behavior.** If a suite's assertion cannot be preserved through the drill (a genuine behavior change, not a navigation one), stop and flag it in the phase record — no silent deviation (AGENTS.md rule 3 spirit).
+
+## Testing & Quality
+- Each updated suite green in isolation: `uv run pytest tests/e2e/test_<suite>.py -v --no-cov` (DB up).
+- Coverage: n/a (tests-only task) — the `app/` gate is untouched.
+
+## Completion Criteria
+- [ ] every suite in the list green in isolation; the asserted document behavior preserved (the drill is the only change)
+- [ ] no app/frontend source changes from this task (tests only)

.agents/phases/todo/97_kb_tree_catalog/08_catalog_suites_viewer_nav.md

@@ -0,0 +1,33 @@
+# Task 08 — The existing viewer / nav / UX story suites + the regression run
+
+**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
+
+## Objective
+The remaining story suites that touch the catalog table (document viewer, navigation, UX) are green on the tree view, and the phase's full validation gate is run end to end.
+
+## Work
+- **The same recipe as task 07** (drill before asserting nested rows; per-level counts; drill before `.first` waits; the empty-state semantic) for:
+  - `test_document_viewer.py` — the `gitlab-compose.yaml` / `xss-fixture.md` rows (drill, then the row link opens the modal — the viewer flow itself is unchanged).
+  - `test_document_back_navigation.py` — the `kubernetes.md` row (drill).
+  - `test_edit_summaries.py` — the DOC_PATH row (drill — the phase-57 summary-edit flow on the VIEWER is untouched by this phase; only the catalog navigation to reach the doc changes).
+  - `test_summary_in_viewer.py` — the YAML/MD rows (drill).
+  - `test_markdown_tables.py` — the TABLES_PATH row (drill).
+  - `test_responsive_polish.py` — the `backup_rotation` row + the desktop/mobile `.first` waits (drill).
+  - the `.first`-visible waits against all-nested KBs: `test_chat_persistence.py`, `test_navbar_refresh.py`, `test_nav_consistency.py`, `test_nav_switch_keeps_stream.py`, `test_shared_header.py`, `test_sources_midstream_bug.py` — drill to a level that has a file before the wait (or assert the source/folder row where the intent is just "the catalog rendered").
+  - anything else that turns red in the regression run.
+- **The regression run (the phase gate, the AGENTS.md rule-9 commands):**
+  1. `uv run pytest --cov=app --cov-report=term-missing` — unit + integration green, TOTAL > 90%.
+  2. `uv run ruff check . && uv run pyright` — clean.
+  3. The catalog-touching E2E list in isolation — at minimum the task 07/08 suites above PLUS the completed phases' suites this phase's area of change can touch: `test_ls_tree_drilldown.py` (the agent's `ls` — must be byte-identical for the same data), `test_kb_overview.py`, `test_edit_summaries.py`, `test_import_documents.py`, `test_sync_button.py`, `test_admin_auth.py`.
+  4. `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` (task 06's suite) green in isolation.
+  5. The full `tests/e2e/` sweep in isolation for any suite the run above did not cover and that references the RAG view — the final pass (the `00_phase.md` completion criteria) is the authority; this task leaves nothing red.
+
+## Testing & Quality
+- Each updated suite green in isolation; the gates above green.
+- Coverage: the `app/` TOTAL > 90% (this task is tests-only — the coverage is the tasks 01–03 code, already gated).
+
+## Completion Criteria
+- [ ] every suite in the list green in isolation; the asserted behavior preserved (drill = the only change)
+- [ ] unit + integration green, TOTAL coverage > 90%, ruff + pyright clean
+- [ ] `test_ls_tree_drilldown.py` green — the agent's tree is byte-identical (the only schema change is the additive flag column)
+- [ ] no app/frontend source changes from this task (tests only)