phase: 162_search_recall_small_sources

**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).

.agents/phases/complete/162_search_recall_small_sources/00_phase.md

@@ -0,0 +1,70 @@
+# Phase 162 — Search recall: accent-folded name hits & grep, KB-overview source inventory, search-discipline teaching
+
+**Source:** owner live report, 2026-10-07 — brain.reeseapps.com, shared conversation "How do I build a Besta?" (share token `58525626-4462-4baf-bcf2-3abc81f18284`). The owner asked "Why did you miss it? Let's try to improve your search techniques and system prompt." The agent's post-mortem identified **four stacked failures**, three of which are code-fixable:
+
+1. **The `<knowledge_base>` outline is lossy and is the only map the model gets.** The outline listed 3 groups (Gufo, Brain-of-Reese, Deployments); the live index had 14 sources. `ikea_instructions` (1 doc) and `players_handbook` (1 doc) were **invisible** — a whole class of source-level questions could not be surfaced by the outline at all. The model answered from the outline instead of running `ls`.
+2. **Notes poisoned the model with a true-but-different fact** (the phase-140/148 note "the IKEA tar is gitignored, on disk only" was over-generalized to "the brain has no copy of this PDF"). Model behavior — mitigated here by prompt teaching (D4), plus an owner runtime follow-up (A6, recorded, not code).
+3. **The Å/ASCII accent-folding gap.** `grep "besta"` returned only phase docs and never the PDF, because the only product name in the PDF's text layer is `BESTÅ` (U+00C5): the filename is ASCII (`besta-frame-white__…`) but the content is not, and the exact-substring grep cannot bridge that. On the name-hit side the gap is symmetric: the question token `BESTÅ` normalizes (lower + alnum-only) to `best` — not `besta` — so an accented question never name-matches the ASCII filename's stem sub-components.
+4. **Retrieval under-ranked the PDF (the phase-119 failure class).** The doc's title is the generic "Assembly Instructions" (title ranking gives nothing); by phase-119 LOCKED A2 the digitless token `besta` (≥4 chars) equals a stem sub-component of `besta-frame-white__AA-1272068-11-100` and should have name-matched, but the name-hit row lost the fused walk to the phase docs whose bodies literally contain "besta". Flagged by the post-mortem as a hypothesis needing a test — **this phase pins it with the battery (D5)** rather than assuming a finding.
+
+**Story:** n/a (owner-directed RAG behavior change; extends the phase-119 name-hit mechanism, the phase-31 KB overview, the phase-68/70 grep, and the phase-117 prompt pins).
+
+## Objective
+Accented names (`BESTÅ` ↔ `besta`) are found **both ways** — by the name-hit path (question side) and by `grep` (content side); **no source can be invisible** to `<knowledge_base>` (a deterministic, non-LLM source-inventory line is appended to the stored overview); the model is **taught the search discipline** the post-mortem adopted (outline ≠ inventory; `ls` first; part-number/filename-token greps; gitignored ≠ unindexed); and the battery question "How do I build a Besta?" **pins** the IKEA doc into the suggested top-5 — turning the anecdote into a regression test.
+
+## Dependencies
+- `119_name_signal_read_chips` (complete) — the phase-119 name-hit rule (LOCKED A2), the `name_hit` flag, the bonus (LOCKED A3), and the `suggested:` eval-battery line this phase extends.
+- `31`-line KB overview (complete; `app/rag/overview.py`) — the `regenerate_overview` best-effort/change-gated generation this phase appends the inventory to.
+- `68_search_tool` / `70_harness_aligned_tools` (complete) — the `grep` contract (case-insensitive fixed substring, locator, teaching refusals) this phase extends.
+- `117_tool_call_compact` (complete) — the `TOOLS_SECTION` byte-pins this phase re-cuts.
+
+## Design (shared by all tasks — the executor reads this, not the chat)
+
+**D1 — accent-folded name normalization (task 01, LOCKED A1).** One new helper in `app/rag/retriever.py`: `_fold(s)` = `unicodedata` NFKD normalize + drop every char with a non-zero combining class (stdlib only; no new dependency). `_normalize_name` becomes the existing `[a-z0-9]+` scan of `_fold(s.lower())`. Because `_normalize_name` is the **single** normalization both sides of name matching pass through — the question tokens in `name_hit_tokens` AND the path parts / file stem / stem sub-components in `_name_parts` — the one change bridges both directions: the ASCII question `besta` keeps matching the ASCII filename's sub-components (unchanged behavior), the accented question `BESTÅ` now normalizes to `besta` (was `best` — the 2026-10-07 gap), and an accented **filename** (`BÅsta.md` → `basta`) matches an ASCII question. Folding cases the tests pin: `Å/å → a`, `Ä/ä/Ö/ö/Ü/ü → a/o/u` (the letter, decomposition preserved), `ß → ss`, the `fi` ligature → `fi`, `ç → c`; characters with no NFKD decomposition (CJK, `①`) still drop out under the alnum scan, exactly as today. **Everything else in the A2 mechanism is untouched**: the digit-bearing-prefix / digitless-equality match classes, titles-never-matched, the (matched-token count, catalog order) ranking, `NAME_TOKEN_MIN_LEN`, the versioned-name join, `NAME_HIT_LIMIT`, the `name_hit` flag, the phase-119 bonus (LOCKED A3), and the FTS / cosine / A8 gate (LOCKED A7/A8 — the `tsvector` 'english' pipeline is NOT modified; same-accents FTS already bridges, accent-vs-ASCII FTS bridging is out of scope by the lock).
+
+**D2 — the KB-overview source inventory (task 02, LOCKED A2).** The overview is `lite`-generated and lossy (D1-failure #1); the fix is **deterministic, non-LLM**: `regenerate_overview` (happy path only) appends one line to the stored content, after the model's outline: `Sources: <name> (<k> doc|docs), …` — one entry per DISTINCT source, catalog order (`source, path`), exact per-source document counts. Computed from the same catalogue rows the prompt builder already reads (no new query). `build_overview_prompt` / `OVERVIEW_INSTRUCTION` are **byte-identical** (the model never sees the tail — it is mechanical, not a model burden); `load_kb_overview` / `build_kb_section` are untouched (the existing longest-prefix cap applies to the whole stored text). Budget: the model caps the outline at ~1500 chars per the instruction; one inventory line is ~15 chars/source — at the default `BOR_KB_OVERVIEW_MAX_CHARS` (4000) the tail always fits; a smaller owner-set cap cuts the tail first (accepted; the cap is owner-tunable, the record here is the constraint). The zero-doc early return and the `LLMError` path are **byte-identical** (the previous row — tail included, it is part of the stored content — stays intact; an old outline is better than none).
+
+**D3 — accent-folding grep (task 03, LOCKED A3).** `grep_document` in `app/rag/agent.py` folds **both sides** — the pattern and each content line — through the SAME `_fold` as D1 (imported from `app.rag.retriever`, which `agent.py` already imports from: grep and the name-hit path can never drift apart) plus lowercase (replacing the current lower-only comparison): `grep "besta"` now matches a content line `The BESTÅ frame`. The contract remains a **fixed substring** (the phase-68 owner lock) — the fold changes the alphabet it ignores, not the matching semantics; still a locator, still ≤20 lines, still line-truncated at 200 chars. The regex teaching (`NO_MATCHES_REGEX` / `_SCOPED`, `looks_like_regex`, the plain-form retry hint) is **byte-identical** — the plain-substring claim it teaches is still true. Copy updates (all in this task): the `agent.py` module docstring grep paragraph, the `AGENT_TOOLS` grep function description + the `pattern` parameter description, and the `TOOLS_SECTION` grep sentence in `app/rag/prompts.py` ("case-insensitive" → "case- and accent-insensitive"), plus D4's new sentence. The phase-117 `test_prompt_lock.py` / `test_prompts.py` / `test_agent.py` pins re-cut in the same task (the phase-124 pattern: copy change + re-cut pins in one task).
+
+**D4 — the search-discipline sentence (task 03, LOCKED A4).** Appended to the `TOOLS_SECTION` tools paragraph **immediately after the grep sentence** (the conversation's placement), LOCKED verbatim:
+
+> `A summary or outline is a hint, never an inventory: before claiming the knowledge base lacks something, run `ls` with no `path` to list every source, and `grep` the artifact's part number or a filename token. `grep` folds accents — `besta` matches `BESTÅ` — but one pattern is one string: a name's split tokens (`frame`, `white`) and its part number (`AA-1272068`) need separate greps. A note saying a file is gitignored describes its repo status, not whether it is indexed as a source.`
+
+(Adapted from the conversation's proposed patch for the D3 fold — the conversation's original sentence assumed a non-folding grep; the owner asked for "improve your search techniques and system prompt", which is the permission source for this copy change.) `PERSONA`, the LOW/`DEFLECT_MODE` body, and every other `TOOLS_SECTION` sentence stay byte-identical. The mock LLM is unaffected (it keys on markers, not on the sentence).
+
+**D5 — verification (task 04).** A new dedicated Playwright suite `tests/e2e/test_search_recall_small_sources.py` (run in isolation; the phase-119 local-directory-source fixture pattern) with a synthetic replica of the live miss: source `ikea_instructions` (ONE doc, `besta-frame-white__AA-1272068-11-100.md`, body containing the name **only** in accented form — no ASCII "besta") + 2–3 distractor docs in other sources whose bodies contain ASCII "besta" (the phase-docs stand-in). The **pre-fix evidence is measured and recorded** in the task file: with the task-01 fold temporarily reverted (git stash), the ACCENTED question "How do I build the BESTÅ?" must lose the accented doc from the suggested top-5 (record the measured ranks); restore the fix. Gated assertions: (1) the accented question's `done`-frame `suggestions` include the `ikea_instructions` doc (the D1 question-side fold); (2) the ASCII question "How do I build a Besta?" includes it too (the phase-119 path regression); (3) the stored `kb_overview` row (read through the fixture session) ends with the `Sources: …` tail naming **every** fixture source with exact counts — the 1-doc source visible (D2); (4) the out-of-KB question's gate verdict is unchanged (D1–D3 must not move the A8 gate). `tests/fixtures/retrieval_battery.txt` (the tracked phase-119 battery) gains `How do I build a Besta?` with its gate comment: on the 1,000-doc live replica (the `restore-test-db` skill; the dump is a local gitignored artifact — if missing, STOP and ask the owner, never fabricate numbers) the battery runs when the replica's `documents` table contains an `ikea_instructions`-class source (check `SELECT DISTINCT source FROM documents`) and the suggested top-5 must include `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`; if the replica lacks it (the snapshot may predate the import), the five existing phase-119 questions still run (their gate must NOT move — the name-hit change is behavior-preserving for them) and the Besta question is recorded as **skipped loudly (source absent in the replica snapshot)** — the phase-140 "skipped loudly" precedent. The under-rank hypothesis (D1-failure #4) is thus either pinned green or handed to the owner with measured ranks — never assumed.
+
+**Invariants (all tasks).** The A8 honesty gate, the FTS `tsvector` pipeline, the cosine path, every SSE frame shape, the LOW prompt, `PERSONA`, the `NO_MATCHES_REGEX`/`_SCOPED` teaching lines, the mock-LLM markers, and the `build_overview_prompt` / `OVERVIEW_INSTRUCTION` inputs are byte-identical. `PLAN.md` untouched (Protocol B — A5).
+
+## Tasks
+1. `01_accent_fold_name_norms.md` — the shared `_fold` helper + the `_normalize_name` fold (D1).
+2. `02_kb_overview_source_inventory.md` — the deterministic `Sources: …` tail (D2).
+3. `03_grep_folding_search_discipline.md` — the folded `grep_document` + the three copy updates + the D4 sentence + the re-cut pins (D3+D4).
+4. `04_e2e_battery_pins.md` — the dedicated E2E, the pre-fix evidence, the battery question + replica run, the final gates (D5).
+
+## Testing & Quality
+- Unit: `tests/unit/test_retriever.py` (D1: the bridge both ways, the fold cases, the A2 non-regression set), `tests/unit/test_overview.py` (D2: the tail format/order/counts, the 1-doc survival, the zero-doc + LLM-failure paths, the byte-identical `build_overview_prompt`), `tests/unit/test_agent.py` (D3: `grep_document` fold both sides, the scoped + unscoped paths, the copy pins), `tests/unit/test_prompts.py` + `tests/unit/test_prompt_lock.py` (D3+D4: the re-cut pins; the LOW + `PERSONA` byte-identity).
+- E2E: `tests/e2e/test_search_recall_small_sources.py`, **in isolation** (`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`); the phase-119/117/31 regression suites green (the re-cut pins included).
+- Battery (task 04): per D5 — the replica run recorded in the task file; the five phase-119 questions' gate unchanged.
+- Coverage: **>90%** on `app/`; `uv run ruff check . && uv run pyright` clean.
+
+## Completion Criteria
+- [ ] `grep "besta"` (unit + the E2E fixture's content layer) matches an accented `BESTÅ` content line, and the accented question "How do I build the BESTÅ?" name-matches the ASCII filename's stem sub-components — the fold is one function, both sides, measured (the pre-fix ranks recorded in `04_…md`).
+- [ ] The stored `kb_overview` row ends with the deterministic `Sources: …` line naming every source with exact counts; a 1-doc source is visible; the zero-doc and LLM-failure paths are byte-identical to the pre-phase behavior.
+- [ ] The `TOOLS_SECTION` carries the folded grep contract + the LOCKED D4 sentence (all other pins byte-identical; the re-cut pins green).
+- [ ] `tests/e2e/test_search_recall_small_sources.py` green in isolation; the battery question is recorded per D5; `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean.
+- [ ] `README.md` + `AGENTS.md` reviewed in full in the final pass (the grep contract line in `AGENTS.md`'s §6 quick reference and `PLAN.md` §6's tool table mention "case-insensitive fixed substring" — the final pass updates `AGENTS.md`'s tool description to "case- and accent-insensitive"; `PLAN.md` stays untouched per A5, folded at the next redo).
+- [ ] One `--no-gpg-sign` commit; the phase dir moves to `complete/` by the pipeline gate.
+
+## Locked decisions
+- **A1 — the fold**: NFKD normalize + combining-class strip (stdlib `unicodedata`) applied inside `_normalize_name` — the single normalization both sides of name matching use; the A2 match classes, ranking, caps, flags, the phase-119 bonus, and the FTS/cosine/A8 pipeline are untouched (owner-confirmed 2026-10-07, from the conversation's post-mortem).
+- **A2 — the overview tail**: `Sources: <name> (<k> doc|docs), …` appended server-side to the stored outline at generation time (happy path only); deterministic, non-LLM, catalog order; `build_overview_prompt` / `OVERVIEW_INSTRUCTION` / `load_kb_overview` / `build_kb_section` byte-identical; the zero-doc and `LLMError` paths untouched (owner-confirmed 2026-10-07).
+- **A3 — the grep fold**: `grep_document` compares the `_fold`+lowercase of the pattern against the `_fold`+lowercase of each line — the SAME `_fold` as A1 (imported); still a fixed substring (phase-68 lock), still a locator, regex teaching byte-identical (owner-confirmed 2026-10-07 — the conversation's "optional grep-side accent folding", accepted).
+- **A4 — the D4 sentence** is appended to the `TOOLS_SECTION` tools paragraph immediately after the grep sentence, verbatim as written in D4; `PERSONA` + LOW body + the other `TOOLS_SECTION` sentences byte-identical (owner-confirmed 2026-10-07 — the owner asked for the system-prompt improvement in the conversation).
+- **A5 — PLAN.md untouched (Protocol B)**: the grep-contract extension (A3) and the prompt sentence (A4) are recorded here (owner-confirmed) and folded into the anchor table at the next redo (the phase-119 A6 precedent).
+- **A6 — the steering-note follow-up is an owner runtime action, not code**: the conversation's proposed tuning rule ("IKEA's BESTÅ assembly PDF is a live source — search it as BESTÅ or AA-1272068, not besta") is a `steering_notes` row the owner adds via the Tuning tab after this phase ships; the phase records it here and does not touch `steering_notes` data.
+
+## Commit
+```bash
+git add app/ tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(rag): accent-fold name hits and grep (BESTÅ <-> besta), a deterministic source inventory in the KB overview, and search-discipline prompt teaching"
+```

.agents/phases/complete/162_search_recall_small_sources/01_accent_fold_name_norms.md

@@ -0,0 +1,28 @@
+# Task 01 — The accent-folded name normalization (phase-162 D1, LOCKED A1)
+
+**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D1 — the 2026-10-07 live finding: `grep "besta"` never hit the IKEA PDF (its text layer carries only `BESTÅ`, U+00C5), and the symmetric name-hit gap: the question token `BESTÅ` normalizes (lower + alnum-only) to `best`, never `besta`, so an accented question cannot name-match the ASCII filename `besta-frame-white__AA-1272068-11-100` (whose stem sub-components include `besta` under phase-119 LOCKED A2).
+**Story:** n/a
+
+## Objective
+`_normalize_name` in `app/rag/retriever.py` folds accents (NFKD + combining-class strip, stdlib `unicodedata`) before the existing alnum scan, so both sides of name matching bridge `BESTÅ` ↔ `besta` — with the A2 mechanism (match classes, ranking, caps, flags, bonus) and the FTS/cosine/A8 pipeline byte-identical.
+
+## Work
+1. `app/rag/retriever.py` — new module-level `_fold(s: str) -> str`: `unicodedata.normalize("NFKD", s)` with every char whose `unicodedata.combining(c)` is non-zero dropped (stdlib only — no new dependency). Docstring: the shared accent fold (phase 162) — NFKD decomposition, combining marks dropped; `Å/å → a`, `ß → ss`, `fi → fi`, `ç → c`; characters with no decomposition (CJK, `①`) survive the fold and still drop out under the caller's alnum scan (behavior unchanged for them).
+2. `app/rag/retriever.py` — `_normalize_name`: `"".join(_ALNUM_RE.findall(_fold(s.lower())))` (was: `…findall(s.lower())`). Docstring update: the folded form (`Qwen 3.8` → `qwen38` unchanged; `BESTÅ` → `besta` new) — this is the **single** normalization both sides pass through (the `name_hit_tokens` question tokens and the `_name_parts` path parts / stem / stem sub-components), so the one change bridges both directions.
+3. `app/rag/retriever.py` — `name_hit_tokens` / `_name_parts` / `_name_token_matches`: **no code change** (they consume `_normalize_name`); their docstrings gain one clause each: candidates/parts are accent-folded (phase 162) — `besta` matches `BESTÅ`-class names both ways.
+4. `app/rag/retriever.py` — module docstring: the normalization paragraph now states the fold (the 2026-10-07 gap: the ASCII filename vs the accented content layer).
+5. `tests/unit/test_retriever.py` — the fold, on the file's existing fixture pattern:
+   - **question side**: the accented question "How do I build the BESTÅ?" produces a `besta` token (was `best` pre-phase — pin the exact token list); `Ä`/`Ö`/`Ü`/`ß`/`fi`/`ç` fold cases (`straße`-class → `strasse`-class, ligature `fine` → `fine`);
+   - **path side**: `_name_parts` on `besta-frame-white__AA-1272068-11-100` yields `besta` in the equal set (the phase-119 case, unchanged) AND on the accented filename `BÅsta.md` yields `basta` — the ASCII question `basta` name-matches it;
+   - **the bridge, end to end**: `_name_hit_chunks`-class assertion — an accented-question token hits the ASCII-stem doc AND an ASCII token hits the accented-stem doc;
+   - **A2 non-regression (all existing behavior, re-pinned after the change)**: digitless precision (the `deploy`-titled `Deployments/` doc does NOT hit for `deploy`), digit-bearing prefix (`qwen38` → `qwen3.8-27b-…`, and the mid-word non-match), the versioned join (`Qwen 3.8` → `qwen38`), min-length on the **folded** form, ranking (matched-token count, catalog order), `NAME_HIT_LIMIT`, `name_hit=True`/`False` convention;
+   - **no-decomposition characters**: a CJK / `①` token still drops out (the alnum scan behavior for non-folding chars is unchanged);
+   - the FTS/`retrieve()`/gate rows are untouched (the existing `test_retriever.py` / `test_retriever_recency.py` suite stays green without modification).
+
+## Testing & Quality
+- Unit: the cases above; coverage **>90%** on `app/`.
+
+## Completion Criteria
+- [ ] `BESTÅ` (question) → `besta` token → name-hits the ASCII filename's sub-components, and `besta` (question) → name-hits the accented filename `BÅsta.md` — one function, both directions.
+- [ ] Every phase-119 A2 case (precision, prefix, join, ranking, cap, flag) passes UNCHANGED after the fold; the FTS/gate suites are untouched.
+- [ ] `uv run pytest tests/unit/test_retriever.py tests/unit/test_retriever_recency.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.

.agents/phases/complete/162_search_recall_small_sources/02_kb_overview_source_inventory.md

@@ -0,0 +1,29 @@
+# Task 02 — The deterministic source inventory in the KB overview (phase-162 D2, LOCKED A2)
+
+**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D2 — the 2026-10-07 live finding: the `lite`-generated `<knowledge_base>` outline listed 3 groups of a 14-source index; the 1-doc sources `ikea_instructions` and `players_handbook` were invisible, so source-level questions ("You have ikea_instructions in your RAG" — the owner's push-back) could not be surfaced by the outline at all.
+**Story:** n/a
+
+## Objective
+`regenerate_overview` in `app/rag/overview.py` appends one **deterministic, non-LLM** line to the stored outline — `Sources: <name> (<k> doc|docs), …` — so no source can be invisible to `<knowledge_base>`; the model input, the load/inject path, and the zero-doc / LLM-failure paths are byte-identical.
+
+## Work
+1. `app/rag/overview.py` — a new pure helper `_source_inventory(rows: Sequence[tuple[str, str, str, str | None]]) -> str` next to `_first_summary_line`: one entry per **distinct** source, in catalog order (first occurrence in the `source, path`-ordered rows), each `<name> (<k> doc|docs)` — singular `doc` at k=1, `docs` otherwise — joined with `, ` behind the prefix `Sources: `. (The `rows` shape is the existing `(source, path, title, summary)` projection `regenerate_overview` already reads — no new query.) Docstring: the phase-162 deterministic inventory (LOCKED A2) — mechanical, non-LLM; the `lite` outline stays the model's job.
+2. `app/rag/overview.py` — `regenerate_overview` **happy path only**: after the model text is obtained and before the upsert, the stored content = the model text (trailing newlines rstripped) + `"\n"` + the inventory line. The upsert, the `overview: regenerated docs=… chars=…` log (chars = the FULL stored content, tail included), the `True` return, and the change-gating contract are otherwise unchanged.
+3. `app/rag/overview.py` — the **zero-doc early return** (`False`, row untouched) and the **`LLMError` path** (logged, `False`, previous row intact) are UNCHANGED — the previous row keeps its earlier tail, it is part of the stored content; an old outline is better than none. `build_overview_prompt`, `OVERVIEW_INSTRUCTION`, `SYSTEM_PROMPT`, `load_kb_overview`, and `app/rag/prompts.py::build_kb_section` are **untouched** (the model never sees the tail; the existing longest-prefix cap applies to the whole stored text — at the default `BOR_KB_OVERVIEW_MAX_CHARS` (4000) the ~1500-char outline + the one-line tail always fit; a smaller owner-set cap cuts the tail first — accepted, recorded in the docstring).
+4. `app/rag/overview.py` — module docstring: a paragraph on the inventory line (phase 162 — the 2026-10-07 finding: the `lite` outline is lossy; small sources must be visible without a model call; the line is deterministic so the mock/real model outputs never affect it).
+5. `tests/unit/test_overview.py` — the helper + the paths (the file's existing duck-typed-`OverviewLLM` fake pattern):
+   - the format: `Sources: a (1 doc), b (2 docs), c (3 docs)` — exact, incl. singular/plural and the catalog order;
+   - the **1-doc survival** (the `ikea_instructions` case): a 1-doc source among many appears with `(1 doc)`;
+   - the happy path: the stored row = model text + newline + the tail (exact content pin on the fake LLM's fixed output);
+   - zero-doc: `False`, the existing row (a pre-phase row, tail-less) byte-identical;
+   - `LLMError`: `False`, the **previous** row byte-identical — INCLUDING a row that already carried a tail (the tail is content, preserved on failure);
+   - `build_overview_prompt` byte-identical to the pre-phase output for the same rows (the model input never gains the tail);
+   - the regenerated log line carries the full stored-content char count.
+
+## Testing & Quality
+- Unit: the cases above; coverage **>90%** on `app/`.
+
+## Completion Criteria
+- [ ] A regenerated `kb_overview` row ends with `Sources: …` naming every source with exact counts (a 1-doc source visible); the model input is byte-identical to the pre-phase prompt.
+- [ ] The zero-doc and `LLMError` paths leave the row byte-identical (an existing tail preserved on failure); the char log is exact.
+- [ ] `uv run pytest tests/unit/test_overview.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.

.agents/phases/complete/162_search_recall_small_sources/03_grep_folding_search_discipline.md

@@ -0,0 +1,27 @@
+# Task 03 — The accent-folding grep + the search-discipline prompt teaching (phase-162 D3+D4, LOCKED A3+A4)
+
+**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D3+D4 — the 2026-10-07 live finding: `grep "besta"` never hit the PDF whose text layer carries only `BESTÅ` ("the tool is documented as 'a plain substring' and silently missing Å is the nastiest of the four"), and the owner's ask: "Let's try to improve your search techniques and system prompt."
+**Story:** n/a
+
+## Objective
+`grep_document` folds accents on **both** pattern and content (the SAME `_fold` as task 01, imported — grep and the name-hit path can never drift); the contract copy in all three places states the folded contract; the LOCKED D4 search-discipline sentence joins the `TOOLS_SECTION` tools paragraph; the phase-117 pins re-cut in this task.
+
+## Work
+1. `app/rag/agent.py` — `grep_document(content, pattern)`: the comparison is `_fold(pattern.lower()) in _fold(line.lower())` per (rstripped) line — importing `_fold` from `app.rag.retriever` (the module already imports `TRUNCATION_MARKER` from there). Everything else UNCHANGED: the rstrip, the `lineno` numbering, the 20-match cap, the 200-char line truncation, the scoped/unscoped call sites (`all_documents` / the one-doc path at the existing call site), the locator semantics (no context add — `holder.read_docs` untouched). Docstring update: case- AND accent-insensitive fixed substring (phase 162, LOCKED A3) — the fold changes the alphabet it ignores, not the matching semantics; still NEVER a regex.
+2. `app/rag/agent.py` — the module docstring grep paragraph (the "case-insensitive fixed substring" sentence): "case- and accent-insensitive fixed substring (accents fold — `besta` matches `BESTÅ`)".
+3. `app/rag/agent.py` — `AGENT_TOOLS` the grep function description: "(case-insensitive)" → "(case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`)", and the `pattern` parameter description gains the same clause (the "plain substring, NEVER a regex" teaching unchanged). The `NO_MATCHES_REGEX` / `NO_MATCHES_REGEX_SCOPED` teaching lines and `looks_like_regex` / `plain_form` are **byte-identical** (the plain-substring claim they teach is still true).
+4. `app/rag/prompts.py` — `TOOLS_SECTION`, the grep sentence: "locates an exact string (case-insensitive)" → "locates an exact string (case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`)" (the sentence's remainder, incl. the plain-substring + retry teaching, unchanged).
+5. `app/rag/prompts.py` — `TOOLS_SECTION`: the LOCKED D4 sentence (verbatim from the overview) appended to the tools paragraph **immediately after the grep sentence**, i.e. after the `…retry with the plain text you expect to see.` clause:
+   `A summary or outline is a hint, never an inventory: before claiming the knowledge base lacks something, run `ls` with no `path` to list every source, and `grep` the artifact's part number or a filename token. `grep` folds accents — `besta` matches `BESTÅ` — but one pattern is one string: a name's split tokens (`frame`, `white`) and its part number (`AA-1272068`) need separate greps. A note saying a file is gitignored describes its repo status, not whether it is indexed as a source.`
+   `PERSONA`, the LOW/`DEFLECT_MODE` body, the `<documents>` blocks, and every other `TOOLS_SECTION` sentence are byte-identical. (The E2E mock is unaffected — it keys on markers, not on the sentence.)
+6. Pins re-cut, **in this task** (the phase-124 pattern): `tests/unit/test_prompt_lock.py` (the `TOOLS_SECTION` pin + its derived cross-pins), `tests/unit/test_prompts.py` (the grounded-prompt pins that embed the grep sentence), `tests/unit/test_agent.py` (the `AGENT_TOOLS` description pin, if present — grep the file for the old description text; the `NO_MATCHES_REGEX` pins stay green unmodified).
+7. `tests/unit/test_agent.py` — `grep_document` fold coverage (the existing case-insensitive test's pattern): `grep_document("The BESTÅ frame (AA-1272068-11-100)", "besta")` hits; `grep_document("…BÅsta…", "basta")` hits; `grep_document("café", "cafe")` hits; the pattern's own accent folds too — `grep_document("The besta frame", "BESTÅ")` ALSO hits (the pattern `BESTÅ` folds to `besta`); a genuinely non-matching folded pattern still returns no lines (`grep_document("The besta frame", "bestb")` → `[]`); line numbers / rstrip / cap / truncation unchanged (existing tests stay green unmodified).
+
+## Testing & Quality
+- Unit: the cases above; coverage **>90%** on `app/`.
+
+## Completion Criteria
+- [ ] `grep_document` matches accented ↔ ASCII both ways through the imported `_fold` (no second copy of the fold); the regex teaching lines are byte-identical.
+- [ ] The three copy sites (module docstring, `AGENT_TOOLS`, `TOOLS_SECTION`) state the folded contract; the D4 sentence sits immediately after the grep sentence, LOCKED verbatim.
+- [ ] The re-cut `test_prompt_lock.py` / `test_prompts.py` / `test_agent.py` pins green; the LOW + `PERSONA` byte-identity pins green unmodified.
+- [ ] `uv run pytest tests/unit/test_agent.py tests/unit/test_prompt_lock.py tests/unit/test_prompts.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.

.agents/phases/complete/162_search_recall_small_sources/04_e2e_battery_pins.md

@@ -0,0 +1,31 @@
+# Task 04 — The dedicated E2E, the measured pre-fix evidence, and the battery pin (phase-162 D5)
+
+**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D5 — the 2026-10-07 finding must turn from anecdote into a regression test: the accented question "How do I build the BESTÅ?" surfaces the accented-content doc whose ASCII filename carries the name's sub-components, over ASCII "besta"-mentioning distractors; the 1-doc source is visible in the stored overview; the A8 gate and the five phase-119 battery questions do not move.
+**Story:** n/a
+
+## Objective
+`tests/e2e/test_search_recall_small_sources.py` (run in isolation) gates the fold end-to-end on a synthetic replica of the live miss; the pre-fix ranks are **measured and recorded** in this file (the under-rank hypothesis is pinned green or handed to the owner with evidence — never assumed); `tests/fixtures/retrieval_battery.txt` gains the Besta question per D5; the final gates run.
+
+## Work
+1. `tests/e2e/test_search_recall_small_sources.py` — new dedicated suite, **run in isolation** (`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`), on the phase-119 local-directory-source fixture pattern (fixture module truncates/re-imports; the deterministic mock LLM by default). Fixture KB — a synthetic replica of the live miss:
+   - source **`ikea_instructions`** (ONE doc): `besta-frame-white__AA-1272068-11-100.md` — body carrying the product name **only in accented form** (e.g. "The BESTÅ frame (AA-1272068-11-100)… cam bolt 130618 … wall bracket 141413 … leveling foot …") — **no ASCII "besta" anywhere in the body** (the live PDF's exact situation);
+   - 2–3 **distractor docs** in other sources whose bodies contain ASCII "besta" (the phase-docs stand-in — e.g. `brain-of-reese/.agents/phases/complete/140_pictorial_pdf_pages/00_phase.md`-shaped text: "the besta fixture drove the pictorial phases"). Tune the distractor count/bodies so the pre-phase accented question loses the `ikea_instructions` doc from the suggested top-5 (step 2 measures this — if it does NOT lose it, add distractors until it does, and record the final fixture shape here).
+   Assertions (mock LLM; the `done`-frame `suggestions` list is the observable tier — the phase-119 done-frame shape):
+   - **(1) the accented question** `POST /api/chat` "How do I build the BESTÅ?" → the `done` frame's `suggestions` include `ikea_instructions/besta-frame-white__AA-1272068-11-100.md` (the D1 question-side fold — `BESTÅ` → `besta` → the stem sub-component name-hit);
+   - **(2) the ASCII regression** "How do I build a Besta?" → the same doc in `suggestions` (the phase-119 path, unchanged by the fold);
+   - **(3) the overview tail** — after the fixture import (the mock's `KB_OVERVIEW_MODE` regeneration ran), the stored `kb_overview` row (read through the fixture's DB session, the phase-119 pattern) ends with `Sources: …` naming **every** fixture source with exact counts — the 1-doc `ikea_instructions` source visible (the D2 gate);
+   - **(4) the gate sanity** — an out-of-KB question (e.g. "What is the capital of Mongolia?") returns the deflected terminal (the D1–D3 changes must not move the A8 gate verdict; the mock's cosine behavior makes this deterministic — follow the phase-119 deflection-sanity assertion).
+2. **The pre-fix evidence — measured, recorded in THIS file (before the final gates):** `git stash` the task-01 `_normalize_name` change (tasks 02–03 changes may stay — they do not affect the name-hit path), run assertion (1) against the fixture, record the measured suggested top-5 ranks for the accented question (the `ikea_instructions` doc's rank or its absence), `git stash pop`. The recorded block must show the doc OUT of the suggested top-5 pre-fix (if the fixture as built does not show that, go back to step 1's tuning and re-measure — the recorded block is the evidence the fold fixes a real miss, not a no-op).
+3. `tests/fixtures/retrieval_battery.txt` — append `How do I build a Besta?` with a gate comment (the file's existing comment style): on the 1,000-doc live replica (the `restore-test-db` skill — the dump is a local gitignored artifact; **if missing, STOP and ask the owner, never fabricate numbers**) the battery runs when `SELECT DISTINCT source FROM documents` shows an `ikea_instructions`-class source, and the printed `suggested:` top-5 must include `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`; if the replica lacks that source (the snapshot may predate the import), the five EXISTING phase-119 questions still run — their recorded gate (the canonical docs' ranks, the qwen3.8 regression, the deflection sanity `HIGH, fts_hits=5`) must be **unchanged** by this phase — and the Besta question is recorded in the task file as **skipped loudly (source absent in the replica snapshot)** — the phase-140 "skipped loudly" precedent. The under-rank hypothesis (the overview's failure #4) is thus either pinned green or handed to the owner with the measured ranks.
+4. Regression + final gates: the phase-119 (`test_name_signal_read_chips.py`), phase-117, and phase-31-era regression suites green; `uv run pytest` (full suite) green; `uv run pytest --cov=app --cov-report=term-missing` coverage **>90%** on `app/`; `uv run ruff check . && uv run pyright` clean.
+
+## Testing & Quality
+- E2E: the dedicated suite in isolation (per AGENTS.md rule 4 — one file per phase, run in isolation); unit pins from tasks 01–03 already in place.
+- Battery: per step 3, recorded in this file.
+- Coverage: **>90%** on `app/`; lint + types clean.
+
+## Completion Criteria
+- [ ] The recorded pre-fix block in this file shows the accented question losing the `ikea_instructions` doc from the suggested top-5; the post-fix E2E (assertions 1–4) is green in isolation.
+- [ ] The battery question is recorded per step 3 (pinned green, or skipped loudly with the five phase-119 questions' gate verified unchanged).
+- [ ] `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean.
+- [ ] The owner follow-up (A6) is noted in the final pass's summary for the owner (the steering-note "rule 13" is a runtime Tuning-tab action, not code).

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.md

@@ -0,0 +1,16 @@
+**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**
+
+- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
+- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
+- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
+- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
+- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
+- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
+- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
+- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
+- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
+- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
+- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
+- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
+- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
+- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.md.status

@@ -0,0 +1 @@
+ok

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.validate

@@ -0,0 +1,124 @@
+........................................................................ [  1%]
+........................................................................ [  3%]
+........................................................................ [  5%]
+........................................................................ [  7%]
+........................................................................ [  9%]
+........................................................................ [ 11%]
+........................................................................ [ 13%]
+........................................................................ [ 15%]
+........................................................................ [ 17%]
+........................................................................ [ 19%]
+........................................................................ [ 21%]
+........................................................................ [ 23%]
+........................................................................ [ 24%]
+........................................................................ [ 26%]
+........................................................................ [ 28%]
+........................................................................ [ 30%]
+........................................................................ [ 32%]
+........................................................................ [ 34%]
+........................................................................ [ 36%]
+........................................................................ [ 38%]
+........................................................................ [ 40%]
+........................................................................ [ 42%]
+........................................................................ [ 44%]
+........................................................................ [ 46%]
+........................................................................ [ 47%]
+........................................................................ [ 49%]
+........................................................................ [ 51%]
+........................................................................ [ 53%]
+........................................................................ [ 55%]
+........................................................................ [ 57%]
+........................................................................ [ 59%]
+........................................................................ [ 61%]
+........................................................................ [ 63%]
+........................................................................ [ 65%]
+........................................................................ [ 67%]
+........................................................................ [ 69%]
+........................................................................ [ 70%]
+........................................................................ [ 72%]
+........................................................................ [ 74%]
+........................................................................ [ 76%]
+........................................................................ [ 78%]
+........................................................................ [ 80%]
+........................................................................ [ 82%]
+........................................................................ [ 84%]
+........................................................................ [ 86%]
+........................................................................ [ 88%]
+........................................................................ [ 90%]
+........................................................................ [ 92%]
+........................................................................ [ 93%]
+........................................................................ [ 95%]
+........................................................................ [ 97%]
+........................................................................ [ 99%]
+............                                                             [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                           Stmts   Miss  Cover
+--------------------------------------------------
+app/__init__.py                    1      0   100%
+app/api/__init__.py                0      0   100%
+app/api/auth.py                   52      0   100%
+app/api/chat.py                  277      1    99%
+app/api/chat_images.py            50      0   100%
+app/api/chats.py                 110      0   100%
+app/api/config.py                 13      0   100%
+app/api/doc_drafts.py             99      0   100%
+app/api/docs.py                  196      1    99%
+app/api/git_sources.py           259      0   100%
+app/api/health.py                 10      0   100%
+app/api/steering.py               42      0   100%
+app/api/suggestions.py            33      0   100%
+app/api/sync.py                  204      0   100%
+app/api/tokens.py                 40      0   100%
+app/api/ui_settings.py            55      0   100%
+app/config.py                    266      0   100%
+app/core/__init__.py               0      0   100%
+app/core/auth.py                  45      0   100%
+app/core/caching.py              124      0   100%
+app/core/debugging.py             29      2    93%
+app/core/docs_push.py             39      0   100%
+app/core/errors.py                 5      0   100%
+app/core/logging.py               13      0   100%
+app/core/rate_limit.py            44      0   100%
+app/core/security_headers.py      20      0   100%
+app/core/theming.py               38      0   100%
+app/core/tokens.py                44      0   100%
+app/db.py                         22      0   100%
+app/main.py                       68      0   100%
+app/models.py                    135      0   100%
+app/rag/__init__.py                0      0   100%
+app/rag/agent.py                 454      1    99%
+app/rag/archive_upload.py        134      0   100%
+app/rag/chunker.py               220      4    98%
+app/rag/doc_dates.py              18      0   100%
+app/rag/folder_summaries.py      134      0   100%
+app/rag/git_sources.py            77      0   100%
+app/rag/importer.py              764      4    99%
+app/rag/llm.py                   246      1    99%
+app/rag/md_images.py              63      0   100%
+app/rag/overview.py               78      0   100%
+app/rag/pdf.py                   192      0   100%
+app/rag/prompts.py               156      0   100%
+app/rag/retriever.py             252      3    99%
+app/rag/scaffolding.py            55      0   100%
+app/rag/source_removal.py         41      0   100%
+app/rag/sources_meta.py           16      0   100%
+app/rag/suggestions.py            27      0   100%
+app/rag/summarizer.py             58      0   100%
+app/schemas.py                   470      0   100%
+--------------------------------------------------
+TOTAL                           5788     17    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
+Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
+
+validation OK

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.md

@@ -0,0 +1,13 @@
+All gates green. Final verification complete.
+
+**Report**
+
+- **Implemented (task 162/01, D1 / LOCKED A1):**
+  - `app/rag/retriever.py`: new module-level `_fold()` (stdlib NFKD + combining-class strip, shared with task 03's grep); `_normalize_name` = alnum scan of `_fold(s.lower())` — the single normalization both sides pass through; `name_hit_tokens`/`_name_parts`/`_name_token_matches` consumed it unchanged (docstring clauses only); module docstring normalization paragraph states the 2026-10-07 gap.
+  - `tests/unit/test_retriever.py`: +9 tests — `_fold` cases (Å/ÄÖÜ/ß/fi/ç, CJK, ①), `BESTÅ`→`besta` token pinned exactly (`["build", "besta"]`), `_name_parts` (`besta` in equal set unchanged; `BÅsta.md`→`basta`), end-to-end bridge both directions via `_name_hit_chunks`, min-length on the folded form, digitless-precision non-regression (`déploy` ≠ `Deployments`).
+- **Tests/lint/coverage:** `uv run pytest tests/unit/test_retriever.py tests/unit/test_retriever_recency.py -v` → 94 passed; full `uv run pytest` → **3747 passed**; `--cov=app` → 99% TOTAL, retriever 99% (3 pre-existing misses); `uv run ruff check .` + `uv run pyright` clean; phase-119 E2E `tests/e2e/test_name_signal_read_chips.py` → 4 passed (FTS/gate suites untouched).
+- **Docs:** README "How retrieval works" — lexical list now names the name-hit path + accent fold (phase 162). No AGENTS.md impact (its grep-contract line is task 03/final-pass scope).
+- **Decisions/deviations:**
+  1. NFKD alone leaves `ß` intact; the task-pinned `ß → ss` case requires `str.casefold()` — applied inside `_fold` (still stdlib-only, single shared function, A1 intent intact; documented).
+  2. `①` is compat-decomposable (NFKD → `1`), contrary to the task's "no decomposition" note: it still drops as a token (<4 chars); its only footprint is a prefix-class versioned-join artifact (`the1`) no path can match — pinned as real behavior and documented.
+- **Next pending task:** `162_search_recall_small_sources/02_kb_overview_source_inventory.md`.

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.md.status

@@ -0,0 +1 @@
+ok

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.validate

@@ -0,0 +1,124 @@
+........................................................................ [  1%]
+........................................................................ [  3%]
+........................................................................ [  5%]
+........................................................................ [  7%]
+........................................................................ [  9%]
+........................................................................ [ 11%]
+........................................................................ [ 13%]
+........................................................................ [ 15%]
+........................................................................ [ 17%]
+........................................................................ [ 19%]
+........................................................................ [ 21%]
+........................................................................ [ 23%]
+........................................................................ [ 24%]
+........................................................................ [ 26%]
+........................................................................ [ 28%]
+........................................................................ [ 30%]
+........................................................................ [ 32%]
+........................................................................ [ 34%]
+........................................................................ [ 36%]
+........................................................................ [ 38%]
+........................................................................ [ 40%]
+........................................................................ [ 42%]
+........................................................................ [ 44%]
+........................................................................ [ 46%]
+........................................................................ [ 48%]
+........................................................................ [ 49%]
+........................................................................ [ 51%]
+........................................................................ [ 53%]
+........................................................................ [ 55%]
+........................................................................ [ 57%]
+........................................................................ [ 59%]
+........................................................................ [ 61%]
+........................................................................ [ 63%]
+........................................................................ [ 65%]
+........................................................................ [ 67%]
+........................................................................ [ 69%]
+........................................................................ [ 71%]
+........................................................................ [ 73%]
+........................................................................ [ 74%]
+........................................................................ [ 76%]
+........................................................................ [ 78%]
+........................................................................ [ 80%]
+........................................................................ [ 82%]
+........................................................................ [ 84%]
+........................................................................ [ 86%]
+........................................................................ [ 88%]
+........................................................................ [ 90%]
+........................................................................ [ 92%]
+........................................................................ [ 94%]
+........................................................................ [ 96%]
+........................................................................ [ 97%]
+........................................................................ [ 99%]
+...                                                                      [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                           Stmts   Miss  Cover
+--------------------------------------------------
+app/__init__.py                    1      0   100%
+app/api/__init__.py                0      0   100%
+app/api/auth.py                   52      0   100%
+app/api/chat.py                  277      1    99%
+app/api/chat_images.py            50      0   100%
+app/api/chats.py                 110      0   100%
+app/api/config.py                 13      0   100%
+app/api/doc_drafts.py             99      0   100%
+app/api/docs.py                  196      1    99%
+app/api/git_sources.py           259      0   100%
+app/api/health.py                 10      0   100%
+app/api/steering.py               42      0   100%
+app/api/suggestions.py            33      0   100%
+app/api/sync.py                  204      0   100%
+app/api/tokens.py                 40      0   100%
+app/api/ui_settings.py            55      0   100%
+app/config.py                    266      0   100%
+app/core/__init__.py               0      0   100%
+app/core/auth.py                  45      0   100%
+app/core/caching.py              124      0   100%
+app/core/debugging.py             29      2    93%
+app/core/docs_push.py             39      0   100%
+app/core/errors.py                 5      0   100%
+app/core/logging.py               13      0   100%
+app/core/rate_limit.py            44      0   100%
+app/core/security_headers.py      20      0   100%
+app/core/theming.py               38      0   100%
+app/core/tokens.py                44      0   100%
+app/db.py                         22      0   100%
+app/main.py                       68      0   100%
+app/models.py                    135      0   100%
+app/rag/__init__.py                0      0   100%
+app/rag/agent.py                 454      1    99%
+app/rag/archive_upload.py        134      0   100%
+app/rag/chunker.py               220      4    98%
+app/rag/doc_dates.py              18      0   100%
+app/rag/folder_summaries.py      134      0   100%
+app/rag/git_sources.py            77      0   100%
+app/rag/importer.py              764      4    99%
+app/rag/llm.py                   246      1    99%
+app/rag/md_images.py              63      0   100%
+app/rag/overview.py               71      0   100%
+app/rag/pdf.py                   192      0   100%
+app/rag/prompts.py               155      0   100%
+app/rag/retriever.py             252      3    99%
+app/rag/scaffolding.py            55      0   100%
+app/rag/source_removal.py         41      0   100%
+app/rag/sources_meta.py           16      0   100%
+app/rag/suggestions.py            27      0   100%
+app/rag/summarizer.py             58      0   100%
+app/schemas.py                   470      0   100%
+--------------------------------------------------
+TOTAL                           5780     17    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
+Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
+
+validation OK

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.md

@@ -0,0 +1,17 @@
+All gates are green. Task 02 is complete.
+
+**Report — task 02 `02_kb_overview_source_inventory.md`**
+
+- **Implemented** (verified/completed in working tree — a prior interrupted run had left the code/tests uncommitted; I audited them against D2/A2, confirmed byte-identity of `build_overview_prompt`/`OVERVIEW_INSTRUCTION`/`load_kb_overview`/`build_kb_section`, and finished the doc work):
+  - `_source_inventory(rows)` in `app/rag/overview.py` — deterministic, non-LLM `Sources: <name> (<k> doc|docs), …`, distinct sources, catalog order, singular/plural.
+  - `regenerate_overview` happy path stores model text (rstripped `\n`) + `\n` + tail; log `chars=` = full stored content; zero-doc and `LLMError` paths untouched (previous row + tail preserved).
+  - Module + function docstrings record phase 162 / A2 / the `BOR_KB_OVERVIEW_MAX_CHARS` budget note.
+  - Unit tests in `tests/unit/test_overview.py`: format/order/counts, 1-doc survival (`ikea_instructions` case), exact happy-path stored content, tail-less zero-doc row preserved, `LLMError` preserves an existing tail, model input never gains the tail, exact char log; re-cut pins in `tests/integration/test_import_docs_overview.py`, `tests/e2e/test_kb_overview.py`, `tests/e2e/test_kb_overview_freshness.py`.
+- **Tests / lint / coverage** (DB container started — it was down):
+  - `uv run pytest tests/unit/test_overview.py -v` → 29 passed
+  - `uv run pytest --cov=app --cov-report=term-missing` → **3753 passed**, `app/` TOTAL **99%**, `app/rag/overview.py` **100%**
+  - `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed; `tests/e2e/test_kb_overview_freshness.py` → 3 passed
+  - `uv run ruff check .` → clean; `uv run pyright` → 0 errors
+- **Docs**: `README.md` — import section now documents the deterministic `Sources: …` tail (phase 162) and the failure-path preservation; `AGENTS.md` — no doc impact (the grep-contract line there is task 03/04's scope).
+- **Decisions**: inventory order via `sorted()` on source names = catalog order regardless of the recency-first row order the generator reads (proven by the scrambled-input test); no other e2e asserts exact overview-row content (checked).
+- **Next pending task**: `03_grep_folding_search_discipline.md`.

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.md.status

@@ -0,0 +1 @@
+ok

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.validate

@@ -0,0 +1,124 @@
+........................................................................ [  1%]
+........................................................................ [  3%]
+........................................................................ [  5%]
+........................................................................ [  7%]
+........................................................................ [  9%]
+........................................................................ [ 11%]
+........................................................................ [ 13%]
+........................................................................ [ 15%]
+........................................................................ [ 17%]
+........................................................................ [ 19%]
+........................................................................ [ 21%]
+........................................................................ [ 23%]
+........................................................................ [ 24%]
+........................................................................ [ 26%]
+........................................................................ [ 28%]
+........................................................................ [ 30%]
+........................................................................ [ 32%]
+........................................................................ [ 34%]
+........................................................................ [ 36%]
+........................................................................ [ 38%]
+........................................................................ [ 40%]
+........................................................................ [ 42%]
+........................................................................ [ 44%]
+........................................................................ [ 46%]
+........................................................................ [ 47%]
+........................................................................ [ 49%]
+........................................................................ [ 51%]
+........................................................................ [ 53%]
+........................................................................ [ 55%]
+........................................................................ [ 57%]
+........................................................................ [ 59%]
+........................................................................ [ 61%]
+........................................................................ [ 63%]
+........................................................................ [ 65%]
+........................................................................ [ 67%]
+........................................................................ [ 69%]
+........................................................................ [ 70%]
+........................................................................ [ 72%]
+........................................................................ [ 74%]
+........................................................................ [ 76%]
+........................................................................ [ 78%]
+........................................................................ [ 80%]
+........................................................................ [ 82%]
+........................................................................ [ 84%]
+........................................................................ [ 86%]
+........................................................................ [ 88%]
+........................................................................ [ 90%]
+........................................................................ [ 92%]
+........................................................................ [ 94%]
+........................................................................ [ 95%]
+........................................................................ [ 97%]
+........................................................................ [ 99%]
+.........                                                                [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                           Stmts   Miss  Cover
+--------------------------------------------------
+app/__init__.py                    1      0   100%
+app/api/__init__.py                0      0   100%
+app/api/auth.py                   52      0   100%
+app/api/chat.py                  277      1    99%
+app/api/chat_images.py            50      0   100%
+app/api/chats.py                 110      0   100%
+app/api/config.py                 13      0   100%
+app/api/doc_drafts.py             99      0   100%
+app/api/docs.py                  196      1    99%
+app/api/git_sources.py           259      0   100%
+app/api/health.py                 10      0   100%
+app/api/steering.py               42      0   100%
+app/api/suggestions.py            33      0   100%
+app/api/sync.py                  204      0   100%
+app/api/tokens.py                 40      0   100%
+app/api/ui_settings.py            55      0   100%
+app/config.py                    266      0   100%
+app/core/__init__.py               0      0   100%
+app/core/auth.py                  45      0   100%
+app/core/caching.py              124      0   100%
+app/core/debugging.py             29      2    93%
+app/core/docs_push.py             39      0   100%
+app/core/errors.py                 5      0   100%
+app/core/logging.py               13      0   100%
+app/core/rate_limit.py            44      0   100%
+app/core/security_headers.py      20      0   100%
+app/core/theming.py               38      0   100%
+app/core/tokens.py                44      0   100%
+app/db.py                         22      0   100%
+app/main.py                       68      0   100%
+app/models.py                    135      0   100%
+app/rag/__init__.py                0      0   100%
+app/rag/agent.py                 454      1    99%
+app/rag/archive_upload.py        134      0   100%
+app/rag/chunker.py               220      4    98%
+app/rag/doc_dates.py              18      0   100%
+app/rag/folder_summaries.py      134      0   100%
+app/rag/git_sources.py            77      0   100%
+app/rag/importer.py              764      4    99%
+app/rag/llm.py                   246      1    99%
+app/rag/md_images.py              63      0   100%
+app/rag/overview.py               78      0   100%
+app/rag/pdf.py                   192      0   100%
+app/rag/prompts.py               155      0   100%
+app/rag/retriever.py             252      3    99%
+app/rag/scaffolding.py            55      0   100%
+app/rag/source_removal.py         41      0   100%
+app/rag/sources_meta.py           16      0   100%
+app/rag/suggestions.py            27      0   100%
+app/rag/summarizer.py             58      0   100%
+app/schemas.py                   470      0   100%
+--------------------------------------------------
+TOTAL                           5787     17    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
+Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
+
+validation OK

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.md

@@ -0,0 +1,22 @@
+Task 03 complete. Report:
+
+**Implemented**
+- `app/rag/agent.py` — `grep_document` now folds **both** pattern and each line through the shared `_fold` imported from `app.rag.retriever` (one fold, no second copy; LOCKED A3): `_fold(pattern.lower()) in _fold(line.lower())`; rstrip/line-numbers/20-cap/200-char truncation/locator semantics untouched.
+- Copy updates (LOCKED A3): module-docstring grep paragraph, `AGENT_TOOLS` grep function description, and `pattern` parameter description now state "case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`"; `NO_MATCHES_REGEX`/`_SCOPED` + `looks_like_regex`/`plain_form` byte-identical.
+- `app/rag/prompts.py` — `TOOLS_SECTION` grep sentence folded-contract re-worded; new `SEARCH_DISCIPLINE_RULE` constant (LOCKED A4 sentence, verbatim) inserted immediately after the grep sentence; `PERSONA`/LOW/other sentences byte-identical.
+- Pins re-cut (phase-124 pattern): `test_prompt_lock.py` (new sha `988642f9…`/len 3818, pre-phase-162 anchors, chained phase-162→145 diff-style pin, discipline-rule position pin, grep-contract pin), `test_prompts.py` (grep clause + D4 boundary pins), `test_agent.py` (`AGENT_TOOLS` + pattern-param description pins).
+- New fold tests in `test_agent.py`: accented↔ASCII both ways (`besta`→`BESTÅ`, `basta`→`BÅsta`, `cafe`→`café`, `BESTÅ` pattern→ASCII content, `strasse`→`Straße`), non-match still `[]`.
+- `README.md` — grep tool table row updated to the folded contract.
+
+**Tests / lint / coverage**
+- `uv run pytest tests/unit/test_agent.py tests/unit/test_prompt_lock.py tests/unit/test_prompts.py` → **292 passed**
+- `uv run pytest` (full suite) → **3756 passed**
+- Related E2E spot-checks: `test_grep_regex_teaching.py`, `test_tool_call_compact.py`, `test_kb_overview*.py`, `test_name_signal_read_chips.py` → **green** (11 passed in the last batch)
+- `uv run pytest --cov=app` → TOTAL **99%** (`agent.py` 99%, `prompts.py` 100%) — >90% gate met
+- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
+
+**Docs**: `README.md` grep-tool row updated (case- and accent-insensitive, fold example); `AGENTS.md` has no grep-contract line to change (the phase-162 final pass handles the §6 wording per the phase overview).
+
+**Decisions**: reworked the phase-145 "only byte change" pin into a chained phase-162→145 revert pin so both historical anchors stay enforced; `SEARCH_DISCIPLINE_RULE` added as a module constant mirroring the `ORIGIN_RULE` pattern. No deviations from LOCKED A3/A4.
+
+**Next pending task**: `162_search_recall_small_sources/04_e2e_battery_pins.md`

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.md.status

@@ -0,0 +1 @@
+ok

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.validate

@@ -0,0 +1,124 @@
+........................................................................ [  1%]
+........................................................................ [  3%]
+........................................................................ [  5%]
+........................................................................ [  7%]
+........................................................................ [  9%]
+........................................................................ [ 11%]
+........................................................................ [ 13%]
+........................................................................ [ 15%]
+........................................................................ [ 17%]
+........................................................................ [ 19%]
+........................................................................ [ 21%]
+........................................................................ [ 23%]
+........................................................................ [ 24%]
+........................................................................ [ 26%]
+........................................................................ [ 28%]
+........................................................................ [ 30%]
+........................................................................ [ 32%]
+........................................................................ [ 34%]
+........................................................................ [ 36%]
+........................................................................ [ 38%]
+........................................................................ [ 40%]
+........................................................................ [ 42%]
+........................................................................ [ 44%]
+........................................................................ [ 46%]
+........................................................................ [ 47%]
+........................................................................ [ 49%]
+........................................................................ [ 51%]
+........................................................................ [ 53%]
+........................................................................ [ 55%]
+........................................................................ [ 57%]
+........................................................................ [ 59%]
+........................................................................ [ 61%]
+........................................................................ [ 63%]
+........................................................................ [ 65%]
+........................................................................ [ 67%]
+........................................................................ [ 69%]
+........................................................................ [ 70%]
+........................................................................ [ 72%]
+........................................................................ [ 74%]
+........................................................................ [ 76%]
+........................................................................ [ 78%]
+........................................................................ [ 80%]
+........................................................................ [ 82%]
+........................................................................ [ 84%]
+........................................................................ [ 86%]
+........................................................................ [ 88%]
+........................................................................ [ 90%]
+........................................................................ [ 92%]
+........................................................................ [ 93%]
+........................................................................ [ 95%]
+........................................................................ [ 97%]
+........................................................................ [ 99%]
+............                                                             [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                           Stmts   Miss  Cover
+--------------------------------------------------
+app/__init__.py                    1      0   100%
+app/api/__init__.py                0      0   100%
+app/api/auth.py                   52      0   100%
+app/api/chat.py                  277      1    99%
+app/api/chat_images.py            50      0   100%
+app/api/chats.py                 110      0   100%
+app/api/config.py                 13      0   100%
+app/api/doc_drafts.py             99      0   100%
+app/api/docs.py                  196      1    99%
+app/api/git_sources.py           259      0   100%
+app/api/health.py                 10      0   100%
+app/api/steering.py               42      0   100%
+app/api/suggestions.py            33      0   100%
+app/api/sync.py                  204      0   100%
+app/api/tokens.py                 40      0   100%
+app/api/ui_settings.py            55      0   100%
+app/config.py                    266      0   100%
+app/core/__init__.py               0      0   100%
+app/core/auth.py                  45      0   100%
+app/core/caching.py              124      0   100%
+app/core/debugging.py             29      2    93%
+app/core/docs_push.py             39      0   100%
+app/core/errors.py                 5      0   100%
+app/core/logging.py               13      0   100%
+app/core/rate_limit.py            44      0   100%
+app/core/security_headers.py      20      0   100%
+app/core/theming.py               38      0   100%
+app/core/tokens.py                44      0   100%
+app/db.py                         22      0   100%
+app/main.py                       68      0   100%
+app/models.py                    135      0   100%
+app/rag/__init__.py                0      0   100%
+app/rag/agent.py                 454      1    99%
+app/rag/archive_upload.py        134      0   100%
+app/rag/chunker.py               220      4    98%
+app/rag/doc_dates.py              18      0   100%
+app/rag/folder_summaries.py      134      0   100%
+app/rag/git_sources.py            77      0   100%
+app/rag/importer.py              764      4    99%
+app/rag/llm.py                   246      1    99%
+app/rag/md_images.py              63      0   100%
+app/rag/overview.py               78      0   100%
+app/rag/pdf.py                   192      0   100%
+app/rag/prompts.py               156      0   100%
+app/rag/retriever.py             252      3    99%
+app/rag/scaffolding.py            55      0   100%
+app/rag/source_removal.py         41      0   100%
+app/rag/sources_meta.py           16      0   100%
+app/rag/suggestions.py            27      0   100%
+app/rag/summarizer.py             58      0   100%
+app/schemas.py                   470      0   100%
+--------------------------------------------------
+TOTAL                           5788     17    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
+Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
+
+validation OK

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.md

@@ -0,0 +1,18 @@
+**Phase 162 / task 04 — complete, all gates green.**
+
+**Added** `tests/e2e/test_search_recall_small_sources.py` (7 tests), fixture KB `tests/fixtures/recallkb/` (3 sources: `ikea_instructions` 1 doc with the name **only** as `BESTÅ`, `phase_docs` 4 + `players_handbook` 1 ASCII-`besta`/`build` distractors), the Besta pair + gate comment in `tests/fixtures/retrieval_battery.txt`, and evidence in `.agents/reports/162_search_recall_small_sources/task04_prefix_evidence_and_battery.md`.
+
+**Commands / outcomes**
+- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated, run twice incl. after the DB restore).
+- `uv run pytest tests/e2e/test_name_signal_read_chips.py tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
+- `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, 3756 tests, app coverage 99%**.
+- `uv run ruff check .` → passed; `uv run pyright` → **0 errors**.
+- Replica restored per the `restore-test-db` skill (fingerprint verified docs=1000/chunks=8866/sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic 0011→0026), then `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` → **GATE PASS 1–5**: gitea README rank 4, forgejo 5/5 with README #1, gateway README rank 3, qwen3.8-27b quadlets #1/#2, Mongolia HIGH/fts_hits=5 — the five `suggested:` lists are **byte-identical** to the phase-119 record, and a re-run with the fold reverted is **byte-identical** to the post-fix run.
+
+**Pre-fix evidence (measured, both directions).** With `app/rag/retriever.py::_normalize_name` reverted by a one-line reversible edit, `How do I build the BESTÅ?` loses the doc: tokens `['build','best']` → no name hit, suggested 5 = the five ASCII distractors, IKEA doc **rank 6** — visible on the wire as its `done.related` entry and its absence from the folder-map echo (gate still HIGH, cosine 0.382, on the wrong docs). Shipped code: tokens `['build','besta']` → IKEA doc **suggested #1**, related = `phase_docs/156_history_source_filter.md`, cosine unchanged (0.382). The hypothesis is now re-measured on every run by `test_under_rank_is_measured_not_assumed`.
+
+**Decisions / deviations** (harness mechanics win): task/phase files untouched, so the evidence lives in the report file above; no `git stash` (reversible source revert instead, verified restored, `grep -c TEMP` = 0); `done.suggestions` is the deflection-chip contract and never carries paths, so the suggested tier is asserted via the durable `query_log.sources` record (suggested 5 + remainder), `done.related`, and the mock's folder-map echo; tuning re-worded two bodies to drop stopword density so the out-of-KB control deflects (cosine 0.3345 → 0.2083) — both token controls are pinned as assertions; the battery carries both `How do I build a Besta?` (task text) and `How do I build the BESTÅ?` (the live miss).
+
+**Skipped loudly:** the 2026-09-05 replica snapshot has no `ikea_instructions`-class source (`grep -c -i ikea` on the dump = 0), so the Besta rank gate stays with the owner's live KB; both questions still ran without a false grounding on a doc naming the product.
+
+**Docs:** no README/AGENTS change needed (task 04 is tests + fixtures + battery; user-visible behavior was documented by tasks 01–03). DB left as found (pre-restore backup restored); no other pending task for this phase — task 04's file is ready for the harness to move, and the phase commit is still owed.

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.md.status

@@ -0,0 +1 @@
+ok

.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.validate

@@ -0,0 +1,124 @@
+........................................................................ [  1%]
+........................................................................ [  3%]
+........................................................................ [  5%]
+........................................................................ [  7%]
+........................................................................ [  9%]
+........................................................................ [ 11%]
+........................................................................ [ 13%]
+........................................................................ [ 15%]
+........................................................................ [ 17%]
+........................................................................ [ 19%]
+........................................................................ [ 21%]
+........................................................................ [ 23%]
+........................................................................ [ 24%]
+........................................................................ [ 26%]
+........................................................................ [ 28%]
+........................................................................ [ 30%]
+........................................................................ [ 32%]
+........................................................................ [ 34%]
+........................................................................ [ 36%]
+........................................................................ [ 38%]
+........................................................................ [ 40%]
+........................................................................ [ 42%]
+........................................................................ [ 44%]
+........................................................................ [ 46%]
+........................................................................ [ 47%]
+........................................................................ [ 49%]
+........................................................................ [ 51%]
+........................................................................ [ 53%]
+........................................................................ [ 55%]
+........................................................................ [ 57%]
+........................................................................ [ 59%]
+........................................................................ [ 61%]
+........................................................................ [ 63%]
+........................................................................ [ 65%]
+........................................................................ [ 67%]
+........................................................................ [ 69%]
+........................................................................ [ 70%]
+........................................................................ [ 72%]
+........................................................................ [ 74%]
+........................................................................ [ 76%]
+........................................................................ [ 78%]
+........................................................................ [ 80%]
+........................................................................ [ 82%]
+........................................................................ [ 84%]
+........................................................................ [ 86%]
+........................................................................ [ 88%]
+........................................................................ [ 90%]
+........................................................................ [ 92%]
+........................................................................ [ 93%]
+........................................................................ [ 95%]
+........................................................................ [ 97%]
+........................................................................ [ 99%]
+............                                                             [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                           Stmts   Miss  Cover
+--------------------------------------------------
+app/__init__.py                    1      0   100%
+app/api/__init__.py                0      0   100%
+app/api/auth.py                   52      0   100%
+app/api/chat.py                  277      1    99%
+app/api/chat_images.py            50      0   100%
+app/api/chats.py                 110      0   100%
+app/api/config.py                 13      0   100%
+app/api/doc_drafts.py             99      0   100%
+app/api/docs.py                  196      1    99%
+app/api/git_sources.py           259      0   100%
+app/api/health.py                 10      0   100%
+app/api/steering.py               42      0   100%
+app/api/suggestions.py            33      0   100%
+app/api/sync.py                  204      0   100%
+app/api/tokens.py                 40      0   100%
+app/api/ui_settings.py            55      0   100%
+app/config.py                    266      0   100%
+app/core/__init__.py               0      0   100%
+app/core/auth.py                  45      0   100%
+app/core/caching.py              124      0   100%
+app/core/debugging.py             29      2    93%
+app/core/docs_push.py             39      0   100%
+app/core/errors.py                 5      0   100%
+app/core/logging.py               13      0   100%
+app/core/rate_limit.py            44      0   100%
+app/core/security_headers.py      20      0   100%
+app/core/theming.py               38      0   100%
+app/core/tokens.py                44      0   100%
+app/db.py                         22      0   100%
+app/main.py                       68      0   100%
+app/models.py                    135      0   100%
+app/rag/__init__.py                0      0   100%
+app/rag/agent.py                 454      1    99%
+app/rag/archive_upload.py        134      0   100%
+app/rag/chunker.py               220      4    98%
+app/rag/doc_dates.py              18      0   100%
+app/rag/folder_summaries.py      134      0   100%
+app/rag/git_sources.py            77      0   100%
+app/rag/importer.py              764      4    99%
+app/rag/llm.py                   246      1    99%
+app/rag/md_images.py              63      0   100%
+app/rag/overview.py               78      0   100%
+app/rag/pdf.py                   192      0   100%
+app/rag/prompts.py               156      0   100%
+app/rag/retriever.py             252      3    99%
+app/rag/scaffolding.py            55      0   100%
+app/rag/source_removal.py         41      0   100%
+app/rag/sources_meta.py           16      0   100%
+app/rag/suggestions.py            27      0   100%
+app/rag/summarizer.py             58      0   100%
+app/schemas.py                   470      0   100%
+--------------------------------------------------
+TOTAL                           5788     17    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
+Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
+
+validation OK

.agents/reports/162_search_recall_small_sources/task04_prefix_evidence_and_battery.md

@@ -0,0 +1,174 @@
+# Task 04 record — pre-fix evidence + battery (phase 162, D5/D6)
+
+**Deviation note (mechanics):** the task file says "record the measured
+pre-fix ranks in THIS file" and to revert the fold with `git stash`, but
+the execution harness forbids editing phase/task files and forbids
+index/history-touching Git commands. Per the conflict rule the harness
+wins on mechanics and the intent is preserved: the evidence lives HERE,
+the fold was reverted by a one-line, reversible edit of
+`app/rag/retriever.py` (restored and byte-verified afterwards —
+`grep -c TEMP app/rag/retriever.py` = 0), and the summary goes in the
+task's final report. The permanent, non-throwaway form of the evidence is
+`tests/e2e/test_search_recall_small_sources.py::test_under_rank_is_measured_not_assumed`,
+which re-measures BOTH directions on every run.
+
+## 1. The fixture (tests/fixtures/recallkb — three local-directory sources)
+
+| source | docs | role |
+| --- | --- | --- |
+| `ikea_instructions` | **1** (`besta-frame-white__AA-1272068-11-100.md`) | the live miss: body names the product **only** as accented `BESTÅ` |
+| `phase_docs` | 4 (`140_pictorial_pdf_pages`, `148_pdf_text_backfill`, `152_seaweedfs_blob_ingest`, `156_history_source_filter`) | ASCII `besta` + `build` distractors |
+| `players_handbook` | 1 (`game_night_setup.md`) | ASCII `besta` + `build` distractor in a second small source |
+
+Final shape tuning (the task's "add distractors until it loses"): the
+initial 4-doc draft already lost the tier, but the deflection control
+failed (best cosine 0.3345 — the small source's `Assembly` section was
+dense in the stopword `the`, which the mock's tokenizer keeps). Two
+bodies were re-worded (`ikea_instructions` Assembly paragraph,
+`players_handbook/game_night_setup.md`) to drop stopword density: the
+out-of-KB cosine fell to **0.2083** (< 0.30 threshold → deflects) while
+the on-topic questions kept 0.38 / 0.44. Two token controls are pinned
+as assertions in `synced_kb`: (a) no ASCII `besta` and no `build` anywhere
+in the small source, (b) the accented name sits **past** the mock's
+24-token summary-digest window, so the ASCII-only digest never leaks a
+`best` lexeme the pre-phase FTS could have hit.
+
+## 2. Measured pre-fix evidence — the E2E fixture, both directions
+
+Lens: the app's REAL `retrieve()` + `select_suggested()` over the synced
+KB with the mock's embeddings. Gate: threshold 0.30, lexical-support
+floor 0.15 (the conftest mock calibration). Bonus 0.005, recency 0.0007,
+suggested 5 / related 2 (code defaults, `.env`-free like production).
+
+Question `How do I build the BESTÅ?`
+
+**PRE-FIX — the real revert** (`_normalize_name` →
+`"".join(_ALNUM_RE.findall(s.lower()))`, the exact task-01 before-state),
+`pytest tests/e2e/test_search_recall_small_sources.py` run against the
+fixture (full log: `/tmp/prefix_e2e_run.log`, `/tmp/prefix_wire_run.log`):
+
+```
+name tokens : ['build', 'best']                  → NO name hit
+suggested 5 : phase_docs/140_pictorial_pdf_pages.md
+              phase_docs/148_pdf_text_backfill.md
+              phase_docs/152_seaweedfs_blob_ingest.md
+              players_handbook/game_night_setup.md
+              phase_docs/156_history_source_filter.md
+related     : ikea_instructions/besta-frame-white__AA-1272068-11-100.md   ← RANK 6, the miss
+gate        : best cosine 0.382, fts_hits 8 → grounded (on the WRONG docs)
+done frame  : {'source': 'ikea_instructions',
+               'path': 'besta-frame-white__AA-1272068-11-100.md',
+               'title': 'Assembly Instructions'}   ← the rank-6+ "Nearby docs" slot
+folder echo : (folders: phase_docs/: 148_…, 152_…, 156_…; players_handbook/:)
+              → NO `ikea_instructions/` line at all: the small source never
+                reached the HIGH prompt
+```
+
+**POST-FIX (shipped code)** — same fixture, same measurement:
+
+```
+name tokens : ['build', 'besta']                 → name hit
+suggested 5 : ikea_instructions/besta-frame-white__AA-1272068-11-100.md  ← #1
+              phase_docs/140_pictorial_pdf_pages.md
+              phase_docs/148_pdf_text_backfill.md
+              phase_docs/152_seaweedfs_blob_ingest.md
+              players_handbook/game_night_setup.md
+related     : phase_docs/156_history_source_filter.md
+gate        : best cosine 0.382, fts_hits 9 (the fold adds the LEXICAL term;
+              cosine untouched — LOCKED A1)
+folder echo : … includes `ikea_instructions/:` (header alone — its only
+              document IS the suggested doc)
+```
+
+Mechanism (measured, and why the fix is the name signal and nothing
+else): every distractor matches the question's other FTS token `build`,
+so each fuses TWO RRF terms (vector + FTS ≈ 0.032) while the small source
+— no `build`, and `BESTÅ` lexes as `bestå`, which the ASCII-`best`
+tsquery cannot match — fuses ONE (≈ 0.016) and lands rank 6. The folded
+name hit is the only signal big enough to lift it: name hits LEAD the
+lexical list (`fuse` rank 1 = 1/61) plus the per-document
+`BOR_NAME_HIT_BONUS` (0.005). ASCII question `How do I build a Besta?`
+measures the SAME tier before and after (suggested #1, cosine 0.4404,
+related `phase_docs/156_history_source_filter.md`) — the phase-119 path
+is untouched.
+
+Under-rank hypothesis (D1 failure #4): **pinned green with measured
+ranks**, not assumed.
+
+## 3. The dedicated suite
+
+`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`
+→ **7 passed** (task-04 cases 1–4 plus the measurement test, the grep
+fold at the content layer, and the folder-line prompt witness).
+
+Interpretation note: the task text says the `done`-frame `suggestions`
+list is the observable tier. On the shipped wire `done.suggestions` is
+the DEFLECTION contract (2–3 title-derived "Maybe try" chips, `[]` on
+grounded turns — `app/api/chat.py`, phase 04) and never carries document
+paths. The faithful observables of the SUGGESTED tier are (a) the durable
+`query_log.sources` record = suggested 5 then the rank-6+ remainder
+(phase 118 A3) and (b) `done.related` = the rank-6+ tier. The suite
+asserts both (plus `done.sources == []` — read-docs-only, phase 119
+LOCKED A1 — and the mock's folder-map echo for the prompt-level witness).
+
+Neighbour suites re-run (the retriever + overview were touched by this
+phase): `uv run pytest tests/e2e/test_name_signal_read_chips.py
+tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
+
+## 4. Battery — `tests/fixtures/retrieval_battery.txt`
+
+The file gains the Besta pair (`How do I build a Besta?` — the task's
+line — and `How do I build the BESTÅ?`, the live miss itself) with the
+gate comment. The five phase-119 questions keep positions 1–5.
+
+Preconditions (the `restore-test-db` skill, run in-container): safety
+backup → `DROP SCHEMA public CASCADE` → single-transaction load of
+`data/bor_live_kb_replica.dump.sql` → **fingerprint verified:
+docs=1000, chunks=8866,
+sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic=0011**
+→ `uv run alembic upgrade head` (0011 → **0026**) → docs=1000 re-verified.
+
+`uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt`
+(full output `/tmp/battery_postfix.log`, pre-fix comparison
+`/tmp/battery_prefix.log`):
+
+| # | question | verdict |
+| --- | --- | --- |
+| 1 | How do I deploy gitea? | HIGH, `fts_hits=38`; suggested top-5 **byte-identical** to the phase-119 record — `gitea/README.md` **rank 4** ✓ |
+| 2 | How do I deploy forgejo? | HIGH, `fts_hits=37`; forgejo tree 5/5, `forgejo/README.md` **#1** ✓ |
+| 3 | How do I set up the gateway? | HIGH, `fts_hits=37`; `gateway/README.md` **rank 3** ✓ |
+| 4 | Do you have notes on the qwen 3.8 27b model? | HIGH, `fts_hits=31`; both `qwen3.8-27b-*` quadlets at #1/#2 ✓ (the digit-prefix regression holds) |
+| 5 | What is the capital of Mongolia? | **HIGH, `fts_hits=5`** — unchanged ✓ |
+| 6 | How do I build a Besta? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
+| 7 | How do I build the BESTÅ? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
+
+`SELECT DISTINCT source FROM documents` on the restored replica lists
+`deploy, homelab, homepage, ServMon, skills, vibe-bot` — NO
+`ikea_instructions`-class source (`grep -c -i ikea
+data/bor_live_kb_replica.dump.sql` = 0: the 2026-09-05 snapshot predates
+the IKEA import, exactly the case D5 anticipated; phase-140's
+"skipped loudly" precedent). The two questions still ran and did not
+crash; both return the `build.sh`/`bifrost`-class top-5 that a KB without
+the product can only return, and neither grounds on a document that names
+the product. **The rank gate therefore stays with the owner** — it can
+only close on a KB that has the source (the live KB).
+
+Behavior-preservation, measured (not argued):
+
+* the five phase-119 `suggested:` lists are **byte-identical** to the
+  recorded 2026-09-16 run in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md`;
+* the battery re-run with the task-01 fold REVERTED is **byte-identical**
+  to the post-fix run (`diff` of the question/gate/suggested lines: no
+  difference) — the fold cannot move an ASCII-only corpus, as designed;
+* `best_cosine` moves by ≤ 0.0004 between the September record and today
+  (real-`embed` model jitter, not code).
+
+**Verdict: GATE PASS** for questions 1–5 (unchanged), 6–7 recorded as
+skipped loudly with the reason and the re-check command.
+
+## 5. Environment left behind
+
+The replica was restored only for the battery. The pre-restore DB state
+(backup `/tmp/pre_restore_host.dump`, 13 leftover E2E fixture docs,
+alembic 0026) is restored afterwards — reload the replica any time with
+the `restore-test-db` skill.