phase: 162_search_recall_small_sources

**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).

app/rag/retriever.py

@@ -28,9 +28,21 @@
   order ``(source, path)`` — the old total-matched-length tie-break
   is retired, it outranked 5-char product names by 6-char common
   tokens; capped at :data:`NAME_HIT_LIMIT`),
-  the FTS rows follow. This is what finds name-your-tool questions
-  ("gitlab") that vector similarity buries — and, the 2026-09-05
-  incident, the versioned-name case the default parser lexes
+  the FTS rows follow. NAME NORMALIZATION IS ACCENT-FOLDED (phase 162,
+  LOCKED A1): :func:`_normalize_name` runs :func:`_fold` (NFKD
+  decomposition, combining marks dropped — stdlib only) before the
+  alnum scan, and it is the SINGLE normalization both sides of name
+  matching pass through (question tokens and path parts alike) — the
+  2026-10-07 live gap this closes: the IKEA filename is ASCII
+  (``besta-frame-white__AA-1272068-11-100``) while its content layer
+  spells the name only as ``BESTÅ`` (U+00C5), and the old lower-only
+  normalization turned the accented question token into ``best`` —
+  never a match. With the fold, ``BESTÅ`` → ``besta`` (was ``best``)
+  and an accented filename (``BÅsta.md`` → ``basta``) answers to the
+  ASCII question — one function, both directions. This is what finds
+  name-your-tool questions ("gitlab") that vector similarity buries —
+  and, the 2026-09-05 incident, the versioned-name case the default
+  parser lexes
   incompatibly (question "Qwen 3.8" → ``qwen``/``3``/``8`` can never
   match a document's ``qwen3``/``8``/``27b`` tokens, while every
   unrelated llama.cpp quadlet out-ranks the target on the shared
@@ -81,6 +93,7 @@ from __future__ import annotations
 
 import math
 import re
+import unicodedata
 import uuid
 from collections.abc import Sequence
 from dataclasses import dataclass, replace
@@ -130,9 +143,42 @@ _ALNUM_RE = re.compile(r"[a-z0-9]+")
 _STEM_SPLIT_RE = re.compile(r"[^a-z0-9]+")
 
 
+def _fold(s: str) -> str:
+    """The shared accent fold (phase 162, LOCKED A1): NFKD decomposition,
+    combining marks dropped (stdlib ``unicodedata`` + ``str.casefold``
+    only — no new dependency).
+
+    ``Å/å → a``, ``Ä/Ö/Ü → a/o/u``, ``ß → ss``, ``fi → fi``, ``ç → c``
+    (``casefold`` is the Unicode fold — the ``ß → ss`` case rides it:
+    NFKD alone leaves ``ß`` intact — and ``str.lower`` leaves the ``fi``
+    ligature). Characters with no decomposition (CJK) survive the fold
+    and still drop out under the caller's alnum scan (behavior
+    unchanged for them); compatibility-decomposable characters (``①`` →
+    ``1``) fold to their ASCII base, which :data:`NAME_TOKEN_MIN_LEN`
+    already excludes as a name token (a digit base can still feed the
+    versioned join as the numeric NEXT token — the only route left is a
+    prefix-class candidate no real path matches). Case is folded too
+    (the caller lowercases anyway). Imported by ``app.rag.agent`` for
+    the folded ``grep`` (phase 162, LOCKED A3) — one fold, never
+    drifting apart.
+    """
+    decomposed = unicodedata.normalize("NFKD", s.casefold())
+    return "".join(c for c in decomposed if not unicodedata.combining(c))
+
+
 def _normalize_name(s: str) -> str:
-    """Lowercased, alnum-only form of *s* (``Qwen 3.8`` → ``qwen38``)."""
-    return "".join(_ALNUM_RE.findall(s.lower()))
+    """Lowercased, ACCENT-FOLDED, alnum-only form of *s* (``Qwen 3.8`` →
+    ``qwen38`` unchanged; ``BESTÅ`` → ``besta`` — the phase-162 fold).
+
+    This is the SINGLE normalization both sides of name matching pass
+    through — the :func:`name_hit_tokens` question tokens and the
+    :func:`_name_parts` path parts / file stem / stem sub-components —
+    so the one change bridges both directions (the ASCII question
+    ``besta`` keeps matching the ASCII filename, the accented question
+    ``BESTÅ`` now matches it too, and an accented filename ``BÅsta.md``
+    answers to the ASCII question ``basta``).
+    """
+    return "".join(_ALNUM_RE.findall(_fold(s.lower())))
 
 
 def name_hit_tokens(question: str) -> list[str]:
@@ -166,7 +212,10 @@ def name_hit_tokens(question: str) -> list[str]:
     Order of first appearance, de-duplicated. Matching applies the
     A2 rule: digit-bearing candidates prefix a normalized path part
     or file stem; digitless candidates equal a part, stem, or stem
-    sub-component — titles are never matched.
+    sub-component — titles are never matched. Candidates are
+    ACCENT-FOLDED through :func:`_normalize_name` (phase 162) — the
+    token ``besta`` drawn from ``BESTÅ`` name-matches ``BESTÅ``-class
+    names both ways.
     """
     tokens = question.split()
     out: list[str] = []
@@ -456,6 +505,10 @@ def _name_parts(path: str) -> tuple[set[str], set[str]]:
       (the stem lowercased, split on non-alphanumeric runs, each piece
       normalized, empties dropped: ``kubernetes_gitea`` →
       ``kubernetes`` / ``gitea``): DIGITLESS tokens exact-match these.
+
+    The parts are ACCENT-FOLDED through :func:`_normalize_name` (phase
+    162) — ``besta`` name-matches ``BESTÅ``-class names both ways (the
+    accented stem ``BÅsta`` enters the equal set as ``basta``).
     """
     p = Path(path)
     prefix = {_normalize_name(part) for part in p.parts}
@@ -479,6 +532,10 @@ def _name_token_matches(token: str, prefix: set[str], equal: set[str]) -> bool:
     * the token HAS NO DIGIT → it EQUALS a normalized path part, file
       stem, or stem sub-component (``gitea`` → the ``gitea/`` folder,
       ``gitea.md``, ``kubernetes_gitea``, ``gitea-values``).
+
+    Both the token and the path sets arrive ACCENT-FOLDED from
+    :func:`_normalize_name` (phase 162) — ``besta`` matches
+    ``BESTÅ``-class names both ways.
     """
     if any(ch.isdigit() for ch in token):
         return any(part.startswith(token) for part in prefix)