phase: 162_search_recall_small_sources

**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).

tests/fixtures/recallkb/ikea_instructions/besta-frame-white__AA-1272068-11-100.md

@@ -0,0 +1,25 @@
+# Assembly Instructions
+
+Article number 127.206.11 — revision 4
+
+Cam bolt 130618 · wall bracket 141413 · leveling foot 303154 · dowel 518041
+
+## Package contents
+
+- 1 x frame, white, 60 x 20 x 60 cm
+- 2 x side panels
+- 1 x back panel, hardboard
+- 12 x cam bolts with sleeve nuts
+- 1 bag of dowels
+
+## Assembly
+
+Insert sleeve nuts into the drilled holes, draw both side panels against the
+frame, tighten each cam bolt a quarter turn. Attach back panel with nails
+supplied, keeping it square and level. Fix wall bracket using plugs matched to
+wall material — furniture of this depth must always be secured.
+
+## Product name
+
+BESTÅ frame, white, 60 x 20 x 60 cm, AA 1272068 11 100. Wipe with a soft
+cloth dampened in water or a mild cleaner. Never soak the edges.

tests/fixtures/recallkb/phase_docs/140_pictorial_pdf_pages.md

@@ -0,0 +1,11 @@
+# Pictorial PDF pages
+
+How do I build a retrievable text layer for a scanned manual? Build one page
+image per PDF page, build an image description for each page, then build the
+index from those descriptions. The ASCII besta assembly PDF was the first
+manual to go through this path.
+
+- build page images at 1400 px
+- build one description per page image
+- build chunks from descriptions, never from pixel guesses
+- besta manual: 24 scanned pages, 24 descriptions

tests/fixtures/recallkb/phase_docs/148_pdf_text_backfill.md

@@ -0,0 +1,9 @@
+# PDF text backfill
+
+Backfill plan for scanned manuals. A manual that arrives without a text layer
+needs one before it can answer anything: run the pictorial page path, build
+descriptions for every page, keep the original page order.
+
+- build: render, describe, index
+- besta shelf PDF: 24 scanned pages, no embedded text
+- question asked twice in review: how do I build this layer?

tests/fixtures/recallkb/phase_docs/152_seaweedfs_blob_ingest.md

@@ -0,0 +1,8 @@
+# SeaweedFS blob ingest
+
+Object-store notes. Mount first, upload second.
+
+- mount fs://manuals before the first upload
+- build the volume list, then copy the scan batch
+- besta scan batch: fs://manuals/2026-09/besta-scans.tar
+- checksum every archive after the copy

tests/fixtures/recallkb/phase_docs/156_history_source_filter.md

@@ -0,0 +1,8 @@
+# History source filter
+
+Turn history keeps the source each answer came from, so a filter can narrow
+the log to one source at a time.
+
+- filter chip per source, sourced from turn history
+- a build log entry records the source it indexed
+- rerunning a question with a filter never rewrites history

tests/fixtures/recallkb/players_handbook/game_night_setup.md

@@ -0,0 +1,7 @@
+# Game night setup
+
+Room notes, games side of the shelf wall.
+
+- besta unit holds box library; top shelf for dice trays
+- build tray insert before next session
+- keep rules binder beside clock, never stacked on besta top

tests/fixtures/retrieval_battery.txt

@@ -1,4 +1,6 @@
-# Phase 119 retrieval battery (task 06, D6) — the 1,000-doc live replica.
+# Retrieval battery — phase 119 (task 06, D6) + phase 162 (task 04, D5),
+# run against the 1,000-doc live replica:
+#   uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt
 #
 # Gate (per the phase overview, post-phase measured numbers):
 #   1-3. each name question's suggested top-5 contains its canonical doc:
@@ -10,8 +12,32 @@
 #   5. deflection sanity: the out-of-KB question's gate verdict + fts
 #      count match the pre-phase replica run (HIGH, fts_hits=5) — the
 #      name-hit change must not move it.
+#
+# Phase 162 appends the Besta pair (the 2026-10-07 live miss). Gate: the
+# suggested top-5 must contain the IKEA doc — ASCII filename
+# `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`, body that
+# spells the product `BESTÅ` only — for BOTH spellings: the accented one
+# needs the phase-162 question-side fold (D1) to bridge ASCII filename ←
+# accented question, the ASCII one is the phase-119 path (regression).
+# Either way ASCII `besta` documents must not outrank the doc that names
+# the product.
+#
+#   SKIPPED LOUDLY on the local replica (the phase-140 precedent): the
+#   2026-09-05 snapshot carries NO `ikea_instructions`-class source —
+#   `grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0, and
+#   `SELECT DISTINCT source FROM documents` after the restore lists the
+#   sources without it (verify, don't assume). The two questions still
+#   run and must behave like any out-of-KB ask (no crash, no false
+#   grounding on a doc that never named the product); the gate on the
+#   rank itself can only close on a KB that has the source (owner's live
+#   KB). The fixture that DOES reproduce the miss end-to-end is
+#   tests/e2e/test_search_recall_small_sources.py over
+#   tests/fixtures/recallkb (pre-fix ranks measured there, both
+#   directions).
 How do I deploy gitea?
 How do I deploy forgejo?
 How do I set up the gateway?
 Do you have notes on the qwen 3.8 27b model?
 What is the capital of Mongolia?
+How do I build a Besta?
+How do I build the BESTÅ?