phase: 162_search_recall_small_sources

**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).

app/rag/overview.py

@@ -28,6 +28,21 @@ an import added/updated at least one document (or no row exists yet),
 and a ``lite`` failure logs and leaves the previous outline intact — an
 old outline is better than none (phase locked decisions).
 
+After the model's outline, :func:`regenerate_overview` appends ONE
+deterministic, non-LLM line — ``Sources: <name> (<k> doc|docs), …``
+(phase 162, LOCKED A2). The 2026-10-07 live finding: the ``lite``
+outline is lossy — it listed 3 groups of a 14-source index, so the
+1-doc sources (``ikea_instructions``, ``players_handbook``) were
+invisible to ``<knowledge_base>``. Small sources must be visible
+without a model call, so the line is computed mechanically from the
+catalogue rows (:func:`_source_inventory`) — the mock/real model
+outputs never affect it, and ``build_overview_prompt`` never sees the
+tail (the model input stays byte-identical). The longest-prefix cap in
+``build_kb_section`` applies to the WHOLE stored text: at the default
+``BOR_KB_OVERVIEW_MAX_CHARS`` (4000) the ~1500-char outline plus the
+one-line tail (~15 chars/source) always fit; a smaller owner-set cap
+cuts the tail first — accepted, the cap is owner-tunable.
+
 The ``KB_OVERVIEW_MODE`` marker follows the ``SUMMARY_MODE`` /
 ``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on it
 in the system prompt (wired in phase 31, task 05).
@@ -101,6 +116,35 @@ def _first_summary_line(summary: str | None) -> str:
     return ""
 
 
+def _source_inventory(rows: Sequence[tuple[str, str, str, str | None]]) -> str:
+    """The phase-162 deterministic source inventory (LOCKED A2).
+
+    Mechanical, non-LLM: one entry per **distinct** source in catalog
+    order (``source, path`` — first occurrence in the ``source,
+    path``-ordered rows), each ``<name> (<k> doc|docs)`` — singular
+    ``doc`` at k=1, ``docs`` otherwise — joined with ``, `` behind the
+    ``Sources: `` prefix. The ``lite`` outline stays the model's job;
+    this line guarantees no source — not even a 1-doc one like the
+    2026-10-07 ``ikea_instructions`` case — can be invisible to
+    ``<knowledge_base>``, whatever the model outputs. The mock/real
+    model outputs never affect it.
+
+    *rows* is the existing ``(source, path, title, summary)``
+    projection :func:`regenerate_overview` already reads — no new
+    query. At the default ``BOR_KB_OVERVIEW_MAX_CHARS`` (4000) the
+    outline plus this one line always fit; a smaller owner-set cap
+    cuts the tail first (accepted — the cap is owner-tunable).
+    """
+    counts: dict[str, int] = {}
+    for source, _path, _title, _summary in rows:
+        counts[source] = counts.get(source, 0) + 1
+    entries = ", ".join(
+        f"{name} ({count} {'doc' if count == 1 else 'docs'})"
+        for name, count in sorted(counts.items())
+    )
+    return f"Sources: {entries}"
+
+
 def build_overview_prompt(
     rows: Sequence[tuple[str, str, str, str | None]], max_chars: int | None = None
 ) -> tuple[str, str]:
@@ -166,10 +210,16 @@ async def regenerate_overview(
 
     * Zero documents → the existing row (if any) is left untouched and
       ``False`` is returned — no KB, no outline, no wasted model call.
-    * Happy path → the model's text is upserted into the single row
-      (``id = 1``, fresh UTC ``updated_at``), committed, and
-      ``overview: regenerated docs=… chars=…`` is logged; returns
-      ``True``. Idempotent by construction: one row, always ``id = 1``.
+    * Happy path → the model's text (trailing newlines rstripped) plus
+      the deterministic ``Sources: …`` inventory line (phase 162,
+      LOCKED A2 — :func:`_source_inventory`) is upserted into the
+      single row (``id = 1``, fresh UTC ``updated_at``), committed, and
+      ``overview: regenerated docs=… chars=…`` is logged (chars = the
+      FULL stored content, tail included); returns ``True``. Idempotent
+      by construction: one row, always ``id = 1``. The model input
+      never sees the tail; the zero-doc and ``LLMError`` paths below
+      leave a previous row byte-identical — an existing tail is part of
+      the stored content and is preserved on failure.
     * :class:`LLMError` → ``overview: regeneration failed — …`` is
       logged and ``False`` is returned with the previous row intact —
       an old outline is better than none.
@@ -209,15 +259,21 @@ async def regenerate_overview(
             logger.error("overview: regeneration failed — %s", e)
             return False
 
+        # Phase 162 (LOCKED A2): the deterministic source inventory is
+        # appended to the STORED content after the model's outline —
+        # mechanical, non-LLM, so no source can be invisible to
+        # `<knowledge_base>`. The model input above never sees the tail.
+        content = text.rstrip("\n") + "\n" + _source_inventory(rows)
+
         now = datetime.now(UTC)
         row = session.get(KbOverview, 1)
         if row is None:
-            session.add(KbOverview(id=1, content=text, updated_at=now))
+            session.add(KbOverview(id=1, content=content, updated_at=now))
         else:
-            row.content = text
+            row.content = content
             row.updated_at = now
         session.commit()
-        logger.info("overview: regenerated docs=%d chars=%d", len(rows), len(text))
+        logger.info("overview: regenerated docs=%d chars=%d", len(rows), len(content))
         return True
     finally:
         if owns_session: