Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 162_search_recall_small_sources
Build and Push Containers / build-and-push-app (push) Successful in 1m50s
Details
Build and Push Containers / build-and-push-db (push) Successful in 14s
Details

Browse Source 
**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).
This commit is contained in:
ducoterra
2026-10-07 01:42:01 -04:00
parent 690fd98036
commit eebc824636
48 changed files with 2685 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.err
View File
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.md
+16
View File
@@ -0,0 +1,16 @@
**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**
- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__00_phase.a1.validate
+124
View File
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.err
View File
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.md
+13
View File
@@ -0,0 +1,13 @@
All gates green. Final verification complete.
**Report**
- **Implemented (task 162/01, D1 / LOCKED A1):**
- `app/rag/retriever.py`: new module-level `_fold()` (stdlib NFKD + combining-class strip, shared with task 03's grep); `_normalize_name` = alnum scan of `_fold(s.lower())` — the single normalization both sides pass through; `name_hit_tokens`/`_name_parts`/`_name_token_matches` consumed it unchanged (docstring clauses only); module docstring normalization paragraph states the 2026-10-07 gap.
- `tests/unit/test_retriever.py`: +9 tests — `_fold` cases (Å/ÄÖÜ/ß/fi/ç, CJK, ①), `BESTÅ`→`besta` token pinned exactly (`["build", "besta"]`), `_name_parts` (`besta` in equal set unchanged; `BÅsta.md`→`basta`), end-to-end bridge both directions via `_name_hit_chunks`, min-length on the folded form, digitless-precision non-regression (`déploy` ≠ `Deployments`).
- **Tests/lint/coverage:** `uv run pytest tests/unit/test_retriever.py tests/unit/test_retriever_recency.py -v` → 94 passed; full `uv run pytest` → **3747 passed**; `--cov=app` → 99% TOTAL, retriever 99% (3 pre-existing misses); `uv run ruff check .` + `uv run pyright` clean; phase-119 E2E `tests/e2e/test_name_signal_read_chips.py` → 4 passed (FTS/gate suites untouched).
- **Docs:** README "How retrieval works" — lexical list now names the name-hit path + accent fold (phase 162). No AGENTS.md impact (its grep-contract line is task 03/final-pass scope).
- **Decisions/deviations:**
1. NFKD alone leaves `ß` intact; the task-pinned `ß → ss` case requires `str.casefold()` — applied inside `_fold` (still stdlib-only, single shared function, A1 intent intact; documented).
2. `①` is compat-decomposable (NFKD → `1`), contrary to the task's "no decomposition" note: it still drops as a token (<4 chars); its only footprint is a prefix-class versioned-join artifact (`the1`) no path can match — pinned as real behavior and documented.
- **Next pending task:** `162_search_recall_small_sources/02_kb_overview_source_inventory.md`.
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__01_accent_fold_name_norms.a1.validate
+124
View File
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 97%]
........................................................................ [ 99%]
... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5780 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.err
View File
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.md
+17
View File
@@ -0,0 +1,17 @@
All gates are green. Task 02 is complete.
**Report — task 02 `02_kb_overview_source_inventory.md`**
- **Implemented** (verified/completed in working tree — a prior interrupted run had left the code/tests uncommitted; I audited them against D2/A2, confirmed byte-identity of `build_overview_prompt`/`OVERVIEW_INSTRUCTION`/`load_kb_overview`/`build_kb_section`, and finished the doc work):
- `_source_inventory(rows)` in `app/rag/overview.py` — deterministic, non-LLM `Sources: <name> (<k> doc|docs), …`, distinct sources, catalog order, singular/plural.
- `regenerate_overview` happy path stores model text (rstripped `\n`) + `\n` + tail; log `chars=` = full stored content; zero-doc and `LLMError` paths untouched (previous row + tail preserved).
- Module + function docstrings record phase 162 / A2 / the `BOR_KB_OVERVIEW_MAX_CHARS` budget note.
- Unit tests in `tests/unit/test_overview.py`: format/order/counts, 1-doc survival (`ikea_instructions` case), exact happy-path stored content, tail-less zero-doc row preserved, `LLMError` preserves an existing tail, model input never gains the tail, exact char log; re-cut pins in `tests/integration/test_import_docs_overview.py`, `tests/e2e/test_kb_overview.py`, `tests/e2e/test_kb_overview_freshness.py`.
- **Tests / lint / coverage** (DB container started — it was down):
- `uv run pytest tests/unit/test_overview.py -v` → 29 passed
- `uv run pytest --cov=app --cov-report=term-missing` → **3753 passed**, `app/` TOTAL **99%**, `app/rag/overview.py` **100%**
- `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed; `tests/e2e/test_kb_overview_freshness.py` → 3 passed
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- **Docs**: `README.md` — import section now documents the deterministic `Sources: …` tail (phase 162) and the failure-path preservation; `AGENTS.md` — no doc impact (the grep-contract line there is task 03/04's scope).
- **Decisions**: inventory order via `sorted()` on source names = catalog order regardless of the recency-first row order the generator reads (proven by the scrambled-input test); no other e2e asserts exact overview-row content (checked).
- **Next pending task**: `03_grep_folding_search_discipline.md`.
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__02_kb_overview_source_inventory.a1.validate
+124
View File
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
......... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5787 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.err
View File
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.md
+22
View File
@@ -0,0 +1,22 @@
Task 03 complete. Report:
**Implemented**
- `app/rag/agent.py` — `grep_document` now folds **both** pattern and each line through the shared `_fold` imported from `app.rag.retriever` (one fold, no second copy; LOCKED A3): `_fold(pattern.lower()) in _fold(line.lower())`; rstrip/line-numbers/20-cap/200-char truncation/locator semantics untouched.
- Copy updates (LOCKED A3): module-docstring grep paragraph, `AGENT_TOOLS` grep function description, and `pattern` parameter description now state "case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`"; `NO_MATCHES_REGEX`/`_SCOPED` + `looks_like_regex`/`plain_form` byte-identical.
- `app/rag/prompts.py` — `TOOLS_SECTION` grep sentence folded-contract re-worded; new `SEARCH_DISCIPLINE_RULE` constant (LOCKED A4 sentence, verbatim) inserted immediately after the grep sentence; `PERSONA`/LOW/other sentences byte-identical.
- Pins re-cut (phase-124 pattern): `test_prompt_lock.py` (new sha `988642f9…`/len 3818, pre-phase-162 anchors, chained phase-162→145 diff-style pin, discipline-rule position pin, grep-contract pin), `test_prompts.py` (grep clause + D4 boundary pins), `test_agent.py` (`AGENT_TOOLS` + pattern-param description pins).
- New fold tests in `test_agent.py`: accented↔ASCII both ways (`besta`→`BESTÅ`, `basta`→`BÅsta`, `cafe`→`café`, `BESTÅ` pattern→ASCII content, `strasse`→`Straße`), non-match still `[]`.
- `README.md` — grep tool table row updated to the folded contract.
**Tests / lint / coverage**
- `uv run pytest tests/unit/test_agent.py tests/unit/test_prompt_lock.py tests/unit/test_prompts.py` → **292 passed**
- `uv run pytest` (full suite) → **3756 passed**
- Related E2E spot-checks: `test_grep_regex_teaching.py`, `test_tool_call_compact.py`, `test_kb_overview*.py`, `test_name_signal_read_chips.py` → **green** (11 passed in the last batch)
- `uv run pytest --cov=app` → TOTAL **99%** (`agent.py` 99%, `prompts.py` 100%) — >90% gate met
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
**Docs**: `README.md` grep-tool row updated (case- and accent-insensitive, fold example); `AGENTS.md` has no grep-contract line to change (the phase-162 final pass handles the §6 wording per the phase overview).
**Decisions**: reworked the phase-145 "only byte change" pin into a chained phase-162→145 revert pin so both historical anchors stay enforced; `SEARCH_DISCIPLINE_RULE` added as a module constant mirroring the `ORIGIN_RULE` pattern. No deviations from LOCKED A3/A4.
**Next pending task**: `162_search_recall_small_sources/04_e2e_battery_pins.md`
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__03_grep_folding_search_discipline.a1.validate
+124
View File
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.err
View File
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.md
+18
View File
@@ -0,0 +1,18 @@
**Phase 162 / task 04 — complete, all gates green.**
**Added** `tests/e2e/test_search_recall_small_sources.py` (7 tests), fixture KB `tests/fixtures/recallkb/` (3 sources: `ikea_instructions` 1 doc with the name **only** as `BESTÅ`, `phase_docs` 4 + `players_handbook` 1 ASCII-`besta`/`build` distractors), the Besta pair + gate comment in `tests/fixtures/retrieval_battery.txt`, and evidence in `.agents/reports/162_search_recall_small_sources/task04_prefix_evidence_and_battery.md`.
**Commands / outcomes**
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated, run twice incl. after the DB restore).
- `uv run pytest tests/e2e/test_name_signal_read_chips.py tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, 3756 tests, app coverage 99%**.
- `uv run ruff check .` → passed; `uv run pyright` → **0 errors**.
- Replica restored per the `restore-test-db` skill (fingerprint verified docs=1000/chunks=8866/sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic 0011→0026), then `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` → **GATE PASS 1–5**: gitea README rank 4, forgejo 5/5 with README #1, gateway README rank 3, qwen3.8-27b quadlets #1/#2, Mongolia HIGH/fts_hits=5 — the five `suggested:` lists are **byte-identical** to the phase-119 record, and a re-run with the fold reverted is **byte-identical** to the post-fix run.
**Pre-fix evidence (measured, both directions).** With `app/rag/retriever.py::_normalize_name` reverted by a one-line reversible edit, `How do I build the BESTÅ?` loses the doc: tokens `['build','best']` → no name hit, suggested 5 = the five ASCII distractors, IKEA doc **rank 6** — visible on the wire as its `done.related` entry and its absence from the folder-map echo (gate still HIGH, cosine 0.382, on the wrong docs). Shipped code: tokens `['build','besta']` → IKEA doc **suggested #1**, related = `phase_docs/156_history_source_filter.md`, cosine unchanged (0.382). The hypothesis is now re-measured on every run by `test_under_rank_is_measured_not_assumed`.
**Decisions / deviations** (harness mechanics win): task/phase files untouched, so the evidence lives in the report file above; no `git stash` (reversible source revert instead, verified restored, `grep -c TEMP` = 0); `done.suggestions` is the deflection-chip contract and never carries paths, so the suggested tier is asserted via the durable `query_log.sources` record (suggested 5 + remainder), `done.related`, and the mock's folder-map echo; tuning re-worded two bodies to drop stopword density so the out-of-KB control deflects (cosine 0.3345 → 0.2083) — both token controls are pinned as assertions; the battery carries both `How do I build a Besta?` (task text) and `How do I build the BESTÅ?` (the live miss).
**Skipped loudly:** the 2026-09-05 replica snapshot has no `ikea_instructions`-class source (`grep -c -i ikea` on the dump = 0), so the Besta rank gate stays with the owner's live KB; both questions still ran without a false grounding on a doc naming the product.
**Docs:** no README/AGENTS change needed (task 04 is tests + fixtures + battery; user-visible behavior was documented by tasks 01–03). DB left as found (pre-restore backup restored); no other pending task for this phase — task 04's file is ready for the harness to move, and the phase commit is still owed.
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/162_search_recall_small_sources/162_search_recall_small_sources__04_e2e_battery_pins.a1.validate
+124
View File
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/162_search_recall_small_sources/task04_prefix_evidence_and_battery.md
+174
View File
@@ -0,0 +1,174 @@
# Task 04 record — pre-fix evidence + battery (phase 162, D5/D6)
**Deviation note (mechanics):** the task file says "record the measured
pre-fix ranks in THIS file" and to revert the fold with `git stash`, but
the execution harness forbids editing phase/task files and forbids
index/history-touching Git commands. Per the conflict rule the harness
wins on mechanics and the intent is preserved: the evidence lives HERE,
the fold was reverted by a one-line, reversible edit of
`app/rag/retriever.py` (restored and byte-verified afterwards —
`grep -c TEMP app/rag/retriever.py` = 0), and the summary goes in the
task's final report. The permanent, non-throwaway form of the evidence is
`tests/e2e/test_search_recall_small_sources.py::test_under_rank_is_measured_not_assumed`,
which re-measures BOTH directions on every run.
## 1. The fixture (tests/fixtures/recallkb — three local-directory sources)
| source | docs | role |
| --- | --- | --- |
| `ikea_instructions` | **1** (`besta-frame-white__AA-1272068-11-100.md`) | the live miss: body names the product **only** as accented `BESTÅ` |
| `phase_docs` | 4 (`140_pictorial_pdf_pages`, `148_pdf_text_backfill`, `152_seaweedfs_blob_ingest`, `156_history_source_filter`) | ASCII `besta` + `build` distractors |
| `players_handbook` | 1 (`game_night_setup.md`) | ASCII `besta` + `build` distractor in a second small source |
Final shape tuning (the task's "add distractors until it loses"): the
initial 4-doc draft already lost the tier, but the deflection control
failed (best cosine 0.3345 — the small source's `Assembly` section was
dense in the stopword `the`, which the mock's tokenizer keeps). Two
bodies were re-worded (`ikea_instructions` Assembly paragraph,
`players_handbook/game_night_setup.md`) to drop stopword density: the
out-of-KB cosine fell to **0.2083** (< 0.30 threshold → deflects) while
the on-topic questions kept 0.38 / 0.44. Two token controls are pinned
as assertions in `synced_kb`: (a) no ASCII `besta` and no `build` anywhere
in the small source, (b) the accented name sits **past** the mock's
24-token summary-digest window, so the ASCII-only digest never leaks a
`best` lexeme the pre-phase FTS could have hit.
## 2. Measured pre-fix evidence — the E2E fixture, both directions
Lens: the app's REAL `retrieve()` + `select_suggested()` over the synced
KB with the mock's embeddings. Gate: threshold 0.30, lexical-support
floor 0.15 (the conftest mock calibration). Bonus 0.005, recency 0.0007,
suggested 5 / related 2 (code defaults, `.env`-free like production).
Question `How do I build the BESTÅ?`
**PRE-FIX — the real revert** (`_normalize_name` →
`"".join(_ALNUM_RE.findall(s.lower()))`, the exact task-01 before-state),
`pytest tests/e2e/test_search_recall_small_sources.py` run against the
fixture (full log: `/tmp/prefix_e2e_run.log`, `/tmp/prefix_wire_run.log`):
```
name tokens : ['build', 'best'] → NO name hit
suggested 5 : phase_docs/140_pictorial_pdf_pages.md
phase_docs/148_pdf_text_backfill.md
phase_docs/152_seaweedfs_blob_ingest.md
players_handbook/game_night_setup.md
phase_docs/156_history_source_filter.md
related : ikea_instructions/besta-frame-white__AA-1272068-11-100.md ← RANK 6, the miss
gate : best cosine 0.382, fts_hits 8 → grounded (on the WRONG docs)
done frame : {'source': 'ikea_instructions',
'path': 'besta-frame-white__AA-1272068-11-100.md',
'title': 'Assembly Instructions'} ← the rank-6+ "Nearby docs" slot
folder echo : (folders: phase_docs/: 148_…, 152_…, 156_…; players_handbook/:)
→ NO `ikea_instructions/` line at all: the small source never
reached the HIGH prompt
```
**POST-FIX (shipped code)** — same fixture, same measurement:
```
name tokens : ['build', 'besta'] → name hit
suggested 5 : ikea_instructions/besta-frame-white__AA-1272068-11-100.md ← #1
phase_docs/140_pictorial_pdf_pages.md
phase_docs/148_pdf_text_backfill.md
phase_docs/152_seaweedfs_blob_ingest.md
players_handbook/game_night_setup.md
related : phase_docs/156_history_source_filter.md
gate : best cosine 0.382, fts_hits 9 (the fold adds the LEXICAL term;
cosine untouched — LOCKED A1)
folder echo : … includes `ikea_instructions/:` (header alone — its only
document IS the suggested doc)
```
Mechanism (measured, and why the fix is the name signal and nothing
else): every distractor matches the question's other FTS token `build`,
so each fuses TWO RRF terms (vector + FTS ≈ 0.032) while the small source
— no `build`, and `BESTÅ` lexes as `bestå`, which the ASCII-`best`
tsquery cannot match — fuses ONE (≈ 0.016) and lands rank 6. The folded
name hit is the only signal big enough to lift it: name hits LEAD the
lexical list (`fuse` rank 1 = 1/61) plus the per-document
`BOR_NAME_HIT_BONUS` (0.005). ASCII question `How do I build a Besta?`
measures the SAME tier before and after (suggested #1, cosine 0.4404,
related `phase_docs/156_history_source_filter.md`) — the phase-119 path
is untouched.
Under-rank hypothesis (D1 failure #4): **pinned green with measured
ranks**, not assumed.
## 3. The dedicated suite
`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`
→ **7 passed** (task-04 cases 1–4 plus the measurement test, the grep
fold at the content layer, and the folder-line prompt witness).
Interpretation note: the task text says the `done`-frame `suggestions`
list is the observable tier. On the shipped wire `done.suggestions` is
the DEFLECTION contract (2–3 title-derived "Maybe try" chips, `[]` on
grounded turns — `app/api/chat.py`, phase 04) and never carries document
paths. The faithful observables of the SUGGESTED tier are (a) the durable
`query_log.sources` record = suggested 5 then the rank-6+ remainder
(phase 118 A3) and (b) `done.related` = the rank-6+ tier. The suite
asserts both (plus `done.sources == []` — read-docs-only, phase 119
LOCKED A1 — and the mock's folder-map echo for the prompt-level witness).
Neighbour suites re-run (the retriever + overview were touched by this
phase): `uv run pytest tests/e2e/test_name_signal_read_chips.py
tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
## 4. Battery — `tests/fixtures/retrieval_battery.txt`
The file gains the Besta pair (`How do I build a Besta?` — the task's
line — and `How do I build the BESTÅ?`, the live miss itself) with the
gate comment. The five phase-119 questions keep positions 1–5.
Preconditions (the `restore-test-db` skill, run in-container): safety
backup → `DROP SCHEMA public CASCADE` → single-transaction load of
`data/bor_live_kb_replica.dump.sql` → **fingerprint verified:
docs=1000, chunks=8866,
sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic=0011**
→ `uv run alembic upgrade head` (0011 → **0026**) → docs=1000 re-verified.
`uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt`
(full output `/tmp/battery_postfix.log`, pre-fix comparison
`/tmp/battery_prefix.log`):
| # | question | verdict |
| --- | --- | --- |
| 1 | How do I deploy gitea? | HIGH, `fts_hits=38`; suggested top-5 **byte-identical** to the phase-119 record — `gitea/README.md` **rank 4** ✓ |
| 2 | How do I deploy forgejo? | HIGH, `fts_hits=37`; forgejo tree 5/5, `forgejo/README.md` **#1** ✓ |
| 3 | How do I set up the gateway? | HIGH, `fts_hits=37`; `gateway/README.md` **rank 3** ✓ |
| 4 | Do you have notes on the qwen 3.8 27b model? | HIGH, `fts_hits=31`; both `qwen3.8-27b-*` quadlets at #1/#2 ✓ (the digit-prefix regression holds) |
| 5 | What is the capital of Mongolia? | **HIGH, `fts_hits=5`** — unchanged ✓ |
| 6 | How do I build a Besta? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
| 7 | How do I build the BESTÅ? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
`SELECT DISTINCT source FROM documents` on the restored replica lists
`deploy, homelab, homepage, ServMon, skills, vibe-bot` — NO
`ikea_instructions`-class source (`grep -c -i ikea
data/bor_live_kb_replica.dump.sql` = 0: the 2026-09-05 snapshot predates
the IKEA import, exactly the case D5 anticipated; phase-140's
"skipped loudly" precedent). The two questions still ran and did not
crash; both return the `build.sh`/`bifrost`-class top-5 that a KB without
the product can only return, and neither grounds on a document that names
the product. **The rank gate therefore stays with the owner** — it can
only close on a KB that has the source (the live KB).
Behavior-preservation, measured (not argued):
* the five phase-119 `suggested:` lists are **byte-identical** to the
recorded 2026-09-16 run in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md`;
* the battery re-run with the task-01 fold REVERTED is **byte-identical**
to the post-fix run (`diff` of the question/gate/suggested lines: no
difference) — the fold cannot move an ASCII-only corpus, as designed;
* `best_cosine` moves by ≤ 0.0004 between the September record and today
(real-`embed` model jitter, not code).
**Verdict: GATE PASS** for questions 1–5 (unchanged), 6–7 recorded as
skipped loudly with the reason and the re-check command.
## 5. Environment left behind
The replica was restored only for the battery. The pre-restore DB state
(backup `/tmp/pre_restore_host.dump`, 13 leftover E2E fixture docs,
alembic 0026) is restored afterwards — reload the replica any time with
the `restore-test-db` skill.