phase: 162_search_recall_small_sources
Build and Push Containers / build-and-push-app (push) Successful in 1m50s
Build and Push Containers / build-and-push-db (push) Successful in 14s

**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).
This commit is contained in:
2026-10-07 01:42:01 -04:00
parent 690fd98036
commit eebc824636
48 changed files with 2685 additions and 80 deletions
@@ -0,0 +1,70 @@
# Phase 162 — Search recall: accent-folded name hits & grep, KB-overview source inventory, search-discipline teaching
**Source:** owner live report, 2026-10-07 — brain.reeseapps.com, shared conversation "How do I build a Besta?" (share token `58525626-4462-4baf-bcf2-3abc81f18284`). The owner asked "Why did you miss it? Let's try to improve your search techniques and system prompt." The agent's post-mortem identified **four stacked failures**, three of which are code-fixable:
1. **The `<knowledge_base>` outline is lossy and is the only map the model gets.** The outline listed 3 groups (Gufo, Brain-of-Reese, Deployments); the live index had 14 sources. `ikea_instructions` (1 doc) and `players_handbook` (1 doc) were **invisible** — a whole class of source-level questions could not be surfaced by the outline at all. The model answered from the outline instead of running `ls`.
2. **Notes poisoned the model with a true-but-different fact** (the phase-140/148 note "the IKEA tar is gitignored, on disk only" was over-generalized to "the brain has no copy of this PDF"). Model behavior — mitigated here by prompt teaching (D4), plus an owner runtime follow-up (A6, recorded, not code).
3. **The Å/ASCII accent-folding gap.** `grep "besta"` returned only phase docs and never the PDF, because the only product name in the PDF's text layer is `BESTÅ` (U+00C5): the filename is ASCII (`besta-frame-white__…`) but the content is not, and the exact-substring grep cannot bridge that. On the name-hit side the gap is symmetric: the question token `BESTÅ` normalizes (lower + alnum-only) to `best` — not `besta` — so an accented question never name-matches the ASCII filename's stem sub-components.
4. **Retrieval under-ranked the PDF (the phase-119 failure class).** The doc's title is the generic "Assembly Instructions" (title ranking gives nothing); by phase-119 LOCKED A2 the digitless token `besta` (≥4 chars) equals a stem sub-component of `besta-frame-white__AA-1272068-11-100` and should have name-matched, but the name-hit row lost the fused walk to the phase docs whose bodies literally contain "besta". Flagged by the post-mortem as a hypothesis needing a test — **this phase pins it with the battery (D5)** rather than assuming a finding.
**Story:** n/a (owner-directed RAG behavior change; extends the phase-119 name-hit mechanism, the phase-31 KB overview, the phase-68/70 grep, and the phase-117 prompt pins).
## Objective
Accented names (`BESTÅ` ↔ `besta`) are found **both ways** — by the name-hit path (question side) and by `grep` (content side); **no source can be invisible** to `<knowledge_base>` (a deterministic, non-LLM source-inventory line is appended to the stored overview); the model is **taught the search discipline** the post-mortem adopted (outline ≠ inventory; `ls` first; part-number/filename-token greps; gitignored ≠ unindexed); and the battery question "How do I build a Besta?" **pins** the IKEA doc into the suggested top-5 — turning the anecdote into a regression test.
## Dependencies
- `119_name_signal_read_chips` (complete) — the phase-119 name-hit rule (LOCKED A2), the `name_hit` flag, the bonus (LOCKED A3), and the `suggested:` eval-battery line this phase extends.
- `31`-line KB overview (complete; `app/rag/overview.py`) — the `regenerate_overview` best-effort/change-gated generation this phase appends the inventory to.
- `68_search_tool` / `70_harness_aligned_tools` (complete) — the `grep` contract (case-insensitive fixed substring, locator, teaching refusals) this phase extends.
- `117_tool_call_compact` (complete) — the `TOOLS_SECTION` byte-pins this phase re-cuts.
## Design (shared by all tasks — the executor reads this, not the chat)
**D1 — accent-folded name normalization (task 01, LOCKED A1).** One new helper in `app/rag/retriever.py`: `_fold(s)` = `unicodedata` NFKD normalize + drop every char with a non-zero combining class (stdlib only; no new dependency). `_normalize_name` becomes the existing `[a-z0-9]+` scan of `_fold(s.lower())`. Because `_normalize_name` is the **single** normalization both sides of name matching pass through — the question tokens in `name_hit_tokens` AND the path parts / file stem / stem sub-components in `_name_parts` — the one change bridges both directions: the ASCII question `besta` keeps matching the ASCII filename's sub-components (unchanged behavior), the accented question `BESTÅ` now normalizes to `besta` (was `best` — the 2026-10-07 gap), and an accented **filename** (`BÅsta.md` → `basta`) matches an ASCII question. Folding cases the tests pin: `Å/å → a`, `Ä/ä/Ö/ö/Ü/ü → a/o/u` (the letter, decomposition preserved), `ß → ss`, the `fi` ligature → `fi`, `ç → c`; characters with no NFKD decomposition (CJK, `①`) still drop out under the alnum scan, exactly as today. **Everything else in the A2 mechanism is untouched**: the digit-bearing-prefix / digitless-equality match classes, titles-never-matched, the (matched-token count, catalog order) ranking, `NAME_TOKEN_MIN_LEN`, the versioned-name join, `NAME_HIT_LIMIT`, the `name_hit` flag, the phase-119 bonus (LOCKED A3), and the FTS / cosine / A8 gate (LOCKED A7/A8 — the `tsvector` 'english' pipeline is NOT modified; same-accents FTS already bridges, accent-vs-ASCII FTS bridging is out of scope by the lock).
**D2 — the KB-overview source inventory (task 02, LOCKED A2).** The overview is `lite`-generated and lossy (D1-failure #1); the fix is **deterministic, non-LLM**: `regenerate_overview` (happy path only) appends one line to the stored content, after the model's outline: `Sources: <name> (<k> doc|docs), …` — one entry per DISTINCT source, catalog order (`source, path`), exact per-source document counts. Computed from the same catalogue rows the prompt builder already reads (no new query). `build_overview_prompt` / `OVERVIEW_INSTRUCTION` are **byte-identical** (the model never sees the tail — it is mechanical, not a model burden); `load_kb_overview` / `build_kb_section` are untouched (the existing longest-prefix cap applies to the whole stored text). Budget: the model caps the outline at ~1500 chars per the instruction; one inventory line is ~15 chars/source — at the default `BOR_KB_OVERVIEW_MAX_CHARS` (4000) the tail always fits; a smaller owner-set cap cuts the tail first (accepted; the cap is owner-tunable, the record here is the constraint). The zero-doc early return and the `LLMError` path are **byte-identical** (the previous row — tail included, it is part of the stored content — stays intact; an old outline is better than none).
**D3 — accent-folding grep (task 03, LOCKED A3).** `grep_document` in `app/rag/agent.py` folds **both sides** — the pattern and each content line — through the SAME `_fold` as D1 (imported from `app.rag.retriever`, which `agent.py` already imports from: grep and the name-hit path can never drift apart) plus lowercase (replacing the current lower-only comparison): `grep "besta"` now matches a content line `The BESTÅ frame`. The contract remains a **fixed substring** (the phase-68 owner lock) — the fold changes the alphabet it ignores, not the matching semantics; still a locator, still ≤20 lines, still line-truncated at 200 chars. The regex teaching (`NO_MATCHES_REGEX` / `_SCOPED`, `looks_like_regex`, the plain-form retry hint) is **byte-identical** — the plain-substring claim it teaches is still true. Copy updates (all in this task): the `agent.py` module docstring grep paragraph, the `AGENT_TOOLS` grep function description + the `pattern` parameter description, and the `TOOLS_SECTION` grep sentence in `app/rag/prompts.py` ("case-insensitive" → "case- and accent-insensitive"), plus D4's new sentence. The phase-117 `test_prompt_lock.py` / `test_prompts.py` / `test_agent.py` pins re-cut in the same task (the phase-124 pattern: copy change + re-cut pins in one task).
**D4 — the search-discipline sentence (task 03, LOCKED A4).** Appended to the `TOOLS_SECTION` tools paragraph **immediately after the grep sentence** (the conversation's placement), LOCKED verbatim:
> `A summary or outline is a hint, never an inventory: before claiming the knowledge base lacks something, run `ls` with no `path` to list every source, and `grep` the artifact's part number or a filename token. `grep` folds accents — `besta` matches `BESTÅ` — but one pattern is one string: a name's split tokens (`frame`, `white`) and its part number (`AA-1272068`) need separate greps. A note saying a file is gitignored describes its repo status, not whether it is indexed as a source.`
(Adapted from the conversation's proposed patch for the D3 fold — the conversation's original sentence assumed a non-folding grep; the owner asked for "improve your search techniques and system prompt", which is the permission source for this copy change.) `PERSONA`, the LOW/`DEFLECT_MODE` body, and every other `TOOLS_SECTION` sentence stay byte-identical. The mock LLM is unaffected (it keys on markers, not on the sentence).
**D5 — verification (task 04).** A new dedicated Playwright suite `tests/e2e/test_search_recall_small_sources.py` (run in isolation; the phase-119 local-directory-source fixture pattern) with a synthetic replica of the live miss: source `ikea_instructions` (ONE doc, `besta-frame-white__AA-1272068-11-100.md`, body containing the name **only** in accented form — no ASCII "besta") + 2–3 distractor docs in other sources whose bodies contain ASCII "besta" (the phase-docs stand-in). The **pre-fix evidence is measured and recorded** in the task file: with the task-01 fold temporarily reverted (git stash), the ACCENTED question "How do I build the BESTÅ?" must lose the accented doc from the suggested top-5 (record the measured ranks); restore the fix. Gated assertions: (1) the accented question's `done`-frame `suggestions` include the `ikea_instructions` doc (the D1 question-side fold); (2) the ASCII question "How do I build a Besta?" includes it too (the phase-119 path regression); (3) the stored `kb_overview` row (read through the fixture session) ends with the `Sources: …` tail naming **every** fixture source with exact counts — the 1-doc source visible (D2); (4) the out-of-KB question's gate verdict is unchanged (D1–D3 must not move the A8 gate). `tests/fixtures/retrieval_battery.txt` (the tracked phase-119 battery) gains `How do I build a Besta?` with its gate comment: on the 1,000-doc live replica (the `restore-test-db` skill; the dump is a local gitignored artifact — if missing, STOP and ask the owner, never fabricate numbers) the battery runs when the replica's `documents` table contains an `ikea_instructions`-class source (check `SELECT DISTINCT source FROM documents`) and the suggested top-5 must include `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`; if the replica lacks it (the snapshot may predate the import), the five existing phase-119 questions still run (their gate must NOT move — the name-hit change is behavior-preserving for them) and the Besta question is recorded as **skipped loudly (source absent in the replica snapshot)** — the phase-140 "skipped loudly" precedent. The under-rank hypothesis (D1-failure #4) is thus either pinned green or handed to the owner with measured ranks — never assumed.
**Invariants (all tasks).** The A8 honesty gate, the FTS `tsvector` pipeline, the cosine path, every SSE frame shape, the LOW prompt, `PERSONA`, the `NO_MATCHES_REGEX`/`_SCOPED` teaching lines, the mock-LLM markers, and the `build_overview_prompt` / `OVERVIEW_INSTRUCTION` inputs are byte-identical. `PLAN.md` untouched (Protocol B — A5).
## Tasks
1. `01_accent_fold_name_norms.md` — the shared `_fold` helper + the `_normalize_name` fold (D1).
2. `02_kb_overview_source_inventory.md` — the deterministic `Sources: …` tail (D2).
3. `03_grep_folding_search_discipline.md` — the folded `grep_document` + the three copy updates + the D4 sentence + the re-cut pins (D3+D4).
4. `04_e2e_battery_pins.md` — the dedicated E2E, the pre-fix evidence, the battery question + replica run, the final gates (D5).
## Testing & Quality
- Unit: `tests/unit/test_retriever.py` (D1: the bridge both ways, the fold cases, the A2 non-regression set), `tests/unit/test_overview.py` (D2: the tail format/order/counts, the 1-doc survival, the zero-doc + LLM-failure paths, the byte-identical `build_overview_prompt`), `tests/unit/test_agent.py` (D3: `grep_document` fold both sides, the scoped + unscoped paths, the copy pins), `tests/unit/test_prompts.py` + `tests/unit/test_prompt_lock.py` (D3+D4: the re-cut pins; the LOW + `PERSONA` byte-identity).
- E2E: `tests/e2e/test_search_recall_small_sources.py`, **in isolation** (`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`); the phase-119/117/31 regression suites green (the re-cut pins included).
- Battery (task 04): per D5 — the replica run recorded in the task file; the five phase-119 questions' gate unchanged.
- Coverage: **>90%** on `app/`; `uv run ruff check . && uv run pyright` clean.
## Completion Criteria
- [ ] `grep "besta"` (unit + the E2E fixture's content layer) matches an accented `BESTÅ` content line, and the accented question "How do I build the BESTÅ?" name-matches the ASCII filename's stem sub-components — the fold is one function, both sides, measured (the pre-fix ranks recorded in `04_…md`).
- [ ] The stored `kb_overview` row ends with the deterministic `Sources: …` line naming every source with exact counts; a 1-doc source is visible; the zero-doc and LLM-failure paths are byte-identical to the pre-phase behavior.
- [ ] The `TOOLS_SECTION` carries the folded grep contract + the LOCKED D4 sentence (all other pins byte-identical; the re-cut pins green).
- [ ] `tests/e2e/test_search_recall_small_sources.py` green in isolation; the battery question is recorded per D5; `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` + `AGENTS.md` reviewed in full in the final pass (the grep contract line in `AGENTS.md`'s §6 quick reference and `PLAN.md` §6's tool table mention "case-insensitive fixed substring" — the final pass updates `AGENTS.md`'s tool description to "case- and accent-insensitive"; `PLAN.md` stays untouched per A5, folded at the next redo).
- [ ] One `--no-gpg-sign` commit; the phase dir moves to `complete/` by the pipeline gate.
## Locked decisions
- **A1 — the fold**: NFKD normalize + combining-class strip (stdlib `unicodedata`) applied inside `_normalize_name` — the single normalization both sides of name matching use; the A2 match classes, ranking, caps, flags, the phase-119 bonus, and the FTS/cosine/A8 pipeline are untouched (owner-confirmed 2026-10-07, from the conversation's post-mortem).
- **A2 — the overview tail**: `Sources: <name> (<k> doc|docs), …` appended server-side to the stored outline at generation time (happy path only); deterministic, non-LLM, catalog order; `build_overview_prompt` / `OVERVIEW_INSTRUCTION` / `load_kb_overview` / `build_kb_section` byte-identical; the zero-doc and `LLMError` paths untouched (owner-confirmed 2026-10-07).
- **A3 — the grep fold**: `grep_document` compares the `_fold`+lowercase of the pattern against the `_fold`+lowercase of each line — the SAME `_fold` as A1 (imported); still a fixed substring (phase-68 lock), still a locator, regex teaching byte-identical (owner-confirmed 2026-10-07 — the conversation's "optional grep-side accent folding", accepted).
- **A4 — the D4 sentence** is appended to the `TOOLS_SECTION` tools paragraph immediately after the grep sentence, verbatim as written in D4; `PERSONA` + LOW body + the other `TOOLS_SECTION` sentences byte-identical (owner-confirmed 2026-10-07 — the owner asked for the system-prompt improvement in the conversation).
- **A5 — PLAN.md untouched (Protocol B)**: the grep-contract extension (A3) and the prompt sentence (A4) are recorded here (owner-confirmed) and folded into the anchor table at the next redo (the phase-119 A6 precedent).
- **A6 — the steering-note follow-up is an owner runtime action, not code**: the conversation's proposed tuning rule ("IKEA's BESTÅ assembly PDF is a live source — search it as BESTÅ or AA-1272068, not besta") is a `steering_notes` row the owner adds via the Tuning tab after this phase ships; the phase records it here and does not touch `steering_notes` data.
## Commit
```bash
git add app/ tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(rag): accent-fold name hits and grep (BESTÅ <-> besta), a deterministic source inventory in the KB overview, and search-discipline prompt teaching"
```
@@ -0,0 +1,28 @@
# Task 01 — The accent-folded name normalization (phase-162 D1, LOCKED A1)
**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D1 — the 2026-10-07 live finding: `grep "besta"` never hit the IKEA PDF (its text layer carries only `BESTÅ`, U+00C5), and the symmetric name-hit gap: the question token `BESTÅ` normalizes (lower + alnum-only) to `best`, never `besta`, so an accented question cannot name-match the ASCII filename `besta-frame-white__AA-1272068-11-100` (whose stem sub-components include `besta` under phase-119 LOCKED A2).
**Story:** n/a
## Objective
`_normalize_name` in `app/rag/retriever.py` folds accents (NFKD + combining-class strip, stdlib `unicodedata`) before the existing alnum scan, so both sides of name matching bridge `BESTÅ` ↔ `besta` — with the A2 mechanism (match classes, ranking, caps, flags, bonus) and the FTS/cosine/A8 pipeline byte-identical.
## Work
1. `app/rag/retriever.py` — new module-level `_fold(s: str) -> str`: `unicodedata.normalize("NFKD", s)` with every char whose `unicodedata.combining(c)` is non-zero dropped (stdlib only — no new dependency). Docstring: the shared accent fold (phase 162) — NFKD decomposition, combining marks dropped; `Å/å → a`, `ß → ss`, `fi → fi`, `ç → c`; characters with no decomposition (CJK, `①`) survive the fold and still drop out under the caller's alnum scan (behavior unchanged for them).
2. `app/rag/retriever.py` — `_normalize_name`: `"".join(_ALNUM_RE.findall(_fold(s.lower())))` (was: `…findall(s.lower())`). Docstring update: the folded form (`Qwen 3.8` → `qwen38` unchanged; `BESTÅ` → `besta` new) — this is the **single** normalization both sides pass through (the `name_hit_tokens` question tokens and the `_name_parts` path parts / stem / stem sub-components), so the one change bridges both directions.
3. `app/rag/retriever.py` — `name_hit_tokens` / `_name_parts` / `_name_token_matches`: **no code change** (they consume `_normalize_name`); their docstrings gain one clause each: candidates/parts are accent-folded (phase 162) — `besta` matches `BESTÅ`-class names both ways.
4. `app/rag/retriever.py` — module docstring: the normalization paragraph now states the fold (the 2026-10-07 gap: the ASCII filename vs the accented content layer).
5. `tests/unit/test_retriever.py` — the fold, on the file's existing fixture pattern:
- **question side**: the accented question "How do I build the BESTÅ?" produces a `besta` token (was `best` pre-phase — pin the exact token list); `Ä`/`Ö`/`Ü`/`ß`/`fi`/`ç` fold cases (`straße`-class → `strasse`-class, ligature `fine` → `fine`);
- **path side**: `_name_parts` on `besta-frame-white__AA-1272068-11-100` yields `besta` in the equal set (the phase-119 case, unchanged) AND on the accented filename `BÅsta.md` yields `basta` — the ASCII question `basta` name-matches it;
- **the bridge, end to end**: `_name_hit_chunks`-class assertion — an accented-question token hits the ASCII-stem doc AND an ASCII token hits the accented-stem doc;
- **A2 non-regression (all existing behavior, re-pinned after the change)**: digitless precision (the `deploy`-titled `Deployments/` doc does NOT hit for `deploy`), digit-bearing prefix (`qwen38` → `qwen3.8-27b-…`, and the mid-word non-match), the versioned join (`Qwen 3.8` → `qwen38`), min-length on the **folded** form, ranking (matched-token count, catalog order), `NAME_HIT_LIMIT`, `name_hit=True`/`False` convention;
- **no-decomposition characters**: a CJK / `①` token still drops out (the alnum scan behavior for non-folding chars is unchanged);
- the FTS/`retrieve()`/gate rows are untouched (the existing `test_retriever.py` / `test_retriever_recency.py` suite stays green without modification).
## Testing & Quality
- Unit: the cases above; coverage **>90%** on `app/`.
## Completion Criteria
- [ ] `BESTÅ` (question) → `besta` token → name-hits the ASCII filename's sub-components, and `besta` (question) → name-hits the accented filename `BÅsta.md` — one function, both directions.
- [ ] Every phase-119 A2 case (precision, prefix, join, ranking, cap, flag) passes UNCHANGED after the fold; the FTS/gate suites are untouched.
- [ ] `uv run pytest tests/unit/test_retriever.py tests/unit/test_retriever_recency.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,29 @@
# Task 02 — The deterministic source inventory in the KB overview (phase-162 D2, LOCKED A2)
**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D2 — the 2026-10-07 live finding: the `lite`-generated `<knowledge_base>` outline listed 3 groups of a 14-source index; the 1-doc sources `ikea_instructions` and `players_handbook` were invisible, so source-level questions ("You have ikea_instructions in your RAG" — the owner's push-back) could not be surfaced by the outline at all.
**Story:** n/a
## Objective
`regenerate_overview` in `app/rag/overview.py` appends one **deterministic, non-LLM** line to the stored outline — `Sources: <name> (<k> doc|docs), …` — so no source can be invisible to `<knowledge_base>`; the model input, the load/inject path, and the zero-doc / LLM-failure paths are byte-identical.
## Work
1. `app/rag/overview.py` — a new pure helper `_source_inventory(rows: Sequence[tuple[str, str, str, str | None]]) -> str` next to `_first_summary_line`: one entry per **distinct** source, in catalog order (first occurrence in the `source, path`-ordered rows), each `<name> (<k> doc|docs)` — singular `doc` at k=1, `docs` otherwise — joined with `, ` behind the prefix `Sources: `. (The `rows` shape is the existing `(source, path, title, summary)` projection `regenerate_overview` already reads — no new query.) Docstring: the phase-162 deterministic inventory (LOCKED A2) — mechanical, non-LLM; the `lite` outline stays the model's job.
2. `app/rag/overview.py` — `regenerate_overview` **happy path only**: after the model text is obtained and before the upsert, the stored content = the model text (trailing newlines rstripped) + `"\n"` + the inventory line. The upsert, the `overview: regenerated docs=… chars=…` log (chars = the FULL stored content, tail included), the `True` return, and the change-gating contract are otherwise unchanged.
3. `app/rag/overview.py` — the **zero-doc early return** (`False`, row untouched) and the **`LLMError` path** (logged, `False`, previous row intact) are UNCHANGED — the previous row keeps its earlier tail, it is part of the stored content; an old outline is better than none. `build_overview_prompt`, `OVERVIEW_INSTRUCTION`, `SYSTEM_PROMPT`, `load_kb_overview`, and `app/rag/prompts.py::build_kb_section` are **untouched** (the model never sees the tail; the existing longest-prefix cap applies to the whole stored text — at the default `BOR_KB_OVERVIEW_MAX_CHARS` (4000) the ~1500-char outline + the one-line tail always fit; a smaller owner-set cap cuts the tail first — accepted, recorded in the docstring).
4. `app/rag/overview.py` — module docstring: a paragraph on the inventory line (phase 162 — the 2026-10-07 finding: the `lite` outline is lossy; small sources must be visible without a model call; the line is deterministic so the mock/real model outputs never affect it).
5. `tests/unit/test_overview.py` — the helper + the paths (the file's existing duck-typed-`OverviewLLM` fake pattern):
- the format: `Sources: a (1 doc), b (2 docs), c (3 docs)` — exact, incl. singular/plural and the catalog order;
- the **1-doc survival** (the `ikea_instructions` case): a 1-doc source among many appears with `(1 doc)`;
- the happy path: the stored row = model text + newline + the tail (exact content pin on the fake LLM's fixed output);
- zero-doc: `False`, the existing row (a pre-phase row, tail-less) byte-identical;
- `LLMError`: `False`, the **previous** row byte-identical — INCLUDING a row that already carried a tail (the tail is content, preserved on failure);
- `build_overview_prompt` byte-identical to the pre-phase output for the same rows (the model input never gains the tail);
- the regenerated log line carries the full stored-content char count.
## Testing & Quality
- Unit: the cases above; coverage **>90%** on `app/`.
## Completion Criteria
- [ ] A regenerated `kb_overview` row ends with `Sources: …` naming every source with exact counts (a 1-doc source visible); the model input is byte-identical to the pre-phase prompt.
- [ ] The zero-doc and `LLMError` paths leave the row byte-identical (an existing tail preserved on failure); the char log is exact.
- [ ] `uv run pytest tests/unit/test_overview.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,27 @@
# Task 03 — The accent-folding grep + the search-discipline prompt teaching (phase-162 D3+D4, LOCKED A3+A4)
**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D3+D4 — the 2026-10-07 live finding: `grep "besta"` never hit the PDF whose text layer carries only `BESTÅ` ("the tool is documented as 'a plain substring' and silently missing Å is the nastiest of the four"), and the owner's ask: "Let's try to improve your search techniques and system prompt."
**Story:** n/a
## Objective
`grep_document` folds accents on **both** pattern and content (the SAME `_fold` as task 01, imported — grep and the name-hit path can never drift); the contract copy in all three places states the folded contract; the LOCKED D4 search-discipline sentence joins the `TOOLS_SECTION` tools paragraph; the phase-117 pins re-cut in this task.
## Work
1. `app/rag/agent.py` — `grep_document(content, pattern)`: the comparison is `_fold(pattern.lower()) in _fold(line.lower())` per (rstripped) line — importing `_fold` from `app.rag.retriever` (the module already imports `TRUNCATION_MARKER` from there). Everything else UNCHANGED: the rstrip, the `lineno` numbering, the 20-match cap, the 200-char line truncation, the scoped/unscoped call sites (`all_documents` / the one-doc path at the existing call site), the locator semantics (no context add — `holder.read_docs` untouched). Docstring update: case- AND accent-insensitive fixed substring (phase 162, LOCKED A3) — the fold changes the alphabet it ignores, not the matching semantics; still NEVER a regex.
2. `app/rag/agent.py` — the module docstring grep paragraph (the "case-insensitive fixed substring" sentence): "case- and accent-insensitive fixed substring (accents fold — `besta` matches `BESTÅ`)".
3. `app/rag/agent.py` — `AGENT_TOOLS` the grep function description: "(case-insensitive)" → "(case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`)", and the `pattern` parameter description gains the same clause (the "plain substring, NEVER a regex" teaching unchanged). The `NO_MATCHES_REGEX` / `NO_MATCHES_REGEX_SCOPED` teaching lines and `looks_like_regex` / `plain_form` are **byte-identical** (the plain-substring claim they teach is still true).
4. `app/rag/prompts.py` — `TOOLS_SECTION`, the grep sentence: "locates an exact string (case-insensitive)" → "locates an exact string (case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`)" (the sentence's remainder, incl. the plain-substring + retry teaching, unchanged).
5. `app/rag/prompts.py` — `TOOLS_SECTION`: the LOCKED D4 sentence (verbatim from the overview) appended to the tools paragraph **immediately after the grep sentence**, i.e. after the `…retry with the plain text you expect to see.` clause:
`A summary or outline is a hint, never an inventory: before claiming the knowledge base lacks something, run `ls` with no `path` to list every source, and `grep` the artifact's part number or a filename token. `grep` folds accents — `besta` matches `BESTÅ` — but one pattern is one string: a name's split tokens (`frame`, `white`) and its part number (`AA-1272068`) need separate greps. A note saying a file is gitignored describes its repo status, not whether it is indexed as a source.`
`PERSONA`, the LOW/`DEFLECT_MODE` body, the `<documents>` blocks, and every other `TOOLS_SECTION` sentence are byte-identical. (The E2E mock is unaffected — it keys on markers, not on the sentence.)
6. Pins re-cut, **in this task** (the phase-124 pattern): `tests/unit/test_prompt_lock.py` (the `TOOLS_SECTION` pin + its derived cross-pins), `tests/unit/test_prompts.py` (the grounded-prompt pins that embed the grep sentence), `tests/unit/test_agent.py` (the `AGENT_TOOLS` description pin, if present — grep the file for the old description text; the `NO_MATCHES_REGEX` pins stay green unmodified).
7. `tests/unit/test_agent.py` — `grep_document` fold coverage (the existing case-insensitive test's pattern): `grep_document("The BESTÅ frame (AA-1272068-11-100)", "besta")` hits; `grep_document("…BÅsta…", "basta")` hits; `grep_document("café", "cafe")` hits; the pattern's own accent folds too — `grep_document("The besta frame", "BESTÅ")` ALSO hits (the pattern `BESTÅ` folds to `besta`); a genuinely non-matching folded pattern still returns no lines (`grep_document("The besta frame", "bestb")` → `[]`); line numbers / rstrip / cap / truncation unchanged (existing tests stay green unmodified).
## Testing & Quality
- Unit: the cases above; coverage **>90%** on `app/`.
## Completion Criteria
- [ ] `grep_document` matches accented ↔ ASCII both ways through the imported `_fold` (no second copy of the fold); the regex teaching lines are byte-identical.
- [ ] The three copy sites (module docstring, `AGENT_TOOLS`, `TOOLS_SECTION`) state the folded contract; the D4 sentence sits immediately after the grep sentence, LOCKED verbatim.
- [ ] The re-cut `test_prompt_lock.py` / `test_prompts.py` / `test_agent.py` pins green; the LOW + `PERSONA` byte-identity pins green unmodified.
- [ ] `uv run pytest tests/unit/test_agent.py tests/unit/test_prompt_lock.py tests/unit/test_prompts.py -v` green; full suite green; `uv run ruff check . && uv run pyright` clean.
@@ -0,0 +1,31 @@
# Task 04 — The dedicated E2E, the measured pre-fix evidence, and the battery pin (phase-162 D5)
**Phase:** `162_search_recall_small_sources` · **Source:** phase overview D5 — the 2026-10-07 finding must turn from anecdote into a regression test: the accented question "How do I build the BESTÅ?" surfaces the accented-content doc whose ASCII filename carries the name's sub-components, over ASCII "besta"-mentioning distractors; the 1-doc source is visible in the stored overview; the A8 gate and the five phase-119 battery questions do not move.
**Story:** n/a
## Objective
`tests/e2e/test_search_recall_small_sources.py` (run in isolation) gates the fold end-to-end on a synthetic replica of the live miss; the pre-fix ranks are **measured and recorded** in this file (the under-rank hypothesis is pinned green or handed to the owner with evidence — never assumed); `tests/fixtures/retrieval_battery.txt` gains the Besta question per D5; the final gates run.
## Work
1. `tests/e2e/test_search_recall_small_sources.py` — new dedicated suite, **run in isolation** (`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`), on the phase-119 local-directory-source fixture pattern (fixture module truncates/re-imports; the deterministic mock LLM by default). Fixture KB — a synthetic replica of the live miss:
- source **`ikea_instructions`** (ONE doc): `besta-frame-white__AA-1272068-11-100.md` — body carrying the product name **only in accented form** (e.g. "The BESTÅ frame (AA-1272068-11-100)… cam bolt 130618 … wall bracket 141413 … leveling foot …") — **no ASCII "besta" anywhere in the body** (the live PDF's exact situation);
- 2–3 **distractor docs** in other sources whose bodies contain ASCII "besta" (the phase-docs stand-in — e.g. `brain-of-reese/.agents/phases/complete/140_pictorial_pdf_pages/00_phase.md`-shaped text: "the besta fixture drove the pictorial phases"). Tune the distractor count/bodies so the pre-phase accented question loses the `ikea_instructions` doc from the suggested top-5 (step 2 measures this — if it does NOT lose it, add distractors until it does, and record the final fixture shape here).
Assertions (mock LLM; the `done`-frame `suggestions` list is the observable tier — the phase-119 done-frame shape):
- **(1) the accented question** `POST /api/chat` "How do I build the BESTÅ?" → the `done` frame's `suggestions` include `ikea_instructions/besta-frame-white__AA-1272068-11-100.md` (the D1 question-side fold — `BESTÅ` → `besta` → the stem sub-component name-hit);
- **(2) the ASCII regression** "How do I build a Besta?" → the same doc in `suggestions` (the phase-119 path, unchanged by the fold);
- **(3) the overview tail** — after the fixture import (the mock's `KB_OVERVIEW_MODE` regeneration ran), the stored `kb_overview` row (read through the fixture's DB session, the phase-119 pattern) ends with `Sources: …` naming **every** fixture source with exact counts — the 1-doc `ikea_instructions` source visible (the D2 gate);
- **(4) the gate sanity** — an out-of-KB question (e.g. "What is the capital of Mongolia?") returns the deflected terminal (the D1–D3 changes must not move the A8 gate verdict; the mock's cosine behavior makes this deterministic — follow the phase-119 deflection-sanity assertion).
2. **The pre-fix evidence — measured, recorded in THIS file (before the final gates):** `git stash` the task-01 `_normalize_name` change (tasks 02–03 changes may stay — they do not affect the name-hit path), run assertion (1) against the fixture, record the measured suggested top-5 ranks for the accented question (the `ikea_instructions` doc's rank or its absence), `git stash pop`. The recorded block must show the doc OUT of the suggested top-5 pre-fix (if the fixture as built does not show that, go back to step 1's tuning and re-measure — the recorded block is the evidence the fold fixes a real miss, not a no-op).
3. `tests/fixtures/retrieval_battery.txt` — append `How do I build a Besta?` with a gate comment (the file's existing comment style): on the 1,000-doc live replica (the `restore-test-db` skill — the dump is a local gitignored artifact; **if missing, STOP and ask the owner, never fabricate numbers**) the battery runs when `SELECT DISTINCT source FROM documents` shows an `ikea_instructions`-class source, and the printed `suggested:` top-5 must include `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`; if the replica lacks that source (the snapshot may predate the import), the five EXISTING phase-119 questions still run — their recorded gate (the canonical docs' ranks, the qwen3.8 regression, the deflection sanity `HIGH, fts_hits=5`) must be **unchanged** by this phase — and the Besta question is recorded in the task file as **skipped loudly (source absent in the replica snapshot)** — the phase-140 "skipped loudly" precedent. The under-rank hypothesis (the overview's failure #4) is thus either pinned green or handed to the owner with the measured ranks.
4. Regression + final gates: the phase-119 (`test_name_signal_read_chips.py`), phase-117, and phase-31-era regression suites green; `uv run pytest` (full suite) green; `uv run pytest --cov=app --cov-report=term-missing` coverage **>90%** on `app/`; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- E2E: the dedicated suite in isolation (per AGENTS.md rule 4 — one file per phase, run in isolation); unit pins from tasks 01–03 already in place.
- Battery: per step 3, recorded in this file.
- Coverage: **>90%** on `app/`; lint + types clean.
## Completion Criteria
- [ ] The recorded pre-fix block in this file shows the accented question losing the `ikea_instructions` doc from the suggested top-5; the post-fix E2E (assertions 1–4) is green in isolation.
- [ ] The battery question is recorded per step 3 (pinned green, or skipped loudly with the five phase-119 questions' gate verified unchanged).
- [ ] `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] The owner follow-up (A6) is noted in the final pass's summary for the owner (the steering-note "rule 13" is a runtime Tuning-tab action, not code).
@@ -0,0 +1,16 @@
**Phase 162 — final verification pass (all tasks already complete; verified + docs pass)**
- Verified: `_fold` (NFKD+casefold, stdlib) inside `_normalize_name` — `BESTÅ`→`besta` question side, `BÅsta.md`→`basta` path side; A2 match classes/ranking/caps/bonus untouched; FTS/cosine/A8 untouched.
- Verified: `grep_document` folds both sides via the imported `_fold` (no second copy); regex-teaching lines byte-identical; 3 copy sites updated.
- Verified: `regenerate_overview` appends the deterministic `Sources: <name> (<k> doc|docs)` tail (happy path only; zero-doc + `LLMError` rows byte-identical; `build_overview_prompt`/`load_kb_overview`/`build_kb_section` untouched).
- Verified: `TOOLS_SECTION` grep sentence updated + D4 search-discipline sentence present **verbatim**, immediately after the grep sentence; `PERSONA`/LOW pins byte-identical (`test_prompt_lock.py`).
- Measured evidence (in `.agents/reports/162_…/task04_prefix_evidence_and_battery.md` + re-measured every run by `test_under_rank_is_measured_not_assumed`): pre-fix tokens `['build','best']` → IKEA doc rank 6 (outside top-5); post-fix `['build','besta']` → suggested #1, cosine unchanged 0.382.
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated); neighbours `test_kb_overview{,_freshness} test_name_signal_read_chips test_grep_regex_teaching test_harness_aligned_tools test_agent_document_tools test_tool_call_compact` → **24 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **3756 passed**, app coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**.
- Completion criteria: fold both ways ✅ · `Sources:` tail ✅ · TOOLS_SECTION + D4 verbatim + re-cut pins ✅ · E2E green/battery recorded/full suite/coverage/lint ✅ · docs pass ✅ · commit + dir move = harness.
- Battery: 5 phase-119 questions byte-identical to the phase-119 record (GATE PASS); the 2 Besta questions **skipped loudly** — verified the 2026-09-05 replica dump has zero `ikea` matches (`grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0); rank gate stays with the owner's live KB.
- Docs fixed: README — overview-tail doc + `BOR_KB_OVERVIEW_MAX_CHARS` tail-cut constraint, the phase-162 search-discipline paragraph, per-phase (not per-story) E2E wording, new "Retrieval regression battery" section; AGENTS.md — rule 4 per-phase wording + battery command in Quick reference. (The overview's expected "AGENTS.md §6 tool table" does not exist in this repo — the grep contract line lives in README's tool table, already updated.)
- Deviations (recorded, not assumed): `_fold` adds `casefold` (required for the pinned `ß → ss`); `①` NFKD-decomposes to `1` (the overview's parenthetical was wrong) and is excluded by `NAME_TOKEN_MIN_LEN`; pre-fix evidence lives in the reports dir because task files are read-only for this pass.
- Owner follow-up A6 (runtime, not code): add the Tuning-tab steering note "IKEA's BESTÅ assembly PDF is a live source — search it as `BESTÅ` or `AA-1272068`".
- Next pending phase: none — 162 is the only entry in `todo/` (harness moves it to `complete/`).
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,13 @@
All gates green. Final verification complete.
**Report**
- **Implemented (task 162/01, D1 / LOCKED A1):**
- `app/rag/retriever.py`: new module-level `_fold()` (stdlib NFKD + combining-class strip, shared with task 03's grep); `_normalize_name` = alnum scan of `_fold(s.lower())` — the single normalization both sides pass through; `name_hit_tokens`/`_name_parts`/`_name_token_matches` consumed it unchanged (docstring clauses only); module docstring normalization paragraph states the 2026-10-07 gap.
- `tests/unit/test_retriever.py`: +9 tests — `_fold` cases (Å/ÄÖÜ/ß/fi/ç, CJK, ①), `BESTÅ`→`besta` token pinned exactly (`["build", "besta"]`), `_name_parts` (`besta` in equal set unchanged; `BÅsta.md`→`basta`), end-to-end bridge both directions via `_name_hit_chunks`, min-length on the folded form, digitless-precision non-regression (`déploy` ≠ `Deployments`).
- **Tests/lint/coverage:** `uv run pytest tests/unit/test_retriever.py tests/unit/test_retriever_recency.py -v` → 94 passed; full `uv run pytest` → **3747 passed**; `--cov=app` → 99% TOTAL, retriever 99% (3 pre-existing misses); `uv run ruff check .` + `uv run pyright` clean; phase-119 E2E `tests/e2e/test_name_signal_read_chips.py` → 4 passed (FTS/gate suites untouched).
- **Docs:** README "How retrieval works" — lexical list now names the name-hit path + accent fold (phase 162). No AGENTS.md impact (its grep-contract line is task 03/final-pass scope).
- **Decisions/deviations:**
1. NFKD alone leaves `ß` intact; the task-pinned `ß → ss` case requires `str.casefold()` — applied inside `_fold` (still stdlib-only, single shared function, A1 intent intact; documented).
2. `①` is compat-decomposable (NFKD → `1`), contrary to the task's "no decomposition" note: it still drops as a token (<4 chars); its only footprint is a prefix-class versioned-join artifact (`the1`) no path can match — pinned as real behavior and documented.
- **Next pending task:** `162_search_recall_small_sources/02_kb_overview_source_inventory.md`.
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 97%]
........................................................................ [ 99%]
... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5780 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,17 @@
All gates are green. Task 02 is complete.
**Report — task 02 `02_kb_overview_source_inventory.md`**
- **Implemented** (verified/completed in working tree — a prior interrupted run had left the code/tests uncommitted; I audited them against D2/A2, confirmed byte-identity of `build_overview_prompt`/`OVERVIEW_INSTRUCTION`/`load_kb_overview`/`build_kb_section`, and finished the doc work):
- `_source_inventory(rows)` in `app/rag/overview.py` — deterministic, non-LLM `Sources: <name> (<k> doc|docs), …`, distinct sources, catalog order, singular/plural.
- `regenerate_overview` happy path stores model text (rstripped `\n`) + `\n` + tail; log `chars=` = full stored content; zero-doc and `LLMError` paths untouched (previous row + tail preserved).
- Module + function docstrings record phase 162 / A2 / the `BOR_KB_OVERVIEW_MAX_CHARS` budget note.
- Unit tests in `tests/unit/test_overview.py`: format/order/counts, 1-doc survival (`ikea_instructions` case), exact happy-path stored content, tail-less zero-doc row preserved, `LLMError` preserves an existing tail, model input never gains the tail, exact char log; re-cut pins in `tests/integration/test_import_docs_overview.py`, `tests/e2e/test_kb_overview.py`, `tests/e2e/test_kb_overview_freshness.py`.
- **Tests / lint / coverage** (DB container started — it was down):
- `uv run pytest tests/unit/test_overview.py -v` → 29 passed
- `uv run pytest --cov=app --cov-report=term-missing` → **3753 passed**, `app/` TOTAL **99%**, `app/rag/overview.py` **100%**
- `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed; `tests/e2e/test_kb_overview_freshness.py` → 3 passed
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
- **Docs**: `README.md` — import section now documents the deterministic `Sources: …` tail (phase 162) and the failure-path preservation; `AGENTS.md` — no doc impact (the grep-contract line there is task 03/04's scope).
- **Decisions**: inventory order via `sorted()` on source names = catalog order regardless of the recency-first row order the generator reads (proven by the scrambled-input test); no other e2e asserts exact overview-row content (checked).
- **Next pending task**: `03_grep_folding_search_discipline.md`.
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
......... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5787 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,22 @@
Task 03 complete. Report:
**Implemented**
- `app/rag/agent.py` — `grep_document` now folds **both** pattern and each line through the shared `_fold` imported from `app.rag.retriever` (one fold, no second copy; LOCKED A3): `_fold(pattern.lower()) in _fold(line.lower())`; rstrip/line-numbers/20-cap/200-char truncation/locator semantics untouched.
- Copy updates (LOCKED A3): module-docstring grep paragraph, `AGENT_TOOLS` grep function description, and `pattern` parameter description now state "case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`"; `NO_MATCHES_REGEX`/`_SCOPED` + `looks_like_regex`/`plain_form` byte-identical.
- `app/rag/prompts.py` — `TOOLS_SECTION` grep sentence folded-contract re-worded; new `SEARCH_DISCIPLINE_RULE` constant (LOCKED A4 sentence, verbatim) inserted immediately after the grep sentence; `PERSONA`/LOW/other sentences byte-identical.
- Pins re-cut (phase-124 pattern): `test_prompt_lock.py` (new sha `988642f9…`/len 3818, pre-phase-162 anchors, chained phase-162→145 diff-style pin, discipline-rule position pin, grep-contract pin), `test_prompts.py` (grep clause + D4 boundary pins), `test_agent.py` (`AGENT_TOOLS` + pattern-param description pins).
- New fold tests in `test_agent.py`: accented↔ASCII both ways (`besta`→`BESTÅ`, `basta`→`BÅsta`, `cafe`→`café`, `BESTÅ` pattern→ASCII content, `strasse`→`Straße`), non-match still `[]`.
- `README.md` — grep tool table row updated to the folded contract.
**Tests / lint / coverage**
- `uv run pytest tests/unit/test_agent.py tests/unit/test_prompt_lock.py tests/unit/test_prompts.py` → **292 passed**
- `uv run pytest` (full suite) → **3756 passed**
- Related E2E spot-checks: `test_grep_regex_teaching.py`, `test_tool_call_compact.py`, `test_kb_overview*.py`, `test_name_signal_read_chips.py` → **green** (11 passed in the last batch)
- `uv run pytest --cov=app` → TOTAL **99%** (`agent.py` 99%, `prompts.py` 100%) — >90% gate met
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
**Docs**: `README.md` grep-tool row updated (case- and accent-insensitive, fold example); `AGENTS.md` has no grep-contract line to change (the phase-162 final pass handles the §6 wording per the phase overview).
**Decisions**: reworked the phase-145 "only byte change" pin into a chained phase-162→145 revert pin so both historical anchors stay enforced; `SEARCH_DISCIPLINE_RULE` added as a module constant mirroring the `ORIGIN_RULE` pattern. No deviations from LOCKED A3/A4.
**Next pending task**: `162_search_recall_small_sources/04_e2e_battery_pins.md`
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,18 @@
**Phase 162 / task 04 — complete, all gates green.**
**Added** `tests/e2e/test_search_recall_small_sources.py` (7 tests), fixture KB `tests/fixtures/recallkb/` (3 sources: `ikea_instructions` 1 doc with the name **only** as `BESTÅ`, `phase_docs` 4 + `players_handbook` 1 ASCII-`besta`/`build` distractors), the Besta pair + gate comment in `tests/fixtures/retrieval_battery.txt`, and evidence in `.agents/reports/162_search_recall_small_sources/task04_prefix_evidence_and_battery.md`.
**Commands / outcomes**
- `uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov` → **7 passed** (isolated, run twice incl. after the DB restore).
- `uv run pytest tests/e2e/test_name_signal_read_chips.py tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
- `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, 3756 tests, app coverage 99%**.
- `uv run ruff check .` → passed; `uv run pyright` → **0 errors**.
- Replica restored per the `restore-test-db` skill (fingerprint verified docs=1000/chunks=8866/sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic 0011→0026), then `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` → **GATE PASS 1–5**: gitea README rank 4, forgejo 5/5 with README #1, gateway README rank 3, qwen3.8-27b quadlets #1/#2, Mongolia HIGH/fts_hits=5 — the five `suggested:` lists are **byte-identical** to the phase-119 record, and a re-run with the fold reverted is **byte-identical** to the post-fix run.
**Pre-fix evidence (measured, both directions).** With `app/rag/retriever.py::_normalize_name` reverted by a one-line reversible edit, `How do I build the BESTÅ?` loses the doc: tokens `['build','best']` → no name hit, suggested 5 = the five ASCII distractors, IKEA doc **rank 6** — visible on the wire as its `done.related` entry and its absence from the folder-map echo (gate still HIGH, cosine 0.382, on the wrong docs). Shipped code: tokens `['build','besta']` → IKEA doc **suggested #1**, related = `phase_docs/156_history_source_filter.md`, cosine unchanged (0.382). The hypothesis is now re-measured on every run by `test_under_rank_is_measured_not_assumed`.
**Decisions / deviations** (harness mechanics win): task/phase files untouched, so the evidence lives in the report file above; no `git stash` (reversible source revert instead, verified restored, `grep -c TEMP` = 0); `done.suggestions` is the deflection-chip contract and never carries paths, so the suggested tier is asserted via the durable `query_log.sources` record (suggested 5 + remainder), `done.related`, and the mock's folder-map echo; tuning re-worded two bodies to drop stopword density so the out-of-KB control deflects (cosine 0.3345 → 0.2083) — both token controls are pinned as assertions; the battery carries both `How do I build a Besta?` (task text) and `How do I build the BESTÅ?` (the live miss).
**Skipped loudly:** the 2026-09-05 replica snapshot has no `ikea_instructions`-class source (`grep -c -i ikea` on the dump = 0), so the Besta rank gate stays with the owner's live KB; both questions still ran without a false grounding on a doc naming the product.
**Docs:** no README/AGENTS change needed (task 04 is tests + fixtures + battery; user-visible behavior was documented by tasks 01–03). DB left as found (pre-restore backup restored); no other pending task for this phase — task 04's file is ready for the harness to move, and the phase commit is still owed.
@@ -0,0 +1,124 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............ [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 277 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 196 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 204 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 266 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 135 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 454 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 764 4 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 156 0 100%
app/rag/retriever.py 252 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 470 0 100%
--------------------------------------------------
TOTAL 5788 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,174 @@
# Task 04 record — pre-fix evidence + battery (phase 162, D5/D6)
**Deviation note (mechanics):** the task file says "record the measured
pre-fix ranks in THIS file" and to revert the fold with `git stash`, but
the execution harness forbids editing phase/task files and forbids
index/history-touching Git commands. Per the conflict rule the harness
wins on mechanics and the intent is preserved: the evidence lives HERE,
the fold was reverted by a one-line, reversible edit of
`app/rag/retriever.py` (restored and byte-verified afterwards —
`grep -c TEMP app/rag/retriever.py` = 0), and the summary goes in the
task's final report. The permanent, non-throwaway form of the evidence is
`tests/e2e/test_search_recall_small_sources.py::test_under_rank_is_measured_not_assumed`,
which re-measures BOTH directions on every run.
## 1. The fixture (tests/fixtures/recallkb — three local-directory sources)
| source | docs | role |
| --- | --- | --- |
| `ikea_instructions` | **1** (`besta-frame-white__AA-1272068-11-100.md`) | the live miss: body names the product **only** as accented `BESTÅ` |
| `phase_docs` | 4 (`140_pictorial_pdf_pages`, `148_pdf_text_backfill`, `152_seaweedfs_blob_ingest`, `156_history_source_filter`) | ASCII `besta` + `build` distractors |
| `players_handbook` | 1 (`game_night_setup.md`) | ASCII `besta` + `build` distractor in a second small source |
Final shape tuning (the task's "add distractors until it loses"): the
initial 4-doc draft already lost the tier, but the deflection control
failed (best cosine 0.3345 — the small source's `Assembly` section was
dense in the stopword `the`, which the mock's tokenizer keeps). Two
bodies were re-worded (`ikea_instructions` Assembly paragraph,
`players_handbook/game_night_setup.md`) to drop stopword density: the
out-of-KB cosine fell to **0.2083** (< 0.30 threshold → deflects) while
the on-topic questions kept 0.38 / 0.44. Two token controls are pinned
as assertions in `synced_kb`: (a) no ASCII `besta` and no `build` anywhere
in the small source, (b) the accented name sits **past** the mock's
24-token summary-digest window, so the ASCII-only digest never leaks a
`best` lexeme the pre-phase FTS could have hit.
## 2. Measured pre-fix evidence — the E2E fixture, both directions
Lens: the app's REAL `retrieve()` + `select_suggested()` over the synced
KB with the mock's embeddings. Gate: threshold 0.30, lexical-support
floor 0.15 (the conftest mock calibration). Bonus 0.005, recency 0.0007,
suggested 5 / related 2 (code defaults, `.env`-free like production).
Question `How do I build the BESTÅ?`
**PRE-FIX — the real revert** (`_normalize_name` →
`"".join(_ALNUM_RE.findall(s.lower()))`, the exact task-01 before-state),
`pytest tests/e2e/test_search_recall_small_sources.py` run against the
fixture (full log: `/tmp/prefix_e2e_run.log`, `/tmp/prefix_wire_run.log`):
```
name tokens : ['build', 'best'] → NO name hit
suggested 5 : phase_docs/140_pictorial_pdf_pages.md
phase_docs/148_pdf_text_backfill.md
phase_docs/152_seaweedfs_blob_ingest.md
players_handbook/game_night_setup.md
phase_docs/156_history_source_filter.md
related : ikea_instructions/besta-frame-white__AA-1272068-11-100.md ← RANK 6, the miss
gate : best cosine 0.382, fts_hits 8 → grounded (on the WRONG docs)
done frame : {'source': 'ikea_instructions',
'path': 'besta-frame-white__AA-1272068-11-100.md',
'title': 'Assembly Instructions'} ← the rank-6+ "Nearby docs" slot
folder echo : (folders: phase_docs/: 148_…, 152_…, 156_…; players_handbook/:)
→ NO `ikea_instructions/` line at all: the small source never
reached the HIGH prompt
```
**POST-FIX (shipped code)** — same fixture, same measurement:
```
name tokens : ['build', 'besta'] → name hit
suggested 5 : ikea_instructions/besta-frame-white__AA-1272068-11-100.md ← #1
phase_docs/140_pictorial_pdf_pages.md
phase_docs/148_pdf_text_backfill.md
phase_docs/152_seaweedfs_blob_ingest.md
players_handbook/game_night_setup.md
related : phase_docs/156_history_source_filter.md
gate : best cosine 0.382, fts_hits 9 (the fold adds the LEXICAL term;
cosine untouched — LOCKED A1)
folder echo : … includes `ikea_instructions/:` (header alone — its only
document IS the suggested doc)
```
Mechanism (measured, and why the fix is the name signal and nothing
else): every distractor matches the question's other FTS token `build`,
so each fuses TWO RRF terms (vector + FTS ≈ 0.032) while the small source
— no `build`, and `BESTÅ` lexes as `bestå`, which the ASCII-`best`
tsquery cannot match — fuses ONE (≈ 0.016) and lands rank 6. The folded
name hit is the only signal big enough to lift it: name hits LEAD the
lexical list (`fuse` rank 1 = 1/61) plus the per-document
`BOR_NAME_HIT_BONUS` (0.005). ASCII question `How do I build a Besta?`
measures the SAME tier before and after (suggested #1, cosine 0.4404,
related `phase_docs/156_history_source_filter.md`) — the phase-119 path
is untouched.
Under-rank hypothesis (D1 failure #4): **pinned green with measured
ranks**, not assumed.
## 3. The dedicated suite
`uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov`
→ **7 passed** (task-04 cases 1–4 plus the measurement test, the grep
fold at the content layer, and the folder-line prompt witness).
Interpretation note: the task text says the `done`-frame `suggestions`
list is the observable tier. On the shipped wire `done.suggestions` is
the DEFLECTION contract (2–3 title-derived "Maybe try" chips, `[]` on
grounded turns — `app/api/chat.py`, phase 04) and never carries document
paths. The faithful observables of the SUGGESTED tier are (a) the durable
`query_log.sources` record = suggested 5 then the rank-6+ remainder
(phase 118 A3) and (b) `done.related` = the rank-6+ tier. The suite
asserts both (plus `done.sources == []` — read-docs-only, phase 119
LOCKED A1 — and the mock's folder-map echo for the prompt-level witness).
Neighbour suites re-run (the retriever + overview were touched by this
phase): `uv run pytest tests/e2e/test_name_signal_read_chips.py
tests/e2e/test_kb_overview.py -q --no-cov` → **8 passed**.
## 4. Battery — `tests/fixtures/retrieval_battery.txt`
The file gains the Besta pair (`How do I build a Besta?` — the task's
line — and `How do I build the BESTÅ?`, the live miss itself) with the
gate comment. The five phase-119 questions keep positions 1–5.
Preconditions (the `restore-test-db` skill, run in-container): safety
backup → `DROP SCHEMA public CASCADE` → single-transaction load of
`data/bor_live_kb_replica.dump.sql` → **fingerprint verified:
docs=1000, chunks=8866,
sources=deploy,homelab,homepage,ServMon,skills,vibe-bot, alembic=0011**
→ `uv run alembic upgrade head` (0011 → **0026**) → docs=1000 re-verified.
`uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt`
(full output `/tmp/battery_postfix.log`, pre-fix comparison
`/tmp/battery_prefix.log`):
| # | question | verdict |
| --- | --- | --- |
| 1 | How do I deploy gitea? | HIGH, `fts_hits=38`; suggested top-5 **byte-identical** to the phase-119 record — `gitea/README.md` **rank 4** ✓ |
| 2 | How do I deploy forgejo? | HIGH, `fts_hits=37`; forgejo tree 5/5, `forgejo/README.md` **#1** ✓ |
| 3 | How do I set up the gateway? | HIGH, `fts_hits=37`; `gateway/README.md` **rank 3** ✓ |
| 4 | Do you have notes on the qwen 3.8 27b model? | HIGH, `fts_hits=31`; both `qwen3.8-27b-*` quadlets at #1/#2 ✓ (the digit-prefix regression holds) |
| 5 | What is the capital of Mongolia? | **HIGH, `fts_hits=5`** — unchanged ✓ |
| 6 | How do I build a Besta? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
| 7 | How do I build the BESTÅ? | **SKIPPED LOUDLY — source absent in the replica snapshot** |
`SELECT DISTINCT source FROM documents` on the restored replica lists
`deploy, homelab, homepage, ServMon, skills, vibe-bot` — NO
`ikea_instructions`-class source (`grep -c -i ikea
data/bor_live_kb_replica.dump.sql` = 0: the 2026-09-05 snapshot predates
the IKEA import, exactly the case D5 anticipated; phase-140's
"skipped loudly" precedent). The two questions still ran and did not
crash; both return the `build.sh`/`bifrost`-class top-5 that a KB without
the product can only return, and neither grounds on a document that names
the product. **The rank gate therefore stays with the owner** — it can
only close on a KB that has the source (the live KB).
Behavior-preservation, measured (not argued):
* the five phase-119 `suggested:` lists are **byte-identical** to the
recorded 2026-09-16 run in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md`;
* the battery re-run with the task-01 fold REVERTED is **byte-identical**
to the post-fix run (`diff` of the question/gate/suggested lines: no
difference) — the fold cannot move an ASCII-only corpus, as designed;
* `best_cosine` moves by ≤ 0.0004 between the September record and today
(real-`embed` model jitter, not code).
**Verdict: GATE PASS** for questions 1–5 (unchanged), 6–7 recorded as
skipped loudly with the reason and the re-check command.
## 5. Environment left behind
The replica was restored only for the battery. The pre-restore DB state
(backup `/tmp/pre_restore_host.dump`, 13 leftover E2E fixture docs,
alembic 0026) is restored afterwards — reload the replica any time with
the `restore-test-db` skill.
+4 -2
View File
@@ -9,8 +9,9 @@
3. **Strictly adhere to the LOCKED DECISIONS** (PLAN §2). If you believe a
LOCKED decision is wrong, stop and flag it — do not silently deviate.
4. **"One Story, One Phase":** each user story in `.agents/user_stories/`
corresponds to a distinct execution phase that includes its own dedicated
Playwright E2E test suite (one test file per story, run in isolation).
corresponds to a distinct execution phase, and **every** phase — story-driven
or owner-directed — includes its own dedicated Playwright E2E test suite
(one test file for the phase, run in isolation).
5. **"UI Structure Check":** before finalizing any UI component, verify that
it follows the layout principles in `.agents/PLAN.md` §7 (proper container
usage, the 72rem centered reading column — every page rides the 72rem
@@ -57,4 +58,5 @@ uv run pytest --cov=app --cov-report=term-missing
uv run playwright install chromium # once
uv run pytest tests/e2e/test_smoke.py -v --no-cov
uv run ruff check . && uv run pyright # lint + types
uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt # retrieval regression battery (phases 119/162)
```
+43 -8
View File
@@ -280,7 +280,12 @@ reflect the new files.
- A run that changed the KB also regenerates the **KB overview** and the
**folder summaries** (the drill-down catalog's stored per-folder
descriptions) with the `lite` model — best-effort: a summary failure
never fails the import. Because the sync only rewrites the folders a
never fails the import. The stored overview ends with a deterministic
`Sources: <name> (<k> doc|docs), …` line — every source with its exact
document count, catalog order, computed without a model call — so no
source (not even a 1-doc one) is invisible to `<knowledge_base>`
(phase 162); the `LLMError`/zero-doc paths leave the previous row
(tail included) untouched. Because the sync only rewrites the folders a
change touches, a one-shot
`uv run python -m scripts.regen_folder_summaries` refreshes EVERY
stored folder summary in one run (full mode: every candidate folder
@@ -492,11 +497,20 @@ server-side tools — on **grounded** (high-relevance) turns only:
|------|-------------|
| **`ls`** | Lists the knowledge base as a **drill-down tree**, one level at a time: no `path` = every source (document count + stored description), a source name = that source's top level, a `source/folder` path = one level deeper; file lines are `source: X \| path: Y \| title: Z` |
| **`read(path, offset?, limit?)`** | Appends one indexed document's text to context by its combined `source/path`; optional `offset` (1-based first line) + `limit` (line count) read just that part of a large document; very large reads are truncated at `BOR_READ_MAX_CHARS` with a notice that teaches the workflow — grep reports line numbers, then read that part with `offset` + `limit` |
| **`grep(pattern, path?)`** | Searches indexed documents for an exact string (case-insensitive fixed substring); returns up to 20 `source/path:line: text` matches; an optional `path` limits to one document |
| **`grep(pattern, path?)`** | Searches indexed documents for an exact string (case- and accent-insensitive fixed substring — accents fold: `besta` matches `BESTÅ`, phase 162); returns up to 20 `source/path:line: text` matches; an optional `path` limits to one document |
| **`show_image(path)`** | *(only when `BOR_ANSWER_IMAGES=1`)* shows one standalone image document as part of the answer — image documents only, at most one per answer |
(See [Answer images](#answer-images-phase-124) for the `show_image` tool.)
**The prompt teaches the search discipline (phase 162).** A summary or an
outline is a hint, never an inventory: the system prompt tells the model to run
`ls` with no `path` — the full source list — before ever claiming the knowledge
base lacks something, and to `grep` the artifact's part number or a filename
token. One grep pattern is one string, so a name's split tokens (`frame`,
`white`) and its part number (`AA-1272068`) each need their own grep; and a
note saying a file is gitignored describes its **repo** status, not whether it
is indexed as a source.
**Git-source provenance (phase 145).** When a document comes from a git
source, its `read` result also carries an `origin:` line (the repository's
URL, credentials stripped) and, when the source pins one, a `branch:` line.
@@ -583,7 +597,12 @@ stemming) and searched **twice** against Postgres:
1. **Vector** — pgvector cosine top-N (default `BOR_HYBRID_VECTOR_CANDIDATES=100`)
2. **Lexical** — a stored `tsvector` (GIN-indexed) matched with `to_tsquery`,
top-N by `ts_rank` (default `BOR_HYBRID_LEXICAL_CANDIDATES=30`)
top-N by `ts_rank` (default `BOR_HYBRID_LEXICAL_CANDIDATES=30`), led by
**name hits** — documents whose *path* matches a question token (`gitea`
→ `gitea/…`, `qwen38` → `qwen3.8-27b-…`; phase 119). Name normalization
is **accent-folded** (phase 162): a question's `BESTÅ` folds to `besta`
and matches the ASCII filename, and an accented filename (`BÅsta.md`)
matches an ASCII question — both directions, stdlib fold only.
The two ranked lists are fused with **Reciprocal Rank Fusion**
(`score = Σ 1/(k + rank)`, `BOR_RRF_K=60`) — a chunk in both lists scores
@@ -917,7 +936,7 @@ to `.env` and adjust.
| `BOR_MAX_OUTPUT_TOKENS` | `32768` | Max answer length in tokens (answers must not be cut off) |
| `BOR_STEERING_MAX_CHARS` | `8000` | Char budget for the `<tuning>` (steering notes) prompt section |
| `BOR_SUMMARY_MAX_CHARS` | `12000` | Cap on document content sent to the `lite` summary model |
| `BOR_KB_OVERVIEW_MAX_CHARS` | `4000` | Char budget for the `<knowledge_base>` prompt section |
| `BOR_KB_OVERVIEW_MAX_CHARS` | `4000` | Char budget for the `<knowledge_base>` prompt section — the stored outline **plus** its deterministic `Sources: …` tail (phase 162); the cap keeps the longest-fitting prefix, so a value below the default clips the tail first (the default always holds both) |
| `BOR_OVERVIEW_INPUT_MAX_CHARS` | `40000` | Cap on the document list sent to `lite` for KB overview generation |
| `BOR_FOLDER_SUMMARY_INPUT_MAX_CHARS` | `8000` | Cap on a folder's document list sent to `lite` for folder summaries (phase 94) |
@@ -1058,8 +1077,9 @@ normally until you attach. Override the port with `DEBUGPY_PORT`.
## QA / Testing
Three layers — the project rule is **one story, one phase, one Playwright
suite** (see `AGENTS.md`):
Three layers — the project rule is **one phase, one Playwright suite** (a
phase may implement a user story or an owner-directed change; see
`AGENTS.md`):
```bash
# Unit + integration (FastAPI TestClient)
@@ -1075,12 +1095,27 @@ uv run pyright
# Playwright E2E — install the browser once:
uv run playwright install chromium
# Each story's E2E runs IN ISOLATION:
# Each phase's E2E runs IN ISOLATION:
uv run pytest tests/e2e/test_import_documents.py -v --no-cov
uv run pytest tests/e2e/test_chat_rag.py -v --no-cov
# ...one file per story in .agents/user_stories/ (see .agents/phases/todo/)
# ...one file per phase in tests/e2e/ (the stories live in
# .agents/user_stories/, see .agents/phases/todo/)
```
**Retrieval regression battery.** `tests/fixtures/retrieval_battery.txt` is
the tracked retrieval gate (phases 119 + 162): the name-your-tool questions,
the versioned-name (`qwen 3.8 27b`) regression, the out-of-KB deflection
sanity, and the `besta` / `BESTÅ` accent-fold pair. Run it against a KB with
`scripts.eval_retrieval` (the real pipeline, real embeddings):
```bash
uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt
```
The exact expected ranks are recorded in the fixture's header comments; a
question whose source is missing from the KB snapshot is recorded as skipped
loudly, never silently dropped.
**Deterministic E2E:** by default the E2E app talks to a local **mock
aipi** (`tests/e2e/mock_llm.py`) whose embeddings are real token-overlap
vectors — so the cosine relevance threshold behaves like production. To
+21 -11
View File
@@ -131,8 +131,10 @@ task 04):
the only capped read (the phase-95 cap unchanged). And ``grep``
greps the
indexed documents (or the one document a combined ``source/path``
names) for a case-insensitive fixed substring and returns up to 20
``source/path:line: text`` match lines (owner-locked A5, phase 68),
names) for a case- and accent-insensitive fixed substring (accents
fold — ``besta`` matches ``BESTÅ``; phase 162, LOCKED A3) and
returns up to 20 ``source/path:line: text`` match lines
(owner-locked A5, phase 68),
each line truncated to 200 chars. A grep is a **locator**, not a
context-adder: it never appends to the answer context (only
``read`` does — ``holder.read_docs`` is untouched by a grep).
@@ -312,7 +314,7 @@ from app.rag.llm import (
ToolResultPiece,
chat_stream_retried,
)
from app.rag.retriever import TRUNCATION_MARKER
from app.rag.retriever import TRUNCATION_MARKER, _fold
from app.rag.scaffolding import ScaffoldingFilter
from app.rag.source_removal import resolve_source_name
@@ -437,7 +439,8 @@ AGENT_TOOLS: list[dict[str, Any]] = [
"name": "grep",
"description": (
"Search the indexed documents for an exact string "
"(case-insensitive) and return up to 20 matching lines "
"(case- and accent-insensitive — accents fold: `besta` "
"matches `BESTÅ`) and return up to 20 matching lines "
"as `source/path:line: text` — a locator, not a "
"context-adder: read the winner with `read` — for a "
"large document, use the reported line number as "
@@ -458,7 +461,9 @@ AGENT_TOOLS: list[dict[str, Any]] = [
"type": "string",
"description": (
"The exact text to search for (a plain "
"substring, not a regex — no '.*', no "
"substring — case- and accent-insensitive, "
"accents fold: `besta` matches `BESTÅ` — "
"not a regex — no '.*', no "
"'\\.', no character classes)"
),
},
@@ -1560,16 +1565,21 @@ def format_covered_spans(spans: Sequence[tuple[int, int]]) -> str:
def grep_document(content: str, pattern: str) -> list[tuple[int, str]]:
"""Every line of *content* that contains *pattern*, in file order.
Case-insensitive **fixed substring** (owner-locked A5: no regex — no
ReDoS surface, a simple contract for the model). Returns
``(1-based line number, line.rstrip())`` pairs; an empty *content*
never matches a non-empty pattern.
Case- **and accent-insensitive fixed substring** (owner-locked A5:
no regex — no ReDoS surface, a simple contract for the model). Both
sides fold through the shared :func:`app.rag.retriever._fold` plus
lowercase (phase 162, LOCKED A3 — the fold changes the alphabet it
ignores, not the matching semantics: ``grep "besta"`` matches a
content line ``The BESTÅ frame``, and a pattern carrying accents
folds too), so grep and the name-hit path can never drift apart.
Returns ``(1-based line number, line.rstrip())`` pairs; an empty
*content* never matches a non-empty pattern.
"""
needle = pattern.lower()
needle = _fold(pattern.lower())
return [
(number, line.rstrip())
for number, line in enumerate(content.split("\n"), start=1)
if needle in line.lower()
if needle in _fold(line.lower())
]
+63 -7
View File
@@ -28,6 +28,21 @@ an import added/updated at least one document (or no row exists yet),
and a ``lite`` failure logs and leaves the previous outline intact — an
old outline is better than none (phase locked decisions).
After the model's outline, :func:`regenerate_overview` appends ONE
deterministic, non-LLM line — ``Sources: <name> (<k> doc|docs), …``
(phase 162, LOCKED A2). The 2026-10-07 live finding: the ``lite``
outline is lossy — it listed 3 groups of a 14-source index, so the
1-doc sources (``ikea_instructions``, ``players_handbook``) were
invisible to ``<knowledge_base>``. Small sources must be visible
without a model call, so the line is computed mechanically from the
catalogue rows (:func:`_source_inventory`) — the mock/real model
outputs never affect it, and ``build_overview_prompt`` never sees the
tail (the model input stays byte-identical). The longest-prefix cap in
``build_kb_section`` applies to the WHOLE stored text: at the default
``BOR_KB_OVERVIEW_MAX_CHARS`` (4000) the ~1500-char outline plus the
one-line tail (~15 chars/source) always fit; a smaller owner-set cap
cuts the tail first — accepted, the cap is owner-tunable.
The ``KB_OVERVIEW_MODE`` marker follows the ``SUMMARY_MODE`` /
``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on it
in the system prompt (wired in phase 31, task 05).
@@ -101,6 +116,35 @@ def _first_summary_line(summary: str | None) -> str:
return ""
def _source_inventory(rows: Sequence[tuple[str, str, str, str | None]]) -> str:
"""The phase-162 deterministic source inventory (LOCKED A2).
Mechanical, non-LLM: one entry per **distinct** source in catalog
order (``source, path`` — first occurrence in the ``source,
path``-ordered rows), each ``<name> (<k> doc|docs)`` — singular
``doc`` at k=1, ``docs`` otherwise — joined with ``, `` behind the
``Sources: `` prefix. The ``lite`` outline stays the model's job;
this line guarantees no source — not even a 1-doc one like the
2026-10-07 ``ikea_instructions`` case — can be invisible to
``<knowledge_base>``, whatever the model outputs. The mock/real
model outputs never affect it.
*rows* is the existing ``(source, path, title, summary)``
projection :func:`regenerate_overview` already reads — no new
query. At the default ``BOR_KB_OVERVIEW_MAX_CHARS`` (4000) the
outline plus this one line always fit; a smaller owner-set cap
cuts the tail first (accepted — the cap is owner-tunable).
"""
counts: dict[str, int] = {}
for source, _path, _title, _summary in rows:
counts[source] = counts.get(source, 0) + 1
entries = ", ".join(
f"{name} ({count} {'doc' if count == 1 else 'docs'})"
for name, count in sorted(counts.items())
)
return f"Sources: {entries}"
def build_overview_prompt(
rows: Sequence[tuple[str, str, str, str | None]], max_chars: int | None = None
) -> tuple[str, str]:
@@ -166,10 +210,16 @@ async def regenerate_overview(
* Zero documents → the existing row (if any) is left untouched and
``False`` is returned — no KB, no outline, no wasted model call.
* Happy path → the model's text is upserted into the single row
(``id = 1``, fresh UTC ``updated_at``), committed, and
``overview: regenerated docs=… chars=…`` is logged; returns
``True``. Idempotent by construction: one row, always ``id = 1``.
* Happy path → the model's text (trailing newlines rstripped) plus
the deterministic ``Sources: …`` inventory line (phase 162,
LOCKED A2 — :func:`_source_inventory`) is upserted into the
single row (``id = 1``, fresh UTC ``updated_at``), committed, and
``overview: regenerated docs=… chars=…`` is logged (chars = the
FULL stored content, tail included); returns ``True``. Idempotent
by construction: one row, always ``id = 1``. The model input
never sees the tail; the zero-doc and ``LLMError`` paths below
leave a previous row byte-identical — an existing tail is part of
the stored content and is preserved on failure.
* :class:`LLMError` → ``overview: regeneration failed — …`` is
logged and ``False`` is returned with the previous row intact —
an old outline is better than none.
@@ -209,15 +259,21 @@ async def regenerate_overview(
logger.error("overview: regeneration failed — %s", e)
return False
# Phase 162 (LOCKED A2): the deterministic source inventory is
# appended to the STORED content after the model's outline —
# mechanical, non-LLM, so no source can be invisible to
# `<knowledge_base>`. The model input above never sees the tail.
content = text.rstrip("\n") + "\n" + _source_inventory(rows)
now = datetime.now(UTC)
row = session.get(KbOverview, 1)
if row is None:
session.add(KbOverview(id=1, content=text, updated_at=now))
session.add(KbOverview(id=1, content=content, updated_at=now))
else:
row.content = text
row.content = content
row.updated_at = now
session.commit()
logger.info("overview: regenerated docs=%d chars=%d", len(rows), len(text))
logger.info("overview: regenerated docs=%d chars=%d", len(rows), len(content))
return True
finally:
if owns_session:
+30 -2
View File
@@ -265,6 +265,31 @@ ORIGIN_RULE: str = (
"again for the source."
)
#: The search-discipline rule (phase 162, task 03, LOCKED A4) — appended
#: to the ``TOOLS_SECTION`` tools paragraph immediately after the grep
#: sentence (i.e. after the ``…retry with the plain text you expect to
#: see.`` clause, the placement from the 2026-10-07 owner conversation
#: where the agent answered from the lossy ``<knowledge_base>`` outline
#: instead of running ``ls``, and where the exact-substring grep silently
#: missed the accented ``BESTÅ`` in the IKEA PDF's text layer). Adapted
#: verbatim from the conversation's proposed system-prompt patch for the
#: folded grep (LOCKED A3). The three lessons: (1) an outline/summary is
#: a hint, never an inventory — ``ls`` (no ``path``) before claiming the
#: KB lacks something; (2) grep folds accents but one pattern is one
#: string — split filename tokens and the part number need separate
#: greps; (3) a gitignored note describes repo status, not index status.
#: The mock E2E is unaffected (it keys on markers, not this sentence).
SEARCH_DISCIPLINE_RULE: str = (
"A summary or outline is a hint, never an inventory: before claiming "
"the knowledge base lacks something, run `ls` with no `path` to list "
"every source, and `grep` the artifact's part number or a filename "
"token. `grep` folds accents — `besta` matches `BESTÅ` — but one "
"pattern is one string: a name's split tokens (`frame`, `white`) and "
"its part number (`AA-1272068`) need separate greps. A note saying a "
"file is gitignored describes its repo status, not whether it is "
"indexed as a source."
)
TOOLS_SECTION: str = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -305,12 +330,15 @@ TOOLS_SECTION: str = (
"path as plain text otherwise. The user should be able to open the "
"complete original from your answer in one step, not ask again for "
"the source. `grep` locates an exact string "
"(case-insensitive) in the indexed documents and returns up to 20 "
"(case- and accent-insensitive — accents fold: `besta` matches "
"`BESTÅ`) in the indexed documents and returns up to 20 "
"matching `source/path:line: text` lines — a locator, not a "
"context-adder: read the winner with `read`. A grep pattern is a "
"plain substring, NEVER a regex — '.*' and '\\.' are literal text "
"there; if such a pattern returns no matches, retry with the plain "
"text you expect to see. For a normal search pass only `pattern` — "
"text you expect to see. "
+ SEARCH_DISCIPLINE_RULE
+ " For a normal search pass only `pattern` — "
"its optional `path` argument "
"limits the search to one document you already know, by the same "
"combined `source/path` string; never a source name — a bare "
+63 -6
View File
@@ -28,9 +28,21 @@
order ``(source, path)`` — the old total-matched-length tie-break
is retired, it outranked 5-char product names by 6-char common
tokens; capped at :data:`NAME_HIT_LIMIT`),
the FTS rows follow. This is what finds name-your-tool questions
("gitlab") that vector similarity buries — and, the 2026-09-05
incident, the versioned-name case the default parser lexes
the FTS rows follow. NAME NORMALIZATION IS ACCENT-FOLDED (phase 162,
LOCKED A1): :func:`_normalize_name` runs :func:`_fold` (NFKD
decomposition, combining marks dropped — stdlib only) before the
alnum scan, and it is the SINGLE normalization both sides of name
matching pass through (question tokens and path parts alike) — the
2026-10-07 live gap this closes: the IKEA filename is ASCII
(``besta-frame-white__AA-1272068-11-100``) while its content layer
spells the name only as ``BESTÅ`` (U+00C5), and the old lower-only
normalization turned the accented question token into ``best`` —
never a match. With the fold, ``BESTÅ`` → ``besta`` (was ``best``)
and an accented filename (``BÅsta.md`` → ``basta``) answers to the
ASCII question — one function, both directions. This is what finds
name-your-tool questions ("gitlab") that vector similarity buries —
and, the 2026-09-05 incident, the versioned-name case the default
parser lexes
incompatibly (question "Qwen 3.8" → ``qwen``/``3``/``8`` can never
match a document's ``qwen3``/``8``/``27b`` tokens, while every
unrelated llama.cpp quadlet out-ranks the target on the shared
@@ -81,6 +93,7 @@ from __future__ import annotations
import math
import re
import unicodedata
import uuid
from collections.abc import Sequence
from dataclasses import dataclass, replace
@@ -130,9 +143,42 @@ _ALNUM_RE = re.compile(r"[a-z0-9]+")
_STEM_SPLIT_RE = re.compile(r"[^a-z0-9]+")
def _fold(s: str) -> str:
"""The shared accent fold (phase 162, LOCKED A1): NFKD decomposition,
combining marks dropped (stdlib ``unicodedata`` + ``str.casefold``
only — no new dependency).
``Å/å → a``, ``Ä/Ö/Ü → a/o/u``, ``ß → ss``, ``fi → fi``, ``ç → c``
(``casefold`` is the Unicode fold — the ``ß → ss`` case rides it:
NFKD alone leaves ``ß`` intact — and ``str.lower`` leaves the ``fi``
ligature). Characters with no decomposition (CJK) survive the fold
and still drop out under the caller's alnum scan (behavior
unchanged for them); compatibility-decomposable characters (``①`` →
``1``) fold to their ASCII base, which :data:`NAME_TOKEN_MIN_LEN`
already excludes as a name token (a digit base can still feed the
versioned join as the numeric NEXT token — the only route left is a
prefix-class candidate no real path matches). Case is folded too
(the caller lowercases anyway). Imported by ``app.rag.agent`` for
the folded ``grep`` (phase 162, LOCKED A3) — one fold, never
drifting apart.
"""
decomposed = unicodedata.normalize("NFKD", s.casefold())
return "".join(c for c in decomposed if not unicodedata.combining(c))
def _normalize_name(s: str) -> str:
"""Lowercased, alnum-only form of *s* (``Qwen 3.8`` → ``qwen38``)."""
return "".join(_ALNUM_RE.findall(s.lower()))
"""Lowercased, ACCENT-FOLDED, alnum-only form of *s* (``Qwen 3.8`` →
``qwen38`` unchanged; ``BESTÅ`` → ``besta`` — the phase-162 fold).
This is the SINGLE normalization both sides of name matching pass
through — the :func:`name_hit_tokens` question tokens and the
:func:`_name_parts` path parts / file stem / stem sub-components —
so the one change bridges both directions (the ASCII question
``besta`` keeps matching the ASCII filename, the accented question
``BESTÅ`` now matches it too, and an accented filename ``BÅsta.md``
answers to the ASCII question ``basta``).
"""
return "".join(_ALNUM_RE.findall(_fold(s.lower())))
def name_hit_tokens(question: str) -> list[str]:
@@ -166,7 +212,10 @@ def name_hit_tokens(question: str) -> list[str]:
Order of first appearance, de-duplicated. Matching applies the
A2 rule: digit-bearing candidates prefix a normalized path part
or file stem; digitless candidates equal a part, stem, or stem
sub-component — titles are never matched.
sub-component — titles are never matched. Candidates are
ACCENT-FOLDED through :func:`_normalize_name` (phase 162) — the
token ``besta`` drawn from ``BESTÅ`` name-matches ``BESTÅ``-class
names both ways.
"""
tokens = question.split()
out: list[str] = []
@@ -456,6 +505,10 @@ def _name_parts(path: str) -> tuple[set[str], set[str]]:
(the stem lowercased, split on non-alphanumeric runs, each piece
normalized, empties dropped: ``kubernetes_gitea`` →
``kubernetes`` / ``gitea``): DIGITLESS tokens exact-match these.
The parts are ACCENT-FOLDED through :func:`_normalize_name` (phase
162) — ``besta`` name-matches ``BESTÅ``-class names both ways (the
accented stem ``BÅsta`` enters the equal set as ``basta``).
"""
p = Path(path)
prefix = {_normalize_name(part) for part in p.parts}
@@ -479,6 +532,10 @@ def _name_token_matches(token: str, prefix: set[str], equal: set[str]) -> bool:
* the token HAS NO DIGIT → it EQUALS a normalized path part, file
stem, or stem sub-component (``gitea`` → the ``gitea/`` folder,
``gitea.md``, ``kubernetes_gitea``, ``gitea-values``).
Both the token and the path sets arrive ACCENT-FOLDED from
:func:`_normalize_name` (phase 162) — ``besta`` matches
``BESTÅ``-class names both ways.
"""
if any(ch.isdigit() for ch in token):
return any(part.startswith(token) for part in prefix)
+4 -1
View File
@@ -307,8 +307,11 @@ def test_mock_generated_outline_is_stored_and_echoed(
assert user.splitlines()[0].startswith(
"docs — homelab/tables.md — Service Port Table"
), "the generator's list must lead with the NEWEST doc (recency-first)"
# Phase 162 (LOCKED A2): the stored content ends with the
# deterministic source-inventory line — one source, 13 docs.
expected = (
"Knowledge base outline:\n- docs homelab tables md service port table this"
"Knowledge base outline:\n- docs homelab tables md service port table this\n"
"Sources: docs (13 docs)"
)
assert row.content == expected
first_bullet = row.content.splitlines()[1].removeprefix("- ").strip()
+4 -2
View File
@@ -329,8 +329,10 @@ def test_regenerated_outline_leads_with_newest_doc(
assert row is not None and row.id == 1
# The stored outline is the mock's 8-token digest of the capped list
# — all 8 tokens from the NEWEST doc's source/path/title (computed
# from the fixture above, not hardcoded wording):
assert row.content == EXPECTED_DIGEST
# from the fixture above, not hardcoded wording) — and phase 162
# (LOCKED A2) appends the deterministic source-inventory line after
# it: every fixture source named with its exact count.
assert row.content == EXPECTED_DIGEST + "\nSources: reese (5 docs)"
assert "old" not in row.content # no oldest-batch token survives the cut
@@ -0,0 +1,864 @@
"""Phase 162 E2E (Playwright, mock-only): a 1-document source is BOTH
visible to ``<knowledge_base>`` and winnable by its accented question.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_search_recall_small_sources.py -v --no-cov
The 2026-10-07 live miss, replayed deterministically: the owner asked
``How do I build the BESTÅ?``; the KB held ``ikea_instructions/
besta-frame-white__AA-1272068-11-100.md`` (1 document in the source, its
body carrying the name ONLY as ``BESTÅ``) and the question came back
empty-handed — ASCII ``besta`` documents outranked the one document that
actually named the product. Three separate losses, three pins here:
* **D1 (the question-side fold)** — name matching never bridged the
accent: ``BESTÅ`` normalized to ``best``, ``besta-frame-…`` to
``besta``, and the digitless rule EQUALS a stem component. Test 1
pins the fix, test 2 MEASURES the miss.
* **D2 (the deterministic source inventory)** — the ``lite`` outline is
lossy, so a 1-doc source could be invisible to the model. Test 4 pins
the mechanical ``Sources:`` tail on the stored row.
* **D3 (the grep fold + search discipline)** — the stored accented body
stayed grep-matchable by the ASCII name. Test 6 pins the content
layer; the turn-level prose discipline is the prompt teaching
(``app/rag/prompts.py``, unit-pinned in task 03).
KB fixture — ``tests/fixtures/recallkb`` (tracked), THREE local-directory
sources registered through the authenticated API + the REAL in-process
``POST /api/sync`` (the ``test_name_signal_read_chips.py`` pattern), six
markdown documents whose text is token-controlled so the hybrid gate, the
FTS list and the name-hit bonus pick the intended tiers deterministically
(``synced_kb`` pins the tiers with the app's REAL retrieval — a
fixture-text regression breaks at setup with a measured message):
* ``ikea_instructions/`` — the small source: ONE document, the live
incident's own filename, its body naming the product ONLY as the
accented ``BESTÅ`` (no ASCII ``besta`` anywhere in the file). Two
token controls make the mechanism visible: no ``build`` (the question's
other FTS token) anywhere, and the accented name deliberately placed
PAST the 24-token digest window, so the mock summary chunk leaks no
ASCII ``best`` lexeme that a pre-fix FTS could have hit;
* ``phase_docs/`` (4 docs) + ``players_handbook/`` (1 doc) — the ASCII
distractors of the live index: every body carries ASCII ``besta`` AND
``build``, so each of the five outranks the small source on
pre-fix retrieval (two fused terms — FTS + vector — against its one).
Measured, never assumed (test 2 re-measures both directions; the same
numbers are recorded in
``.agents/reports/162_search_recall_small_sources/``). Accented question
``How do I build the BESTÅ?``, the app's real ``retrieve()`` +
``select_suggested()`` over this KB with the mock's embeddings (gate:
threshold 0.30, lexical-support floor 0.15 — the conftest mock
calibration; ``best cosine`` / ``fts`` are the A8 inputs):
PRE-FIX (the phase-119 ``_normalize_name``, no accent fold)
name tokens ['build', 'best'] → NO name hit
suggested the five ASCII distractors
related ikea_instructions/besta-frame-white__… (rank 6)
best cosine 0.382, fts 8 (grounded, on the WRONG documents —
the live 2026-10-07 shape: an answer
about the ASCII ``besta`` notes)
POST-FIX (shipped)
name tokens ['build', 'besta'] → name hit
suggested THE SMALL SOURCE #1, then four distractors
related phase_docs/156_history_source_filter.md
best cosine 0.382, fts 9 (the fold adds the lexical term; the
cosine is untouched — LOCKED A1)
The mechanism is the term COUNT: every distractor matches the question's
other FTS token ``build``, so each fuses TWO terms (vector + FTS) while
the small source — no ``build``, no accent-bridgeable lexeme — fuses ONE
and lands rank 6. The folded name hit is the only signal that can lift
it: name hits LEAD the lexical list (``fuse`` rank 1 ⇒ 1/61) and the
per-document ``BOR_NAME_HIT_BONUS`` (0.005) rides on top. The ASCII
question ``How do I build a Besta?`` keeps the same document suggested
#1 with the same cosine, before and after (the untouched phase-119
path, test 3).
Test → contract mapping (task 04 cases 1–4):
1. ``test_accented_question_suggests_the_small_source`` — (1): the
accented question grounds (``deflected: false``), the durable
record's SUGGESTED slice (entries 1–5, LOCKED A3) contains
``ikea_instructions/besta-frame-white__AA-1272068-11-100.md``, and
the ``done`` frame's ``related`` list — the rank-6+ remainder —
does NOT (the doc moved from the rank-6 miss into the tier). A
zero-read grounded turn chips nothing (phase 119 LOCKED A1).
2. ``test_under_rank_is_measured_not_assumed`` — (1, the hypothesis):
with the PRE-FIX normalizer re-installed (``monkeypatch``), the same
retrieval over the same KB puts the doc OUT of the top-5 (and the
``BESTÅ?`` token degrades to ``best``); with the shipped normalizer
it is suggested #1. The under-rank is measured in both directions,
so the pin can never silently rot.
3. ``test_ascii_question_still_name_matches`` — (2): the ASCII
regression (the phase-119 path untouched) — same document in the
suggested tier, grounded, same related remainder.
4. ``test_overview_tail_names_every_source_with_counts`` — (3): the
stored ``kb_overview`` row ENDS with the deterministic
``Sources: …`` line, one entry per source with exact counts — the
1-doc sources visible whatever the ``lite`` digest said.
5. ``test_out_of_kb_question_still_deflects`` — (4): the gate verdict
for the out-of-KB question is UNCHANGED (``deflected: true``,
``fts_hits == 0``, no chips, ``Maybe try`` chips only) — the fold
must not turn a real miss into a false grounding.
6. ``test_grep_folds_the_accented_body`` — (3, content layer):
``grep_document`` with the ASCII pattern ``besta`` matches the
stored accented line (D3), and the accented pattern matches too.
7. ``test_suggested_folder_lines_carry_the_small_source`` — the
prompt-level witness: the mock's ``repeat your folder map`` echo
ends with the suggested-folder lines, and the 1-doc source's line
``ikea_instructions/:`` is among them (its suggested doc owns the
line, so the header stands alone).
"""
from __future__ import annotations
import json
import os
import re
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Locator, Page, expect
from sqlalchemy import select, text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document, KbOverview, QueryLog
from app.rag import retriever
from app.rag.agent import grep_document
from app.rag.retriever import retrieve, select_related, select_suggested
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
from e2e.mock_llm import TOKEN_RE, embed_text
REPO = Path(__file__).resolve().parents[2]
#: Own port (the phase-79 full-inventory rule: a module app binds its
#: own port so a combined run never drives the session app).
APP_PORT = int(os.environ.get("E2E_APP_PORT_RECALL", "8162"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
FIXTURES = REPO / "tests" / "fixtures" / "recallkb"
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
# --------------------------------------------------------------------------
# The fixture KB (tracked files on disk — read back byte-for-byte)
# --------------------------------------------------------------------------
#: source name → document paths. THE SMALL SOURCE is the live incident:
#: one document whose body carries the product name only accented.
SOURCE_DOCS: dict[str, list[str]] = {
"ikea_instructions": ["besta-frame-white__AA-1272068-11-100.md"],
"phase_docs": [
"140_pictorial_pdf_pages.md",
"148_pdf_text_backfill.md",
"152_seaweedfs_blob_ingest.md",
"156_history_source_filter.md",
],
"players_handbook": ["game_night_setup.md"],
}
IKEA_SOURCE = "ikea_instructions"
IKEA_DOC = "besta-frame-white__AA-1272068-11-100.md"
IKEA = f"{IKEA_SOURCE}/{IKEA_DOC}"
#: Every fixture doc in ``(source, path)`` catalog order.
ALL_DOCS = [f"{source}/{path}" for source in SOURCE_DOCS for path in SOURCE_DOCS[source]]
#: The deterministic source inventory tail (LOCKED A2 order: name-sorted,
#: singular ``doc`` at k=1) — the D2 assertion.
INVENTORY = "Sources: ikea_instructions (1 doc), phase_docs (4 docs), players_handbook (1 doc)"
# --------------------------------------------------------------------------
# The scripted questions
# --------------------------------------------------------------------------
#: The live miss, byte-for-byte (the ``Å`` is the whole point — the file
#: spells it ``BESTÅ``, the pre-phase name matcher read ``best``).
ACCENTED_QUESTION = "How do I build the BESTÅ?"
#: The phase-119 regression: the same product, spelled in ASCII. This
#: question ALWAYS name-matched (the untouched path).
ASCII_QUESTION = "How do I build a Besta?"
#: The accented question + the mock's folder-map trigger — the prompt
#: lens on the suggested tier (the folder lines are composed from the
#: suggested documents only).
FOLDER_QUESTION = ACCENTED_QUESTION + " repeat your folder map"
#: The out-of-KB control (the phase-119 battery's deflection sanity).
DEFLECT_QUESTION = "What is the capital of Mongolia?"
def _expected_summary(content: str, source: str, path: str) -> str:
"""The mock lite model's byte-stable digest + the code pointer line
(mirrors ``mock_llm.compose_answer``'s ``SUMMARY_MODE`` branch — the
first 24 tokens of the document content — plus the summarizer's
deterministic ``Source:`` line; no model output is ever trusted)."""
digest = " ".join(TOKEN_RE.findall(content.lower())[:24])
return f"This document covers {digest}.\nSource: {source}/{path}"
def _prefix_normalizer(value: str) -> str:
"""The PRE-PHASE-162 name normalizer (the measured before-state):
lower + alnum scan with NO accent fold — the exact body
``_normalize_name`` had before task 01. Installed by monkeypatch in
test 2 so the miss is MEASURED, not asserted from reasoning."""
return "".join(retriever._ALNUM_RE.findall(value.lower()))
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def seed_dirs() -> list[Path]:
"""The three local-directory sources (the registered directory's
BASENAME is the source name, phase 38). The small source must be a
directory of its own — that is what makes it a 1-doc source."""
dirs = []
for source, paths in SOURCE_DOCS.items():
directory = FIXTURES / source
assert directory.is_dir(), f"{directory} is missing"
for path in paths:
assert (directory / path).is_file(), f"fixture doc {source}/{path} is missing"
dirs.append(directory)
return dirs
@pytest.fixture(scope="module")
def app_server(mock_llm: int, seed_dirs: list[Path]) -> Iterator[str]:
"""The real app under test — per-module app on its own port (the
phase-79 inventory pattern), the mock LLM, the mock-calibrated gate,
and the code defaults for everything the tiers depend on (suggested
5 / related 2 / name-hit bonus 0.005 / recency 0.0007, exactly the
production ``.env``-free defaults)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (the conftest pattern): the accented and the
# ASCII turns ground on the distractors' cosine (≈0.35); the
# out-of-KB control stays under it.
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env is the owner's — empty the git list so the registry
# holds EXACTLY the three directories this suite registers, and pin
# the leak-guarded corpus settings (the conftest pattern).
env["BOR_GIT_SOURCES"] = ""
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the tier margins
are pinned against EXACTLY these six documents."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the cookie-authenticated status endpoint to a terminal state
(the ``test_summary_seed_context`` pattern over plain httpx)."""
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
return body
time.sleep(0.5)
raise AssertionError(f"sync did not reach a terminal state: {body}")
def _tiers(question: str) -> tuple[list[str], list[str], float, int]:
"""The measured tiers of one question over the synced KB, with the
app's REAL hybrid retrieval and the mock's embeddings (the
phase-119 measurement lens): ``(suggested, related, top cosine,
fts hits)``."""
with SessionLocal() as db:
chunks = retrieve(db, question, embed_text(question))
suggested = select_suggested(chunks)
related = select_related(
chunks,
{d.id for d in suggested},
_Settings.model_fields["related_max_docs"].default,
)
return (
[f"{d.source}/{d.path}" for d in suggested],
[f"{d.source}/{d.path}" for d in related],
max((c.cosine for c in chunks), default=0.0),
sum(1 for c in chunks if c.fts_hit),
)
def _assert_tiers(
question: str, suggested: list[str], related: list[str]
) -> None:
"""Pin the engineered invariant for one scripted question, with a
MEASURED message (a fixture-text regression fails at setup, not at
the wire assertions)."""
got_suggested, got_related, cosine, fts = _tiers(question)
assert got_suggested == suggested, (
f"suggested tier drifted for {question!r}: {got_suggested} "
f"(expected {suggested}; best cosine {cosine:.3f}, fts {fts})"
)
assert got_related == related, (
f"related tier drifted for {question!r}: {got_related} "
f"(expected {related})"
)
@pytest.fixture(scope="module")
def synced_kb(app_server: str, seed_dirs: list[Path]) -> None:
"""The story's precondition: the three-source KB synced under the
deterministic mock. Registers the directories through the
authenticated API, runs the REAL in-process sync, pins the stored
content byte-identical to the tracked fixture files (including the
accented ``BESTÅ`` surviving the round trip untouched), pins the
fixture's own token controls (the mechanism the miss is reproduced
with), and pins the engineered tiers for all three scripted
questions."""
_truncate_all()
with httpx.Client(base_url=app_server, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
for directory in seed_dirs:
r = client.post(
"/api/git-sources", json={"kind": "local", "path": str(directory)}
)
assert r.status_code == 201, r.text
r = client.post("/api/sync")
assert r.status_code == 202, r.text
body = _wait_sync_done_http(client)
assert body["state"] == "success", body
assert body["detail"]["added"] == len(ALL_DOCS), body["detail"]
assert body["detail"]["pruned"] == 0, body["detail"]
with SessionLocal() as db:
for source, paths in SOURCE_DOCS.items():
for path in paths:
on_disk = (FIXTURES / source / path).read_text(encoding="utf-8")
stored = db.scalar(
select(Document).where(
Document.source == source, Document.path == path
)
)
assert stored is not None, f"fixture doc {source}/{path} missing"
assert stored.content == on_disk, f"content drifted for {source}/{path}"
assert stored.summary == _expected_summary(on_disk, source, path), (
f"summary is not the mock digest for {source}/{path}"
)
schunks = [c for c in stored.chunks if c.is_summary]
assert len(schunks) == 1 and schunks[0].embedding is not None, (
f"expected one embedded is_summary chunk for {source}/{path}"
)
# ---- The fixture's token controls (the reproduction mechanism) ----
ikea_text = (FIXTURES / IKEA_SOURCE / IKEA_DOC).read_text(encoding="utf-8")
# (a) the name exists ONLY accented — no ASCII ``besta`` in the body,
# so nothing but the phase-162 fold can bridge question ↔ doc;
assert "BESTÅ" in ikea_text
assert "besta" not in ikea_text.lower().replace("bestå", ""), ikea_text
# (b) no ``build`` either: the accented question's only OTHER FTS
# token — the small source must win on the NAME signal, never on
# a content-word accident;
assert "build" not in ikea_text.lower()
# (c) the accented name sits PAST the 24-token digest window, so the
# mock's ASCII-only digest never leaks a ``best`` lexeme the
# pre-phase FTS could have hit (a real ``lite`` summary of the
# live PDF never wrote the name in ASCII either).
digest = TOKEN_RE.findall(ikea_text.lower())[:24]
assert "best" not in digest and "besta" not in digest, digest
# ---- The engineered tiers (measured, stable across re-imports) ----
_assert_tiers(ACCENTED_QUESTION, SUGGESTED_ACCENTED, RELATED_ACCENTED)
_assert_tiers(ASCII_QUESTION, SUGGESTED_ASCII, RELATED_ASCII)
_assert_tiers(FOLDER_QUESTION, SUGGESTED_FOLDER, RELATED_FOLDER)
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped — the
scripted turns never change it)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# The pinned tiers (MEASURED with the app's real retrieval over the mock
# embeddings — see the module docstring's before/after record)
# --------------------------------------------------------------------------
#: POST-FIX, accented question: the small source LEADS the tier (its
#: name hit leads the lexical list ⇒ ``fuse`` rank 1 = 1/61, plus the
#: per-document name-hit bonus 0.005), the five ASCII distractors
#: follow in measured retrieval order, one of them falls to rank 6.
SUGGESTED_ACCENTED = [
IKEA,
"phase_docs/140_pictorial_pdf_pages.md",
"phase_docs/148_pdf_text_backfill.md",
"phase_docs/152_seaweedfs_blob_ingest.md",
"players_handbook/game_night_setup.md",
]
RELATED_ACCENTED = ["phase_docs/156_history_source_filter.md"]
#: POST-FIX, ASCII question (the untouched phase-119 path): same doc,
#: same #1 slot — the only difference is the measured order of two
#: distractors that trade places on the extra ``besta`` FTS term.
SUGGESTED_ASCII = [
IKEA,
"phase_docs/140_pictorial_pdf_pages.md",
"phase_docs/148_pdf_text_backfill.md",
"players_handbook/game_night_setup.md",
"phase_docs/152_seaweedfs_blob_ingest.md",
]
RELATED_ASCII = ["phase_docs/156_history_source_filter.md"]
#: The folder-map turn (the trigger's own tokens dilute the cosine to
#: 0.296 — lexical support carries the gate) keeps the same tier.
SUGGESTED_FOLDER = list(SUGGESTED_ACCENTED)
RELATED_FOLDER = list(RELATED_ACCENTED)
#: PRE-FIX (the measured miss, test 2): the folded token never reaches
#: ``besta``, so the small source has ONE fused term (vector) against
#: every distractor's TWO (vector + FTS ``build``) and the five
#: distractors fill the tier — the IKEA doc is rank 6, i.e. the rank-6+
#: ``related`` remainder. Exactly the live 2026-10-07 under-rank.
PRE_SUGGESTED_ACCENTED = [
"phase_docs/140_pictorial_pdf_pages.md",
"phase_docs/148_pdf_text_backfill.md",
"phase_docs/152_seaweedfs_blob_ingest.md",
"players_handbook/game_night_setup.md",
"phase_docs/156_history_source_filter.md",
]
PRE_RELATED_ACCENTED = [IKEA]
# --------------------------------------------------------------------------
# Page helpers (the phase-119 house patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream (a
#: response clone read in the background) — wire-level assertions on the
#: ``done`` frame, independent of the rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page) -> list[dict[str, Any]]:
"""The SSE frames captured since the last submit, once the hook's
background read settles."""
deadline = time.monotonic() + 30.0
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _done(page: Page) -> dict[str, Any]:
return next(f for f in _frames(page) if f.get("type") == "done")
def _tool_frames(frames: list[dict[str, Any]]) -> list[dict[str, Any]]:
return [f for f in frames if f.get("type") == "tool"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
def _last_brain(page: Page) -> Locator:
return page.locator(".msg.brain").last
def _last_query_log() -> QueryLog:
with SessionLocal() as db:
rows = db.scalars(select(QueryLog)).all()
assert len(rows) == 1, f"expected exactly one query_log row, got {len(rows)}"
return rows[0]
def _recorded_tiers(row: QueryLog) -> tuple[list[str], list[str]]:
"""The durable record's tiers (LOCKED A3: the record is the suggested
tier followed by the rank-6+ remainder — the wire's only faithful
witness of the suggested set, since ``done.sources`` is the READ-DOCS
citation surface and ``done.suggestions`` is the deflection chips)."""
recorded = [s for s in row.sources.split(", ") if s]
return recorded[:5], recorded[5:]
# --------------------------------------------------------------------------
# 1. (task 04 case 1) the accented question puts the 1-doc source in the
# suggested top-5 — and it is no longer the rank-6 miss
# --------------------------------------------------------------------------
def test_accented_question_suggests_the_small_source(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""``How do I build the BESTÅ?`` grounds, and the suggested tier of
the turn carries ``ikea_instructions/besta-frame-white__AA-1272068-
11-100.md`` — SUGGESTED #1, measured. The suggested set is pinned on
the wire through the durable record (LOCKED A3: ``sources`` = the
suggested 5, then the rank-6+ remainder): the IKEA doc sits inside
the first five entries, and the ``done`` frame's ``related`` list —
the rank-6+ tier, the slot it occupied as the live miss — is EMPTY
of it. A zero-read grounded turn chips nothing (phase 119 LOCKED
A1); the answer renders grounded."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, ACCENTED_QUESTION)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = _done(page)
assert done["deflected"] is False, done
assert done["sources"] == [], done["sources"]
# The miss was the IKEA doc landing in the rank-6+ remainder. It is
# not there anymore — the tier it belongs to is the suggested one.
related = [(s["source"], s["path"]) for s in done["related"]]
assert (IKEA_SOURCE, IKEA_DOC) not in related, done["related"]
# The ``suggestions`` field is the DEFLECTION contract (title chips
# under a weak turn) — a grounded turn carries none, so it can never
# be the tier witness (the durable record is, and is asserted below).
assert done["suggestions"] == [], done["suggestions"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# The de-emphasized rank-6+ row renders — and the doc it used to
# hold (the miss) is a distractor now.
row = _last_query_log()
related_links = page.locator(".msg.brain .related-docs .related-doc")
expect(related_links).to_have_count(len(RELATED_ACCENTED))
for i, path in enumerate(RELATED_ACCENTED):
expect(related_links.nth(i)).to_contain_text(path)
expect(related_links.filter(has_text=IKEA)).to_have_count(0)
assert row.question == ACCENTED_QUESTION
assert row.deflected is False
suggested, related_record = _recorded_tiers(row)
assert suggested == SUGGESTED_ACCENTED, row.sources
assert related_record == RELATED_ACCENTED, row.sources
assert suggested[0] == IKEA, row.sources
# --------------------------------------------------------------------------
# 2. (task 04 case 1, the hypothesis) the under-rank is MEASURED in both
# directions: pre-fix the doc loses the tier, post-fix it wins it
# --------------------------------------------------------------------------
def test_under_rank_is_measured_not_assumed(
monkeypatch: pytest.MonkeyPatch, synced_kb: None, db_ready: None
) -> None:
"""The phase-162 hypothesis, measured rather than asserted: with the
PRE-phase normalizer re-installed (``lower`` + alnum scan, no accent
fold) the accented question degrades to the token ``best`` — no name
hit — and the SAME retrieval over the SAME KB drops
``ikea_instructions/besta-frame-white__AA-1272068-11-100.md`` out of
the suggested top-5 into the rank-6+ remainder (the live miss,
reproduced). With the shipped normalizer the folded ``besta`` token
name-matches the filename and the doc is suggested #1. Re-running
the measurement is the guard: if a future fixture or retriever
change ever makes the pre-phase path win, this test fails and the
recorded evidence is re-examined instead of trusted."""
# The token side of the miss, measured.
assert retriever.name_hit_tokens(ACCENTED_QUESTION) == ["build", "besta"]
monkeypatch.setattr(retriever, "_normalize_name", _prefix_normalizer)
assert retriever.name_hit_tokens(ACCENTED_QUESTION) == ["build", "best"]
pre_suggested, pre_related, _cosine, _fts = _tiers(ACCENTED_QUESTION)
assert IKEA not in pre_suggested, (
f"the pre-phase path WINS the tier — the fixture no longer "
f"reproduces the miss: {pre_suggested}"
)
assert pre_suggested == PRE_SUGGESTED_ACCENTED, pre_suggested
assert IKEA in pre_related, (
f"the miss should surface as the rank-6+ remainder: {pre_related}"
)
assert pre_related == PRE_RELATED_ACCENTED, pre_related
monkeypatch.undo()
# And the same measurement with the shipped fold: the doc is #1.
post_suggested, post_related, _cosine, _fts = _tiers(ACCENTED_QUESTION)
assert post_suggested == SUGGESTED_ACCENTED
assert post_related == RELATED_ACCENTED
# --------------------------------------------------------------------------
# 3. (task 04 case 2) the ASCII question keeps its phase-119 name hit
# --------------------------------------------------------------------------
def test_ascii_question_still_name_matches(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""``How do I build a Besta?`` — the question the phase-119 rule
already answered — still grounds on the same document (the fold must
be behavior-preserving for the ASCII path: ``besta`` normalizes to
``besta`` either way). Same tier position, same durable record,
grounded, nothing chipped on a zero-read turn."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, ASCII_QUESTION)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = _done(page)
assert done["deflected"] is False, done
assert done["sources"] == [], done["sources"]
expect(_last_brain(page).locator(".bubble")).to_contain_text(
MOCK_ANSWER_MARKER, timeout=30_000
)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
row = _last_query_log()
assert row.question == ASCII_QUESTION
assert row.deflected is False
suggested, related_record = _recorded_tiers(row)
assert suggested == SUGGESTED_ASCII, row.sources
assert suggested[0] == IKEA, row.sources
assert related_record == RELATED_ASCII, row.sources
# --------------------------------------------------------------------------
# 4. (task 04 case 3) the deterministic source inventory — the D2 tail
# --------------------------------------------------------------------------
def test_overview_tail_names_every_source_with_counts(
synced_kb: None, db_ready: None
) -> None:
"""The stored ``kb_overview`` row ENDS with the mechanical
``Sources: …`` line — one entry per source, exact counts, singular
``doc`` at k=1 — so the 1-document sources are visible to
``<knowledge_base>`` whatever the lossy ``lite`` digest said (the
2026-10-07 live finding: a 14-source index outlined as 3 groups).
The tail is ONE line, appears exactly once, and sits after the
model's outline (the model input never sees it)."""
with SessionLocal() as db:
rows = db.scalars(select(KbOverview)).all()
assert len(rows) == 1, rows
content = rows[0].content
lines = content.splitlines()
assert lines[-1] == INVENTORY, content
assert content.count("Sources: ") == 1, content
# The ``lite`` outline still leads (the tail is appended, never a
# replacement).
assert len(lines) >= 2, content
assert lines[0].startswith("Knowledge base outline:"), content
for source in SOURCE_DOCS:
assert source in lines[-1], content
# --------------------------------------------------------------------------
# 5. (task 04 case 4) the out-of-KB control — the gate verdict unchanged
# --------------------------------------------------------------------------
def test_out_of_kb_question_still_deflects(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""``What is the capital of Mongolia?`` matches nothing in this KB:
the A8 verdict stays DEFLECTED (``deflected: true``, ``fts_hits ==
0``, weak cosine under the mock-calibrated threshold), the answer
carries no citation chips, and the only chips are the ``Maybe try``
suggestions. The accent fold must never turn a real miss into a
false grounding (the phase-119 battery's deflection sanity,
localised)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, DEFLECT_QUESTION)
_wait_settled(page)
done = _done(page)
assert done["deflected"] is True, done
assert done["sources"] == [], done["sources"]
assert 2 <= len(done["suggestions"]) <= 3, done["suggestions"]
bubble = _last_brain(page)
expect(bubble).to_have_class(re.compile(r"is-deflected"))
expect(bubble.locator(".source-chip")).to_have_count(0)
expect(
bubble.locator(".maybe-try .suggestion-chip")
).to_have_count(len(done["suggestions"]))
row = _last_query_log()
assert row.question == DEFLECT_QUESTION
assert row.deflected is True
assert row.fts_hits == 0, row.fts_hits
assert row.top_score < 0.30, row.top_score
# --------------------------------------------------------------------------
# 6. (task 04 case 3 / D3) the stored accented body stays grep-matchable
# --------------------------------------------------------------------------
def test_grep_folds_the_accented_body(synced_kb: None, db_ready: None) -> None:
"""The content layer of the same miss: the stored body spells the
product ``BESTÅ`` only, and the agent's ``grep`` tool folds BOTH the
pattern and the line — so the ASCII pattern ``besta`` finds the
accented line (the pre-fold tool returned nothing, which is why the
live turn gave up on a document it had already opened), and the
accented pattern still matches itself."""
with SessionLocal() as db:
stored = db.scalar(
select(Document).where(
Document.source == IKEA_SOURCE, Document.path == IKEA_DOC
)
)
assert stored is not None
assert "BESTÅ" in stored.content
ascii_hits = grep_document(stored.content, "besta")
assert any("BESTÅ" in line for _no, line in ascii_hits), ascii_hits
accented_hits = grep_document(stored.content, "BESTÅ")
assert [line for _no, line in accented_hits] == [line for _no, line in ascii_hits]
# A pattern that is not the name still finds nothing (the fold is an
# accent bridge, not a wildcard).
assert grep_document(stored.content, "bestaa") == []
# --------------------------------------------------------------------------
# 7. the prompt-level witness — the 1-doc source's folder line rides the
# HIGH prompt with the suggested tier
# --------------------------------------------------------------------------
def test_suggested_folder_lines_carry_the_small_source(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""The accented question + the mock's ``repeat your folder map``
trigger ⇒ the answer ends with the suggested-folder lines VERBATIM.
The 1-doc source contributes ``ikea_instructions/:`` — a header with
no sibling entries, because its only document IS the suggested doc —
the visible proof the small source reached the prompt with the
tier."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, FOLDER_QUESTION)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = _done(page)
assert done["deflected"] is False, done
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(bubble).to_contain_text("ikea_instructions/:")
row = _last_query_log()
assert row.deflected is False
suggested, _related = _recorded_tiers(row)
assert suggested == SUGGESTED_FOLDER, row.sources
@@ -0,0 +1,25 @@
# Assembly Instructions
Article number 127.206.11 — revision 4
Cam bolt 130618 · wall bracket 141413 · leveling foot 303154 · dowel 518041
## Package contents
- 1 x frame, white, 60 x 20 x 60 cm
- 2 x side panels
- 1 x back panel, hardboard
- 12 x cam bolts with sleeve nuts
- 1 bag of dowels
## Assembly
Insert sleeve nuts into the drilled holes, draw both side panels against the
frame, tighten each cam bolt a quarter turn. Attach back panel with nails
supplied, keeping it square and level. Fix wall bracket using plugs matched to
wall material — furniture of this depth must always be secured.
## Product name
BESTÅ frame, white, 60 x 20 x 60 cm, AA 1272068 11 100. Wipe with a soft
cloth dampened in water or a mild cleaner. Never soak the edges.
@@ -0,0 +1,11 @@
# Pictorial PDF pages
How do I build a retrievable text layer for a scanned manual? Build one page
image per PDF page, build an image description for each page, then build the
index from those descriptions. The ASCII besta assembly PDF was the first
manual to go through this path.
- build page images at 1400 px
- build one description per page image
- build chunks from descriptions, never from pixel guesses
- besta manual: 24 scanned pages, 24 descriptions
@@ -0,0 +1,9 @@
# PDF text backfill
Backfill plan for scanned manuals. A manual that arrives without a text layer
needs one before it can answer anything: run the pictorial page path, build
descriptions for every page, keep the original page order.
- build: render, describe, index
- besta shelf PDF: 24 scanned pages, no embedded text
- question asked twice in review: how do I build this layer?
@@ -0,0 +1,8 @@
# SeaweedFS blob ingest
Object-store notes. Mount first, upload second.
- mount fs://manuals before the first upload
- build the volume list, then copy the scan batch
- besta scan batch: fs://manuals/2026-09/besta-scans.tar
- checksum every archive after the copy
@@ -0,0 +1,8 @@
# History source filter
Turn history keeps the source each answer came from, so a filter can narrow
the log to one source at a time.
- filter chip per source, sourced from turn history
- a build log entry records the source it indexed
- rerunning a question with a filter never rewrites history
@@ -0,0 +1,7 @@
# Game night setup
Room notes, games side of the shelf wall.
- besta unit holds box library; top shelf for dice trays
- build tray insert before next session
- keep rules binder beside clock, never stacked on besta top
+27 -1
View File
@@ -1,4 +1,6 @@
# Phase 119 retrieval battery (task 06, D6) — the 1,000-doc live replica.
# Retrieval battery — phase 119 (task 06, D6) + phase 162 (task 04, D5),
# run against the 1,000-doc live replica:
# uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt
#
# Gate (per the phase overview, post-phase measured numbers):
# 1-3. each name question's suggested top-5 contains its canonical doc:
@@ -10,8 +12,32 @@
# 5. deflection sanity: the out-of-KB question's gate verdict + fts
# count match the pre-phase replica run (HIGH, fts_hits=5) — the
# name-hit change must not move it.
#
# Phase 162 appends the Besta pair (the 2026-10-07 live miss). Gate: the
# suggested top-5 must contain the IKEA doc — ASCII filename
# `ikea_instructions/besta-frame-white__AA-1272068-11-100.pdf`, body that
# spells the product `BESTÅ` only — for BOTH spellings: the accented one
# needs the phase-162 question-side fold (D1) to bridge ASCII filename ←
# accented question, the ASCII one is the phase-119 path (regression).
# Either way ASCII `besta` documents must not outrank the doc that names
# the product.
#
# SKIPPED LOUDLY on the local replica (the phase-140 precedent): the
# 2026-09-05 snapshot carries NO `ikea_instructions`-class source —
# `grep -c -i ikea data/bor_live_kb_replica.dump.sql` = 0, and
# `SELECT DISTINCT source FROM documents` after the restore lists the
# sources without it (verify, don't assume). The two questions still
# run and must behave like any out-of-KB ask (no crash, no false
# grounding on a doc that never named the product); the gate on the
# rank itself can only close on a KB that has the source (owner's live
# KB). The fixture that DOES reproduce the miss end-to-end is
# tests/e2e/test_search_recall_small_sources.py over
# tests/fixtures/recallkb (pre-fix ranks measured there, both
# directions).
How do I deploy gitea?
How do I deploy forgejo?
How do I set up the gateway?
Do you have notes on the qwen 3.8 27b model?
What is the capital of Mongolia?
How do I build a Besta?
How do I build the BESTÅ?
+11 -5
View File
@@ -239,10 +239,11 @@ def test_changed_import_writes_overview_row(
by_role = {m["role"]: m["content"] for m in llm.chat_calls[3]}
assert "FOLDER_SUMMARY_MODE" in by_role["system"]
assert by_role["user"].splitlines()[0] == "Folder: MyDocs"
# The model's outline lands in the single row.
# The model's outline lands in the single row, followed by the
# phase-162 deterministic source inventory (LOCKED A2).
row = _row(db)
assert row is not None
assert row.content == "Summary of MyDocs"
assert row.content == "Summary of MyDocs\nSources: MyDocs (2 docs)"
assert row.updated_at is not None
# The phase's required log line (PLAN §9).
assert any("overview: regenerated docs=2 chars=" in r.getMessage() for r in records)
@@ -274,7 +275,8 @@ def test_unchanged_reimport_does_not_call_lite(
)
assert len(llm.chat_calls) == 4 # no new lite call
row = _row(db)
assert row is not None and row.content == "Summary of MyDocs"
assert row is not None
assert row.content == "Summary of MyDocs\nSources: MyDocs (2 docs)"
assert _version(db) == 1 # phase 53: an unchanged re-run never bumps
@@ -345,7 +347,8 @@ def test_limit_run_skips_overview(
# lite call; the overview + folder gates skip under --limit.
assert len(llm.chat_calls) == 5
row = _row(db)
assert row is not None and row.content == "Summary of MyDocs"
assert row is not None
assert row.content == "Summary of MyDocs\nSources: MyDocs (2 docs)"
assert _version(db) == 1 # phase 53: --limit debug runs never bump
@@ -462,5 +465,8 @@ def test_overview_outline_names_the_newest_docs_when_the_cap_bites(
# would have been "mydocs old 01 md old 01 summary of" (the oldest
# batch) — the "summaries only up to phase 113" shape this pin
# forbids.
assert row.content == "Knowledge base outline:\n- mydocs zz new 03 md zz new 03"
assert row.content == (
"Knowledge base outline:\n- mydocs zz new 03 md zz new 03\n"
"Sources: MyDocs (6 docs)"
)
assert "old" not in row.content # no oldest-batch token may survive the cut
+28 -3
View File
@@ -367,7 +367,8 @@ def test_agent_tools_names_and_parameters() -> None:
# teaching no-match line instead of a trusted miss.
assert grep["description"] == (
"Search the indexed documents for an exact string "
"(case-insensitive) and return up to 20 matching lines "
"(case- and accent-insensitive — accents fold: `besta` "
"matches `BESTÅ`) and return up to 20 matching lines "
"as `source/path:line: text` — a locator, not a "
"context-adder: read the winner with `read` — for a "
"large document, use the reported line number as "
@@ -387,8 +388,11 @@ def test_agent_tools_names_and_parameters() -> None:
assert set(grep_params["properties"]) == {"pattern", "path"}
assert all(p["type"] == "string" for p in grep_params["properties"].values())
assert grep_params["properties"]["pattern"]["description"] == (
"The exact text to search for (a plain substring, "
"not a regex — no '.*', no '\\.', no character classes)"
"The exact text to search for (a plain "
"substring — case- and accent-insensitive, "
"accents fold: `besta` matches `BESTÅ` — "
"not a regex — no '.*', no "
"'\\.', no character classes)"
)
# Phase 72 (task 02): the bare-path contract is stated up front;
# task 05 (live gate iterations 1-8): the one-known-document clause
@@ -4000,6 +4004,27 @@ def test_origin_lines_image_doc_keeps_marker_position(
# ---------- grep (the phase-68 A5 contract under the new name) ----------
def test_grep_document_folds_accents_both_ways() -> None:
"""Phase 162 (LOCKED A3): grep folds BOTH the pattern and each line
through the shared ``app.rag.retriever._fold`` plus lowercase — an
ASCII pattern hits accented content and an accented pattern hits
ASCII content, one fold, never drifting from the name-hit path."""
# ASCII pattern, accented content.
assert agent.grep_document("The BESTÅ frame (AA-1272068-11-100)", "besta") == [
(1, "The BESTÅ frame (AA-1272068-11-100)")
]
assert agent.grep_document("line one\nBÅsta shelf\nline three", "basta") == [
(2, "BÅsta shelf")
]
assert agent.grep_document("café-au-lait", "cafe") == [(1, "café-au-lait")]
# Accented pattern, ASCII content — the pattern's own accents fold.
assert agent.grep_document("The besta frame", "BESTÅ") == [(1, "The besta frame")]
# A genuinely non-matching folded pattern still returns no lines.
assert agent.grep_document("The besta frame", "bestb") == []
# The fold bridges Ä/Ö/Ü/ß too (the same _fold as name matching).
assert agent.grep_document("Straße", "strasse") == [(1, "Straße")]
def test_grep_document_case_insensitive_line_numbers() -> None:
"""Case-insensitive fixed substring, 1-based line numbers, file order,
repeated matches within a line collapse to one match (grep semantics)."""
+144 -6
View File
@@ -5,7 +5,11 @@ per-document user lines (source/path/title/first summary line), and the
input cap with the shared ``[…truncated…]`` marker. The loader and
regenerator tests run against the local compose Postgres (preferred —
real single-row upsert), skipping with clear instructions when the stack
is not up — same pattern as ``test_importer.py``.
is not up — same pattern as ``test_importer.py``. The phase-162 section
pins the deterministic ``Sources: …`` inventory tail (LOCKED A2): the
format/order/counts, the 1-doc survival, the happy-path stored content,
the byte-identical zero-doc / ``LLMError`` paths, and the byte-identical
model input.
"""
from __future__ import annotations
@@ -15,7 +19,7 @@ import uuid
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import text
from sqlalchemy import select, text
from sqlalchemy.orm import Session
from app.config import Settings, get_settings
@@ -25,6 +29,7 @@ from app.rag.overview import (
KB_OVERVIEW_MODE,
OVERVIEW_INSTRUCTION,
SYSTEM_PROMPT,
_source_inventory,
build_overview_prompt,
load_kb_overview,
regenerate_overview,
@@ -302,11 +307,13 @@ def test_regenerate_happy_path_creates_single_row_id_1(
row = db.get(KbOverview, 1)
assert row is not None, "the single row must be upserted"
assert row.id == 1
assert row.content == REPLY
# Phase 162 (LOCKED A2): the stored content carries the
# deterministic source-inventory tail after the model text.
assert row.content == REPLY + "\nSources: Homelab (2 docs)"
assert row.updated_at is not None
age = datetime.now(UTC) - row.updated_at
assert age.total_seconds() < 300, "updated_at must be a fresh UTC timestamp"
assert f"overview: regenerated docs=2 chars={len(REPLY)}" in caplog.text
assert f"overview: regenerated docs=2 chars={len(row.content)}" in caplog.text
finally:
_truncate_documents(db)
@@ -324,7 +331,7 @@ def test_regenerate_updates_existing_row_in_place(db: Session, clean_overview) -
assert count == 1, "still exactly one row after the re-generation"
row = db.get(KbOverview, 1)
assert row is not None
assert row.content == REPLY # replaced, not appended
assert row.content == REPLY + "\nSources: Homelab (1 doc)" # replaced, not appended
assert row.updated_at is not None and row.updated_at > past
finally:
_truncate_documents(db)
@@ -511,4 +518,135 @@ def test_regenerate_opens_own_session_when_none(db: Session, clean_overview) ->
assert ok is True
row = db.get(KbOverview, 1) # fresh lookup via the test session
assert row is not None and row.content == REPLY
assert row is not None and row.content == REPLY + "\nSources: Homelab (1 doc)"
# ---------- phase 162: the deterministic source inventory (LOCKED A2) -------
def test_source_inventory_format_counts_and_catalog_order() -> None:
"""Exact format, singular/plural, and catalog order: one entry per
DISTINCT source — ``<name> (<k> doc|docs)`` joined with ``, ``
behind ``Sources: ``, ordered ``source, path`` (the scrambled
input rows prove the catalog order is the OUTPUT order, not the
recency-first order of the rows the generator reads)."""
rows = [
("b", "x.md", "X", None),
("a", "y.md", "Y", None),
("c", "p1.md", "P1", None),
("b", "z.md", "Z", None),
("c", "p2.md", "P2", None),
("c", "p3.md", "P3", None),
]
assert _source_inventory(rows) == "Sources: a (1 doc), b (2 docs), c (3 docs)"
def test_source_inventory_keeps_a_one_doc_source_visible() -> None:
"""The 2026-10-07 ``ikea_instructions`` case: a 1-doc source among
many appears with ``(1 doc)`` — no source is invisible to
``<knowledge_base>``, whatever the ``lite`` outline says."""
rows = [
("gufo", "a.md", "A", None),
(
"ikea_instructions",
"besta-frame-white__AA-1272068-11-100.md",
"Assembly Instructions",
None,
),
("gufo", "b.md", "B", None),
("players_handbook", "core.md", "Core", None),
]
assert _source_inventory(rows) == (
"Sources: gufo (2 docs), ikea_instructions (1 doc), "
"players_handbook (1 doc)"
)
def test_regenerate_appends_the_inventory_tail_to_the_stored_content(
db: Session, clean_overview, caplog: pytest.LogCaptureFixture
) -> None:
"""Happy path, exact pin on the fake's fixed output: the stored row
= model text (trailing newlines rstripped) + ``\\n`` + the inventory
line; the log's chars = the FULL stored content, tail included."""
_truncate_documents(db)
try:
_add_doc(db, "players_handbook", "rpg.md", "RPG")
_add_doc(db, "Homelab", "b.md", "B")
_add_doc(db, "Homelab", "a.md", "A")
llm = _FakeLLM(reply=REPLY + "\n\n")
expected = REPLY + "\nSources: Homelab (2 docs), players_handbook (1 doc)"
with caplog.at_level(logging.INFO, logger="app.rag.overview"):
ok = asyncio.run(regenerate_overview(llm, db))
assert ok is True
row = db.get(KbOverview, 1)
assert row is not None
assert row.content == expected # no blank line before the tail
assert f"overview: regenerated docs=3 chars={len(expected)}" in caplog.text
finally:
_truncate_documents(db)
def test_regenerate_model_input_never_gains_the_inventory_tail(
db: Session, clean_overview
) -> None:
"""The model input is byte-identical to the pre-phase prompt: the
user message the fake received equals ``build_overview_prompt`` for
the same catalogue rows — the tail is never sent to ``lite``."""
_truncate_documents(db)
try:
_add_doc(db, "Homelab", "a.md", "A", "A summary.\nSource: Homelab/a.md")
_add_doc(db, "Deployments", "b.yaml", "B")
llm = _FakeLLM()
assert asyncio.run(regenerate_overview(llm, db)) is True
rows = db.execute(
select(Document.source, Document.path, Document.title, Document.summary)
.order_by(Document.created_at.desc(), Document.source, Document.path)
).all()
finally:
_truncate_documents(db)
user = next(m["content"] for m in llm.messages if m["role"] == "user")
_, expected_user = build_overview_prompt([tuple(r) for r in rows])
assert user == expected_user
assert "Sources:" not in user
def test_regenerate_zero_docs_keeps_tailless_pre_phase_row(
db: Session, clean_overview
) -> None:
"""Zero-doc early return: byte-identical pre-phase behavior — the
pre-phase (tail-less) row survives untouched, no tail is invented."""
db.execute(text("DELETE FROM kb_overview"))
pre_phase_row = "- Old outline\n- from before phase 162"
db.add(KbOverview(id=1, content=pre_phase_row))
db.commit()
_truncate_documents(db)
llm = _FakeLLM()
ok = asyncio.run(regenerate_overview(llm, db))
assert ok is False
assert llm.calls == 0
row = db.get(KbOverview, 1)
assert row is not None and row.content == pre_phase_row
assert "Sources:" not in row.content
def test_regenerate_llm_error_preserves_an_existing_inventory_tail(
db: Session, clean_overview
) -> None:
"""The tail is CONTENT: a previous row that already carried a
``Sources: …`` tail survives an ``LLMError`` byte-identically —
including the tail (an old outline is better than none)."""
db.execute(text("DELETE FROM kb_overview"))
previous = "- Old outline\nSources: Homelab (3 docs)"
db.add(KbOverview(id=1, content=previous))
db.commit()
_truncate_documents(db)
try:
_add_doc(db, "Homelab", "a.md", "A")
llm = _FakeLLM(fail=LLMError("simulated lite-model failure"))
ok = asyncio.run(regenerate_overview(llm, db))
assert ok is False
row = db.get(KbOverview, 1)
assert row is not None and row.content == previous
finally:
_truncate_documents(db)
+69 -16
View File
@@ -17,6 +17,7 @@ import hashlib
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SEARCH_DISCIPLINE_RULE,
SUGGEST_INTRO,
TOOLS_SECTION,
_base,
@@ -87,9 +88,34 @@ def test_high_and_low_bases_byte_locked() -> None:
#: inserted — the prefix (the ``ls``-clause opening) and the suffix
#: (the discipline-rules ending) survived byte-identical, so they are
#: the same anchors as pre-phase-145; the pre-phase-145 sha/length are
#: kept below for the diff-style pin.
TOOLS_SECTION_SHA256 = "41a84885be98d4e15b95205694d59d364296a30e27758d39ae737881d5bacaa7"
TOOLS_SECTION_LEN = 3279
#: kept below for the chained diff-style pin. RE-CUT again for phase 162
#: (task 03, LOCKED A3+A4): the ``grep`` sentence's ``(case-insensitive)``
#: parenthetical widened to the folded contract ``(case- and
#: accent-insensitive — accents fold: besta matches BESTÅ)``, and the
#: LOCKED search-discipline sentence (:data:`SEARCH_DISCIPLINE_RULE`) was
#: inserted immediately after the grep sentence (before the ``For a normal
#: search pass`` clause). Only those two byte-groups moved — the prefix
#: (the ``ls``-clause opening) and the suffix (the discipline-rules
#: ending) survive byte-identical; the pre-phase-162 sha/length are kept
#: below for the chained diff-style pin.
TOOLS_SECTION_SHA256 = "988642f9dc6bb2a4a1d44a83b5368ff4cdb9df4e181b2fe3dc2fb241b1857595"
TOOLS_SECTION_LEN = 3818
# The pre-phase-162 anchors — the section BEFORE phase 162's grep-fold +
# search-discipline insertion. Kept for the chained diff-style pin.
TOOLS_SECTION_PRE_PHASE_162_SHA256 = (
"41a84885be98d4e15b95205694d59d364296a30e27758d39ae737881d5bacaa7"
)
TOOLS_SECTION_PRE_PHASE_162_LEN = 3279
# The grep contract parenthetical exactly as it changed in phase 162
# (LOCKED A3): the pre-phase ``(case-insensitive)`` vs the folded one now
# embedded in ``TOOLS_SECTION``. Used to revert phase 162 for the diff pin.
_GREP_CONTRACT_PRE_PHASE_162 = "(case-insensitive)"
_GREP_CONTRACT_PHASE_162 = (
"(case- and accent-insensitive — accents fold: `besta` matches `BESTÅ`)"
)
TOOLS_SECTION_PREFIX = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -159,19 +185,46 @@ def test_tools_section_origin_rule_verbatim_at_its_position() -> None:
) in TOOLS_SECTION
def test_tools_section_origin_rule_is_the_only_byte_change() -> None:
"""Phase 145 (task 02) diff-style pin: the section's OTHER bytes are
byte-identical to pre-phase — the pre-phase section with ONLY the new
sentence-group inserted. Removing the rule (and the single space
that now separates it from the ``grep`` clause) reproduces the
pre-phase-145 section byte-for-byte (sha + length), and the length
delta is exactly the rule's length plus that one inserted space."""
assert len(TOOLS_SECTION) == (
TOOLS_SECTION_PRE_PHASE_145_LEN + len(ORIGIN_RULE) + 1
)
pre_phase = TOOLS_SECTION.replace(ORIGIN_RULE + " ", "")
assert len(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_LEN
assert _sha256(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_SHA256
def test_tools_section_search_discipline_rule_verbatim_at_its_position() -> None:
"""Phase 162 (task 03, LOCKED A4): the search-discipline rule is
LOCKED verbatim and sits at its EXACT position — immediately after
the grep sentence (the ``…retry with the plain text you expect to
see.`` clause) and BEFORE the ``For a normal search pass`` clause.
The rule + its exact boundaries are pinned as one contiguous span, so
a move or re-spelling of ANY byte fails."""
assert SEARCH_DISCIPLINE_RULE in TOOLS_SECTION
assert TOOLS_SECTION.count(SEARCH_DISCIPLINE_RULE) == 1 # exactly once
assert (
"retry with the plain text you expect to see. "
+ SEARCH_DISCIPLINE_RULE
+ " For a normal search pass only `pattern`"
) in TOOLS_SECTION
def test_tools_section_grep_fold_contract_present() -> None:
"""Phase 162 (task 03, LOCKED A3): the grep sentence now states the
FOLDED contract — the ``(case-insensitive)`` parenthetical is replaced
byte-for-byte by the folded one, and the old one is gone."""
assert _GREP_CONTRACT_PHASE_162 in TOOLS_SECTION
assert _GREP_CONTRACT_PRE_PHASE_162 not in TOOLS_SECTION
def test_tools_section_phase162_and_145_are_the_only_byte_changes() -> None:
"""Chained diff-style pin: the ONLY byte changes are phase 162's (the
folded grep contract + :data:`SEARCH_DISCIPLINE_RULE`) and phase 145's
(:data:`ORIGIN_RULE`). Reverting phase 162 (drop the rule + its
separating space, restore the pre-phase grep parenthetical) reproduces
the pre-phase-162 section byte-for-byte; reverting the origin rule on
top of that reproduces the pre-phase-145 section byte-for-byte."""
# Revert phase 162 -> pre-phase-162.
pre_162 = TOOLS_SECTION.replace(SEARCH_DISCIPLINE_RULE + " ", "")
pre_162 = pre_162.replace(_GREP_CONTRACT_PHASE_162, _GREP_CONTRACT_PRE_PHASE_162)
assert len(pre_162) == TOOLS_SECTION_PRE_PHASE_162_LEN
assert _sha256(pre_162) == TOOLS_SECTION_PRE_PHASE_162_SHA256
# Revert phase 145 -> pre-phase-145.
pre_145 = pre_162.replace(ORIGIN_RULE + " ", "")
assert len(pre_145) == TOOLS_SECTION_PRE_PHASE_145_LEN
assert _sha256(pre_145) == TOOLS_SECTION_PRE_PHASE_145_SHA256
# ---------- the LOW prompt's locked DEFLECT_MODE body ----------
+19 -10
View File
@@ -50,6 +50,7 @@ from app.models import Document
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SEARCH_DISCIPLINE_RULE,
SHOW_IMAGE_CLAUSE,
SUGGEST_INTRO,
TOOLS_SECTION,
@@ -286,14 +287,14 @@ def test_tools_section_markers_and_new_tool_names() -> None:
def test_tools_section_teaches_the_harness_shapes() -> None:
"""Copy pins: ``ls``'s phase-63 catalog-line format, ``grep``'s
case-insensitive exact-string locator contract (up to 20
case- and accent-insensitive exact-string locator contract (up to 20
``source/path:line: text`` lines, a locator not a context-adder),
and ``read``'s combined ``source/path`` + full content. (Phase 72:
the source-name scope clause and the combined-identity clause are
pinned byte-for-byte in
:func:`test_tools_section_phase72_contract_clauses`.)"""
assert "source: X | path: Y | title: Z" in TOOLS_SECTION
assert "case-insensitive" in TOOLS_SECTION
assert "case- and accent-insensitive" in TOOLS_SECTION # phase 162 fold
assert "up to 20" in TOOLS_SECTION
assert "source/path:line: text" in TOOLS_SECTION
assert "locator, not a context-adder" in TOOLS_SECTION
@@ -414,20 +415,28 @@ def test_tools_section_phase118_summary_seed_read_clause() -> None:
"| path: Y | title: Z`; to find one specific document without "
"listing, use `grep`."
) in TOOLS_SECTION
# The grep clause — byte-identical.
# The grep clause — folded contract (phase 162, LOCKED A3), up to the
# search-discipline sentence, byte-identical.
assert (
"`grep` locates an exact string (case-insensitive) in the "
"`grep` locates an exact string (case- and accent-insensitive — "
"accents fold: `besta` matches `BESTÅ`) in the "
"indexed documents and returns up to 20 matching "
"`source/path:line: text` lines — a locator, not a "
"context-adder: read the winner with `read`. A grep pattern "
"is a plain substring, NEVER a regex — '.*' and '\\.' are "
"literal text there; if such a pattern returns no matches, "
"retry with the plain text you expect to see. For a normal "
"search pass only `pattern` — its optional `path` argument "
"limits the search to one document you already know, by the "
"same combined `source/path` string; never a source name — a "
"bare document path (without the source name) will not "
"resolve there either."
"retry with the plain text you expect to see. "
) in TOOLS_SECTION
# The LOCKED search-discipline sentence sits immediately after the
# grep sentence, before the path clause (phase 162, LOCKED A4).
assert (
"retry with the plain text you expect to see. "
+ SEARCH_DISCIPLINE_RULE
+ " For a normal search pass only `pattern` — its optional "
"`path` argument limits the search to one document you already "
"know, by the same combined `source/path` string; never a "
"source name — a bare document path (without the source name) "
"will not resolve there either."
) in TOOLS_SECTION
# The discipline rules — byte-identical.
assert (
+153
View File
@@ -1346,3 +1346,156 @@ def test_bonus_lives_in_the_selection_layer_only() -> None:
rc.chunk_id: (rc.score, rc.cosine, rc.fts_hit, rc.name_hit) for rc in chunks
}
assert before == after
# ---------------------------------------------------------------------------
# Phase 162, D1 — the accent-folded name normalization (LOCKED A1): the
# 2026-10-07 live gap, the ASCII filename vs the accented content layer
# (``besta`` <-> ``BESTÅ``, both directions, ONE function)
# ---------------------------------------------------------------------------
from app.rag.retriever import _fold, _name_parts # noqa: E402
def test_fold_accent_ligature_and_sharp_s_cases() -> None:
"""The shared accent fold (phase 162, LOCKED A1): NFKD decomposition,
combining marks dropped — ``Å/å → a``, ``Ä/Ö/Ü → a/o/u`` (the letter
survives, the mark drops), ``ß → ss`` (the Unicode fold —
``str.casefold``; NFKD alone leaves ``ß`` intact), the ``fi`` ligature
``→ fi``, ``ç → c``."""
assert _fold("BESTÅ") == "besta"
assert _fold("Åsa") == "asa"
assert _fold("ÄÖÜäöü") == "aouaou"
assert _fold("Straße") == "strasse"
assert _fold("fine") == "fine"
assert _fold("garçon") == "garcon"
def test_fold_survives_non_decomposing_characters() -> None:
"""Characters with NO decomposition (CJK) survive the fold unchanged —
the caller's alnum scan drops them, exactly as pre-phase. Compat-only
characters (``①``) NFKD to their ASCII base (``1``); that base is
below :data:`NAME_TOKEN_MIN_LEN`, so it never becomes a name token
(the name-signal behavior for such characters is unchanged)."""
assert _fold("漢字") == "漢字"
assert _fold("①") == "1"
def test_normalize_name_is_accent_folded() -> None:
"""``_normalize_name`` = the alnum scan of ``_fold(s.lower())`` —
``Qwen 3.8 → qwen38`` UNCHANGED, ``BESTÅ → besta`` NEW (pre-phase it
was ``best``); non-folding characters still drop out (``漢字 → ""``)."""
assert _normalize_name("Qwen 3.8") == "qwen38"
assert _normalize_name("BESTÅ") == "besta"
assert _normalize_name("BÅsta") == "basta"
assert _normalize_name("straße") == "strasse"
assert _normalize_name("fine") == "fine"
assert _normalize_name("garçon") == "garcon"
assert _normalize_name("漢字") == ""
# Compat-only base survives (``①`` → ``1``) but is a 1-char token —
# below the minimum, never a name candidate (see the token test).
assert _normalize_name("①") == "1"
def test_name_hit_tokens_accented_question_bridges() -> None:
"""The question side of the bridge: the 2026-10-07 battery question
produces the ``besta`` token (pre-phase: ``best`` — the exact list is
pinned; ``how``/``do``/``i``/``the`` are below NAME_TOKEN_MIN_LEN).
The fold covers every pinned case: ``BESTÅ``, umlauts, ``ß``, the
``fi`` ligature, ``ç``."""
assert name_hit_tokens("How do I build the BESTÅ?") == ["build", "besta"]
assert name_hit_tokens("how do I fine the Über strasse") == [
"fine", "uber", "strasse",
]
assert name_hit_tokens("MÖRK ÖVER BÄSTA GARÇON") == [
"mork", "over", "basta", "garcon",
]
def test_name_hit_tokens_min_length_on_the_folded_form() -> None:
"""NAME_TOKEN_MIN_LEN is measured on the FOLDED form (``BÄST`` →
``bast`` (4, kept) while ``BÄ`` → ``ba`` (2, dropped)), and
non-folding tokens still drop out: ``漢字`` normalizes to ``""`` and
``①`` to the 1-char ``1`` — neither can ever be a candidate. The
``1`` is compat-decomposition (NFKD — LOCKED A1): the ``①`` token
itself never survives, and the only footprint a digit-base can leave
is a versioned-JOIN artifact (``the`` + ``1`` → ``the1``) whose sole
match route is PREFIXING a normalized path part — no real path does,
so the name signal for such characters is unchanged."""
assert name_hit_tokens("BÄST BÄ") == ["bast"]
assert name_hit_tokens("tell me about it ① 漢字") == ["tell", "about"]
# The compat-decomposed ``1`` acts as a version token for the join —
# pinned as the real behavior (a prefix-class candidate no path hits).
assert name_hit_tokens("tell me about the ① 漢字") == [
"tell", "about", "the1",
]
def test_name_parts_fold_both_the_ascii_and_accented_stem() -> None:
"""The path side of the bridge: the ASCII IKEA filename yields
``besta`` in the EQUAL set (the stem sub-component — the phase-119
A2 case, unchanged), and the accented filename ``BÅsta.md`` yields
``basta`` (the folded stem — already in the equal set via the
normalized-stem member, no _name_parts code change)."""
_prefix, equal = _name_parts(
"ikea/besta-frame-white__AA-1272068-11-100.pdf"
)
assert "besta" in equal # sub-component, unchanged
_prefix2, equal2 = _name_parts("manuals/BÅsta.md")
assert "basta" in equal2 # folded stem — an ASCII question matches it
def test_name_hit_chunks_accented_question_hits_ascii_doc() -> None:
"""The bridge, end to end (direction 1): the accented question
``BESTÅ`` name-hits the ASCII-stem document — the token ``besta``
EQUALS the stem sub-component of
``besta-frame-white__AA-1272068-11-100`` (the 2026-10-07 live miss;
pre-phase the token was ``best`` and the hit was impossible)."""
ikea = _doc(
"ikea/besta-frame-white__AA-1272068-11-100.pdf",
"MONTERING BÄSTA",
source="ikea_instructions",
)
distractor = _doc("notes/meeting.md", "we discussed besta", source="Brain-of-Reese")
session = _FakeSession(
[_name_row(ikea), _name_row(distractor)],
[_name_hit_lateral_row(ikea)],
)
out = _name_hit_chunks(session, "How do I build the BESTÅ?") # pyright: ignore[reportArgumentType]
assert [rc.document.path for rc in out] == [ikea.path]
assert out[0].name_hit is True
assert out[0].fts_hit is True
assert out[0].cosine == 0.0
def test_name_hit_chunks_ascii_question_hits_accented_doc() -> None:
"""The bridge, end to end (direction 2): the ASCII question token
``basta`` name-hits the ACCENTED filename ``BÅsta.md`` — the folded
stem enters the equal set as ``basta`` (pre-phase the accented stem
normalized to ``bsta`` — the ``å`` simply dropped out of the alnum
scan — so the match was impossible)."""
accented = _doc("manuals/BÅsta.md", "MONTERING", source="ikea_instructions")
miss = _doc("manuals/hesta.md", "MONTERING", source="ikea_instructions")
session = _FakeSession(
[_name_row(accented), _name_row(miss)],
[_name_hit_lateral_row(accented)],
)
out = _name_hit_chunks(session, "How do I assemble the basta shelf?") # pyright: ignore[reportArgumentType]
assert [rc.document.path for rc in out] == [accented.path]
assert out[0].name_hit is True
def test_name_hit_chunks_fold_preserves_digitless_precision() -> None:
"""The fold does NOT relax the A2 match classes: the accented token
``déploy`` folds to ``deploy`` and STILL does not hit the
``Deployments/`` document (the part normalizes to ``deployments`` ≠
``deploy`` — titles are never matched, digits never enter); only the
exact ``gitea`` component hits."""
precision = _doc("Deployments/reeseapps/README.md", "body", title="Deployments")
gitea = _doc("notes/gitea.md", "body")
session = _FakeSession(
[_name_row(precision), _name_row(gitea)],
[_name_hit_lateral_row(gitea)],
)
out = _name_hit_chunks(session, "how do I déploy gitea") # pyright: ignore[reportArgumentType]
assert [rc.document.path for rc in out] == [gitea.path]