phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
@@ -920,6 +920,98 @@ def render_folder_listing(
|
||||
return "\n".join([header, *body])
|
||||
|
||||
|
||||
def suggested_folder_lines(
|
||||
db: Session,
|
||||
suggested: Sequence[Document],
|
||||
max_lines: int = 5,
|
||||
max_entries: int = 8,
|
||||
) -> list[str]:
|
||||
"""The HIGH prompt's suggested-folder context lines (phase 119, D3,
|
||||
LOCKED A4) — pure composition over the existing ``ls`` machinery.
|
||||
|
||||
One line per DISTINCT parent folder of the *suggested* documents —
|
||||
in suggested-doc order, deduped by ``(source, parent prefix)`` (the
|
||||
first suggested doc wins the slot), at most *max_lines* lines:
|
||||
|
||||
* the parent prefix is the path up to (excluding) the last ``/``
|
||||
(``""`` = the source root);
|
||||
* the line is ``<source>/<prefix>/: e1, e2, …`` (the source root
|
||||
renders as ``<source>/: …`` — the filesystem-style folder path,
|
||||
trailing slash included, + the colon) with the folder's direct
|
||||
children in the
|
||||
EXISTING ``ls`` folder-level rendering order — the direct
|
||||
subfolders first (``name/ (N docs)``, the recursive doc count,
|
||||
singular ``(1 doc)``), then the files by relative filename — so
|
||||
the line reads the same as the model's own ``ls`` output of that
|
||||
folder (the :func:`group_folder_listing` grouping, over
|
||||
:func:`_source_document_rows` + :func:`_source_folder_summaries`);
|
||||
* the suggested document that OWNS the line is excluded from the
|
||||
entries (its identity is already in its ``<document>`` block — the
|
||||
line is the folder's OTHER contents, the pre-seed that makes the
|
||||
model ``read`` the right file in round 1 instead of walking the
|
||||
``ls`` drill-downs);
|
||||
* at most *max_entries* entries, then `` +N more`` (N = the
|
||||
remaining count, the true pre-cap folder total — the suggested
|
||||
doc leaves the total even when its row sat past
|
||||
:data:`LS_MAX_FILE_LINES`); a folder whose only entry was the
|
||||
suggested doc renders its header alone (the ``… — 0 documents, 0
|
||||
folders:`` empty-level precedent).
|
||||
|
||||
Empty *suggested* → ``[]`` (the caller then builds the
|
||||
byte-identical phase-118 prompt). Module-level so unit tests can
|
||||
monkeypatch the fetchers without a database (the house style:
|
||||
:func:`ls_top` / :func:`ls_folder` compose the same fetchers).
|
||||
"""
|
||||
lines: list[str] = []
|
||||
seen: set[tuple[str, str]] = set()
|
||||
rows_cache: dict[str, tuple[list[tuple[str, str, str]], dict[str, str]]] = {}
|
||||
for doc in suggested:
|
||||
if len(lines) >= max_lines:
|
||||
break
|
||||
prefix = folder_of(doc.path)
|
||||
key = (doc.source, prefix)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
if doc.source not in rows_cache:
|
||||
rows_cache[doc.source] = (
|
||||
_source_document_rows(db, doc.source),
|
||||
_source_folder_summaries(db, doc.source),
|
||||
)
|
||||
rows, summaries = rows_cache[doc.source]
|
||||
subfolders, files, total_files = group_folder_listing(
|
||||
doc.source, prefix, rows, summaries
|
||||
)
|
||||
entries = [
|
||||
f"{sub}/ ({count} {'doc' if count == 1 else 'docs'})"
|
||||
for sub, count, _summary in subfolders
|
||||
]
|
||||
# The owning suggested doc is a direct file of this folder —
|
||||
# drop it from the entries (its identity is already in its
|
||||
# <document> block); its count leaves the total either way
|
||||
# (even when its row sat past the LS_MAX_FILE_LINES file cap).
|
||||
files = [entry for entry in files if entry[1] != doc.path]
|
||||
# The file entries come AFTER the subfolders (the ls folder-level
|
||||
# order) and ride by RELATIVE filename — the basename within the
|
||||
# folder (the line's ``<source>/<prefix>/:`` header supplies the
|
||||
# folder; combined ``source/prefix/name`` is the read identity).
|
||||
entries.extend(path.rsplit("/", 1)[-1] for _src, path, _title, _date in files)
|
||||
total = len(subfolders) + total_files - 1
|
||||
shown = entries[:max_entries]
|
||||
# The pinned identity shape: ``<source>/<prefix>/:`` (the source
|
||||
# root: ``<source>/:``) — the filesystem-style folder path
|
||||
# (trailing slash included) + the colon.
|
||||
identity = f"{doc.source}/{prefix}/" if prefix else f"{doc.source}/"
|
||||
line = f"{identity}:"
|
||||
if shown:
|
||||
suffix = ", ".join(shown)
|
||||
if total > len(shown):
|
||||
suffix += f" +{total - len(shown)} more"
|
||||
line += f" {suffix}"
|
||||
lines.append(line)
|
||||
return lines
|
||||
|
||||
|
||||
def find_document(db: Session, source: str, path: str) -> Document | None:
|
||||
"""The indexed document at ``(source, path)``, or ``None``.
|
||||
|
||||
|
||||
+49
-14
@@ -132,21 +132,30 @@ _KB_INTRO = (
|
||||
)
|
||||
|
||||
#: The ``<documents>`` section's start-here intro (phase 118, task 03 —
|
||||
#: the owner directive, TODO L3): the seeded blocks are the SUMMARIES of
|
||||
#: the top-ranked documents for the question, opt-in starting points —
|
||||
#: start here if one seems right; ``read`` the document's combined
|
||||
#: ``source/path`` to add its FULL content before answering specifics
|
||||
#: (the full text is NOT in the prompt until you read it); cite the
|
||||
#: document(s) you used, by path. Rendered BEFORE the first block, only
|
||||
#: when at least one block is present. The ``<documents>`` tag and the
|
||||
#: per-block ``<document>`` markup stay byte-stable around it (the E2E
|
||||
#: mock's block parser and the ``read``-tool copy key off both).
|
||||
#: the owner directive, TODO L3; phase 119, task 04, LOCKED A5 — the
|
||||
#: final sentence re-cut to the cite discipline): the seeded blocks are
|
||||
#: the SUMMARIES of the top-ranked documents for the question, opt-in
|
||||
#: starting points — start here if one seems right; ``read`` the
|
||||
#: document's combined ``source/path`` to add its FULL content before
|
||||
#: answering specifics (the full text is NOT in the prompt until you
|
||||
#: read it); cite ONLY the documents you read (or the suggested document
|
||||
#: you answered from without reading it) — never one you neither read
|
||||
#: nor used. The discipline sentence closes the phase-119 live
|
||||
#: confabulation — the model's "Docs used:" line cited a file it never
|
||||
#: read, confabulated from the suggestion blocks sitting in context —
|
||||
#: and retires the phase-118 "cite the document(s) you used, by path"
|
||||
#: sentence. Rendered BEFORE the first block, only when at least one
|
||||
#: block is present. The ``<documents>`` tag and the per-block
|
||||
#: ``<document>`` markup stay byte-stable around it (the E2E mock's
|
||||
#: block parser and the ``read``-tool copy key off both).
|
||||
SUGGEST_INTRO = (
|
||||
"The blocks below are the summaries of the top-ranked documents for "
|
||||
"your question — start here if one seems right to you: call `read` "
|
||||
"with that document's combined `source/path` to add its full content "
|
||||
"before answering specifics (its full text is not in the prompt until "
|
||||
"you read it). Cite the document(s) you used, by path."
|
||||
"you read it). Cite only the document(s) you read — or, if you "
|
||||
"answered from a suggested summary without reading it, cite that "
|
||||
"suggested document — never a document you neither read nor used."
|
||||
)
|
||||
|
||||
#: The ``<tools>`` instructions section — **HIGH prompt only** (phase 37,
|
||||
@@ -416,6 +425,7 @@ def build_high_prompt(
|
||||
documents: Sequence[Document],
|
||||
notes: Sequence[str] | None = None,
|
||||
kb_overview: str | None = None,
|
||||
folder_lines: Sequence[str] = (),
|
||||
) -> str:
|
||||
"""Grounded turn: locked persona (+ steering, + KB overview) + the
|
||||
top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions
|
||||
@@ -423,7 +433,8 @@ def build_high_prompt(
|
||||
``grep`` shapes; phase 72: the document-identity contract up front;
|
||||
phase 118, LOCKED A6 re-revising A7: the ``<documents>`` section
|
||||
seeds SUMMARIES, never full texts — full text enters the context
|
||||
only through the capped ``read`` tool).
|
||||
only through the capped ``read`` tool; phase 119, LOCKED A4: the
|
||||
suggested-folder context lines, below).
|
||||
|
||||
Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
|
||||
→ ``<documents>`` → ``<tools>``; empty steering/overview omit their
|
||||
@@ -435,11 +446,28 @@ def build_high_prompt(
|
||||
start-here :data:`SUGGEST_INTRO` line (before the first block — the
|
||||
phase-15 ``_STEERING_INTRO`` / phase-31 ``_KB_INTRO`` precedent): the
|
||||
blocks are the summaries of the top-ranked documents, ``read`` adds
|
||||
the full text, and the answer cites the document(s) used by path.
|
||||
This is NOT the reverted phase-72 in-context reminder (that copy
|
||||
the full text, and the answer cites only the documents it read (or
|
||||
the suggested document it answered from without reading — the
|
||||
phase-119 cite discipline, LOCKED A5; the phase-118 "cite what you
|
||||
used, by path" sentence is retired). This is NOT the reverted
|
||||
phase-72 in-context reminder (that copy
|
||||
taught the seed documents as already-read context; this one teaches
|
||||
the summary-as-starting-point contract the A6 re-revision requires).
|
||||
|
||||
*folder_lines* (phase 119, D3, LOCKED A4): the suggested-folder
|
||||
context lines (``app.rag.agent.suggested_folder_lines``) — the
|
||||
direct children of each suggested document's parent folder, so the
|
||||
model ``read``s the right file in round 1 instead of walking the
|
||||
``ls`` drill-downs. When blocks are present AND the lines are
|
||||
non-empty, the ``<documents>`` body is ``SUGGEST_INTRO``, then the
|
||||
folder lines (each on its own line, immediately after the intro
|
||||
line), a blank line, then the first ``<document>`` block. They are
|
||||
PLAIN lines — no new markup/tag (the E2E mock keys off the
|
||||
``<documents>`` marker and the LAST block's tail, so the lines must
|
||||
never land after a summary). Empty *folder_lines* ⇒ the phase-118
|
||||
build is byte-identical (pinned in
|
||||
``tests/unit/test_prompts.py``).
|
||||
|
||||
Each ``<document>`` block carries the identity attributes
|
||||
``source`` / ``path`` / ``title`` — and, since phase 106 (D5),
|
||||
``date`` (the row's ``created_at`` UTC date part, ``YYYY-MM-DD``,
|
||||
@@ -466,7 +494,14 @@ def build_high_prompt(
|
||||
"(no documents matched — do not invent specifics)"
|
||||
)
|
||||
if blocks:
|
||||
body = SUGGEST_INTRO + "\n\n" + body
|
||||
if folder_lines:
|
||||
# Phase 119 (D3, LOCKED A4): the folder lines ride immediately
|
||||
# after the intro line, one per line, a blank line, then the
|
||||
# first block — plain lines, no new markup (the <document>
|
||||
# block markup AND body stay byte-identical around them).
|
||||
body = SUGGEST_INTRO + "\n" + "\n".join(folder_lines) + "\n\n" + body
|
||||
else:
|
||||
body = SUGGEST_INTRO + "\n\n" + body
|
||||
prompt = _base("HIGH")
|
||||
for part in (build_kb_section(kb_overview or ""), build_steering_section(notes or [])):
|
||||
if part:
|
||||
|
||||
+272
-89
@@ -4,16 +4,30 @@
|
||||
each carrying its cosine ``1 − distance`` (the honesty-gate input).
|
||||
* **Lexical list** — top-N chunks matching an OR-``tsquery`` over the
|
||||
question's tokens, ordered by ``ts_rank``, UNION the name-hit list:
|
||||
documents whose normalized name (title + path stem, alnum-only,
|
||||
lowercased) contains a DIGIT-BEARING normalized question token of
|
||||
length >= 4 — bare tokens (``1panel``) and the join of adjacent
|
||||
question tokens starting with a letter ("Qwen 3.8" → ``qwen38``).
|
||||
The digit requirement is the precision guard: plain prose words
|
||||
("server", "arguments" — 4+ chars but no digit) never name-match,
|
||||
while versioned product names (the incident's whole point) always
|
||||
carry one. The name-hit documents LEAD the lexical list (ranked by
|
||||
match count, then total matched length, then catalog order; capped
|
||||
at :data:`NAME_HIT_LIMIT`),
|
||||
documents whose PATH components match a question name token under
|
||||
the two-class rule (the 2026-09-05 incident fix, extended
|
||||
2026-09-16 for product names WITHOUT digits — phase 119, LOCKED
|
||||
A2). The token set is class-agnostic — every normalized whitespace
|
||||
token of length >= 4 (dotted kept whole: ``llama.cpp`` →
|
||||
``llamacpp``) plus the versioned join ("Qwen 3.8" → ``qwen38``) —
|
||||
the digit distinction lives on the MATCH side:
|
||||
|
||||
* a DIGIT-BEARING token prefix-matches a normalized path part or
|
||||
file stem (``qwen38`` →
|
||||
``qwen3.8-27b-juggernaut-vulkan.container`` — the incident's
|
||||
original precision guard);
|
||||
* a DIGITLESS token exact-matches a normalized path part, file
|
||||
stem, or stem sub-component (``gitea`` → the ``gitea/`` folder,
|
||||
``gitea.md``, ``kubernetes_gitea``, ``gitea-values``).
|
||||
|
||||
TITLES ARE NEVER MATCHED — titles are prose: ``deploy/Deployments/
|
||||
reeseapps/README.md`` is titled "Deployments" and must not
|
||||
name-match the common token "deploy" (the owner-verified failure
|
||||
mode of the naive relaxation). The name-hit documents LEAD the
|
||||
lexical list (ranked by distinct matched-token count, then catalog
|
||||
order ``(source, path)`` — the old total-matched-length tie-break
|
||||
is retired, it outranked 5-char product names by 6-char common
|
||||
tokens; capped at :data:`NAME_HIT_LIMIT`),
|
||||
the FTS rows follow. This is what finds name-your-tool questions
|
||||
("gitlab") that vector similarity buries — and, the 2026-09-05
|
||||
incident, the versioned-name case the default parser lexes
|
||||
@@ -23,7 +37,8 @@
|
||||
``llama``/``cpp`` tokens). The name-hit rows carry ``fts_hit=True``
|
||||
(they ARE the lexical signal — the A8 honesty gate answers on them
|
||||
only when the best cosine clears ``BOR_LEXICAL_SUPPORT_FLOOR``,
|
||||
A8 revised 2026-09-14) and ``cosine=0.0``; the RRF fusion is
|
||||
A8 revised 2026-09-14), ``cosine=0.0``, and ``name_hit=True`` (the
|
||||
selection-tier bonus input, phase 119 D2); the RRF fusion is
|
||||
unchanged (same lists, same ``1/(k+rank)`` terms).
|
||||
* **Fusion** — Reciprocal Rank Fusion (``score = Σ 1/(k + rank)`` over the
|
||||
lists a chunk appears in; chunks hit by both lists get both terms). The
|
||||
@@ -35,6 +50,19 @@
|
||||
ranked documents (up to ``BOR_RELATED_MAX_DOCS``) become the related
|
||||
tier.
|
||||
|
||||
Phase 119, D2 (LOCKED A3): the SELECTION walks (``select_suggested``,
|
||||
``select_related``, ``weak_hit_titles``) re-rank on an EFFECTIVE score
|
||||
— the document's best fused chunk score plus ``BOR_NAME_HIT_BONUS``
|
||||
when any of its chunks is a name hit (the D1 path match). A
|
||||
product-name question ("How do I deploy gitea?") thus lifts the
|
||||
product's own documents into the seeded suggestion tier even when the
|
||||
name hit only LEADS the lexical list. The bonus lives in the selection
|
||||
layer only (the phase-106 recency-boost pattern: additive, bounded,
|
||||
single apply site): chunk scores, the fusion, ``fuse()``,
|
||||
:func:`retrieve()`, the A8 honesty gate, and ``query_log.top_score``
|
||||
are untouched; ``0`` reproduces the pre-phase walk byte-identically
|
||||
(the kill switch) and a negative value fails startup loudly.
|
||||
|
||||
The product requirement (A7, re-revised by the phase-118 owner directive,
|
||||
LOCKED A6, 2026-09-15): the retrieval path seeds **summaries** — the
|
||||
suggestion tier (:func:`select_suggested`, top-N distinct documents, no
|
||||
@@ -96,6 +124,10 @@ NAME_HIT_LIMIT = 10
|
||||
#: Alphanumeric runs of a lowercased string (name normalization).
|
||||
_ALNUM_RE = re.compile(r"[a-z0-9]+")
|
||||
|
||||
#: Non-alphanumeric runs of a lowercased string (the stem sub-component
|
||||
#: split — ``kubernetes_gitea`` → ``kubernetes`` / ``gitea``).
|
||||
_STEM_SPLIT_RE = re.compile(r"[^a-z0-9]+")
|
||||
|
||||
|
||||
def _normalize_name(s: str) -> str:
|
||||
"""Lowercased, alnum-only form of *s* (``Qwen 3.8`` → ``qwen38``)."""
|
||||
@@ -103,14 +135,23 @@ def _normalize_name(s: str) -> str:
|
||||
|
||||
|
||||
def name_hit_tokens(question: str) -> list[str]:
|
||||
"""The normalized name-match candidates of one question.
|
||||
"""The name-match candidates of one question (class-agnostic,
|
||||
LOCKED A2).
|
||||
|
||||
Only DIGIT-BEARING candidates count (the precision guard — a plain
|
||||
prose word like "server" or "arguments" must never name-match a
|
||||
document; a versioned product name always carries a digit):
|
||||
Every qualifying whitespace token is a candidate — the digit
|
||||
distinction (which CLASS of match a token gets) lives on the
|
||||
matching side (:func:`_name_hit_chunks`), because product names
|
||||
WITHOUT digits ("gitea", "forgejo", "gateway") were the
|
||||
2026-09-16 live finding: the old digit-only list gave them no
|
||||
name signal at all, so an OR-tsquery dominated by a common token
|
||||
("deploy") buried the product's own documents. Prose precision is
|
||||
now carried by the match class itself (a digitless token must
|
||||
EQUAL a whole path component — "server"/"arguments" rarely do),
|
||||
not by filtering the candidate list.
|
||||
|
||||
* the :func:`_normalize_name` form of every whitespace token, kept
|
||||
when at least :data:`NAME_TOKEN_MIN_LEN` chars (``1panel``,
|
||||
when at least :data:`NAME_TOKEN_MIN_LEN` chars — dotted tokens
|
||||
kept whole (``llama.cpp`` → ``llamacpp``, ``1panel``,
|
||||
``qwen38`` from a single written token);
|
||||
* the versioned-name JOIN — the normalized concatenation of every
|
||||
ADJACENT token pair whose SECOND token is purely numeric (a
|
||||
@@ -121,18 +162,17 @@ def name_hit_tokens(question: str) -> list[str]:
|
||||
starts with the first token's text (a letter in practice), so no
|
||||
digit-leading artifact (``38show``) can survive.
|
||||
|
||||
Order of first appearance, de-duplicated.
|
||||
Order of first appearance, de-duplicated. Matching applies the
|
||||
A2 rule: digit-bearing candidates prefix a normalized path part
|
||||
or file stem; digitless candidates equal a part, stem, or stem
|
||||
sub-component — titles are never matched.
|
||||
"""
|
||||
tokens = question.split()
|
||||
out: list[str] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def add(tok: str) -> None:
|
||||
if (
|
||||
len(tok) >= NAME_TOKEN_MIN_LEN
|
||||
and any(ch.isdigit() for ch in tok)
|
||||
and tok not in seen
|
||||
):
|
||||
if len(tok) >= NAME_TOKEN_MIN_LEN and tok not in seen:
|
||||
seen.add(tok)
|
||||
out.append(tok)
|
||||
|
||||
@@ -187,6 +227,13 @@ class RetrievedChunk:
|
||||
resolves to the full source document through the unchanged
|
||||
chunk→document mapping (A7 revised). Default ``False`` keeps every
|
||||
ordinary content chunk valid.
|
||||
* ``name_hit`` — True for the name-hit representative row (phase 119,
|
||||
D1): the chunk came from the document PATH match, not the
|
||||
OR-tsquery — the selection tier's bonus input (task 02, D2).
|
||||
Default ``False`` keeps every ordinary construction valid;
|
||||
:func:`fuse`'s ``replace()`` copies it (the double-hit merge ORs it
|
||||
in — a vector row that is also the name hit's representative chunk
|
||||
keeps the flag).
|
||||
"""
|
||||
|
||||
chunk_id: uuid.UUID
|
||||
@@ -197,6 +244,7 @@ class RetrievedChunk:
|
||||
cosine: float = 0.0
|
||||
fts_hit: bool = False
|
||||
is_summary: bool = False
|
||||
name_hit: bool = False
|
||||
|
||||
|
||||
def lexical_tsquery(question: str) -> str | None:
|
||||
@@ -234,7 +282,9 @@ def fuse(
|
||||
Lexical-only hits (no vector rank) enter with ``cosine=0.0`` and
|
||||
``fts_hit=True``; vector chunks matched by the lexical list get
|
||||
``fts_hit=True`` in place (the input objects are mutated — callers
|
||||
should not reuse them afterwards).
|
||||
should not reuse them afterwards), and ``name_hit=True`` is ORed in
|
||||
(a vector row that is also the name hit's representative chunk is a
|
||||
name-hit row — the phase-119 selection tier must see it).
|
||||
"""
|
||||
if k <= 0:
|
||||
raise ValueError("rrf k must be > 0")
|
||||
@@ -247,7 +297,9 @@ def fuse(
|
||||
term = 1.0 / (k + rank)
|
||||
if rc.chunk_id in by_id:
|
||||
existing = by_id[rc.chunk_id]
|
||||
by_id[rc.chunk_id] = replace(existing, fts_hit=True)
|
||||
by_id[rc.chunk_id] = replace(
|
||||
existing, fts_hit=True, name_hit=existing.name_hit or rc.name_hit
|
||||
)
|
||||
fused[rc.chunk_id] += term
|
||||
else:
|
||||
rc = replace(rc, fts_hit=True)
|
||||
@@ -390,23 +442,76 @@ _NAME_HIT_SQL = text(
|
||||
)
|
||||
|
||||
|
||||
def _name_hit_chunks(db: Session, question: str) -> list[RetrievedChunk]:
|
||||
"""The documents whose NAME matches the question (the 2026-09-05
|
||||
incident, the versioned-name signal).
|
||||
def _name_parts(path: str) -> tuple[set[str], set[str]]:
|
||||
"""The normalized name components of a document path (LOCKED A2).
|
||||
|
||||
A document is a name hit when its normalized name —
|
||||
:func:`_normalize_name` of its ``title`` followed by the normalized
|
||||
path stem (``qwen3.8-27b-juggernaut-vulkan.container`` →
|
||||
``qwen3827bjuggernautvulkan``) — contains at least one
|
||||
:func:`name_hit_tokens` candidate (``qwen38``). Ranked by
|
||||
(distinct matched tokens, total matched length, source, path) —
|
||||
catalog order is the final deterministic tie-break — capped at
|
||||
:data:`NAME_HIT_LIMIT`. Each hit becomes one lexical
|
||||
:class:`RetrievedChunk` (its representative chunk, ``fts_hit=True``,
|
||||
``cosine=0.0``). Two lightweight queries: one projection over
|
||||
(id, source, path, title) in catalog order, one LATERAL chunk
|
||||
fetch for the ≤ :data:`NAME_HIT_LIMIT` winners (no full-content
|
||||
load; the content joins in via the row fetch below).
|
||||
Returns ``(prefix_set, equal_set)``:
|
||||
|
||||
* ``prefix_set`` — the normalized form of every path part plus the
|
||||
normalized file stem (``qwen3.8-27b-juggernaut-vulkan.container``
|
||||
→ ``{…, qwen3827bjuggernautvulkancontainer, qwen3827bjuggernaut…}``):
|
||||
DIGIT-BEARING tokens prefix-match these;
|
||||
* ``equal_set`` — ``prefix_set`` plus the stem's sub-components
|
||||
(the stem lowercased, split on non-alphanumeric runs, each piece
|
||||
normalized, empties dropped: ``kubernetes_gitea`` →
|
||||
``kubernetes`` / ``gitea``): DIGITLESS tokens exact-match these.
|
||||
"""
|
||||
p = Path(path)
|
||||
prefix = {_normalize_name(part) for part in p.parts}
|
||||
prefix.discard("")
|
||||
prefix.add(_normalize_name(p.stem))
|
||||
equal = set(prefix)
|
||||
for piece in _STEM_SPLIT_RE.split(p.stem.lower()):
|
||||
norm = _normalize_name(piece)
|
||||
if norm:
|
||||
equal.add(norm)
|
||||
return prefix, equal
|
||||
|
||||
|
||||
def _name_token_matches(token: str, prefix: set[str], equal: set[str]) -> bool:
|
||||
"""The two-class match of one candidate token against one path
|
||||
(LOCKED A2, phase 119):
|
||||
|
||||
* the token CONTAINS A DIGIT → it is a PREFIX of a normalized path
|
||||
part or file stem (``qwen38`` →
|
||||
``qwen3.8-27b-juggernaut-vulkan.container``);
|
||||
* the token HAS NO DIGIT → it EQUALS a normalized path part, file
|
||||
stem, or stem sub-component (``gitea`` → the ``gitea/`` folder,
|
||||
``gitea.md``, ``kubernetes_gitea``, ``gitea-values``).
|
||||
"""
|
||||
if any(ch.isdigit() for ch in token):
|
||||
return any(part.startswith(token) for part in prefix)
|
||||
return token in equal
|
||||
|
||||
|
||||
def _name_hit_chunks(db: Session, question: str) -> list[RetrievedChunk]:
|
||||
"""The documents whose PATH matches the question (the 2026-09-05
|
||||
incident's versioned-name signal, extended 2026-09-16 for product
|
||||
names without digits — phase 119, D1, LOCKED A2).
|
||||
|
||||
A document is a name hit when at least one :func:`name_hit_tokens`
|
||||
candidate matches its path components under the two-class rule
|
||||
(:func:`_name_token_matches`): DIGIT-BEARING tokens prefix-match a
|
||||
normalized path part or file stem (``qwen38`` →
|
||||
``qwen3.8-27b-juggernaut-vulkan.container``); DIGITLESS tokens
|
||||
exact-match a normalized path part, file stem, or stem
|
||||
sub-component (``gitea`` → the ``gitea/`` folder, ``gitea.md``,
|
||||
``kubernetes_gitea``, ``gitea-values``). **Titles are never
|
||||
matched** — titles are prose: ``deploy/Deployments/reeseapps/
|
||||
README.md`` is titled "Deployments" and must NOT name-match the
|
||||
common token ``deploy`` or the 9 other deployment-titled docs
|
||||
(the owner-verified failure mode of the naive title relaxation).
|
||||
|
||||
Ranked by (distinct matched-token count DESC, then ``(source,
|
||||
path)`` catalog order) — the old total-matched-length tie-break is
|
||||
RETIRED (it systematically outranked 5-char product names by
|
||||
6-char common tokens) — capped at :data:`NAME_HIT_LIMIT`. Each hit
|
||||
becomes one lexical :class:`RetrievedChunk` (its representative
|
||||
chunk, ``fts_hit=True``, ``cosine=0.0``, ``name_hit=True``). Two
|
||||
lightweight queries: one projection over (id, source, path, title)
|
||||
in catalog order (the title is selected but never matched), one
|
||||
LATERAL chunk fetch for the ≤ :data:`NAME_HIT_LIMIT` winners (no
|
||||
full-content load; the content joins in via the row fetch below).
|
||||
"""
|
||||
tokens = name_hit_tokens(question)
|
||||
if not tokens:
|
||||
@@ -416,22 +521,22 @@ def _name_hit_chunks(db: Session, question: str) -> list[RetrievedChunk]:
|
||||
Document.source, Document.path
|
||||
)
|
||||
).all()
|
||||
scored: list[tuple[int, int, uuid.UUID]] = []
|
||||
scored: list[tuple[int, uuid.UUID]] = []
|
||||
by_id: dict[uuid.UUID, tuple[str, str]] = {} # id -> (source, path)
|
||||
for doc_id, source, path, title in rows:
|
||||
for doc_id, source, path, _title in rows:
|
||||
by_id[doc_id] = (source, path)
|
||||
name = _normalize_name(title) + _normalize_name(Path(path).stem)
|
||||
matched = [t for t in tokens if t in name]
|
||||
prefix, equal = _name_parts(path)
|
||||
matched = sum(1 for t in tokens if _name_token_matches(t, prefix, equal))
|
||||
if matched:
|
||||
scored.append((len(matched), sum(len(t) for t in matched), doc_id))
|
||||
scored.append((matched, doc_id))
|
||||
if not scored:
|
||||
return []
|
||||
# Ranked by (distinct matched tokens, total matched length), the
|
||||
# deterministic catalog tie-break (source, path) last.
|
||||
scored.sort(
|
||||
key=lambda s: (-s[0], -s[1], by_id[s[2]][0], by_id[s[2]][1])
|
||||
)
|
||||
ids = [s[2] for s in scored[:NAME_HIT_LIMIT]]
|
||||
# Ranked by (distinct matched-token count DESC), the deterministic
|
||||
# catalog tie-break (source, path) — the old total-matched-length
|
||||
# tie-break is retired (it outranked 5-char product names by
|
||||
# 6-char common tokens).
|
||||
scored.sort(key=lambda s: (-s[0], by_id[s[1]][0], by_id[s[1]][1]))
|
||||
ids = [s[1] for s in scored[:NAME_HIT_LIMIT]]
|
||||
hit_rows = list(db.execute(_NAME_HIT_SQL, {"ids": ids}).all())
|
||||
# The LATERAL query returns winners in id order; re-order by the
|
||||
# ranked order computed above so the lexical list is deterministic.
|
||||
@@ -460,6 +565,7 @@ def _name_hit_chunks(db: Session, question: str) -> list[RetrievedChunk]:
|
||||
cosine=0.0, # no vector rank — name-only hit
|
||||
fts_hit=True, # lexical signal — A8 answers if cosine corroborates
|
||||
is_summary=bool(row.is_summary),
|
||||
name_hit=True, # phase 119 — the selection tier's bonus input
|
||||
)
|
||||
)
|
||||
return out
|
||||
@@ -556,20 +662,92 @@ def retrieve(
|
||||
return fused
|
||||
|
||||
|
||||
def weak_hit_titles(chunks: Sequence[RetrievedChunk]) -> list[str]:
|
||||
"""Distinct parent-document titles of *chunks*, best fused score first.
|
||||
def _selection_order(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
bonus: float,
|
||||
) -> list[tuple[Document, float, float, int]]:
|
||||
"""The shared selection walk of the phase-119 name-hit bonus (D2,
|
||||
LOCKED A3) — one walk for ``select_suggested``, ``select_related``,
|
||||
and ``weak_hit_titles``.
|
||||
|
||||
Returns ``(document, effective_score, best_cosine, first_seen_index)``
|
||||
for each distinct document, where *effective_score* is the
|
||||
document's best fused chunk score plus *bonus* when ANY of its
|
||||
chunks carries ``name_hit`` (the D1 path match — the bonus is per
|
||||
DOCUMENT, applied ONCE no matter how many of the document's chunks
|
||||
are name hits).
|
||||
|
||||
The walk keeps the EXISTING selection semantics (not just the loop):
|
||||
the same stable score-descending order as
|
||||
:func:`select_documents_tiered` / :func:`select_suggested` — a
|
||||
document's rank position is fixed by its FIRST seen chunk — with its
|
||||
best cosine tracked across ALL of its chunks (the tiered walk's
|
||||
tracking).
|
||||
|
||||
* *bonus* ``== 0`` (the kill switch) or no name-hit chunk present:
|
||||
the document order is IDENTICAL to the pre-phase walk — no re-sort
|
||||
happens at all (byte-identical, LOCKED A3);
|
||||
* otherwise the documents are ordered by
|
||||
``(−effective_score, −best_cosine, document.path,
|
||||
first_seen_index)`` — the bounded re-rank: a name-hit document
|
||||
gets a head start on the fused scale, and an effective-score tie
|
||||
resolves by cosine, then path, then the pre-bonus rank.
|
||||
|
||||
The bonus lives in the SELECTION layer only: the chunk objects are
|
||||
never modified — their ``score``/``cosine``/``fts_hit`` (the A8
|
||||
gate's inputs) and ``query_log.top_score`` are untouched.
|
||||
"""
|
||||
order: list[Document] = []
|
||||
best_score: dict[uuid.UUID, float] = {}
|
||||
best_cosine: dict[uuid.UUID, float] = {}
|
||||
any_name_hit: dict[uuid.UUID, bool] = {}
|
||||
first_seen: dict[uuid.UUID, int] = {}
|
||||
for idx, rc in enumerate(sorted(chunks, key=lambda c: c.score, reverse=True)):
|
||||
doc = rc.document
|
||||
if doc.id in best_cosine:
|
||||
if rc.cosine > best_cosine[doc.id]:
|
||||
best_cosine[doc.id] = rc.cosine
|
||||
if rc.name_hit:
|
||||
any_name_hit[doc.id] = True
|
||||
else:
|
||||
order.append(doc)
|
||||
best_score[doc.id] = rc.score
|
||||
best_cosine[doc.id] = rc.cosine
|
||||
any_name_hit[doc.id] = rc.name_hit
|
||||
first_seen[doc.id] = idx
|
||||
effective = {
|
||||
doc.id: best_score[doc.id] + (bonus if any_name_hit[doc.id] else 0.0)
|
||||
for doc in order
|
||||
}
|
||||
if bonus > 0.0 and any(any_name_hit[doc.id] for doc in order):
|
||||
order.sort(
|
||||
key=lambda doc: (
|
||||
-effective[doc.id],
|
||||
-best_cosine[doc.id],
|
||||
doc.path,
|
||||
first_seen[doc.id],
|
||||
)
|
||||
)
|
||||
return [
|
||||
(doc, effective[doc.id], best_cosine[doc.id], first_seen[doc.id])
|
||||
for doc in order
|
||||
]
|
||||
|
||||
|
||||
def weak_hit_titles(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
bonus: float | None = None,
|
||||
) -> list[str]:
|
||||
"""Distinct parent-document titles of *chunks*, best SELECTION score
|
||||
first (the phase-119 name-hit bonus applied — see
|
||||
:func:`_selection_order`, D2, LOCKED A3).
|
||||
|
||||
Deflection mode (PLAN §6, A8) is built from these *titles only* — the
|
||||
LOW prompt and the "Maybe try" chips never see document content.
|
||||
"""
|
||||
titles: list[str] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if rc.document.id in seen:
|
||||
continue
|
||||
seen.add(rc.document.id)
|
||||
titles.append(rc.document.title)
|
||||
return titles
|
||||
if bonus is None:
|
||||
bonus = get_settings().name_hit_bonus
|
||||
return [doc.title for doc, _eff, _cos, _idx in _selection_order(chunks, bonus)]
|
||||
|
||||
|
||||
def select_documents_tiered(
|
||||
@@ -685,54 +863,56 @@ def select_documents(
|
||||
def select_suggested(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
n: int | None = None,
|
||||
bonus: float | None = None,
|
||||
) -> list[Document]:
|
||||
"""Top-N distinct parent documents in fused rank order — the phase-118
|
||||
"start here" suggestion tier (LOCKED A3), with NO cosine floor.
|
||||
"""Top-N distinct parent documents in SELECTION rank order — the
|
||||
phase-118 "start here" suggestion tier (LOCKED A3, re-revised by
|
||||
phase 119: the walk now carries the bounded name-hit bonus, D2),
|
||||
with NO cosine floor.
|
||||
|
||||
Distinct parent documents are walked in the SAME stable score-
|
||||
descending order as :func:`select_documents_tiered` (a document's rank
|
||||
position is fixed by its FIRST seen chunk; dedupe by ``document.id``),
|
||||
and at most *n* of them are returned (default the
|
||||
``BOR_SUGGESTED_DOCS`` setting, 5). Unlike the phase-113 cited tier,
|
||||
the usefulness bar NEVER filters here: a lexical-only hit with
|
||||
cosine 0.0 is suggested when it ranks. Suggestions are opt-in
|
||||
position is fixed by its FIRST seen chunk; dedupe by ``document.id``)
|
||||
— the shared :func:`_selection_order` walk — plus, when the bonus is
|
||||
on AND a name-hit chunk is present, the
|
||||
``(−effective, −best_cosine, path, first_seen)`` re-rank that gives
|
||||
a name-hit document its head start. At most *n* of them are returned
|
||||
(default the ``BOR_SUGGESTED_DOCS`` setting, 5). Unlike the phase-113
|
||||
cited tier, the usefulness bar NEVER filters here: a lexical-only hit
|
||||
with cosine 0.0 is suggested when it ranks. Suggestions are opt-in
|
||||
starting points, not citations — the seeded prompt carries the
|
||||
document's summary, and the LLM decides whether to extend its context
|
||||
by reading the document's full text.
|
||||
|
||||
*bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the
|
||||
owner-tunable starting point); ``0`` reproduces the pre-phase walk
|
||||
byte-identically (the kill switch, LOCKED A3).
|
||||
|
||||
The returned rows carry the full document content, byte-identical —
|
||||
the content is what the agent's ``read`` tool serves later (never
|
||||
truncated; A6 re-revises A7: full text enters the context only through
|
||||
the capped ``read`` tool).
|
||||
"""
|
||||
top_n = n if n is not None else get_settings().suggested_docs
|
||||
|
||||
order: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if len(order) >= top_n:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
order.append(doc)
|
||||
return order
|
||||
if bonus is None:
|
||||
bonus = get_settings().name_hit_bonus
|
||||
return [doc for doc, _eff, _cos, _idx in _selection_order(chunks, bonus)[:top_n]]
|
||||
|
||||
|
||||
def select_related(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
excluded_ids: set[uuid.UUID],
|
||||
cap: int,
|
||||
bonus: float | None = None,
|
||||
) -> list[Document]:
|
||||
"""The documents ranked AFTER *excluded_ids* — the phase-118 related
|
||||
tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap*
|
||||
(``BOR_RELATED_MAX_DOCS``).
|
||||
|
||||
The SAME stable score-descending walk as
|
||||
:func:`select_documents_tiered` / :func:`select_suggested` (a
|
||||
document's rank position is fixed by its FIRST seen chunk; dedupe by
|
||||
``document.id``), skipping every document whose id is in
|
||||
The SAME shared selection walk as :func:`select_documents_tiered` /
|
||||
:func:`select_suggested` (a document's rank position is fixed by its
|
||||
FIRST seen chunk; dedupe by ``document.id`` — the phase-119 name-hit
|
||||
bonus applied, D2, LOCKED A3), skipping every document whose id is in
|
||||
*excluded_ids* and admitting at most *cap* documents. There is NO
|
||||
cosine floor: the related tier is the ranked remainder (a lexical-
|
||||
only cosine 0.0 hit is included) — its job on the ``done`` frame is
|
||||
@@ -741,18 +921,21 @@ def select_related(
|
||||
tier's document ids (LOCKED A3: a contiguous top-N, no floor) —
|
||||
"excluding the suggested" is exactly "rank 6+".
|
||||
|
||||
*bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the
|
||||
owner-tunable starting point); ``0`` reproduces the pre-phase walk
|
||||
byte-identically (the kill switch, LOCKED A3).
|
||||
|
||||
The returned rows carry the full document content, byte-identical
|
||||
(the tier is metadata for the ``done`` frame and the durable
|
||||
record; the prompt and ``read`` contract are untouched).
|
||||
"""
|
||||
if bonus is None:
|
||||
bonus = get_settings().name_hit_bonus
|
||||
out: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
for doc, _eff, _cos, _idx in _selection_order(chunks, bonus):
|
||||
if len(out) >= cap:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen or doc.id in excluded_ids:
|
||||
if doc.id in excluded_ids:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
out.append(doc)
|
||||
return out
|
||||
|
||||
Reference in New Issue
Block a user