phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
+75
-25
@@ -55,12 +55,18 @@ point — and the LLM extends its context by ``read``-ing what it needs
|
||||
(the capped ``read`` tool is the ONLY full-text path). Both tiers are
|
||||
computed once per turn, for BOTH branches: ``suggested_docs`` (no
|
||||
floor, A3) and ``related_docs`` (rank 6+, the done frame's row).
|
||||
``done.sources`` is the citation surface — suggested + agent-read,
|
||||
deduped (A4: a grounded turn always shows chips); ``query_log.sources``
|
||||
and the per-turn log line keep recording the full retrieval
|
||||
(suggested + related + read, LOCKED A3). The deflected branch's prompt
|
||||
is byte-identical (weak-hit titles only — A8 untouched); its TurnPlan
|
||||
still carries both tiers for the durable record.
|
||||
Phase 119 (LOCKED A1, owner decision 2026-09-16 — the phase-118 A4
|
||||
union RETIRED): ``done.sources`` is the citation surface and carries
|
||||
the AGENT-READ documents only (``holder.read_docs``, deduped by
|
||||
``(source, path)``, read order) — a zero-read grounded turn (the
|
||||
summary-only fast path) chips nothing, an accepted, owner-directed
|
||||
consequence (the answer prose names the doc it used per the phase-119
|
||||
cite discipline, A5); ``query_log.sources`` and the per-turn log line
|
||||
keep recording the full retrieval (suggested + related + read, 118-A3
|
||||
stands — the durable record records retrieval, not citations). The
|
||||
deflected branch's prompt is byte-identical (weak-hit titles only —
|
||||
A8 untouched); its TurnPlan still carries both tiers for the durable
|
||||
record.
|
||||
|
||||
KB overview (phase 31): the lite-generated outline of the knowledge
|
||||
base (single ``kb_overview`` row) is read per turn (one indexed PK
|
||||
@@ -198,6 +204,7 @@ from app.rag.agent import (
|
||||
AgentHolder,
|
||||
MalformedReplyError, # phase 71: the recovery policy's terminal signal
|
||||
run_agent,
|
||||
suggested_folder_lines, # phase 119 (D3): the suggested-folder context lines
|
||||
)
|
||||
from app.rag.llm import (
|
||||
EmbeddingError,
|
||||
@@ -282,8 +289,11 @@ class TurnPlan:
|
||||
deflected: bool
|
||||
system_prompt: str
|
||||
# The summary-seeded suggestion tier (phase 118, A6: top-N distinct
|
||||
# documents, NO floor, A3 — the HIGH prompt carries their SUMMARIES;
|
||||
# the done frame's citation surface + run_agent's seed_docs, A4).
|
||||
# documents, NO floor, A3 — the HIGH prompt carries their SUMMARIES
|
||||
# and they seed run_agent). Phase 119 (LOCKED A1): NOT the done
|
||||
# frame's citation surface anymore — done.sources is the
|
||||
# agent-read docs only (the phase-118 A4 union retired); this tier
|
||||
# still rides the durable record (query_log + the log line).
|
||||
suggested_docs: list[Document]
|
||||
# Rank 6+ after the suggested set (phase 118: the next ranked docs
|
||||
# that are not already suggested, at most related_max_docs — the
|
||||
@@ -306,6 +316,7 @@ def plan_turn(
|
||||
settings: Settings,
|
||||
notes: Sequence[str] | None = None,
|
||||
kb_overview: str | None = None,
|
||||
folder_lines: Sequence[str] = (),
|
||||
) -> TurnPlan:
|
||||
"""Apply the honesty gate (A8, revised) and assemble prompt + context.
|
||||
|
||||
@@ -336,6 +347,17 @@ def plan_turn(
|
||||
prompts are byte-identical to the pre-phase text and ``kb_chars``
|
||||
is 0.
|
||||
|
||||
*folder_lines* (phase 119, D3, LOCKED A4): the suggested-folder
|
||||
context lines, computed by the chat endpoint (which holds the DB
|
||||
session) via ``suggested_folder_lines`` over the SAME deterministic
|
||||
suggested walk this function performs internally. The HIGH branch
|
||||
passes them to ``build_high_prompt`` (they ride after
|
||||
``SUGGEST_INTRO``, before the first ``<document>`` block); the LOW
|
||||
branch IGNORES them — the deflected prompt stays byte-identical.
|
||||
Empty *folder_lines* ⇒ the HIGH prompt is byte-identical to the
|
||||
phase-118 shape. The :class:`TurnPlan`, the durable record, and the
|
||||
per-turn log line are untouched.
|
||||
|
||||
``summary_hits`` (phase 30; phase 118 redefinition) counts the hit
|
||||
chunks with ``is_summary`` whose parent document is in the
|
||||
SUGGESTED set — both the HIGH and the LOW branch record it.
|
||||
@@ -380,7 +402,9 @@ def plan_turn(
|
||||
best_cosine,
|
||||
fts_hits,
|
||||
False,
|
||||
build_high_prompt(suggested, notes=steering, kb_overview=kb_text),
|
||||
build_high_prompt(
|
||||
suggested, notes=steering, kb_overview=kb_text, folder_lines=folder_lines
|
||||
),
|
||||
suggested,
|
||||
related_docs,
|
||||
[],
|
||||
@@ -581,8 +605,26 @@ async def chat(
|
||||
kb_overview = load_kb_overview(step_db)
|
||||
with SessionLocal() as step_db:
|
||||
chunks = retrieve(step_db, request.message, question_vec)
|
||||
# Phase 119 (D3, LOCKED A4): the suggested-folder
|
||||
# context — the SAME deterministic suggested walk
|
||||
# plan_turn performs internally (one extra cheap
|
||||
# walk, no shared state); each suggested doc's
|
||||
# parent-folder contents pre-seed the HIGH prompt
|
||||
# so the model reads the right file in round 1
|
||||
# instead of walking ls drill-downs. The LOW branch
|
||||
# ignores them (the deflected prompt stays
|
||||
# byte-identical); empty ⇒ the phase-118 prompt is
|
||||
# byte-identical too.
|
||||
folder_lines = suggested_folder_lines(
|
||||
step_db,
|
||||
select_suggested(chunks, n=settings.suggested_docs),
|
||||
)
|
||||
plan = plan_turn(
|
||||
chunks, settings, notes=steering_notes, kb_overview=kb_overview
|
||||
chunks,
|
||||
settings,
|
||||
notes=steering_notes,
|
||||
kb_overview=kb_overview,
|
||||
folder_lines=folder_lines,
|
||||
)
|
||||
except Exception: # noqa: BLE001 — DB failure mid-turn
|
||||
logger.exception(
|
||||
@@ -852,19 +894,25 @@ async def chat(
|
||||
# "start here" tier, no floor, A3) and the related
|
||||
# docs (``plan.related_docs``, rank 6+ after the
|
||||
# suggested set). The DURABLE record keeps the full
|
||||
# retrieval (LOCKED A3: query_log records retrieval,
|
||||
# retrieval (118-A3: query_log records retrieval,
|
||||
# not citations — even on deflected turns, where the
|
||||
# weak hits are suggested). Phase 112/118:
|
||||
# done.sources is the CITATION surface — it carries the
|
||||
# suggested docs + the agent-read docs (deduped,
|
||||
# LOCKED A4) on grounded turns and [] on deflected
|
||||
# ones (a deflected answer cites nothing; the weak
|
||||
# hits stay in the durable record).
|
||||
# weak hits are suggested). Phase 112/119:
|
||||
# done.sources is the CITATION surface — READ DOCS
|
||||
# ONLY (phase 119, LOCKED A1, owner decision
|
||||
# 2026-09-16; the phase-118 A4 suggested+read union
|
||||
# is RETIRED): on grounded turns it carries
|
||||
# ``holder.read_docs`` deduped by (source, path),
|
||||
# read order — a zero-read grounded turn (the
|
||||
# summary-only fast path) chips nothing, an accepted
|
||||
# consequence (the answer prose names the doc per the
|
||||
# phase-119 cite discipline, A5) — and [] on
|
||||
# deflected ones (a deflected answer cites nothing;
|
||||
# the weak hits stay in the durable record).
|
||||
# A cancelled turn (the generator closed by the
|
||||
# consumer) never reaches this step — no query_log row.
|
||||
cited_docs: list[Document] = []
|
||||
cited_seen: set[tuple[str, str]] = set()
|
||||
for doc in [*plan.suggested_docs, *holder.read_docs]:
|
||||
for doc in holder.read_docs:
|
||||
key = (doc.source, doc.path)
|
||||
if key not in cited_seen:
|
||||
cited_seen.add(key)
|
||||
@@ -925,13 +973,15 @@ async def chat(
|
||||
# citations). The retrieval stays durably recorded above
|
||||
# (query_log.sources + the log line — observability
|
||||
# unchanged).
|
||||
# Phase 118 (A3/A4): done.related carries the related
|
||||
# tier — the ranked documents beyond the suggested set
|
||||
# (rank 6+ after the contiguous top-N suggestion, capped
|
||||
# by related_max_docs in the tiering), deduped against
|
||||
# the cited list, the same (source, path) pattern as
|
||||
# cited_docs: an agent-read related doc is a citation,
|
||||
# never a "nearby doc". The UI renders it as the
|
||||
# Phase 118 (A3) / phase 119 (A1): done.related carries
|
||||
# the related tier — the ranked documents beyond the
|
||||
# suggested set (rank 6+ after the contiguous top-N
|
||||
# suggestion, capped by related_max_docs in the tiering),
|
||||
# deduped against the cited list, the same (source,
|
||||
# path) pattern as cited_docs — which is the READ DOCS
|
||||
# only now: a read related doc is a chip, never a
|
||||
# "nearby doc" (unchanged intent, keyed on read docs
|
||||
# since phase 119). The UI renders the row as the
|
||||
# de-emphasized related-docs row, never a citation chip;
|
||||
# old clients ignore the field.
|
||||
cited_refs: list[SourceRef] = []
|
||||
|
||||
Reference in New Issue
Block a user