phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
@@ -5,7 +5,9 @@ uses (cosine top-N + FTS top-N, RRF-fused — plus the phase-106 recency
|
||||
boost, which ``retrieve()`` applies after the fusion), and prints the
|
||||
top-5 documents with their cosine / fts / fused scores (labelled
|
||||
``effective`` when the recency boost is on — the post-boost score)
|
||||
and the document's creation date, plus the honesty-gate verdict:
|
||||
and the document's creation date, the seeded suggestion tier
|
||||
(``suggested:`` — the ``select_suggested`` walk, the phase-119 name-hit
|
||||
bonus included), plus the honesty-gate verdict:
|
||||
|
||||
uv run python -m scripts.eval_retrieval "How did I install gitlab?"
|
||||
uv run python -m scripts.eval_retrieval --from-file questions.txt
|
||||
@@ -86,7 +88,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
from app.config import get_settings
|
||||
from app.db import SessionLocal, db_available
|
||||
from app.rag.llm import LLMClient
|
||||
from app.rag.retriever import RetrievedChunk, retrieve
|
||||
from app.rag.retriever import RetrievedChunk, retrieve, select_suggested
|
||||
|
||||
settings = get_settings()
|
||||
if not db_available():
|
||||
@@ -101,7 +103,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
f"vector_candidates={settings.hybrid_vector_candidates} "
|
||||
f"lexical_candidates={settings.hybrid_lexical_candidates} rrf_k={settings.rrf_k} "
|
||||
f"recency_boost={settings.recency_boost} "
|
||||
f"recency_half_life_days={settings.recency_half_life_days}"
|
||||
f"recency_half_life_days={settings.recency_half_life_days} "
|
||||
f"name_hit_bonus={settings.name_hit_bonus}"
|
||||
)
|
||||
with SessionLocal() as db:
|
||||
for question, vec in zip(questions, vectors, strict=True):
|
||||
@@ -134,6 +137,21 @@ def main(argv: list[str] | None = None) -> int:
|
||||
f"{score_label}={c.score:.5f} created={c.document.created_at:%Y-%m-%d} "
|
||||
f"({c.document.title})"
|
||||
)
|
||||
# Phase 119, D2: the seeded suggestion tier — the
|
||||
# ``select_suggested`` walk over the same list (the name-hit
|
||||
# bonus included, default settings) — the tuning tool must
|
||||
# report the tier the prompt actually seeds.
|
||||
suggested = select_suggested(chunks)
|
||||
print(
|
||||
" suggested: "
|
||||
+ (
|
||||
" ".join(
|
||||
f"{i}. {d.source}/{d.path}"
|
||||
for i, d in enumerate(suggested, start=1)
|
||||
)
|
||||
or "(none)"
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user