phase: 119_name_signal_read_chips
Build and Push Containers / build-and-push-app (push) Successful in 2m1s
Build and Push Containers / build-and-push-db (push) Successful in 18s

All verification complete. Final report:

**Phase 119 final verification pass — all criteria verified, one stale pin fixed.**
- Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry.
- Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged.
- New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2.
- Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed).
- Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met).
- Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
2026-09-16 15:50:48 -04:00
parent 795fb56425
commit a5b63f83ad
89 changed files with 4377 additions and 655 deletions
+21 -3
View File
@@ -5,7 +5,9 @@ uses (cosine top-N + FTS top-N, RRF-fused — plus the phase-106 recency
boost, which ``retrieve()`` applies after the fusion), and prints the
top-5 documents with their cosine / fts / fused scores (labelled
``effective`` when the recency boost is on — the post-boost score)
and the document's creation date, plus the honesty-gate verdict:
and the document's creation date, the seeded suggestion tier
(``suggested:`` — the ``select_suggested`` walk, the phase-119 name-hit
bonus included), plus the honesty-gate verdict:
uv run python -m scripts.eval_retrieval "How did I install gitlab?"
uv run python -m scripts.eval_retrieval --from-file questions.txt
@@ -86,7 +88,7 @@ def main(argv: list[str] | None = None) -> int:
from app.config import get_settings
from app.db import SessionLocal, db_available
from app.rag.llm import LLMClient
from app.rag.retriever import RetrievedChunk, retrieve
from app.rag.retriever import RetrievedChunk, retrieve, select_suggested
settings = get_settings()
if not db_available():
@@ -101,7 +103,8 @@ def main(argv: list[str] | None = None) -> int:
f"vector_candidates={settings.hybrid_vector_candidates} "
f"lexical_candidates={settings.hybrid_lexical_candidates} rrf_k={settings.rrf_k} "
f"recency_boost={settings.recency_boost} "
f"recency_half_life_days={settings.recency_half_life_days}"
f"recency_half_life_days={settings.recency_half_life_days} "
f"name_hit_bonus={settings.name_hit_bonus}"
)
with SessionLocal() as db:
for question, vec in zip(questions, vectors, strict=True):
@@ -134,6 +137,21 @@ def main(argv: list[str] | None = None) -> int:
f"{score_label}={c.score:.5f} created={c.document.created_at:%Y-%m-%d} "
f"({c.document.title})"
)
# Phase 119, D2: the seeded suggestion tier — the
# ``select_suggested`` walk over the same list (the name-hit
# bonus included, default settings) — the tuning tool must
# report the tier the prompt actually seeds.
suggested = select_suggested(chunks)
print(
" suggested: "
+ (
" ".join(
f"{i}. {d.source}/{d.path}"
for i, d in enumerate(suggested, start=1)
)
or "(none)"
)
)
return 0