phase: 119_name_signal_read_chips
Build and Push Containers / build-and-push-app (push) Successful in 2m1s
Build and Push Containers / build-and-push-db (push) Successful in 18s

All verification complete. Final report:

**Phase 119 final verification pass — all criteria verified, one stale pin fixed.**
- Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry.
- Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged.
- New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2.
- Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed).
- Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met).
- Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
2026-09-16 15:50:48 -04:00
parent 795fb56425
commit a5b63f83ad
89 changed files with 4377 additions and 655 deletions
+19 -18
View File
@@ -540,12 +540,14 @@ def test_old_correct_beats_new_similar(
page: Page, app_url: str, mock_llm: int, db_ready: None, dates_tree: Path
) -> None:
"""``How did I configure the backup retention policy?`` → grounded,
and the FIRST cited source is the OLDER correct doc (2020) — the
NEWER similar one (now, "under review") is cited second. Phase 118
(A4): the chip row IS the suggested tier (top-5, NO floor) — this
four-doc KB chips ALL four docs in rank order — and the related row
is absent (no rank-6+ doc). The real hybrid retriever + the DEFAULT
recency boost (0.0007 / 365 d) over the mock's token-overlap
and the FIRST logged source is the OLDER correct doc (2020) — the
NEWER similar one (now, "under review") is second. Phase 119
(LOCKED A1): the zero-read turn chips NOTHING (the chip row is the
READ docs only — the retired phase-118 A4 suggested-chip row is
gone), so the ordering assertion rides the durable record (LOCKED
A3 — all four docs, suggested rank order, untouched); the related
row is absent (no rank-6+ doc). The real hybrid retriever + the
DEFAULT recency boost (0.0007 / 365 d) over the mock's token-overlap
embeddings (the module docstring records the measured fused scores:
0.032523 vs 0.031498 — margin ≈ 0.001025 WITH the full zero-age
boost on the newer doc)."""
@@ -563,21 +565,20 @@ def test_old_correct_beats_new_similar(
# top_score 0.6222 ≥ the e2e threshold 0.30).
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# Phase 118 (A4): the chip row IS the suggested tier — top-5, NO
# floor, and this KB has exactly four docs, so ALL four are
# suggested (chipped) in fused rank order: the OLDER correct doc
# first, the NEWER similar one (the boost's intended beneficiary)
# second, then the two unrelated docs (no floor filters them — the
# LLM decides what the summaries earn).
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(4, timeout=30_000)
assert chips.nth(0).inner_text() == f"{dates_tree.name}/{RETENTION_MD}"
assert chips.nth(1).inner_text() == f"{dates_tree.name}/{DRAFT_MD}"
assert chips.nth(2).inner_text() == f"{dates_tree.name}/{FORWARD_MD}"
assert chips.nth(3).inner_text() == f"{dates_tree.name}/{OLDDOC_MD}"
# Phase 119 (LOCKED A1): the turn read nothing, so ZERO citation
# chips — the four suggested docs (all of them, top-5 NO floor on a
# four-doc KB: the OLDER correct doc first, the NEWER similar one
# second, then the two unrelated docs) seeded the prompt but never
# chip (the retired phase-118 A4 union is gone). Their rank order
# is pinned by the durable record below (LOCKED A3, untouched).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# No rank-6+ doc exists in this four-doc KB → the related row is
# absent (the de-emphasized row renders only when it has entries).
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
# The button label is the settle sync: the client re-labels Send on
# the done frame, and the server writes the query_log row just
# before yielding it.
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# Durable record: one row, grounded, the FULL retrieval (suggested
# tier + related + read, deduped — here: all four docs) in rank