phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
+21
-17
@@ -97,18 +97,12 @@ def test_on_topic_question_streams_grounded_answer(
|
||||
expect(bubble.first).to_contain_text(QUESTION, timeout=30_000)
|
||||
expect(bubble.first).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||||
|
||||
# Grounded: a kubernetes.md source chip renders under the bubble
|
||||
# (top-N docs can add more chips; the question's doc must be among them).
|
||||
# Phase 26: the chip opens the document in the SAME-PAGE modal — no new
|
||||
# tab; the encoded href stays as the no-JS / context-menu escape hatch
|
||||
# (phase 13's back=/ lets the viewer's back button return to chat).
|
||||
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
|
||||
expect(chip).to_have_count(1)
|
||||
expect(chip.first).to_contain_text("kubernetes.md")
|
||||
expect(chip.first).to_have_attribute(
|
||||
"href", "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
|
||||
)
|
||||
expect(chip.first).not_to_have_attribute("target") # phase 26: modal, not a new tab
|
||||
# Phase 119 (LOCKED A1): a zero-read grounded turn chips NOTHING —
|
||||
# the suggested kubernetes.md is seed context, not a citation chip
|
||||
# (the retired phase-118 A4 suggested+read union is gone; the
|
||||
# pre-phase-26 chip/contract pins retired with it). The grounding is
|
||||
# pinned by the query_log row below (LOCKED A3, untouched).
|
||||
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
|
||||
|
||||
# Button recovers: enabled + "Send" (never stale).
|
||||
expect(page.locator("#send-btn")).to_be_enabled()
|
||||
@@ -121,9 +115,16 @@ def test_chat_logs_query(page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
login(page, app_url, next="/")
|
||||
page.fill("#message-input", QUESTION)
|
||||
page.click("#send-btn")
|
||||
expect(
|
||||
page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
|
||||
).to_have_count(1, timeout=30_000)
|
||||
# The turn settles with the mock answer (phase 119 A1: no chip to
|
||||
# wait on — a zero-read turn chips nothing; the retired phase-118
|
||||
# A4 union is gone). The button recovery is the settle sync: the
|
||||
# client re-enables Send on the done frame, and the server writes
|
||||
# the query_log row just before yielding it.
|
||||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
||||
"Deterministic mock answer for E2E", timeout=30_000
|
||||
)
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
# App still healthy after the turn.
|
||||
r = httpx.get(f"{app_url}/api/health", timeout=5)
|
||||
@@ -180,5 +181,8 @@ def test_sse_stream_shape(app_url: str, mock_llm: int, db_ready: None) -> None:
|
||||
assert frames[-1]["type"] == "done" # done is the final event
|
||||
assert done[0]["deflected"] is False
|
||||
assert done[0]["suggestions"] == []
|
||||
assert done[0]["sources"], "done must carry the cited sources"
|
||||
assert any(s["path"] == "homelab/kubernetes.md" for s in done[0]["sources"])
|
||||
# Phase 119 (LOCKED A1): done.sources = the READ docs only — this
|
||||
# plain turn read nothing, so the frame carries an EMPTY sources
|
||||
# list (the retired phase-118 A4 suggested+read union is gone; the
|
||||
# suggested doc's durable record lives in query_log, not the frame).
|
||||
assert done[0]["sources"] == [], done[0]["sources"]
|
||||
|
||||
Reference in New Issue
Block a user