phase: 119_name_signal_read_chips
Build and Push Containers / build-and-push-app (push) Successful in 2m1s
Build and Push Containers / build-and-push-db (push) Successful in 18s

All verification complete. Final report:

**Phase 119 final verification pass — all criteria verified, one stale pin fixed.**
- Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry.
- Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged.
- New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2.
- Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed).
- Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met).
- Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
2026-09-16 15:50:48 -04:00
parent 795fb56425
commit a5b63f83ad
89 changed files with 4377 additions and 655 deletions
+44 -36
View File
@@ -59,19 +59,21 @@ TODO item — one Playwright file per story, A16):
The inverse of the retired phase-24 pin: NO document's tail
sentinel is in the echoed context — the full content of no
suggested doc ever reached the model (case b). The grounded turn
cites the 5 suggested docs (case e's chip surface, no read yet) and
cites NOTHING (phase 119, LOCKED A1 — a zero-read turn chips
nothing; the retired phase-118 A4 suggested+read union is gone) and
renders the de-emphasized related row (rank 6+); the durable
record carries suggested + related.
record carries suggested + related (118-A3 untouched).
2. ``test_read_suggested_doc_adds_full_text_and_cites`` — cases (c) +
(e): the scripted flow ``read``s the rank-1 suggested doc — the
read SUCCEEDS (phase 118: the seeds are summaries, not full text —
the retired top-2 seed-read refusal is gone) and the mock's
verbatim echo of the read result lands the tail sentinel in the
answer (case c: the full text now arrives through the ``read``
tool, not the seed); the citation chips = suggested + read,
deduped — the read doc is among the suggested, so the chip row is
exactly the 5 suggested docs, and the related row renders rank 6+
(case e); the durable record carries suggested + related + read.
tool, not the seed); the citation chips = the READ docs only
(phase 119, LOCKED A1 — exactly the one read doc; none of the
four other suggested docs chips), and the related row renders
rank 6+ (case e); the durable record carries suggested + related +
read (118-A3 untouched).
"""
from __future__ import annotations
@@ -503,14 +505,16 @@ def _last_query_log() -> QueryLog:
return rows[0]
def _assert_suggested_chips_and_related_row(page: Page, suggested_paths: list[str]) -> None:
"""The citation surface of a grounded turn (LOCKED A4): the chip row
is the suggested set (+ agent reads, deduped — asserted per test)
and the de-emphasized ``related-docs`` row carries the rank-6+
remainder (phase-113 UI, never a citation chip)."""
def _assert_read_chips_and_related_row(page: Page, read_paths: list[str]) -> None:
"""The citation surface of a grounded turn (phase 119, LOCKED A1 —
the retired phase-118 A4 union is gone): the chip row is the
AGENT-READ docs only (deduped, read order — a zero-read turn chips
NOTHING: the suggested docs are seed context, not citations), and
the de-emphasized ``related-docs`` row carries the rank-6+ remainder
(phase-113 UI, untouched — never a citation chip)."""
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(len(suggested_paths))
for path in suggested_paths:
expect(chips).to_have_count(len(read_paths))
for path in read_paths:
expect(
chips.filter(has_text=path), message=f"chip for {path}"
).to_have_count(1)
@@ -553,11 +557,12 @@ def test_summaries_seed_the_prompt_not_the_full_text(
assert _tool_frames(frames) == [], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The grounded turn cites the 5 suggested docs (LOCKED A4 — no read
# yet: the chip row IS the suggested set, in rank order).
assert [
(s["source"], s["path"]) for s in done["sources"]
] == [(SOURCE, p) for p in SUGGESTED_TAIL], done["sources"]
# Phase 119 (LOCKED A1): the grounded done frame cites the READ docs
# only — this turn read NOTHING (the summary-only fast path), so the
# citation surface is empty; the suggested set is seed context, not
# citations (the retired phase-118 A4 union is gone). The durable
# record below still carries suggested + related (118-A3 untouched).
assert done["sources"] == [], done["sources"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
@@ -583,12 +588,13 @@ def test_summaries_seed_the_prompt_not_the_full_text(
expect(bubble).not_to_contain_text(sentinel)
expect(bubble).not_to_contain_text(TRUNCATION_MARKER)
# Case (e)'s chip surface (no read yet): chips = the 5 suggested
# docs; the related row renders rank 6+ (the de-emphasized row).
_assert_suggested_chips_and_related_row(page, SUGGESTED_TAIL)
# Case (e)'s chip surface (phase 119 A1 — no read yet): ZERO chips;
# the related row renders rank 6+ (the de-emphasized row, untouched).
_assert_read_chips_and_related_row(page, [])
# Durable record: grounded; suggested + related (LOCKED A3 — the log
# records retrieval, not citations).
# records retrieval, not citations; phase 119 A1 retires the chip
# union only, not the record).
row = _last_query_log()
assert row.question == TAIL_QUESTION
assert row.deflected is False
@@ -609,10 +615,10 @@ def test_read_suggested_doc_adds_full_text_and_cites(
retired top-2 seed-read refusal is gone) and the mock's verbatim
echo of the read result lands the document's tail sentinel in the
answer (case c: the full text reached the model through the
``read`` tool, not the seed). The citation chips = suggested +
read, deduped — the read doc is among the suggested, so the chip
row is exactly the 5 suggested docs; the related row renders rank
6+ (case e)."""
``read`` tool, not the seed). The citation chips = the READ docs
only (phase 119, LOCKED A1 — exactly the one read doc; none of the
other four suggested docs chips; the retired phase-118 A4 union is
gone); the related row renders rank 6+ (case e, untouched)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
@@ -628,13 +634,15 @@ def test_read_suggested_doc_adds_full_text_and_cites(
], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# LOCKED A4: done.sources = suggested + read, deduped — the read doc
# is already among the suggested, so the citation surface is
# exactly the 5 suggested docs (in rank order).
# Phase 119 (LOCKED A1): done.sources = the READ docs only —
# exactly the one scripted read; the other four suggested docs
# never chip (the retired phase-118 A4 suggested+read union is
# gone).
assert [
(s["source"], s["path"]) for s in done["sources"]
] == [(SOURCE, p) for p in SUGGESTED_READ], done["sources"]
# done.related = the rank-6+ remainder (deduped against the cited).
] == [(SOURCE, READ_TARGET)], done["sources"]
# done.related = the rank-6+ remainder (deduped against the cited —
# the read doc is rank 1, not in the remainder).
assert [
(s["source"], s["path"]) for s in done["related"]
] == [(SOURCE, p) for p in RELATED_PATHS], done["related"]
@@ -658,13 +666,13 @@ def test_read_suggested_doc_adds_full_text_and_cites(
if path != READ_TARGET:
expect(bubble).not_to_contain_text(sentinel)
# Case (e): the UI chip row = suggested + read (deduped — the read
# doc is among the suggested, so chips = the 5 suggested docs), and
# the related row renders rank 6+.
_assert_suggested_chips_and_related_row(page, SUGGESTED_READ)
# Case (e): the UI chip row = the READ docs only (phase 119 A1 —
# exactly the one read doc), and the related row renders rank 6+.
_assert_read_chips_and_related_row(page, [READ_TARGET])
# Durable record: grounded; suggested + related + read (deduped,
# LOCKED A3).
# LOCKED A3 — phase 119 A1 retires the chip union only, not the
# record).
row = _last_query_log()
assert row.question == READ_QUESTION
assert row.deflected is False