phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
@@ -65,9 +65,11 @@ Test → story mapping (Playwright Mapping Rule):
|
||||
``I read <sp1> and <sp2>.`` line; the round cap (default 10) bounds
|
||||
the turn, no budget refusal anywhere.
|
||||
2. ``test_done_sources_include_reads`` — the source chips under the
|
||||
answer list the retrieval doc PLUS both read documents, deduped
|
||||
(the phase-37 ``done.sources`` extension contract, now with 2
|
||||
reads); the same combined list lands in ``query_log.sources``.
|
||||
answer list the BOTH read documents, deduped (phase 119, LOCKED A1:
|
||||
chips cite read docs only — the retrieval doc was never read, so it
|
||||
never chips; the retired phase-118 A4 suggested+read union is
|
||||
gone); the retrieval doc PLUS both reads still land in
|
||||
``query_log.sources`` (LOCKED A3, untouched).
|
||||
3. ``test_relist_allowed`` — the listing tool ran (its line rendered)
|
||||
and no pre-phase-45 budget refusal ("… budget left") appears
|
||||
anywhere in the message bubble or tool lines: the old
|
||||
@@ -195,14 +197,22 @@ BUDGET_REFUSAL_FRAGMENTS = (
|
||||
"budget left",
|
||||
)
|
||||
|
||||
# The combined source list the app reports (app/api/chat.py): retrieval
|
||||
# docs first, then the agent's read docs, deduped by (source, path).
|
||||
# The combined source list the DURABLE record reports
|
||||
# (app/api/chat.py, LOCKED A3 — suggested + related + read, deduped):
|
||||
# retrieval doc first, then the agent's read docs, deduped by
|
||||
# (source, path). The citation surface (done.sources, the chips) is
|
||||
# the READ set only (phase 119, LOCKED A1 — the retired phase-118 A4
|
||||
# suggested+read union is gone): the retrieval doc was never read.
|
||||
EXPECTED_SOURCES = [
|
||||
(SEED_SOURCE, SEED_PATH),
|
||||
(READ1_SOURCE, READ1_PATH),
|
||||
(READ2_SOURCE, READ2_PATH),
|
||||
]
|
||||
EXPECTED_SOURCES_LINE = ", ".join(f"{s}/{p}" for s, p in EXPECTED_SOURCES)
|
||||
READ_ONLY_SOURCES = [
|
||||
(READ1_SOURCE, READ1_PATH),
|
||||
(READ2_SOURCE, READ2_PATH),
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -508,7 +518,7 @@ def test_multi_read_turn(
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 2. done.sources / source chips: retrieval doc + BOTH reads, deduped
|
||||
# 2. done.sources / source chips: BOTH reads, deduped (phase 119 A1)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -523,23 +533,28 @@ def test_done_sources_include_reads(
|
||||
_submit(page, MULTI_QUESTION)
|
||||
_wait_settled(page)
|
||||
|
||||
# Wire level: done.sources is the retrieval doc FIRST, then both
|
||||
# read documents — deduped (the retrieval doc was never read, the
|
||||
# reads are each read once; nothing appears twice).
|
||||
# Wire level: done.sources is the READ documents only, in read
|
||||
# order — deduped (the reads are each read once; nothing appears
|
||||
# twice). Phase 119, LOCKED A1: the retrieval doc was never read,
|
||||
# so it is NOT in the citation surface (the retired phase-118 A4
|
||||
# suggested+read union is gone); it still lands in the durable
|
||||
# record (LOCKED A3, pinned in test 1).
|
||||
frames = _frames(page)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert [(s["source"], s["path"]) for s in done["sources"]] == EXPECTED_SOURCES
|
||||
assert [
|
||||
(s["source"], s["path"]) for s in done["sources"]
|
||||
] == READ_ONLY_SOURCES, done["sources"]
|
||||
pairs = [(s["source"], s["path"]) for s in done["sources"]]
|
||||
assert len(pairs) == len(set(pairs)), "done.sources must be deduped"
|
||||
|
||||
# UI: exactly three source chips under the answer, in the same
|
||||
# order, each a viewer link — no duplicated chip.
|
||||
# UI: exactly two source chips under the answer (the read docs,
|
||||
# read order), each a viewer link — no duplicated chip, and no
|
||||
# chip for the never-read retrieval doc (phase 119 A1).
|
||||
chips = page.locator(".msg.brain .source-chip")
|
||||
expect(chips).to_have_count(3)
|
||||
expect(chips.nth(0)).to_contain_text(SEED_SP)
|
||||
expect(chips.nth(1)).to_contain_text(READ1_SP)
|
||||
expect(chips.nth(2)).to_contain_text(READ2_SP)
|
||||
for i, (source, path) in enumerate(EXPECTED_SOURCES):
|
||||
expect(chips).to_have_count(2)
|
||||
expect(chips.nth(0)).to_contain_text(READ1_SP)
|
||||
expect(chips.nth(1)).to_contain_text(READ2_SP)
|
||||
for i, (source, path) in enumerate(READ_ONLY_SOURCES):
|
||||
expect(chips.nth(i)).to_have_attribute(
|
||||
"href", f"/document.html?source={source}&path={path}&back=%2F"
|
||||
)
|
||||
@@ -631,12 +646,13 @@ def test_single_tool_flow_regression(
|
||||
expect(bubble).not_to_contain_text(BOTH_READS_LINE)
|
||||
expect(bubble).not_to_contain_text(READ2_SP)
|
||||
|
||||
# done: non-deflected; sources = retrieval doc + the single read
|
||||
# (READ2 absent — it was never read).
|
||||
# done: non-deflected; sources = the single READ doc only (phase
|
||||
# 119, LOCKED A1 — READ2 absent: never read; the retrieval doc
|
||||
# absent: never read — the retired phase-118 A4 union is gone; the
|
||||
# durable record below keeps retrieval + read, LOCKED A3).
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["deflected"] is False
|
||||
assert [(s["source"], s["path"]) for s in done["sources"]] == [
|
||||
(SEED_SOURCE, SEED_PATH),
|
||||
(READ1_SOURCE, READ1_PATH),
|
||||
]
|
||||
|
||||
|
||||
Reference in New Issue
Block a user