phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
@@ -633,6 +633,13 @@ class _FakeSession:
|
||||
def get(self, _model: Any, _pk: Any) -> Any:
|
||||
return None
|
||||
|
||||
def execute(self, *args: Any, **kwargs: Any) -> list[Any]:
|
||||
# Phase 119 (D3): the endpoint's suggested-folder lines run the
|
||||
# ls catalog fetchers on this session — the fake catalog is
|
||||
# empty (header-only lines), keeping the prompt builds
|
||||
# deterministic here.
|
||||
return []
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _admin_signed_in(client: TestClient) -> None:
|
||||
@@ -736,17 +743,19 @@ def test_shape_4_meta_question_deflected_frame_has_no_chips_or_row(
|
||||
assert "108_history_wire_check/00_phase.md" in row.sources
|
||||
|
||||
|
||||
def test_done_frame_carries_suggested_refs_strong_plus_weak(
|
||||
def test_done_frame_zero_read_grounded_turn_carries_no_chips(
|
||||
client: TestClient,
|
||||
chip_env: tuple[_FakeSession, _CannedLLM],
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Shape 2 on the wire under phase 118 (LOCKED A4): the citation
|
||||
surface is the suggested tier + the agent's reads (deduped) —
|
||||
with two retrieved docs and no read, BOTH docs are ``sources``
|
||||
refs (no floor — A3); nothing reaches rank 6+, so ``related`` is
|
||||
empty; the tiers stay disjoint (the done frame's dedupe). The
|
||||
durable record keeps the FULL retrieval (LOCKED A3)."""
|
||||
"""Shape 2 on the wire under phase 119 (LOCKED A1 — the phase-118
|
||||
A4 union retired): the citation surface is the agent's READ DOCS
|
||||
only — with two retrieved docs and no read, ``sources`` is empty
|
||||
(the zero-read grounded turn chips nothing — an accepted,
|
||||
owner-directed consequence); the never-read suggested docs appear
|
||||
NOWHERE in the frame (the explicit contrast against the retired
|
||||
A4 union); nothing reaches rank 6+, so ``related`` is empty. The
|
||||
durable record keeps the FULL retrieval (118-A3 stands)."""
|
||||
session, _llm = chip_env
|
||||
validate = _doc("brain-of-reese", ".agents/validate.sh", "validate.sh",
|
||||
"VALIDATE_SH_CONTENT")
|
||||
@@ -766,14 +775,11 @@ def test_done_frame_carries_suggested_refs_strong_plus_weak(
|
||||
frames = _ask(client, "How does the phase gate decide to validate?")
|
||||
done = frames[-1]
|
||||
assert done["deflected"] is False
|
||||
assert [(s["source"], s["path"]) for s in done["sources"]] == [
|
||||
("brain-of-reese", ".agents/validate.sh"),
|
||||
("ServMon", "README.md"), # A4: suggested + read — both suggested (A3)
|
||||
]
|
||||
# A1: read docs only — the agent read nothing ⇒ no chips, and the
|
||||
# never-read suggested docs appear NOWHERE in the frame (the retired
|
||||
# A4 union would have carried both).
|
||||
assert done["sources"] == []
|
||||
assert done["related"] == [] # nothing reaches rank 6+ for 2 docs
|
||||
cited = {(s["source"], s["path"]) for s in done["sources"]}
|
||||
related = {(s["source"], s["path"]) for s in done["related"]}
|
||||
assert cited.isdisjoint(related)
|
||||
|
||||
(row,) = session.added
|
||||
assert isinstance(row, QueryLog)
|
||||
@@ -788,14 +794,15 @@ def test_agent_read_related_doc_is_cited_not_related(
|
||||
chip_env: tuple[_FakeSession, _CannedLLM],
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The agent-read exemption (LOCKED A4, phase-118 tiering): a
|
||||
rank-6+ doc — the related tier ("nearby docs") — that the agent
|
||||
``read`` via the tool is cited by definition: the model read it, so
|
||||
it was used. It joins ``sources`` (after the suggested docs — it
|
||||
was not suggested, so the read appends it last, deduped) and is
|
||||
EXCLUDED from ``related`` (a used doc must never read as "nearby");
|
||||
the other rank-6+ doc stays in the tier. The read content reached
|
||||
the model (the tool result in the follow-up request)."""
|
||||
"""The agent-read exemption (phase-119 A1 × phase-118 tiering):
|
||||
``sources`` is the READ DOCS only — a rank-6+ doc (the related
|
||||
tier, "nearby docs") that the agent ``read`` via the tool is the
|
||||
frame's ONLY chip: the model read it, so it was used. It is EXCLUDED
|
||||
from ``related`` (a used doc must never read as "nearby" — unchanged
|
||||
intent, the dedupe now keyed on read docs); the other rank-6+ doc
|
||||
stays in the tier; the never-read suggested docs appear NOWHERE in
|
||||
the frame (contrast against the retired A4 union). The read content
|
||||
reached the model (the tool result in the follow-up request)."""
|
||||
session, _default_llm = chip_env
|
||||
strong = _doc("docs", "strong.md", "Strong", "STRONG_DOC_CONTENT")
|
||||
fillers = [
|
||||
@@ -838,10 +845,10 @@ def test_agent_read_related_doc_is_cited_not_related(
|
||||
assert "WEAK_B_READ_BY_AGENT" in tool_msgs[0]["content"]
|
||||
|
||||
sources = [(s["source"], s["path"]) for s in done["sources"]]
|
||||
# A4: suggested (5) + the read doc (last — it was not suggested).
|
||||
assert sources[-1] == ("docs", "weak-b.md") # read ⇒ cited, last
|
||||
assert len(sources) == 6
|
||||
assert ("docs", "weak-c.md") not in sources # never suggested, never read
|
||||
# A1: read docs only — the read related doc is the ONLY chip (the
|
||||
# never-read suggested docs are nowhere in the frame).
|
||||
assert sources == [("docs", "weak-b.md")] # read ⇒ chip
|
||||
assert ("docs", "weak-c.md") not in sources # never read
|
||||
related = [(s["source"], s["path"]) for s in done["related"]]
|
||||
assert related == [("docs", "weak-c.md")] # the read doc is not "nearby"
|
||||
assert set(sources).isdisjoint(set(related))
|
||||
|
||||
Reference in New Issue
Block a user