phase: 119_name_signal_read_chips
All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
@@ -25,15 +25,17 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from sqlalchemy import select, text
|
||||
from sqlalchemy import text
|
||||
from test_chat_api import FakeRagLLM, _stream_chat, _token_vec
|
||||
|
||||
from app.api import chat as chat_api
|
||||
from app.config import get_settings
|
||||
from app.main import app as fastapi_app
|
||||
from app.models import Document, KbOverview
|
||||
from app.rag.agent import suggested_folder_lines
|
||||
from app.rag.importer import import_sources
|
||||
from app.rag.prompts import build_deflect_prompt, build_high_prompt
|
||||
from app.rag.retriever import retrieve, weak_hit_titles
|
||||
from app.rag.retriever import retrieve, select_suggested, weak_hit_titles
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "docs"
|
||||
@@ -90,15 +92,27 @@ def _seed_overview(db) -> None:
|
||||
db.commit()
|
||||
|
||||
|
||||
def _cited_docs(db, frames: list[dict]) -> list[Document]:
|
||||
"""The documents the done event cited, in citation order — the same
|
||||
list ``plan_turn`` passed to the prompt builder."""
|
||||
docs = []
|
||||
for s in frames[-1]["sources"]:
|
||||
doc = db.scalar(select(Document).where(Document.path == s["path"]))
|
||||
assert doc is not None, f"done source {s['path']!r} missing from the KB"
|
||||
docs.append(doc)
|
||||
return docs
|
||||
def _suggested_docs(db) -> list[Document]:
|
||||
"""The documents the grounded turn's HIGH prompt seeded, in the same
|
||||
order ``plan_turn`` walked them — the SAME deterministic suggested
|
||||
walk over the retrieval. Phase 119 (A1) moved the citation surface
|
||||
to the agent's READ docs, so the seeded tier can no longer be read
|
||||
off the done frame (the canned LLM here reads nothing ⇒
|
||||
``done.sources`` is empty) — reconstructing from the retrieval is
|
||||
the faithful source now."""
|
||||
chunks = retrieve(db, QUESTION, _token_vec(QUESTION))
|
||||
return list(select_suggested(chunks, n=get_settings().suggested_docs))
|
||||
|
||||
|
||||
def _suggested_folder_lines(db) -> list[str]:
|
||||
"""The suggested-folder context lines the endpoint computed (phase
|
||||
119, D3, LOCKED A4) — the SAME deterministic suggested walk
|
||||
``plan_turn`` performs internally (one extra walk, no shared
|
||||
state), over the real seeded catalog."""
|
||||
chunks = retrieve(db, QUESTION, _token_vec(QUESTION))
|
||||
return suggested_folder_lines(
|
||||
db, select_suggested(chunks, n=get_settings().suggested_docs)
|
||||
)
|
||||
|
||||
|
||||
def _turn_log_lines(caplog: pytest.LogCaptureFixture) -> list[str]:
|
||||
@@ -124,7 +138,14 @@ def test_no_row_high_prompt_byte_identical_to_pre_phase(
|
||||
assert frames[-1]["deflected"] is False
|
||||
(system, user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
|
||||
assert user["content"] == QUESTION
|
||||
expected = build_high_prompt(_cited_docs(db, frames), notes=[], kb_overview=None)
|
||||
# Phase 119 (D3): the endpoint's suggested-folder lines ride the
|
||||
# HIGH prompt — reconstructed the same deterministic way.
|
||||
expected = build_high_prompt(
|
||||
_suggested_docs(db),
|
||||
notes=[],
|
||||
kb_overview=None,
|
||||
folder_lines=_suggested_folder_lines(db),
|
||||
)
|
||||
assert system["content"] == expected
|
||||
assert "<knowledge_base>" not in system["content"]
|
||||
|
||||
@@ -179,8 +200,14 @@ def test_row_high_prompt_carries_kb_section_exactly(
|
||||
|
||||
assert frames[-1]["deflected"] is False
|
||||
(system, _user) = seeded_kb.seen_messages[0][0], seeded_kb.seen_messages[0][1]
|
||||
# Phase 119 (D3): the suggested-folder lines ride the HIGH prompt
|
||||
# alongside the <knowledge_base> section — reconstructed the same
|
||||
# deterministic way.
|
||||
expected = build_high_prompt(
|
||||
_cited_docs(db, frames), notes=[], kb_overview=OVERVIEW
|
||||
_suggested_docs(db),
|
||||
notes=[],
|
||||
kb_overview=OVERVIEW,
|
||||
folder_lines=_suggested_folder_lines(db),
|
||||
)
|
||||
assert system["content"] == expected
|
||||
|
||||
|
||||
Reference in New Issue
Block a user