phase: 118_summary_seed_context
**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed.
- **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0).
- **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`.
- **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed.
- **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits).
- **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step).
- **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
This commit is contained in:
@@ -36,16 +36,14 @@ the same host) registered as a local-directory source (the
|
||||
``test_local_directory_sources.py`` registration + real-Sync pattern —
|
||||
registration through the authenticated API, the real in-process
|
||||
``POST /api/sync`` pipeline; no git anywhere), with THREE documents
|
||||
whose bodies are token-controlled so the phase-72 ALREADY_IN_CONTEXT
|
||||
dedupe (the read target is refused when it is a top-2 retrieval seed)
|
||||
NEVER fires — each scripted read target must actually execute, not be
|
||||
refused:
|
||||
whose bodies are token-controlled so every scripted turn stays
|
||||
GROUNDED (the hybrid gate HIGH — the mock's read-cap flow needs the
|
||||
``<tools>`` section):
|
||||
|
||||
* ``anchor-2024.md`` — the retrieval ANCHOR: a digit-bearing name.
|
||||
Both questions name "anchor 2024", whose joined normalized token
|
||||
(``anchor2024``) name-hits this document — the name-hit list LEADS
|
||||
the lexical side, so the anchor is the #1 seed of BOTH turns
|
||||
(grounding: the name hit's ``fts_hit`` keeps the gate HIGH, the
|
||||
(``anchor2024``) name-hits this document — the name hit's
|
||||
``fts_hit`` keeps the gate HIGH for every turn (grounding: the
|
||||
``<tools>`` section is present, and the anchor is never a read
|
||||
target);
|
||||
* ``zz-capped.md`` — the SUBJECT: ~3 100 chars of varied rotation
|
||||
@@ -53,18 +51,19 @@ refused:
|
||||
``chars_total`` the assertions use is ``len(CAP_DOC)`` of this very
|
||||
string, and ``synced_kb`` pins the stored content byte-identical to
|
||||
it). Its body avoids EVERY token of both questions, so on its own
|
||||
(turn 1) question it has zero lexical hits and only the common-word
|
||||
cosine — it is the #3 fused candidate, NOT a seed;
|
||||
question it has zero lexical hits and only the common-word cosine —
|
||||
it ranks below the anchor (irrelevant to the cap's contract: phase
|
||||
118, A6 — the seeds are SUMMARIES, so a first ``read`` of ANY
|
||||
document, suggested or not, succeeds, and the cap applies to
|
||||
everything ``read`` returns — the retired top-2 seed-read refusal
|
||||
that used to constrain this design is gone);
|
||||
* ``aa-short.md`` — the CONTROL: ~200 chars, under the cap.
|
||||
|
||||
The turn-specific flavor words pick the #2 seed: turn 1's question
|
||||
carries ``note, long form`` (planted in the SHORT doc's body) and turn
|
||||
2's carries ``quick pass`` (planted in the CAPPED doc's body), so each
|
||||
turn's NON-target document wins the second seed slot on its own
|
||||
question and the target stays #3/#6. ``synced_kb`` pins this design
|
||||
with the app's real hybrid retrieval (``_assert_target_not_seed`` — a
|
||||
fixture-text regression that makes a target a seed fails at setup with
|
||||
a clear message, not at the wire assertions).
|
||||
The turn-specific flavor words (``note, long form`` planted in the
|
||||
SHORT doc's body, ``quick pass`` in the CAPPED doc's) are inert
|
||||
leftovers of the retired not-a-seed design — they no longer select a
|
||||
seed slot (phase 118: a read of a suggested document succeeds, so the
|
||||
target's seed status never changes the wire contract).
|
||||
|
||||
Test → story mapping (Playwright Mapping Rule; the story is the owner
|
||||
TODO item — one Playwright file per story, A16):
|
||||
@@ -116,7 +115,6 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.mock_llm import embed_text
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -163,9 +161,10 @@ def _cap_doc() -> str:
|
||||
every token of BOTH questions (``read``, ``capped``, ``document``,
|
||||
``capkb``, ``zz``, ``md``, ``anchor``, ``2024``, ``note``,
|
||||
``long``, ``form``, ``aa``, ``short``, ``quick``, ``pass``,
|
||||
``wire``, ``save``, ``control``, ``check``) — the target must have
|
||||
zero lexical hits on its own turn so the dedupe cannot refuse the
|
||||
read."""
|
||||
``wire``, ``save``, ``control``, ``check``) — the target has zero
|
||||
lexical hits on its own turn, so it ranks below the anchor (phase
|
||||
118: its seed status is irrelevant — a first read of any document
|
||||
succeeds; the anchor keeps the gate HIGH)."""
|
||||
topics = [
|
||||
"vault", "mirror", "raid", "pool", "drive",
|
||||
"chain", "slot", "cycle", "guard", "probe",
|
||||
@@ -226,7 +225,8 @@ NOTICE = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=CAP_DOC_TOTAL)
|
||||
|
||||
# The scripted turns (the mock's ``READ_CAP_TRIGGER`` questions — each
|
||||
# carries its own tool call after the colon; the turn-specific flavor
|
||||
# words are the #2-seed selectors, see the module docstring).
|
||||
# words are inert leftovers of the retired #2-seed design — see the
|
||||
# module docstring).
|
||||
Q_WIRE = (
|
||||
f"Read the capped document: read {CAPPED_SP} — "
|
||||
"anchor 2024 note, long form, wire check"
|
||||
@@ -359,26 +359,6 @@ def _truncate_all() -> None:
|
||||
db.commit()
|
||||
|
||||
|
||||
def _assert_target_not_seed(question: str, target_rel: str) -> None:
|
||||
"""Pin the retrieval-anchor design (see the module docstring) with
|
||||
the app's REAL hybrid retrieval over the mock's embeddings
|
||||
(deterministic): the scripted read target must NOT be a top-2 seed
|
||||
for its own question — the phase-72 ALREADY_IN_CONTEXT dedupe would
|
||||
refuse the read and the cap would never fire (the turn would echo
|
||||
the refusal instead). A fixture-text regression that breaks this
|
||||
fails here, at setup, with a clear message."""
|
||||
from app.rag.retriever import retrieve, select_documents
|
||||
|
||||
with SessionLocal() as db:
|
||||
seeds = select_documents(retrieve(db, question, embed_text(question)))
|
||||
paths = [f"{d.source}/{d.path}" for d in seeds]
|
||||
assert f"{SOURCE}/{target_rel}" not in paths, (
|
||||
f"the read target {SOURCE}/{target_rel} is a top-2 seed for its own "
|
||||
f"question — the ALREADY_IN_CONTEXT dedupe would refuse the read and "
|
||||
f"the cap would never fire (seeds: {paths})"
|
||||
)
|
||||
|
||||
|
||||
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
|
||||
"""Poll the (cookie-authenticated) status endpoint until the run
|
||||
reaches a terminal state (the test_ls_tree_drilldown pattern, over
|
||||
@@ -403,8 +383,10 @@ def synced_kb(app_server: str, cap_dirs: Path) -> None:
|
||||
runs the REAL in-process sync (``POST /api/sync`` — walk → chunk →
|
||||
embed → overview → version bump), pins the stored content
|
||||
byte-identical to the fixture strings (the ``chars_total``
|
||||
assumption), and pins the retrieval-anchor design for all three
|
||||
scripted questions (the dedupe never fires)."""
|
||||
assumption). Phase 118: the retrieval-anchor design only pins the
|
||||
gate's grounding (the anchor keeps every turn HIGH) — a read of a
|
||||
suggested document succeeds, so the targets' seed status no longer
|
||||
constrains the suite."""
|
||||
_truncate_all()
|
||||
with httpx.Client(base_url=app_server, timeout=30.0) as client:
|
||||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
@@ -436,11 +418,6 @@ def synced_kb(app_server: str, cap_dirs: Path) -> None:
|
||||
)
|
||||
)
|
||||
assert stored == expected, f"stored content drifted for {rel}"
|
||||
# The retrieval-anchor design (the module docstring): every scripted
|
||||
# read target stays OUT of its own question's top-2 seeds.
|
||||
_assert_target_not_seed(Q_WIRE, CAPPED_REL)
|
||||
_assert_target_not_seed(Q_SAVE, CAPPED_REL)
|
||||
_assert_target_not_seed(Q_CONTROL, SHORT_REL)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -681,9 +658,10 @@ def test_truncated_read_frame_order_live_marker_and_llm_notice(
|
||||
assert i_tool < i_result < i_delta, (i_tool, i_result, i_delta)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["deflected"] is False, done
|
||||
# The read document is the turn's cited source even though it was
|
||||
# NOT a retrieval seed (the anchor design) — retrieval + agent-read,
|
||||
# deduped (the grounded-turn record).
|
||||
# The read document is the turn's cited source (retrieval +
|
||||
# agent-read, deduped — the grounded-turn record; phase 118: the
|
||||
# target may be a suggested (summary-seeded) document — its seed
|
||||
# status never changes the cap's wire contract).
|
||||
assert any(
|
||||
s["path"] == CAPPED_REL and s["source"] == SOURCE for s in done["sources"]
|
||||
), done["sources"]
|
||||
|
||||
Reference in New Issue
Block a user