"""Phase 118 task 06 E2E (Playwright, mock-only): the summary-seed context — grounded turns seed SUMMARIES, full text arrives only via ``read`` (the owner directive, TODO.md L3; the retired phase-24 full-text seeding is gone — locked A1/A6). Run in isolation (DB must be up: ``podman compose up -d db``): uv run pytest tests/e2e/test_summary_seed_context.py -v --no-cov MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate is the deterministic contract, not the real model's behavior: the mock LLM's tail-echo trigger (``END_OF_NOTES_TRIGGER``) quotes the last 160 chars of the seeded ```` block, and the scripted summary-read flow (``SUMMARY_SEED_READ_TRIGGER``) echoes the whole read tool result — both are the house scripted-turn lenses on the LLM's context (the mock is the only E2E lens), so what reached the prompt is assertable byte-exactly. KB fixture — one host temp dir (``tmp_path_factory``; the app runs on the same host) registered as a local-directory source (the ``test_ls_tree_drilldown.py`` / ``test_read_truncation_cap.py`` registration + real-Sync pattern — registration through the authenticated API, the real in-process ``POST /api/sync`` pipeline; no git anywhere), with SEVEN markdown documents (every file markdown — the locked A2 end-to-end proof: the mock ``SUMMARY_MODE`` digest must land on every doc, markdown included) whose bodies are token-controlled so the hybrid gate picks the intended suggested set deterministically (``synced_kb`` pins it with the app's REAL retrieval — a fixture-text regression that breaks the design fails at setup with a clear message, not at the wire assertions): * each doc = a neutral 24-token prelude (no question tokens — the mock's first-24-token digest therefore shares NOTHING with either question, so a doc's embedded summary chunk carries no strength tokens and never outranks that doc's own content chunks) + ``seed vault rotation notes`` repeated *i* times (the strength gradient) + six unique filler tokens + a unique tail SENTINEL on the document's LAST line (outside the digest — the full-text marker); * the measured rank order (probe, stable across re-imports) is ``doc-a > doc-d > doc-b > doc-c > doc-e > doc-f > doc-g`` for the tail question (×1 strength gradient — md5-collision-reordered) and a clean ``doc-a > doc-b > doc-c > doc-d > doc-e > doc-f > doc-g`` for the read question (×3 gradient — the collision noise cannot cross the wider gaps); the suggested tier (LOCKED A3: top-5, NO floor) is the first five of each, the related tier (rank 6+, ``related_max_docs`` = 2) is ``doc-f``/``doc-g`` in both. Test → contract mapping (the task's cases a–e; the story is the owner TODO item — one Playwright file per story, A16): 1. ``test_summaries_seed_the_prompt_not_the_full_text`` — cases (a) + (b) + (d): the tail-echo question's answer quotes the last 160 chars of the seeded ```` block, which end in the LAST suggested document's SUMMARY — the mock's byte-stable ``SUMMARY_MODE`` digest tail + pointer line (case a: summaries reached the prompt; case d: the markdown doc's block carries the digest, not a content preview — the phase-30 digest shape, and the ``synced_kb`` pin proves every doc's stored summary IS the digest). The inverse of the retired phase-24 pin: NO document's tail sentinel is in the echoed context — the full content of no suggested doc ever reached the model (case b). The grounded turn cites NOTHING (phase 119, LOCKED A1 — a zero-read turn chips nothing; the retired phase-118 A4 suggested+read union is gone) and renders the de-emphasized related row (rank 6+); the durable record carries suggested + related (118-A3 untouched). 2. ``test_read_suggested_doc_adds_full_text_and_cites`` — cases (c) + (e): the scripted flow ``read``s the rank-1 suggested doc — the read SUCCEEDS (phase 118: the seeds are summaries, not full text — the retired top-2 seed-read refusal is gone) and the mock's verbatim echo of the read result lands the tail sentinel in the answer (case c: the full text now arrives through the ``read`` tool, not the seed); the citation chips = the READ docs only (phase 119, LOCKED A1 — exactly the one read doc; none of the four other suggested docs chips), and the related row renders rank 6+ (case e); the durable record carries suggested + related + read (118-A3 untouched). """ from __future__ import annotations import json import os import re import subprocess import sys import time from collections.abc import Iterator from pathlib import Path from typing import Any import httpx import pytest from playwright.sync_api import Locator, Page, expect from sqlalchemy import select, text from app.config import Settings as _Settings from app.db import SessionLocal from app.models import Document, QueryLog from app.rag.retriever import TRUNCATION_MARKER, retrieve, select_related, select_suggested from e2e.auth_helpers import login from e2e.conftest import ( ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http, ) from e2e.mock_llm import TOKEN_RE, embed_text REPO = Path(__file__).resolve().parents[2] # Phase 79 (task 04, full inventory): the conftest session app owns its # port in a combined run — this module app binds its own port instead # (a same-port second uvicorn dies on bind and would drive the wrong # server). Env-overridable. APP_PORT = int(os.environ.get("E2E_APP_PORT_SEEDCTX", "8138")) APP_URL = f"http://127.0.0.1:{APP_PORT}" SOURCE = "seedkb" # the local directory's basename = the source name MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E" # -------------------------------------------------------------------------- # Fixture documents (deterministic, token-controlled — see the module # docstring for the design and the measured rank order) # -------------------------------------------------------------------------- #: 22 neutral tokens — together with the 2-token title they ARE the #: mock's first-24-token digest, so no summary chunk shares a token #: with either question (the summary chunks rank below every content #: chunk; the digest is the byte-stable assertion surface). PRELUDE = ( "fixture preamble block one two three four five six seven eight nine ten " "eleven twelve thirteen fourteen fifteen sixteen seventeen eighteen nineteen" ) assert len(TOKEN_RE.findall(PRELUDE.lower())) == 22 #: The strength gradient (each token a question token — ``seed``, #: ``vault``, ``rotation``, ``notes``) — repeated *i* times in doc *i*. PHRASE = "seed vault rotation notes" #: (path, title, phrase repeats, filler token, tail sentinel) — the #: MEASURED rank order (probe, stable across re-imports) is #: doc-a > doc-d > doc-b > doc-c > doc-e > doc-f > doc-g: the suggested #: tier is the first five, the related tier the last two. DOCS: list[tuple[str, str, int, str, str]] = [ ("doc-a.md", "Zeta Alfa", 7, "fillera", "SEEDA-TAIL-7f3a"), ("doc-b.md", "Zeta Bravo", 6, "fillerb", "SEEDB-TAIL-8c4d"), ("doc-c.md", "Zeta Charlie", 5, "fillerc", "SEEDC-TAIL-9d5e"), ("doc-d.md", "Zeta Delta", 4, "fillerd", "SEEDD-TAIL-0e6f"), ("doc-e.md", "Zeta Echo", 3, "fillere", "SEEDE-TAIL-1f7a"), ("doc-f.md", "Zeta Foxtrot", 2, "fillerf", "SEEDF-TAIL-2a8b"), ("doc-g.md", "Zeta Golf", 1, "fillerg", "SEEDG-TAIL-3b9c"), ] #: The suggested tier per scripted question (the MEASURED rank order — #: probe-verified, stable across re-imports): the tail question's ×1 #: strength gradient is md5-collision-reordered (d before b), the read #: question's ×3 gradient is clean. Related is the rank-6+ remainder in #: both (the gradient never puts a strength doc below the filler docs). SUGGESTED_TAIL = ["doc-a.md", "doc-d.md", "doc-b.md", "doc-c.md", "doc-e.md"] SUGGESTED_READ = ["doc-a.md", "doc-b.md", "doc-c.md", "doc-d.md", "doc-e.md"] RELATED_PATHS = ["doc-f.md", "doc-g.md"] LAST_SUGGESTED = "doc-e.md" # the tail-echo target (case a/d) READ_TARGET = "doc-a.md" # rank 1 — a suggested doc (case c) SENTINELS = [sentinel for _p, _t, _i, _f, sentinel in DOCS] READ_SENTINEL = next(s for p, _t, _i, _f, s in DOCS if p == READ_TARGET) def _doc_content(title: str, i: int, filler: str, sentinel: str) -> str: return ( f"# {title}\n" f"\n{PRELUDE}\n" f"\n{' '.join([PHRASE] * i)}\n" f"\n{' '.join([filler] * 6)}\n" f"\n{sentinel}\n" ) def _expected_summary(content: str, path: str) -> str: """The mock lite model's byte-stable digest + the code pointer line (mirrors ``mock_llm.compose_answer``'s ``SUMMARY_MODE`` branch — first 24 tokens of the document content — plus the summarizer's deterministic ``Source:`` line; no model output is ever trusted).""" digest = " ".join(TOKEN_RE.findall(content.lower())[:24]) return f"This document covers {digest}.\nSource: {SOURCE}/{path}" #: The turn's questions (the mock's trigger phrases — see the module #: docstring). The tail question carries ``END_OF_NOTES_TRIGGER``; its #: strength tokens ground the turn (best cosine ≈ 0.55 ≥ the E2E 0.30 #: threshold). The read question carries ``SUMMARY_SEED_READ_TRIGGER`` #: with its own scripted call after the colon (the house scripted-turn #: convention — ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE`` shape), #: ``seed vault rotation notes`` × 3 for the same grounding. TAIL_QUESTION = "Show the end of your notes about the seed vault rotation, please." READ_QUESTION = ( f"Read the suggested document: read {SOURCE}/{READ_TARGET} — " f"{PHRASE} {PHRASE} {PHRASE}" ) assert "show the end of your notes" in TAIL_QUESTION.lower() assert "read the suggested document" in READ_QUESTION.lower() assert re.fullmatch(r"[a-z0-9_./-]+", f"{SOURCE}/{READ_TARGET}") # -------------------------------------------------------------------------- # Fixtures # -------------------------------------------------------------------------- @pytest.fixture(scope="module") def seed_dirs(tmp_path_factory: pytest.TempPathFactory) -> Path: """The story's local-directory source: one host temp dir (the app server runs on the same host, so the path is visible to it) holding the seven token-controlled markdown documents. The directory's basename is the source name (``kind=local``, phase 38).""" root = tmp_path_factory.mktemp("bor_seed_ctx") / SOURCE root.mkdir() for path, title, i, filler, sentinel in DOCS: (root / path).write_text( _doc_content(title, i, filler, sentinel), encoding="utf-8" ) assert (root / READ_TARGET).is_file() return root @pytest.fixture(scope="module") def app_server(mock_llm: int, seed_dirs: Path) -> Iterator[str]: """The real app under test — per-module app (the conftest pattern, cf. ``test_read_truncation_cap.py``): NO ``BOR_GIT_SOURCES`` (the env fallback is git-only — the source here is a DB-registered local directory), the mock LLM, the mock-calibrated threshold, and the leak-guarded code defaults (the suggested/related tier settings ride their code defaults — 5 / 2 — exactly like the production ``.env``-free defaults). The session app is never started in this isolated run, so no port clash.""" env = dict(os.environ) env.pop("DEBUGPY", None) env["BOR_ENVIRONMENT"] = "e2e" env["BOR_STATIC_DIR"] = str(REPO / "frontend") env["BOR_LLM_BASE_URL"] = ( "https://aipi.reeseapps.com/v1" if USE_REAL_LLM else f"http://127.0.0.1:{mock_llm}/v1" ) # Mock-calibrated gate (conftest pattern): the strength gradient # keeps every scripted turn grounded (best cosine ≈ 0.55 / ≈ 0.80). env["BOR_RELEVANCE_THRESHOLD"] = "0.30" env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15" # Phase 67: instant retry waits + the code-default budget (the # conftest leak-guard pattern). env["BOR_LLM_RETRY_DELAY"] = "0" env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default) env.setdefault( "BOR_DATABASE_URL", "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese", ) # Phase 16: admin auth must be set or create_app() refuses to boot. env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD env["BOR_SESSION_SECRET"] = SESSION_SECRET # The repo's .env file carries the owner's BOR_GIT_SOURCES (the app # reads it from cwd) — override it with an EMPTY value (the env var # beats the .env file): the registry must hold EXACTLY the local # directory this suite registers (a leftover env git list would # pollute the KB the scripted turns run against). env["BOR_GIT_SOURCES"] = "" # Leak guards (conftest pattern): an operator's local (gitignored) # .env cannot leak corpus-specific settings into the app under test. env["BOR_DOCS_REPO"] = "" env["BOR_SUGGESTIONS"] = json.dumps( _Settings.model_fields["suggestions"].default ) env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default proc = subprocess.Popen( [sys.executable, "-m", "uvicorn", "app.main:app", "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"], cwd=REPO, env=env, ) try: _wait_http(f"{APP_URL}/api/health") yield APP_URL finally: proc.terminate() try: proc.wait(timeout=10) except subprocess.TimeoutExpired: proc.kill() @pytest.fixture(scope="module") def app_url(app_server: str) -> str: return app_server def _truncate_all() -> None: """Fresh registry + KB (the E2E isolation pattern): the E2E suites share one Postgres, so a leftover git_sources row or document would pollute the retrieval the scripted turns run against (the strength gradient's margins are pinned against EXACTLY these seven documents).""" with SessionLocal() as db: db.execute( text( "TRUNCATE chunks, documents, query_log, steering_notes, " "kb_overview, git_sources, folder_summaries" ) ) db.commit() def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]: """Poll the (cookie-authenticated) status endpoint until the run reaches a terminal state (the test_ls_tree_drilldown pattern, over plain httpx — this fixture has no browser page yet).""" deadline = time.monotonic() + timeout_s body: dict[str, Any] = {} while time.monotonic() < deadline: r = client.get("/api/sync/status") assert r.status_code == 200, r.text body = r.json() if body["state"] in ("success", "failed"): return body time.sleep(0.5) raise AssertionError(f"sync did not reach a terminal state: {body}") def _assert_tiers(question: str, suggested_paths: list[str]) -> None: """Pin the strength-gradient design with the app's REAL hybrid retrieval over the mock's embeddings (deterministic): the suggested tier is exactly the five strength docs in the MEASURED rank order for THIS question (LOCKED A3 — top-5, NO floor) and the related tier is the rank-6+ remainder (``related_max_docs`` = 2). A fixture-text regression that breaks the design fails here, at setup, with a clear message.""" with SessionLocal() as db: chunks = retrieve(db, question, embed_text(question)) suggested = [f"{d.source}/{d.path}" for d in select_suggested(chunks)] related = [ f"{d.source}/{d.path}" for d in select_related( chunks, {d.id for d in select_suggested(chunks)}, _Settings.model_fields["related_max_docs"].default, ) ] expected = [f"{SOURCE}/{p}" for p in suggested_paths] assert suggested == expected, ( f"suggested tier drifted for {question!r}: {suggested} " f"(expected {expected})" ) assert related == [f"{SOURCE}/{p}" for p in RELATED_PATHS], ( f"related tier drifted for {question!r}: {related}" ) @pytest.fixture(scope="module") def synced_kb(app_server: str, seed_dirs: Path) -> None: """The story's precondition: the one-source KB synced under the deterministic mock. Registers the temp directory through the authenticated API (the ``test_local_directory_sources.py`` pattern), runs the REAL in-process sync (``POST /api/sync`` — walk → chunk → embed → summaries → overview → folder summaries → version bump), pins the stored content byte-identical to the fixture strings, pins LOCKED A2 end-to-end (every doc — markdown included — stores the mock's byte-stable digest + exactly one embedded ``is_summary`` chunk), and pins the tier design for both scripted questions.""" _truncate_all() with httpx.Client(base_url=app_server, timeout=30.0) as client: r = client.post("/api/login", json={"password": ADMIN_PASSWORD}) assert r.status_code == 204, r.text r = client.post( "/api/git-sources", json={"kind": "local", "path": str(seed_dirs)} ) assert r.status_code == 201, r.text r = client.post("/api/sync") assert r.status_code == 202, r.text body = _wait_sync_done_http(client) assert body["state"] == "success", body detail = body["detail"] assert detail["added"] == len(DOCS), detail assert detail["pruned"] == 0, detail # The import stored the fixture strings BYTE-IDENTICALLY and, for # EVERY doc (markdown included — locked A2), the mock's byte-stable # digest: the deterministic assertion surface of the whole suite. with SessionLocal() as db: for path, title, i, filler, sentinel in DOCS: stored = db.scalar( select(Document).where( Document.source == SOURCE, Document.path == path ) ) assert stored is not None, f"fixture doc {path} was not imported" expected = _doc_content(title, i, filler, sentinel) assert stored.content == expected, f"stored content drifted for {path}" assert stored.summary == _expected_summary(expected, path), ( f"summary is not the mock digest for {path}: {stored.summary!r}" ) schunks = [c for c in stored.chunks if c.is_summary] assert len(schunks) == 1 and schunks[0].position == -1, ( f"expected exactly one is_summary chunk for {path}" ) assert schunks[0].embedding is not None, ( f"the is_summary chunk of {path} is not embedded" ) _assert_tiers(TAIL_QUESTION, SUGGESTED_TAIL) _assert_tiers(READ_QUESTION, SUGGESTED_READ) @pytest.fixture(autouse=True) def _clean(db_ready: None) -> Iterator[None]: """Per-test query_log isolation (the KB itself is module-scoped — the scripted turns never change it, so the registry and the KB persist across the tests of this module).""" with SessionLocal() as db: db.execute(text("TRUNCATE query_log")) db.commit() yield with SessionLocal() as db: db.execute(text("TRUNCATE query_log")) db.commit() # -------------------------------------------------------------------------- # Page helpers (the test_ls_tree_drilldown / test_read_truncation_cap # house patterns) # -------------------------------------------------------------------------- #: Captures the raw SSE ``data:`` payloads of the /api/chat stream #: (a response clone read in the background) — wire-level assertions #: for the ``tool`` / ``done`` frames, independent of the UI rendering. SSE_HOOK = """ () => { if (window.__sseInstalled) return; window.__sseInstalled = true; window.__sseFrames = []; const origFetch = window.fetch; window.fetch = async function (...args) { const res = await origFetch.apply(this, args); try { const url = typeof args[0] === 'string' ? args[0] : args[0].url; if (url.includes('/api/chat')) { res.clone().text().then((bodyText) => { for (const block of bodyText.split('\\n\\n')) { const line = block.trim(); if (line.startsWith('data: ')) { window.__sseFrames.push(line.slice(6)); } } }); } } catch (e) { /* non-clonable responses: ignored */ } return res; }; } """ def _install_page_hooks(page: Page) -> None: page.evaluate(SSE_HOOK) def _frames(page: Page) -> list[dict]: """The SSE frames captured since the last submit (``_submit`` clears the buffer), once the hook's background read settles.""" deadline = time.monotonic() + 30.0 while True: raw = page.evaluate("() => window.__sseFrames || []") parsed = [json.loads(line) for line in raw if line] if any(f.get("type") == "done" for f in parsed): return parsed if time.monotonic() > deadline: raise AssertionError( f"SSE hook captured no `done` frame (frames so far: " f"{len(parsed)}) — hook install failed?" ) time.sleep(0.05) def _tool_frames(frames: list[dict]) -> list[dict]: return [f for f in frames if f.get("type") == "tool"] def _submit(page: Page, question: str) -> None: page.evaluate("window.__sseFrames = []") page.fill("#message-input", question) page.click("#send-btn") # The user bubble lands synchronously with the submit handler. expect(page.locator(".msg.user .bubble").last).to_contain_text(question) def _wait_settled(page: Page) -> None: """The turn is complete: answer text in the bubble, button recovered (the phase-48 settle wait, the test_agent_document_tools helper).""" expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000) expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000) expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000) def _last_brain(page: Page) -> Locator: return page.locator(".msg.brain").last def _last_query_log() -> QueryLog: with SessionLocal() as db: rows = db.scalars(select(QueryLog)).all() assert len(rows) == 1, f"expected exactly one query_log row, got {len(rows)}" return rows[0] def _assert_read_chips_and_related_row(page: Page, read_paths: list[str]) -> None: """The citation surface of a grounded turn (phase 119, LOCKED A1 — the retired phase-118 A4 union is gone): the chip row is the AGENT-READ docs only (deduped, read order — a zero-read turn chips NOTHING: the suggested docs are seed context, not citations), and the de-emphasized ``related-docs`` row carries the rank-6+ remainder (phase-113 UI, untouched — never a citation chip).""" chips = page.locator(".msg.brain .source-chip") expect(chips).to_have_count(len(read_paths)) for path in read_paths: expect( chips.filter(has_text=path), message=f"chip for {path}" ).to_have_count(1) row = page.locator(".msg.brain .related-docs") expect(row).to_have_count(1) expect(row.first).to_have_attribute("aria-label", "Nearby docs, in case") links = page.locator(".msg.brain .related-docs .related-doc") expect(links).to_have_count(len(RELATED_PATHS)) for i, path in enumerate(RELATED_PATHS): expect(links.nth(i)).to_contain_text(f"{SOURCE}/{path}") # The related links are never citation chips. expect(page.locator(".msg.brain .related-docs .source-chip")).to_have_count(0) # -------------------------------------------------------------------------- # 1. Cases (a) + (b) + (d): summaries seed the prompt — no full text # -------------------------------------------------------------------------- def test_summaries_seed_the_prompt_not_the_full_text( page: Page, app_url: str, synced_kb: None, db_ready: None ) -> None: """The tail-echo question quotes the last 160 chars of the seeded ```` block: they end in the LAST suggested doc's SUMMARY (the mock's byte-stable digest tail + pointer — cases a + d: the markdown doc's block carries the digest, not a content preview), and NO document's tail sentinel is in the echoed context (case b: the full text of no suggested doc reached the model — the inverse of the retired phase-24 pin).""" page.set_default_timeout(30_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, TAIL_QUESTION) _wait_settled(page) # No tools on this turn — the answer is the mock's direct tail echo # of the seeded context (the summary-seed lens). frames = _frames(page) assert _tool_frames(frames) == [], _tool_frames(frames) done = next(f for f in frames if f.get("type") == "done") assert done["deflected"] is False, done # Phase 119 (LOCKED A1): the grounded done frame cites the READ docs # only — this turn read NOTHING (the summary-only fast path), so the # citation surface is empty; the suggested set is seed context, not # citations (the retired phase-118 A4 union is gone). The durable # record below still carries suggested + related (118-A3 untouched). assert done["sources"] == [], done["sources"] bubble = _last_brain(page).locator(".bubble") expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000) # Case (a) + (d): the echoed tail ends in the LAST suggested doc's # summary — the markdown doc-e's byte-stable digest tail + pointer # line (the digest is the mock's SUMMARY_MODE shape, pinned in # ``synced_kb`` — a content preview would carry the prelude/phrase # text instead, and the pointer line only ever exists on a stored # summary). The bubble renders the answer as markdown, which # collapses the summary's newline — so pin each LINE separately # (the digest line's tail sits inside the echoed 160 chars; the # pointer line is single-line too). last_doc = next(d for d in DOCS if d[0] == LAST_SUGGESTED) last_summary = _expected_summary(_doc_content(*last_doc[1:]), LAST_SUGGESTED) digest_line = last_summary.split("\n", 1)[0] expect(bubble).to_contain_text("Source: seedkb/doc-e.md") expect(bubble).to_contain_text(digest_line[-100:]) # The digest is NOT the raw content: the echoed tail cannot carry # any document's tail sentinel — the inverse of the retired # phase-24 full-text pin (case b). for sentinel in SENTINELS: expect(bubble).not_to_contain_text(sentinel) expect(bubble).not_to_contain_text(TRUNCATION_MARKER) # Case (e)'s chip surface (phase 119 A1 — no read yet): ZERO chips; # the related row renders rank 6+ (the de-emphasized row, untouched). _assert_read_chips_and_related_row(page, []) # Durable record: grounded; suggested + related (LOCKED A3 — the log # records retrieval, not citations; phase 119 A1 retires the chip # union only, not the record). row = _last_query_log() assert row.question == TAIL_QUESTION assert row.deflected is False assert row.sources == ", ".join(f"{SOURCE}/{p}" for p in SUGGESTED_TAIL + RELATED_PATHS) # -------------------------------------------------------------------------- # 2. Cases (c) + (e): ``read`` adds the full text; chips = suggested + # read (deduped); related row = rank 6+ # -------------------------------------------------------------------------- def test_read_suggested_doc_adds_full_text_and_cites( page: Page, app_url: str, synced_kb: None, db_ready: None ) -> None: """The scripted flow ``read``s the rank-1 suggested doc: the read SUCCEEDS (phase 118 — the seeds are summaries, not full text; the retired top-2 seed-read refusal is gone) and the mock's verbatim echo of the read result lands the document's tail sentinel in the answer (case c: the full text reached the model through the ``read`` tool, not the seed). The citation chips = the READ docs only (phase 119, LOCKED A1 — exactly the one read doc; none of the other four suggested docs chips; the retired phase-118 A4 union is gone); the related row renders rank 6+ (case e, untouched).""" page.set_default_timeout(30_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, READ_QUESTION) _wait_settled(page) # Exactly ONE executed tool call: the scripted read of the # rank-1 suggested doc. frames = _frames(page) assert _tool_frames(frames) == [ {"type": "tool", "name": "read", "argument": f"{SOURCE}/{READ_TARGET}"} ], _tool_frames(frames) done = next(f for f in frames if f.get("type") == "done") assert done["deflected"] is False, done # Phase 119 (LOCKED A1): done.sources = the READ docs only — # exactly the one scripted read; the other four suggested docs # never chip (the retired phase-118 A4 suggested+read union is # gone). assert [ (s["source"], s["path"]) for s in done["sources"] ] == [(SOURCE, READ_TARGET)], done["sources"] # done.related = the rank-6+ remainder (deduped against the cited — # the read doc is rank 1, not in the remainder). assert [ (s["source"], s["path"]) for s in done["related"] ] == [(SOURCE, p) for p in RELATED_PATHS], done["related"] # The live Reading line carries the combined identity. line = _last_brain(page).locator(".tool-call") expect(line).to_have_count(1) expect(line).to_contain_text(f"Reading {SOURCE}/{READ_TARGET}") # Case (c): the answer is the mock's verbatim echo of the READ # RESULT — its header and the document's tail sentinel (the last # line, outside the digest): the full text reached the model # through the read, not the seed. bubble = _last_brain(page).locator(".bubble") expect(bubble).to_contain_text(f"Document {SOURCE}/{READ_TARGET}:", timeout=30_000) expect(bubble).to_contain_text(READ_SENTINEL) expect(bubble).not_to_contain_text(TRUNCATION_MARKER) # The seed itself carried no full text: none of the OTHER docs' # sentinels are in the answer either. for path, _t, _i, _f, sentinel in DOCS: if path != READ_TARGET: expect(bubble).not_to_contain_text(sentinel) # Case (e): the UI chip row = the READ docs only (phase 119 A1 — # exactly the one read doc), and the related row renders rank 6+. _assert_read_chips_and_related_row(page, [READ_TARGET]) # Durable record: grounded; suggested + related + read (deduped, # LOCKED A3 — phase 119 A1 retires the chip union only, not the # record). row = _last_query_log() assert row.question == READ_QUESTION assert row.deflected is False assert row.sources == ", ".join(f"{SOURCE}/{p}" for p in SUGGESTED_READ + RELATED_PATHS)