Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 95_read_truncation_cap
Build and Push Containers / build-and-push-app (push) Successful in 1m38s
Details
Build and Push Containers / build-and-push-db (push) Successful in 12s
Details

Browse Source 
All gates are green. Final verification is complete — no fixes were needed; the phase is fully implemented and passing.

**Phase 95 final verification report**

- Verified all 3 tasks already implemented (task files in `complete/`): read cap + notice + `ToolResultPiece` (task 01), `tool_result` SSE + live/saved/shared marker (task 02), E2E suite (task 03).
- Code checks passed: `BOR_READ_MAX_CHARS` (default 128 000, validator), pinned `READ_TRUNCATION_NOTICE` + `TRUNCATION_MARKER`, byte-identical at/under-cap read, holder `read_truncations` (no `tool_calls` impact), piece order (after `tool`, before next round), `ChatToolResultEvent`, `ToolCall` back-compat fields, `app.js` live/restore handlers, `shared.js`, `.truncated-note` CSS, `.env.example` entry, mock-LLM echo script.
- **Tests:** `uv run pytest` → 1966 passed; `uv run pytest --cov=app --cov-report=term-missing` → all green, TOTAL **99%** (>90% gate).
- **E2E:** `uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov` → **3 passed** (frame order + live marker + LLM notice via echo; save→shared fidelity; short-read control).
- **Regression (isolated):** `test_agent_document_tools` 4 ✓, `test_chat_history` 5 ✓, `test_share_chat` 4 ✓, `test_big_read_progress` 4 ✓, `test_stop_generation` 3 ✓.
- **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings.

**Completion criteria:** ① over-cap read → first-cap-chars + marker + pinned notice — ✓ (unit-pinned: at-cap/cap+1/notice tests); ② user marker live/saved/shared — ✓ (E2E + frontend tests); ③ at/under cap byte-identical, no frame — ✓ (unit + control E2E); ④ top-2 `<documents>` retrieval untouched — ✓ (`app/rag/retriever.py` unmodified vs HEAD); ⑤ suite green, >90% coverage, ruff+pyright clean — ✓; ⑥ no completed-phase behavior change — ✓ (all gates green; commit left to harness per pass rules).

- No defects found; no changes made this pass. Next pending phase: none in `todo/` (96 is the next free number).
This commit is contained in:
ducoterra
2026-09-11 03:42:51 -04:00
parent d4943b4822
commit bcaef800c5
36 changed files with 2836 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/e2e/test_read_truncation_cap.py
+861
View File
@@ -0,0 +1,861 @@
"""Phase 95 task 03 E2E (Playwright, mock-only): the read cap's
dedicated story suite — the truncated read, end to end.
The whole TODO.md L5 item (``95_read_truncation_cap`` task 03 is the
story gate): a document over the (lowered) cap is read truncated, the
``tool_result`` frame lands, the Reading line carries the marker, the
LLM's context carried ``[…truncated…]`` + the grep pointer, and the
marker survives save → shared.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real
``turbo`` does whatever it does with the tools, while this story's gate
is the deterministic SCRIPTED capped-read flow in
``tests/e2e/mock_llm.py`` (``READ_CAP_TRIGGER``: user message contains
``read the capped document`` — the question carries its own tool call
after the colon, ``read the capped document: read source/path`` —
**and** the system prompt carries the ``<tools>`` section of the HIGH
prompt). The mock ECHOES the ENTIRE read tool result into its final
answer (the house scripted-turn way of asserting on tool results — the
mock is the only E2E lens on the LLM's context), so both the marker's
presence (the truncated turn) and its absence (the short-document
control) land on the rendered answer.
The app under test boots with the LOWERED cap (the
env-override-for-the-app-under-test pattern — this suite's module app,
as in ``test_local_directory_sources.py`` / ``test_ls_tree_drilldown.py``):
``BOR_READ_MAX_CHARS=1500`` — small enough that the ~3 100-char
fixture document truncates deterministically at 1 500, big enough that
the short-document control (~200 chars) never does.
KB fixture — one host temp dir (``tmp_path_factory``; the app runs on
the same host) registered as a local-directory source (the
``test_local_directory_sources.py`` registration + real-Sync pattern —
registration through the authenticated API, the real in-process
``POST /api/sync`` pipeline; no git anywhere), with THREE documents
whose bodies are token-controlled so the phase-72 ALREADY_IN_CONTEXT
dedupe (the read target is refused when it is a top-2 retrieval seed)
NEVER fires — each scripted read target must actually execute, not be
refused:
* ``anchor-2024.md`` — the retrieval ANCHOR: a digit-bearing name.
Both questions name "anchor 2024", whose joined normalized token
(``anchor2024``) name-hits this document — the name-hit list LEADS
the lexical side, so the anchor is the #1 seed of BOTH turns
(grounding: the name hit's ``fts_hit`` keeps the gate HIGH, the
``<tools>`` section is present, and the anchor is never a read
target);
* ``zz-capped.md`` — the SUBJECT: ~3 100 chars of varied rotation
prose (deterministic, pinned below with its exact length — the
``chars_total`` the assertions use is ``len(CAP_DOC)`` of this very
string, and ``synced_kb`` pins the stored content byte-identical to
it). Its body avoids EVERY token of both questions, so on its own
(turn 1) question it has zero lexical hits and only the common-word
cosine — it is the #3 fused candidate, NOT a seed;
* ``aa-short.md`` — the CONTROL: ~200 chars, under the cap.
The turn-specific flavor words pick the #2 seed: turn 1's question
carries ``note, long form`` (planted in the SHORT doc's body) and turn
2's carries ``quick pass`` (planted in the CAPPED doc's body), so each
turn's NON-target document wins the second seed slot on its own
question and the target stays #3/#6. ``synced_kb`` pins this design
with the app's real hybrid retrieval (``_assert_target_not_seed`` — a
fixture-text regression that makes a target a seed fails at setup with
a clear message, not at the wire assertions).
Test → story mapping (Playwright Mapping Rule; the story is the owner
TODO item — one Playwright file per story, A16):
1. ``test_truncated_read_frame_order_live_marker_and_llm_notice`` —
the wire carries the ``tool`` frame THEN exactly one
``tool_result`` frame (the pinned counts) THEN the answer's
``delta``/``done`` frames; the live Reading line carries the pinned
``(truncated — showing 1500 of N chars)`` marker; the mock's echo
proves the LLM context carried ``[…truncated…]`` AND the pinned
``TRUNCATED — … Use grep …`` notice (and the head sentinel reached
the model while the past-the-cap tail sentinel did NOT).
2. ``test_truncated_read_save_and_shared_fidelity`` — the auto-saved
row's (phase 55) brain message ``tools`` record carries
``truncated: true`` + the counts (saved-chats API); the shared
page (``/shared/<token>``) renders the SAME marker on the Reading
line (the phase-50 restore contract, pixel-identical).
3. ``test_short_read_control_no_frame_no_marker`` — the under-cap read
executes (its ``tool`` frame lands) but streams NO ``tool_result``
frame, the Reading line carries NO marker, the echo shows no
``[…truncated…]`` / ``TRUNCATED —``, and the saved ``tools``
record is the plain pre-truncation shape.
"""
from __future__ import annotations
import json
import os
import re
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Browser, BrowserContext, Locator, Page, expect
from sqlalchemy import select, text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document
from app.rag.agent import READ_TRUNCATION_NOTICE
from app.rag.retriever import TRUNCATION_MARKER
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
from e2e.mock_llm import embed_text
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): the conftest session app owns its
# port in a combined run — this module app binds its own port instead
# (a same-port second uvicorn dies on bind and would drive the wrong
# server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_READCAP", "8137"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The lowered read cap the app under test boots with
# (``BOR_READ_MAX_CHARS``): the subject of the suite.
READ_CAP = 1500
SOURCE = "capkb" # the local directory's basename = the source name
ANCHOR_REL = "anchor-2024.md"
CAPPED_REL = "zz-capped.md"
SHORT_REL = "aa-short.md"
CAPPED_SP = f"{SOURCE}/{CAPPED_REL}" # the combined read identity
SHORT_SP = f"{SOURCE}/{SHORT_REL}"
# --------------------------------------------------------------------------
# Fixture documents (deterministic, token-controlled — see the module
# docstring for the retrieval-anchor design)
# --------------------------------------------------------------------------
ANCHOR_DOC = (
"# Anchor 2024\n\n"
"The anchor 2024 record keeps the backup rotation steady: the quiet "
"window, the mirror pool, and the retention label all stay pinned to "
"this year's plan.\n"
)
HEAD_SENTINEL = "CAPDOC-HEAD-9f2a" # inside the first 1 500 chars
TAIL_SENTINEL = "CAPDOC-TAIL-7c3b" # PAST the cap (the rest is NOT shown)
SHORT_SENTINEL = "CAPSHORT-5d1e" # the whole short doc is under the cap
def _cap_doc() -> str:
"""The ~3 100-char subject document: a repeated-but-varied paragraph
block (deterministic), the head sentinel near the top, the planted
``quick pass`` sentence (the turn-2 flavor words — see the module
docstring) and the tail sentinel at the very end. The body avoids
every token of BOTH questions (``read``, ``capped``, ``document``,
``capkb``, ``zz``, ``md``, ``anchor``, ``2024``, ``note``,
``long``, ``form``, ``aa``, ``short``, ``quick``, ``pass``,
``wire``, ``save``, ``control``, ``check``) — the target must have
zero lexical hits on its own turn so the dedupe cannot refuse the
read."""
topics = [
"vault", "mirror", "raid", "pool", "drive",
"chain", "slot", "cycle", "guard", "probe",
]
paras: list[str] = []
i = 0
while True:
t = topics[i % len(topics)]
paras.append(
f"{t.title()} item {i:02d}: at 02:00 the quiet window opens and "
f"the archive job copies the {t} image to the mirror pool, then "
f"the manifest audit verifies the snapshot chain for slot {i:02d}, "
"keeping the retention label clean and the rotation order exact "
"for the whole night cycle."
)
i += 1
if len("\n\n".join(paras)) > 2750:
break
head = (
"# Rotation archive\n\n"
f"{HEAD_SENTINEL}\n\n"
"A quick pass over the rotation confirms the pass order of the "
"guard probe before the quiet window closes, and the audit log "
"records the result.\n\n"
)
tail = f"\n\n{TAIL_SENTINEL}\n"
return head + "\n\n".join(paras) + tail
CAP_DOC = _cap_doc()
CAP_DOC_TOTAL = len(CAP_DOC)
SHORT_DOC = (
"# Rotation plan\n\n"
"This note keeps the long form of the rotation plan in one place: the "
"quiet window, the mirror pool, the retention label, and the guard "
"probe order for every weekly run.\n\n"
f"{SHORT_SENTINEL}\n"
)
SHORT_DOC_TOTAL = len(SHORT_DOC)
# The suite's length contract (the task's pinned-count assumption —
# ``chars_total`` is asserted against ``len(CAP_DOC)`` of the very
# string written to the fixture, and ``synced_kb`` pins the stored
# content byte-identical to it): the subject is OVER the cap with the
# tail sentinel past the cut; the control is UNDER the cap.
assert CAP_DOC_TOTAL > READ_CAP, CAP_DOC_TOTAL
assert CAP_DOC.index(HEAD_SENTINEL) < READ_CAP < CAP_DOC.index(TAIL_SENTINEL)
assert CAP_DOC.count(HEAD_SENTINEL) == 1 and CAP_DOC.count(TAIL_SENTINEL) == 1
assert SHORT_DOC_TOTAL < READ_CAP, SHORT_DOC_TOTAL
assert SHORT_DOC.count(SHORT_SENTINEL) == 1
# The pinned marker copy (task 02 — the app.js/shared.js template, plain
# integers, no thousands separators) and the LLM-side notice (the
# app.rag.agent constant, formatted for this fixture's counts).
MARKER = f" (truncated — showing {READ_CAP} of {CAP_DOC_TOTAL} chars)"
NOTICE = READ_TRUNCATION_NOTICE.format(shown=READ_CAP, total=CAP_DOC_TOTAL)
# The scripted turns (the mock's ``READ_CAP_TRIGGER`` questions — each
# carries its own tool call after the colon; the turn-specific flavor
# words are the #2-seed selectors, see the module docstring).
Q_WIRE = (
f"Read the capped document: read {CAPPED_SP} — "
"anchor 2024 note, long form, wire check"
)
Q_SAVE = (
f"Read the capped document: read {CAPPED_SP} — "
"anchor 2024 note, long form, save check"
)
Q_CONTROL = (
f"Read the capped document: read {SHORT_SP} — "
"anchor 2024, quick pass, control check"
)
#: The invalid/unknown share token shape (test_share_chat.py's
#: ``SHARE_URL_RE`` convention).
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def cap_dirs(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The story's local-directory source: one host temp dir (the app
server runs on the same host, so the path is visible to it) holding
the three token-controlled fixture documents. The directory's
basename is the source name (``kind=local``, phase 38)."""
root = tmp_path_factory.mktemp("bor_read_cap") / SOURCE
root.mkdir()
for rel, doc in (
(ANCHOR_REL, ANCHOR_DOC),
(CAPPED_REL, CAP_DOC),
(SHORT_REL, SHORT_DOC),
):
(root / rel).write_text(doc, encoding="utf-8")
assert (root / rel).read_text(encoding="utf-8") == doc
assert not (root / ".git").exists() # the local-kind story: NOT git
return root
@pytest.fixture(scope="module")
def app_server(mock_llm: int, cap_dirs: Path) -> Iterator[str]:
"""The real app under test — per-module env (the conftest pattern,
cf. ``test_ls_tree_drilldown.py``): NO ``BOR_GIT_SOURCES`` (the env
fallback is git-only — the source here is a DB-registered local
directory), the mock LLM, the mock-calibrated threshold, the
leak-guarded code defaults — and the SUBJECT: the lowered read cap
``BOR_READ_MAX_CHARS=1500`` (the env-override-for-the-app-under-
test pattern; the default 128 000 is unit-pinned in
``tests/unit/test_config.py``). The session app is never started in
this isolated run, so no port clash."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated threshold (conftest pattern) — the retrieval-anchor
# design keeps every scripted turn grounded regardless.
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
# Phase 67: instant retry waits + the code-default budget (the
# conftest leak-guard pattern).
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
# reads it from cwd) — override it with an EMPTY value (the env var
# beats the .env file): the registry must hold EXACTLY the local
# directory this suite registers (a leftover env git list would
# pollute the KB the scripted reads run against).
env["BOR_GIT_SOURCES"] = ""
# Leak guards (conftest pattern): an operator's local (gitignored)
# .env cannot leak corpus-specific settings into the app under test.
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
# THE SUBJECT: the lowered read cap (task 03 — the app under test
# boots with BOR_READ_MAX_CHARS=1500; the production default of
# 128 000 stays pinned in tests/unit/test_config.py).
env["BOR_READ_MAX_CHARS"] = str(READ_CAP)
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
share one Postgres, so a leftover git_sources row or document would
pollute the retrieval the scripted turns run against (the anchor
design's margins are pinned against EXACTLY these three
documents)."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
def _assert_target_not_seed(question: str, target_rel: str) -> None:
"""Pin the retrieval-anchor design (see the module docstring) with
the app's REAL hybrid retrieval over the mock's embeddings
(deterministic): the scripted read target must NOT be a top-2 seed
for its own question — the phase-72 ALREADY_IN_CONTEXT dedupe would
refuse the read and the cap would never fire (the turn would echo
the refusal instead). A fixture-text regression that breaks this
fails here, at setup, with a clear message."""
from app.rag.retriever import retrieve, select_documents
with SessionLocal() as db:
seeds = select_documents(retrieve(db, question, embed_text(question)))
paths = [f"{d.source}/{d.path}" for d in seeds]
assert f"{SOURCE}/{target_rel}" not in paths, (
f"the read target {SOURCE}/{target_rel} is a top-2 seed for its own "
f"question — the ALREADY_IN_CONTEXT dedupe would refuse the read and "
f"the cap would never fire (seeds: {paths})"
)
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the (cookie-authenticated) status endpoint until the run
reaches a terminal state (the test_ls_tree_drilldown pattern, over
plain httpx — this fixture has no browser page yet)."""
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
return body
time.sleep(0.5)
raise AssertionError(f"sync did not reach a terminal state: {body}")
@pytest.fixture(scope="module")
def synced_kb(app_server: str, cap_dirs: Path) -> None:
"""The story's precondition: the one-source KB synced under the
deterministic mock. Registers the temp directory through the
authenticated API (the ``test_local_directory_sources.py`` pattern),
runs the REAL in-process sync (``POST /api/sync`` — walk → chunk →
embed → overview → version bump), pins the stored content
byte-identical to the fixture strings (the ``chars_total``
assumption), and pins the retrieval-anchor design for all three
scripted questions (the dedupe never fires)."""
_truncate_all()
with httpx.Client(base_url=app_server, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
r = client.post(
"/api/git-sources", json={"kind": "local", "path": str(cap_dirs)}
)
assert r.status_code == 201, r.text
r = client.post("/api/sync")
assert r.status_code == 202, r.text
body = _wait_sync_done_http(client)
assert body["state"] == "success", body
detail = body["detail"]
assert detail["added"] == 3, detail
assert detail["pruned"] == 0, detail
# The import stored the fixture strings BYTE-IDENTICALLY — the
# ``chars_total`` every assertion below uses is ``len()`` of the
# very string written to the directory (the task's pinned-count
# assumption).
with SessionLocal() as db:
for rel, expected in (
(ANCHOR_REL, ANCHOR_DOC),
(CAPPED_REL, CAP_DOC),
(SHORT_REL, SHORT_DOC),
):
stored = db.scalar(
select(Document.content).where(
Document.source == SOURCE, Document.path == rel
)
)
assert stored == expected, f"stored content drifted for {rel}"
# The retrieval-anchor design (the module docstring): every scripted
# read target stays OUT of its own question's top-2 seeds.
_assert_target_not_seed(Q_WIRE, CAPPED_REL)
_assert_target_not_seed(Q_SAVE, CAPPED_REL)
_assert_target_not_seed(Q_CONTROL, SHORT_REL)
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped —
the scripted turns never change it, so the registry persists
across the tests of this module)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# Page helpers (the test_ls_tree_drilldown / test_share_chat house
# patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
#: (a response clone read in the background) — wire-level assertions
#: for the ``tool`` / ``tool_result`` frames, independent of the UI
#: rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page) -> list[dict]:
"""The SSE frames captured since the last submit (``_submit``
clears the buffer), once the hook's background read settles."""
deadline = time.monotonic() + 30.0
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _tool_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool"]
def _result_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool_result"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
"""The turn is complete: answer text in the bubble, button recovered
(the phase-48 settle wait, the test_agent_document_tools helper)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
def _last_brain(page: Page) -> Locator:
return page.locator(".msg.brain").last
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
def _admin_cookies(page: Page) -> dict[str, str]:
"""The signed session cookies the browser holds after a form login —
used to call the admin API with plain httpx (the test's API side
sees exactly what the signed-in browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["chats"]
def _find_row(
rows: list[dict[str, Any]], title: str
) -> dict[str, Any] | None:
return next((c for c in rows if c["title"] == title), None)
def _get_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> dict[str, Any]:
r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
assert r.status_code == 200, r.text
return r.json()
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55: auto-saves are SILENT —
A2 — so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
last = _find_row(_chats(app_url, cookies), title)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(test_share_chat.py — the assertion branches on the API's
availability, so a non-secure origin still passes through the
fallback branch deterministically)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
"""Press the chat page's Share pill and pin the owner-locked
outcome on the live region (test_share_chat.py's helper, verbatim
contract): "Share link copied." when ``navigator.clipboard`` is
available in the context, else the inline fallback link field
carrying the ``/shared/<uuid>`` URL."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
expect(page.locator(".share-link-fallback")).to_have_count(0)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
field = page.locator(".share-link-fallback")
expect(field).to_be_visible()
href = field.get_attribute("href")
assert href is not None
assert href.startswith(app_url)
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
# --------------------------------------------------------------------------
# 1. The truncated read: the wire's frame order, the live marker, the
# LLM-visible notice (via the mock's echo)
# --------------------------------------------------------------------------
def test_truncated_read_frame_order_live_marker_and_llm_notice(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_WIRE)
_wait_settled(page)
# --- the wire: tool → tool_result → delta… (exactly one, counts) --
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "read", "argument": CAPPED_SP}
], _tool_frames(frames)
# EXACTLY ONE tool_result frame — the pinned counts (the char-based
# cap: showing the first READ_CAP of the fixture's true length).
assert _result_frames(frames) == [
{
"type": "tool_result",
"name": "read",
"argument": CAPPED_SP,
"truncated": True,
"chars_shown": READ_CAP,
"chars_total": CAP_DOC_TOTAL,
}
], _result_frames(frames)
# Frame order: the marker lands AFTER the call is shown and BEFORE
# the answer's first delta (the phase-37/48 "calling tool" timing
# is untouched — the beat between the two is the truncation).
i_tool = next(i for i, f in enumerate(frames) if f.get("type") == "tool")
i_result = next(
i for i, f in enumerate(frames) if f.get("type") == "tool_result"
)
i_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
assert i_tool < i_result < i_delta, (i_tool, i_result, i_delta)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source even though it was
# NOT a retrieval seed (the anchor design) — retrieval + agent-read,
# deduped (the grounded-turn record).
assert any(
s["path"] == CAPPED_REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
# --- the live DOM: the Reading line carries the pinned marker ----
line = _last_brain(page).locator(".tool-call")
expect(line).to_have_count(1)
expect(line).to_contain_text(f"Reading {CAPPED_SP}")
expect(line.locator("code")).to_have_text(CAPPED_SP)
expect(line.locator("span.truncated-note")).to_have_text(MARKER)
# --- the mock's echo: the LLM's context carried the marker AND ---
# --- the grep-pointer notice (the house scripted-turn lens) ------
bubble = _last_brain(page).locator(".bubble")
text = bubble.text_content() or ""
assert TRUNCATION_MARKER in text, text
assert NOTICE in text, text
# The first cap chars reached the model…
assert HEAD_SENTINEL in text, text
# …and the past-the-cap tail did NOT (the cut is real, not cosmetic).
assert TAIL_SENTINEL not in text, text
# --------------------------------------------------------------------------
# 2. Save → shared: the stored tools record carries the truncation, and
# the shared page renders the SAME marker (pixel-identical)
# --------------------------------------------------------------------------
def test_truncated_read_save_and_shared_fidelity(
page: Page,
browser: Browser,
app_url: str,
synced_kb: None,
db_ready: None,
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_submit(page, Q_SAVE)
_wait_settled(page)
# The live marker landed (regression through the save path — the
# tool_result frame stamped the toolAcc entry the save carries).
line = _last_brain(page).locator(".tool-call")
expect(line).to_have_count(1)
expect(line.locator("span.truncated-note")).to_have_text(MARKER)
cookies = _admin_cookies(page)
title = _auto_title(Q_SAVE)
row = _wait_saved_row(app_url, cookies, title)
chat_id: str = row["id"]
anon_ctx: BrowserContext | None = None
try:
# The saved message's ``tools`` record carries the truncation —
# ``truncated: true`` + the pinned counts (the saved-chats API;
# the model_dump round-trip fills the non-truncated defaults,
# so the full five-key shape is asserted).
detail = _get_chat(app_url, cookies, chat_id)
brain_msgs = [m for m in detail["messages"] if m["who"] == "brain"]
assert len(brain_msgs) == 1, detail["messages"]
assert brain_msgs[0]["tools"] == [
{
"name": "read",
"argument": CAPPED_SP,
"truncated": True,
"chars_shown": READ_CAP,
"chars_total": CAP_DOC_TOTAL,
}
], brain_msgs[0]["tools"]
# Share the auto-saved row (phase 55: settled = already saved —
# the Share click shares the linked row) and read back the link.
_grant_clipboard(page, app_url)
_click_share_and_assert_status(page, app_url)
row = _find_row(_chats(app_url, cookies), title)
assert row is not None and row.get("share_url"), row
share_url: str = row["share_url"]
assert SHARE_URL_RE.fullmatch(share_url)
# A FRESH context (no cookies — the guest's only credential is
# the token in the URL): the shared page renders the SAME
# marker on the Reading line (shared.js renders it from the
# stored record — the phase-50 restore contract).
anon_ctx = browser.new_context()
anon = anon_ctx.new_page()
anon.set_default_timeout(30_000)
anon.goto(app_url + share_url)
expect(anon.locator("#shared-title")).to_have_text(title)
anon_line = anon.locator(".msg.brain .tool-call")
expect(anon_line).to_have_count(1)
expect(anon_line).to_contain_text(f"Reading {CAPPED_SP}")
expect(anon_line.locator("code")).to_have_text(CAPPED_SP)
expect(anon_line.locator("span.truncated-note")).to_have_text(MARKER)
# The shared answer still renders the truncation the LLM was
# told about (same stored text, same renderer).
anon_text = anon.locator(".msg.brain .bubble").first.text_content() or ""
assert TRUNCATION_MARKER in anon_text, anon_text
assert NOTICE in anon_text, anon_text
assert HEAD_SENTINEL in anon_text, anon_text
assert TAIL_SENTINEL not in anon_text, anon_text
anon_ctx.close()
anon_ctx = None
finally:
if anon_ctx is not None:
anon_ctx.close()
_delete_chat(app_url, cookies, chat_id)
# --------------------------------------------------------------------------
# 3. The control: the under-cap read executes, streams NO tool_result
# frame, shows NO marker, and saves the plain tools record
# --------------------------------------------------------------------------
def test_short_read_control_no_frame_no_marker(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_CONTROL)
_wait_settled(page)
# The read EXECUTED (the tool frame landed)…
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "read", "argument": SHORT_SP}
], _tool_frames(frames)
# …but a non-truncated read streams NO tool_result frame (one frame
# = one noteworthy event — the phase-95 additive contract).
assert _result_frames(frames) == [], _result_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
assert any(
s["path"] == SHORT_REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
# No marker on the live Reading line.
line = _last_brain(page).locator(".tool-call")
expect(line).to_have_count(1)
expect(line).to_contain_text(f"Reading {SHORT_SP}")
expect(line.locator("code")).to_have_text(SHORT_SP)
expect(line.locator("span.truncated-note")).to_have_count(0)
# The mock's echo: the PLAIN read shape — the whole short document
# (it is under the cap), no marker, no notice.
bubble = _last_brain(page).locator(".bubble")
text = bubble.text_content() or ""
assert SHORT_SENTINEL in text, text
assert TRUNCATION_MARKER not in text, text
assert "TRUNCATED —" not in text, text
# The saved ``tools`` record is the plain pre-truncation shape
# (``truncated`` default False, the counts null — the
# pre-phase-95 shape validates and renders unchanged).
cookies = _admin_cookies(page)
title = _auto_title(Q_CONTROL)
row = _wait_saved_row(app_url, cookies, title)
try:
detail = _get_chat(app_url, cookies, row["id"])
brain_msgs = [m for m in detail["messages"] if m["who"] == "brain"]
assert len(brain_msgs) == 1, detail["messages"]
assert brain_msgs[0]["tools"] == [
{
"name": "read",
"argument": SHORT_SP,
"truncated": False,
"chars_shown": None,
"chars_total": None,
}
], brain_msgs[0]["tools"]
finally:
_delete_chat(app_url, cookies, row["id"])