Files
brain-of-reese/tests/e2e/test_read_range_dedupe.py
T
ducoterra a8c0402ad2
Build and Push Containers / build-and-push-app (push) Successful in 2m37s
Build and Push Containers / build-and-push-db (push) Successful in 12s
phase: 141_read_range_dedupe_fix
All gates green and verified. Final report:

**Phase 141 final verification pass — complete**

- **Verified (tasks 01–04 already in `complete/`):** `AgentHolder.read_coverage` span recording (all 4 read outcomes, refusals record nothing); coverage dedupe matrix (range read of uncovered lines proceeds, fully-covered re-read → `ALREADY_IN_CONTEXT` with a now-true claim, partial plain re-read → `REREAD_PARTIAL` naming merged spans, `REREAD_TRUNCATED` retired, fast path retired per task 02); `TOOLS_SECTION` re-read sentence re-cut to coverage-true form (prompt lock re-pinned); owner-scenario integration repro (pure + truncated variants) and mock `read two ranges` flow + dedicated E2E — all match the task specs exactly; no code defects found.
- **Tests/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL 99% (5498 stmts, `app/rag/agent.py` 99%) — run twice, both green; `uv run pytest tests/e2e/test_read_range_dedupe.py -v --no-cov` → 2 passed in isolation; `uv run ruff check . && uv run pyright` → All checks passed / 0 errors.
- **Completion criteria:** (1) two different ranges both reach the model, re-covered re-read refused truthfully — ✅ (integration + E2E); (2) partial plain re-read gets the `offset`/`limit` teaching, false claim gone both directions — ✅; (3) plain-read + phase-131 result shapes byte-identical, existing pins pass unchanged — ✅; (4) all gate commands green, >90% — ✅; (5) no behavior change in completed phases (grep/ls, truncation notice, `read_docs`/`read_truncations` signals) — ✅ (whole project suite green).
- **Documentation:** README.md — fixed pre-existing phase-140 gap: `BOR_PDF_PICTORIAL` / `BOR_PDF_PICTORIAL_MIN_CHARS` were in `.env.example`/Settings but undocumented (README claims every `.env.example` var is covered); added the pictorial-PDF-pass bullet to the PDF section + two config-reference rows. AGENTS.md — verified consistent (rules, quick-ref commands, phase refs all check out) — no changes needed.
- **Notable:** overview's "fast path survives for holder-state rows" note is superseded by task 02's authoritative decision (every dedupe row resolves — "full" needs the true line count); implementation follows task 02. No phase files touched; changes left in the working tree for the harness commit.
- **Next pending phase:** `142_kb_overview_freshness`.
2026-10-01 01:27:46 -04:00

666 lines
28 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 141 task 04 E2E (Playwright, mock-only): the read
range-dedupe fix's dedicated story suite — the owner's stuck
scenario, end to end through the SSE stream.
TODO.md L3: "From a chat I was having, the LLM is trying to read
different ranges of a document and keeps saying: 'The system is
telling me the document is already in my context, but I haven't
actually seen the full text … The read calls keep returning
"Already in your context"…' This seems like a critical bug, fix
it." Pre-phase, ``AgentHolder``'s pair-set dedupe recorded ANY
successful read — including a RANGE read — as if the FULL document
were in context, so a second read of a DIFFERENT range of the same
document was refused with a claim that was false. Phase 141 tracks
the covered line SPANS (``AgentHolder.read_coverage``) and refuses
only genuinely-covered re-reads (tasks 01–03, unit/integration
pinned) — this suite is the browser-visible proof: the
deterministic mock LLM drives a two-range turn, and the assertions
show both ``read`` tool frames succeeding and the final answer
carrying content from the SECOND range.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_read_range_dedupe.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — this story's
gate is the deterministic SCRIPTED flow in
``tests/e2e/mock_llm.py`` (the mock is the only E2E lens on the
LLM's context):
* the two-range turn — ``TWO_RANGES_TRIGGER``: a user message
containing ``read two ranges`` **and** the system prompt carrying
the HIGH prompt's ``<tools>`` section. Request 1 reads
``offset=1, limit=100`` (range A) on the question-carried target
(``read two ranges: read source/path``); request 2 (one
``tool``-role read result in the messages, carrying the
``lines: 1-100 of …`` extent header — the mock's own keying,
asserted) issues the SECOND read ``offset=101, limit=100`` (range
B) on the SAME target — pre-phase the server refused it
``ALREADY_IN_CONTEXT`` while only range A was in the prompt; the
flow only works if the second read PROCEEDS. Request 3 (two read
results in the messages, the second carrying the ``lines: 101-…``
header — asserted the same way) answers ``Both parts read.
<first 80 chars of the second read result's content, header lines
skipped>`` (the existing read-echo shape) — the marker line lives
in range B, so an echo of it proves the slice reached the model.
A broken premise (a refusal in the messages — the pre-phase bug —
or an unkeyed extent header) lands as the deterministic
``Premise broke:`` echo, so a suite fails loudly on it.
* the single-range turn — the mock's EXISTING ``READ_LINES_TRIGGER``
flow (``read the lines``, phase 131 — the grep →
read(offset/limit) dance) reused on THIS suite's KB: the
single-range behavior is unchanged by the coverage rework (the
phase-131 expectation still passes against the coverage-tracked
server).
The app under test keeps the DEFAULT read cap (128 000 chars) —
deliberately: NO read truncates in this suite. That is the point —
pre-phase, a second range read of a NON-truncated document already
in the pair set was refused ``ALREADY_IN_CONTEXT`` (the phase-139
suite had to shrink ``BOR_READ_MAX_CHARS`` until the first range
slice truncated just to let a second range read through — that
workaround is retired by the coverage tracking). The still-refused
row (a re-read of FULLY-covered lines) is unit-pinned in tasks
02/03 (``tests/unit/test_agent.py`` — the dedupe matrix); this
suite's job is the HAPPY two-range path, browser-visible.
KB fixture — ONE deterministic 200-line text document
(``tworanges/two.txt``): fixed-length filler lines (``Line <i>: …``
— the line numbers are exact) carrying THREE marker lines —
``TWO_RANGES_A_LINE …`` at line 50 (range A), ``TWO_RANGES_B_LINE …``
at line 101 (the FIRST line of range B — the read-echo quote is the
slice's first 80 chars, so the marker must sit at the range's start
to be echoed), and ``LINE_MARKER …`` at line 30 (the phase-131
flow's grep target — exactly one line carries the pattern). Each
marker line has two contextual neighbor lines repeating its
distinctive words (the mock's token-overlap embeddings are
diluted by the ~19-line chunks — the neighbors keep the marker
chunk's cosine above the mock-calibrated lexical floor, the
house gate pattern). All line positions are computed from the very
string written to the fixture (never a literal in the
assertions), so the suite stays true if the fixture is regenerated.
The app under test is mock-calibrated (conftest pattern):
``BOR_RELEVANCE_THRESHOLD=0.30`` + ``BOR_LEXICAL_SUPPORT_FLOOR=0.15``
+ ``BOR_SOURCE_USEFULNESS_FLOOR=0.15`` — both questions' best-chunk
cosine (the marker chunk wins — ~0.23 / ~0.19 on the mock's
token-overlap distribution, deterministic) clears the floor with
FTS hits (``alpha``/``beta``/``beacon`` and ``trap``/``components``
in the marker chunks), so every scripted turn stays GROUNDED (the
``<tools>`` section is present) and the read document clears the
usefulness bar into ``done.sources``.
Test → observable mapping (Playwright Mapping Rule; the story is
the owner's TODO item — one Playwright file per story, A16):
1. ``test_two_ranges_both_reach_the_model`` — the ``read two
ranges`` question → the SSE ``tool`` frames are EXACTLY the two
range reads of the SAME path (``offset`` 1/``limit`` 100, then
``offset`` 101/``limit`` 100 — asserted on the frames' ARGS; a
refused second read would re-issue the call and add a THIRD
frame), NO ``tool_result`` frames (nothing truncated — the
default cap), the ``done`` frame is non-deflected with the read
document cited (phase 119 A1), the final answer carries
``Both parts read.`` + the ``TWO_RANGES_B_LINE`` marker text
(the second slice reached the model's context), and NO refusal
surface anywhere (neither the ``ALREADY_IN_CONTEXT`` line nor
the mock's broken-premise echo is in the answer).
2. ``test_single_range_flow_unchanged`` — the phase-131 ``read the
lines`` question on this KB → the SSE ``tool`` frames are the
grep (the canned pattern) then the range read (``offset`` = the
marker's true line, ``limit`` = 30) — the phase-131 expectation
unchanged against the coverage-tracked server; the final answer
is the grounded echo ``Found it at line <N>`` + the marker text.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Locator, Page, expect
from sqlalchemy import select, text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document
from app.rag.agent import ALREADY_IN_CONTEXT
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
from e2e.mock_llm import READ_LINES_LIMIT, READ_LINES_PATTERN
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): the conftest session app owns
# its port in a combined run — this module app binds its own port
# instead (a same-port second uvicorn dies on bind and would drive
# the wrong server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_TWORANGES", "8156"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
SOURCE = "tworanges" # the local directory's basename = the source name
REL = "two.txt"
SP = f"{SOURCE}/{REL}" # the combined read identity
# The fixture's layout (deterministic — see the module docstring):
# 200 fixed-length filler lines (60 chars each) carrying three
# marker lines, each flanked by two contextual neighbor lines that
# repeat its distinctive words (the mock-calibrated gate — the
# marker chunk's cosine must clear the 0.15 lexical floor).
TOTAL_LINES = 200
A_MARKER_LINE = 50 # range A (lines 1–100)
B_MARKER_LINE = 101 # range B (lines 101+): the FIRST line of range B
LINE_MARKER_LINE = 30 # the phase-131 flow's grep target (range A)
#: The range A marker line (lines 1–100 carry it — NOT echoed by the
#: mock; its presence in the fixture is the two-range premise).
A_MARKER_TEXT = (
"TWO_RANGES_A_LINE the first range documents the alpha beacon payload"
)
A_CTX_ABOVE = (
"the alpha beacon table starts here: alpha beacon payloads, documented below"
)
A_CTX_BELOW = (
"the alpha beacon entry ends here — alpha beacon payload documented above"
)
#: The range B marker line — the FIRST line of range B, so the
#: mock's read-echo quote (the second read result's first 80 chars,
#: header lines skipped) carries it: an echo of it proves the second
#: slice reached the model.
B_MARKER_TEXT = (
"TWO_RANGES_B_LINE the second range documents the beta beacon payload"
)
B_CTX_ABOVE = (
"the beta beacon table starts here: beta beacon payloads, documented below"
)
B_CTX_BELOW = (
"the beta beacon entry ends here — beta beacon payload documented above"
)
#: The phase-131 flow's marker line: exactly ONE line in the document
#: carries ``READ_LINES_PATTERN`` (imported — the mock can never
#: drift from the fixture). Its true line is computed from the very
#: string written to the fixture (the determinism pin).
LINE_MARKER_TEXT = "LINE_MARKER the trap components are documented here"
LM_CTX_ABOVE = (
"the trap components table starts here: trap components, documented below"
)
LM_CTX_BELOW = (
"the trap components entry ends here — trap components documented above"
)
_FILLER_WORDS = "quiet mirror vault raid pool chain slot cycle guard probe audit "
def _filler_line(i: int) -> str:
"""One fixed-length (60-char) filler line — the line numbers are
exact and every line is the SAME length, so the range slices'
lengths are deterministic."""
prefix = f"Line {i:04d}: "
return prefix + (_FILLER_WORDS * 5)[: 60 - len(prefix)]
def _doc() -> str:
"""The 200-line fixture document (deterministic — module
docstring)."""
lines: list[str] = []
for i in range(1, TOTAL_LINES + 1):
if i == LINE_MARKER_LINE - 1:
lines.append(LM_CTX_ABOVE)
elif i == LINE_MARKER_LINE:
lines.append(LINE_MARKER_TEXT)
elif i == LINE_MARKER_LINE + 1:
lines.append(LM_CTX_BELOW)
elif i == A_MARKER_LINE - 1:
lines.append(A_CTX_ABOVE)
elif i == A_MARKER_LINE:
lines.append(A_MARKER_TEXT)
elif i == A_MARKER_LINE + 1:
lines.append(A_CTX_BELOW)
elif i == B_MARKER_LINE - 1:
lines.append(B_CTX_ABOVE)
elif i == B_MARKER_LINE:
lines.append(B_MARKER_TEXT)
elif i == B_MARKER_LINE + 1:
lines.append(B_CTX_BELOW)
else:
lines.append(_filler_line(i))
return "\n".join(lines)
DOC = _doc()
DOC_LINES = DOC.splitlines()
DOC_TOTAL = len(DOC)
assert len(DOC_LINES) == TOTAL_LINES, len(DOC_LINES)
# The determinism pins (computed from the very string written to the
# fixture — the assertions use these, never literals): the markers
# are unique, the B marker is the FIRST line of range B (the
# read-echo quote reaches it), and the range slices are UNDER the
# default read cap (nothing truncates — no ``tool_result`` frames).
assert DOC.count(LINE_MARKER_TEXT) == 1
assert DOC.count(A_MARKER_TEXT) == 1
assert DOC.count(B_MARKER_TEXT) == 1
assert DOC_LINES[B_MARKER_LINE - 1] == B_MARKER_TEXT
assert DOC_LINES[A_MARKER_LINE - 1] == A_MARKER_TEXT
assert DOC_LINES[LINE_MARKER_LINE - 1] == LINE_MARKER_TEXT
#: The default read cap (128 000 — ``app/config.py``) stays in force
#: for this suite; both 100-line slices must stay under it.
assert len("\n".join(DOC_LINES[:100])) < 128_000
assert len("\n".join(DOC_LINES[100:200])) < 128_000
# The expected answer fragments (the mock's read-echo shape —
# ``Both parts read. <quote>`` with quote the second read result's
# content, the ``date:`` / ``lines:`` header lines skipped, first 80
# chars). The B marker line (70 chars) IS the slice's first line, so
# the full marker text sits inside the 80-char quote.
ANSWER_LEAD = "Both parts read."
assert len(B_MARKER_TEXT) < 80, len(B_MARKER_TEXT)
# The scripted questions. Q_A carries ``TWO_RANGES_TRIGGER`` + the
# target (the question-carried-call convention — the mock parses
# ``read two ranges: read source/path`` from the raw user message);
# Q_B carries the phase-131 ``READ_LINES_TRIGGER`` (the grep →
# read(offset/limit) dance) on this KB. Both carry their marker
# chunks' distinctive words (the mock-calibrated gate stays
# GROUNDED — the ``<tools>`` section is present, so both flows
# fire).
Q_A = (
f"Read two ranges: read {SP} — "
"what documents the alpha and beta beacon payloads?"
)
Q_B = f"Read the lines of {SP} — what are the trap components documented here?"
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def tworanges_dirs(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The story's local-directory source: one host temp dir (the app
server runs on the same host, so the path is visible to it)
holding the ONE generated 200-line text document. The directory's
basename is the source name (``kind=local``, phase 38)."""
root = tmp_path_factory.mktemp("bor_two_ranges") / SOURCE
root.mkdir()
(root / REL).write_text(DOC, encoding="utf-8")
assert (root / REL).read_text(encoding="utf-8") == DOC
assert not (root / ".git").exists() # the local-kind story: NOT git
return root
@pytest.fixture(scope="module")
def app_server(mock_llm: int, tworanges_dirs: Path) -> Iterator[str]:
"""The real app under test — per-module env (the conftest
pattern, cf. ``test_read_line_ranges.py``): NO
``BOR_GIT_SOURCES`` (the env fallback is git-only — the source
here is a DB-registered local directory), the mock LLM, the
mock-calibrated gate (threshold 0.30 + lexical floor 0.15 — the
conftest pair). The DEFAULT read cap stays in force — THE
SUBJECT: no read truncates in this suite, and the second range
read still proceeds (pre-phase, the pair-set dedupe refused a
second range read of a NON-truncated document already in
context — the owner's stuck turn). The session app is never
started in this isolated run, so no port clash."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern): both scripted turns
# stay GROUNDED (the questions' best-chunk cosine clears the
# 0.15 floor with FTS hits — the marker chunks win).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
# The read document must clear the citation bar into
# done.sources (phase 119 A1 — both turns assert it).
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget (the
# conftest leak-guard pattern).
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env file carries the owner's settings (the app reads
# it from cwd) — override the source list with an EMPTY value (the
# env var beats the .env file): the registry must hold EXACTLY the
# local directory this suite registers (a leftover env git list
# would pollute the KB the scripted turns run against).
env["BOR_GIT_SOURCES"] = ""
# Leak guards (conftest pattern): an operator's local (gitignored)
# .env cannot leak corpus-specific settings into the app under
# test.
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E
suites share one Postgres, so a leftover git_sources row or
document would pollute the retrieval the scripted turns run
against (the single-document KB's gate margins are pinned
against EXACTLY this one document)."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the (cookie-authenticated) status endpoint until the run
reaches a terminal state (the test_ls_tree_drilldown pattern, over
plain httpx — this fixture has no browser page yet)."""
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
return body
time.sleep(0.5)
raise AssertionError(f"sync did not reach a terminal state: {body}")
@pytest.fixture(scope="module")
def synced_kb(app_server: str, tworanges_dirs: Path) -> None:
"""The story's precondition: the one-source KB synced under the
deterministic mock. Registers the temp directory through the
authenticated API (the ``test_local_directory_sources.py``
pattern), runs the REAL in-process sync (``POST /api/sync`` —
walk → chunk → embed → overview → version bump), pins the stored
content byte-identical to the fixture string."""
_truncate_all()
with httpx.Client(base_url=app_server, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
r = client.post(
"/api/git-sources", json={"kind": "local", "path": str(tworanges_dirs)}
)
assert r.status_code == 201, r.text
r = client.post("/api/sync")
assert r.status_code == 202, r.text
body = _wait_sync_done_http(client)
assert body["state"] == "success", body
detail = body["detail"]
assert detail["added"] == 1, detail
assert detail["pruned"] == 0, detail
# The import stored the fixture string BYTE-IDENTICALLY — the
# slice extents every assertion uses come from the very string
# written to the directory.
with SessionLocal() as db:
stored = db.scalar(
select(Document.content).where(
Document.source == SOURCE, Document.path == REL
)
)
assert stored == DOC, "stored content drifted from the fixture"
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped —
the scripted turns never change it, so the registry persists
across the tests of this module)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# Page helpers (the test_read_line_ranges / test_read_range_tool_lines
# house patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
#: (a response clone read in the background) — wire-level assertions
#: for the ``tool`` / ``tool_result`` frames, independent of the UI
#: rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page, deadline_s: float = 120.0) -> list[dict]:
"""The SSE frames captured since the last submit (``_submit``
clears the buffer), once the hook's background read settles."""
deadline = time.monotonic() + deadline_s
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _tool_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool"]
def _result_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool_result"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page, timeout_ms: int = 120_000) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (the phase-48 settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text(
"", timeout=timeout_ms
)
expect(page.locator("#send-btn")).to_be_enabled(timeout=timeout_ms)
expect(page.locator("#send-label")).to_have_text("Send", timeout=timeout_ms)
def _last_brain(page: Page) -> Locator:
return page.locator(".msg.brain").last
# --------------------------------------------------------------------------
# 1. The owner's stuck scenario, browser-visible: two DIFFERENT ranges
# of the same document both reach the model (no refusal)
# --------------------------------------------------------------------------
def test_two_ranges_both_reach_the_model(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_A)
_wait_settled(page)
frames = _frames(page)
# THE SUBJECT on the wire — IN ORDER, asserted on the frames'
# ARGS (not a substring): the SAME path, TWO distinct ranges
# (offset 1/limit 100, then offset 101/limit 100 — the phase-131
# additive fields ride the SAME frame). EXACTLY two: a refused
# second read (the pre-phase bug) would re-issue the call and
# add a THIRD frame — or land as the mock's broken-premise echo.
assert _tool_frames(frames) == [
{
"type": "tool",
"name": "read",
"argument": SP,
"offset": 1,
"limit": 100,
},
{
"type": "tool",
"name": "read",
"argument": SP,
"offset": 101,
"limit": 100,
},
], _tool_frames(frames)
# NO ``tool_result`` frame: nothing truncated (the default cap —
# both 100-line slices are far under it; one frame = one
# noteworthy event, the phase-95 additive contract).
assert _result_frames(frames) == [], _result_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source (phase 119 A1).
assert any(
s["path"] == REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
# The GROUNDED echo — the SECOND range reached the model's
# context and landed in the answer: ``Both parts read.`` + the
# ``TWO_RANGES_B_LINE`` marker text (the marker is range B's
# first line — inside the 80-char read-echo quote).
text = _last_brain(page).locator(".bubble").text_content() or ""
assert ANSWER_LEAD in text, text
assert B_MARKER_TEXT in text, text
# NO refusal surface anywhere in the rendered answer: the
# ``ALREADY_IN_CONTEXT`` line (the pre-phase false claim — its
# full text is the refusal the owner quoted) and the mock's
# broken-premise echo are both absent.
assert ALREADY_IN_CONTEXT not in text, text
assert "Premise broke" not in text, text
# --------------------------------------------------------------------------
# 2. The single-range behavior is unchanged: the phase-131 grep →
# read(offset/limit) dance still lands on the coverage-tracked
# server (the phase-131 expectation, this KB)
# --------------------------------------------------------------------------
def test_single_range_flow_unchanged(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_B)
_wait_settled(page)
frames = _frames(page)
# The phase-131 dance on the wire — IN ORDER, asserted on the
# frames' ARGS: the grep (the mock's canned pattern), then the
# range read — path = the doc's combined identity, offset = the
# marker's TRUE line (the mock parsed it from the grep result's
# first match line), limit = READ_LINES_LIMIT (30).
assert _tool_frames(frames) == [
{"type": "tool", "name": "grep", "argument": READ_LINES_PATTERN},
{
"type": "tool",
"name": "read",
"argument": SP,
"offset": LINE_MARKER_LINE,
"limit": READ_LINES_LIMIT,
},
], _tool_frames(frames)
# The 30-line range slice did NOT truncate (the default cap) —
# no ``tool_result`` frame (the phase-131 pinned order contract).
assert _result_frames(frames) == [], _result_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
assert any(
s["path"] == REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
# The GROUNDED echo — the phase-131 answer shape unchanged:
# ``Found it at line <N>`` (N == the marker's true line) + the
# marker text itself (the single-range read reached the model's
# context exactly as before the coverage rework).
text = _last_brain(page).locator(".bubble").text_content() or ""
assert f"Found it at line {LINE_MARKER_LINE}." in text, text
assert LINE_MARKER_TEXT in text, text