Phase 146 final verification pass complete — all green, tree untouched by me.
**Verified (final pass — both tasks already in `complete/`):**
- Scoped grep streams `{"type":"tool","name":"grep","argument":<pattern>,"path":<source/path>}`; bare grep/read/ls frames dump byte-identical (unit matrix in `test_sse_events.py` + builder pins in `test_chat_gate.py` L1364–1448)
- Tool line `🔎 Searching for <pattern> in <source/path>` (two `<code>` spans, `.grep-scope` sibling, no new emoji); bare line byte-identical (unit Node test + E2E)
- Status label "… is searching for X in Y" scoped-only; `path` persisted additively on the saved tool record and re-rendered on `/?chat=<id>` restore; pre-phase records restore byte-identically (E2E tests 2 & 3)
**Results (exact commands):**
- `uv run pytest --cov=app --cov-report=term-missing` → **3419 passed**, `app/` coverage **99%** (>90% ✓)
- `uv run pytest tests/e2e/test_search_tool_scope.py -v --no-cov` → **3 passed** in isolation ✓
- `uv run ruff check . && uv run pyright` → clean (0 errors) ✓
**Completion criteria:** all 6 ✓ (wire shape, UI line, status label, save/restore, gates, no completed-phase behavior change — full suite green includes all prior-phase pins).
**Documentation:** README.md (incl. its phase-146 section) and AGENTS.md reviewed in full against the codebase — verified consistent, no changes needed (`.env.example` ⊆ README config tables; no stale vars; phase-100 72rem rule in AGENTS.md §5 matches shipped CSS; no-CDN/smoke-DB/gpgsign claims all confirmed).
**Notable:** none — no defects found, no live-infrastructure touched, nothing committed per harness protocol.
**Next pending phase:** `147_sync_changed_only`.
669 lines
27 KiB
Python
669 lines
27 KiB
Python
"""Phase 146 (task 02) E2E (Playwright, mock-only): the search tool
|
|
line shows WHERE the LLM is searching — a scoped grep carries its
|
|
document.
|
|
|
|
Source: TODO.md L3 — "The "search" tool in the UI doesn't show where
|
|
the LLM is searching. This results in, what looks like, multiple
|
|
search tool calls with the same value, like the LLM is repeating
|
|
itself." A grep that re-searches a specific document now shows THAT
|
|
document in the chat's tool line and status label — live, persisted on
|
|
the saved turn's tool record (additively), and re-rendered on
|
|
restore — so repeated patterns with different scopes read as distinct
|
|
searches. Every bare-grep rendering (live line, status label, saved
|
|
record shape, restored line) is byte-identical to pre-phase.
|
|
|
|
Run in isolation (DB must be up: ``podman compose up -d db``):
|
|
|
|
uv run pytest tests/e2e/test_search_tool_scope.py -v --no-cov
|
|
|
|
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
|
|
deterministic SCOPED-SEARCH marker flow in ``tests/e2e/mock_llm.py``
|
|
(``SCOPED_SEARCH_TRIGGER`` — user message contains "search within one
|
|
document" **and** the system prompt carries the ``<tools>`` section):
|
|
|
|
1. request 1 (``tools`` offered, no search result yet) → streams ONLY
|
|
``tool_calls`` deltas calling ``grep`` with ``{"pattern":
|
|
SEARCH_PATTERN, "path": SCOPED_SEARCH_PATH}`` (id ``call_0``) —
|
|
the SCOPED form; the app's ``tool`` frame carries the scope as its
|
|
additive ``path`` field (phase 146 task 01);
|
|
2. request 2 (a ``tool``-role search result — the agent's REAL scoped
|
|
grep matched the fixture's sentinel line) → the content answer
|
|
``Found <first matched line's content up to 80 chars>``.
|
|
|
|
The EXISTING bare-grep flow (``SEARCH_TRIGGER`` — "search your
|
|
documents") is untouched and is this suite's byte-identity pin
|
|
(test 2).
|
|
|
|
KB fixture — ONE document, imported through the real importer against
|
|
``tests/fixtures/search_docs/`` with the deterministic mock
|
|
embeddings (the ``test_search_tool.py`` pattern, reused as-is):
|
|
|
|
* ``search_docs/reese-notes.md`` — homelab kubernetes backup notes;
|
|
both questions cosine ≈0.51 against it (mock bag-of-words, well
|
|
past the E2E 0.30 threshold, and they FTS-match too) → grounded, so
|
|
the HIGH prompt carries the ``<tools>`` section. Its line 6 carries
|
|
the sentinel ``reese-sentinel-42`` exactly once — the deterministic
|
|
match of the agent's (bare or scoped) grep.
|
|
|
|
Test → observable mapping (Playwright Mapping Rule; the story is the
|
|
owner's TODO item — one Playwright file per story, A16):
|
|
1. ``test_scoped_grep_line_and_status_show_the_scope`` — the scoped
|
|
question → the SSE ``tool`` frame is
|
|
``{"type": "tool", "name": "grep", "argument": <pattern>, "path":
|
|
<source/path>}`` (the scope rides the wire); while the turn is
|
|
live, the status carries "… is searching for <pattern> in
|
|
<source/path>"; the tool line reads
|
|
``🔎 Searching for <pattern> in <source/path>`` with TWO
|
|
``<code>`` spans (pattern, then scope) and the `` in `` text node
|
|
between them (the textContent structure, not a regex over the
|
|
whole line — the phase-139 range-suffix pin style); the answer
|
|
quotes the matched line (the scoped grep found it).
|
|
2. ``test_bare_grep_is_byte_identical`` — the EXISTING bare-grep
|
|
question (the phase-68/70 scripted flow — the byte-identity pin) →
|
|
the SSE ``tool`` frame has NO ``path`` key (ABSENT, never null);
|
|
the tool line is exactly ``🔎 Searching for <pattern>`` (ONE
|
|
``<code>`` span, NO ``.grep-scope`` span); the status label is the
|
|
pre-phase "… is searching for <pattern>" (no `` in `` extension);
|
|
the auto-saved record keeps the pre-phase shape (NO ``path`` key);
|
|
and the record restored through ``/?chat=<id>`` renders the
|
|
pre-phase line byte-identically (a no-key record IS the pre-phase
|
|
saved shape — the backward-compat pin, end to end).
|
|
3. ``test_scoped_scope_survives_save_restore`` — the scoped question
|
|
→ the live line carries the scope → the conversation auto-saves
|
|
(phase 55, silent) with ``path`` on the tool record →
|
|
``/?chat=<id>`` re-renders the saved record → the scope comes
|
|
back through the SAME structure (two ``<code>`` spans + the
|
|
`` in `` text node + the ``.grep-scope`` span), with the
|
|
phase-117 disclosure FOLDED (a restored turn is one compact
|
|
"Tool call(s) (N)" line — the shape is unchanged).
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import re
|
|
import time
|
|
from pathlib import Path
|
|
from threading import Thread
|
|
from typing import Any
|
|
|
|
import httpx
|
|
from playwright.sync_api import Page, expect
|
|
from sqlalchemy import text
|
|
|
|
from app.config import Settings
|
|
from app.db import SessionLocal
|
|
from app.rag.importer import ImportSummary, import_sources
|
|
from app.rag.llm import LLMClient
|
|
from e2e.auth_helpers import login
|
|
from tests.e2e.mock_llm import SEARCH_PATTERN, SEARCH_TRIGGER
|
|
|
|
REPO = Path(__file__).resolve().parents[2]
|
|
FIXTURES = REPO / "tests" / "fixtures" / "search_docs"
|
|
|
|
#: The one imported document (the importer derives ``source`` from the
|
|
#: fixture root's name and ``path`` from the file's relative position)
|
|
#: — the SAME fixture the bare-grep scripted flow already greps.
|
|
SEED_SOURCE = "search_docs"
|
|
SEED_PATH = "reese-notes.md"
|
|
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
|
|
|
|
#: The fixture's sentinel line (line 6) — the mock's grep (bare or
|
|
#: scoped) matches it exactly once, and its ``text`` part is what the
|
|
#: "Found …" answer quotes. Pinned against the fixture file itself
|
|
#: below.
|
|
SENTINEL_LINE = f"The offsite vault passphrase marker is {SEARCH_PATTERN}."
|
|
FOUND_ANSWER = f"Found {SENTINEL_LINE[:80]}"
|
|
|
|
#: Carries ``SCOPED_SEARCH_TRIGGER`` and is on-topic (the SAME
|
|
#: vocabulary as the bare question — cosine ≈0.51 against the fixture
|
|
#: + FTS hits → HIGH gate, the ``<tools>`` section rides along).
|
|
SCOPED_QUESTION = (
|
|
"Search within one document: the vault passphrase marker in my "
|
|
"homelab kubernetes backup notes?"
|
|
)
|
|
assert "search within one document" in SCOPED_QUESTION.lower()
|
|
|
|
#: The EXISTING bare-grep question (byte-identical to
|
|
#: ``test_search_tool.py``'s — the phase-68/70 scripted flow is the
|
|
#: byte-identity pin, untouched by this phase).
|
|
BARE_QUESTION = (
|
|
"Search your documents for the vault passphrase marker in my homelab "
|
|
"kubernetes backup notes?"
|
|
)
|
|
assert SEARCH_TRIGGER in BARE_QUESTION.lower()
|
|
|
|
# The two questions must each carry EXACTLY their own trigger — no
|
|
# other mock marker flow may fire (the phase-68 collision guard).
|
|
for other in (
|
|
"search within one document",
|
|
"search your documents",
|
|
"use your tools",
|
|
"read two documents",
|
|
"write a long answer",
|
|
"think in paragraphs",
|
|
"think out loud",
|
|
"show the end of your notes",
|
|
"show me a table",
|
|
"show me the blockquote",
|
|
"fail then answer",
|
|
"always fail",
|
|
"embed fail once",
|
|
"fail first turn",
|
|
"partial then fail",
|
|
"emit raw tool markup",
|
|
"list the files in this directory",
|
|
"what are the correct llama.cpp arguments",
|
|
"drill down the tree",
|
|
"read the capped document",
|
|
"read the suggested document",
|
|
"show me the picture of",
|
|
"which diagram is in your notes",
|
|
"quote the image reference",
|
|
"read the lines",
|
|
"read the same file twice",
|
|
"read two ranges",
|
|
"original source link",
|
|
"answer first, then list, then think",
|
|
):
|
|
if other == "search within one document":
|
|
assert other in SCOPED_QUESTION.lower()
|
|
assert other not in BARE_QUESTION.lower()
|
|
elif other == "search your documents":
|
|
assert other not in SCOPED_QUESTION.lower()
|
|
assert other in BARE_QUESTION.lower()
|
|
else:
|
|
assert other not in SCOPED_QUESTION.lower(), other
|
|
assert other not in BARE_QUESTION.lower(), other
|
|
|
|
|
|
def _pin_fixture() -> None:
|
|
"""The fixture carries the sentinel on line 6, exactly once."""
|
|
content = (FIXTURES / SEED_PATH).read_text(encoding="utf-8")
|
|
lines = content.split("\n")
|
|
assert lines[5] == SENTINEL_LINE, lines[5]
|
|
assert sum(SEARCH_PATTERN in line for line in lines) == 1
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# DB seeding (TRUNCATE-then-import via the real importer — the
|
|
# test_search_tool.py pattern, reused as-is: the same admin-import
|
|
# pipeline the admin Sources page drives, against the session app's
|
|
# DB)
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
async def _import_fixtures(mock_port: int) -> ImportSummary:
|
|
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
|
|
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
|
return await import_sources([FIXTURES], LLMClient(settings))
|
|
|
|
|
|
def _run_in_thread(coro: Any) -> Any:
|
|
"""Run a coroutine on a worker thread.
|
|
|
|
Playwright's sync API keeps an asyncio loop running on the test
|
|
thread, so ``asyncio.run`` cannot be called directly from a test
|
|
body (the established house helper).
|
|
"""
|
|
box: dict[str, Any] = {}
|
|
|
|
def runner() -> None:
|
|
try:
|
|
box["value"] = asyncio.run(coro)
|
|
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
|
|
box["error"] = e
|
|
|
|
t = Thread(target=runner)
|
|
t.start()
|
|
t.join()
|
|
if "error" in box:
|
|
raise box["error"]
|
|
return box["value"]
|
|
|
|
|
|
def _reset_db(mock_port: int) -> ImportSummary:
|
|
"""Truncate the KB (plus the prompt-shaping tables), then re-import.
|
|
|
|
``steering_notes`` / ``kb_overview`` are truncated too, so the HIGH
|
|
prompt is exactly ``<relevance>`` + ``<documents>`` + ``<tools>``
|
|
regardless of leftovers from other suites — byte-stable prompts,
|
|
byte-stable answers. ``saved_chats`` is truncated too: the
|
|
auto-save rows are global state, and a leftover row with the same
|
|
auto-title would race ``_wait_saved_row`` (the E2E isolation
|
|
pattern — this suite runs in isolation, A16)."""
|
|
with SessionLocal() as db:
|
|
db.execute(
|
|
text(
|
|
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
|
"kb_overview, saved_chats"
|
|
)
|
|
)
|
|
db.commit()
|
|
summary = _run_in_thread(_import_fixtures(mock_port))
|
|
assert summary is not None and summary.added == 1, summary
|
|
return summary
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# Page + API helpers (the test_search_tool.py / test_read_range_tool_lines
|
|
# house patterns)
|
|
# --------------------------------------------------------------------------
|
|
|
|
#: Records every value #send-status takes during the turn — the
|
|
#: transient "… is searching for <pattern> in <source/path>"
|
|
#: calling-tool state is held only from the first `tool` frame until
|
|
#: the first answer delta, so the pre-submit observer is the
|
|
#: deterministic source of truth for it (no polling race — the
|
|
#: phase-44 task-03 flake fix).
|
|
STATUS_RECORDER = """
|
|
() => {
|
|
if (window.__statusesInstalled) return;
|
|
window.__statusesInstalled = true;
|
|
window.__statuses = [];
|
|
const el = document.querySelector('#send-status');
|
|
if (!el) return;
|
|
const rec = (v) => {
|
|
const l = window.__statuses;
|
|
if (!l.length || l[l.length - 1] !== v) l.push(v);
|
|
};
|
|
rec(el.textContent);
|
|
new MutationObserver(() => rec(el.textContent)).observe(el, {
|
|
childList: true,
|
|
subtree: true,
|
|
});
|
|
}
|
|
"""
|
|
|
|
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
|
|
#: (a response clone read in the background) — wire-level assertions
|
|
#: for the ``tool`` frames, independent of the UI rendering.
|
|
SSE_HOOK = """
|
|
() => {
|
|
if (window.__sseInstalled) return;
|
|
window.__sseInstalled = true;
|
|
window.__sseFrames = [];
|
|
const origFetch = window.fetch;
|
|
window.fetch = async function (...args) {
|
|
const res = await origFetch.apply(this, args);
|
|
try {
|
|
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
|
|
if (url.includes('/api/chat')) {
|
|
res.clone().text().then((bodyText) => {
|
|
for (const block of bodyText.split('\\n\\n')) {
|
|
const line = block.trim();
|
|
if (line.startsWith('data: ')) {
|
|
window.__sseFrames.push(line.slice(6));
|
|
}
|
|
}
|
|
});
|
|
}
|
|
} catch (e) { /* non-clonable responses: ignored */ }
|
|
return res;
|
|
};
|
|
}
|
|
"""
|
|
|
|
|
|
def _install_page_hooks(page: Page) -> None:
|
|
"""Install all hooks on the loaded page (post-goto, pre-submit)."""
|
|
page.evaluate(SSE_HOOK)
|
|
page.evaluate(STATUS_RECORDER)
|
|
|
|
|
|
def _frames(page: Page) -> list[dict]:
|
|
"""The captured SSE frames, once the hook's background read settles.
|
|
|
|
The hook reads ``res.clone().text()`` in a background promise that
|
|
resolves right after the stream closes — poll briefly until the
|
|
final ``done`` frame lands (fail loud if the hook captured nothing).
|
|
"""
|
|
deadline = time.monotonic() + 10.0
|
|
while True:
|
|
raw = page.evaluate("() => window.__sseFrames || []")
|
|
parsed = [json.loads(line) for line in raw if line]
|
|
if any(f.get("type") == "done" for f in parsed):
|
|
return parsed
|
|
if time.monotonic() > deadline:
|
|
raise AssertionError(
|
|
f"SSE hook captured no `done` frame (frames so far: "
|
|
f"{len(parsed)}) — hook install failed?"
|
|
)
|
|
time.sleep(0.05)
|
|
|
|
|
|
def _tool_frames(frames: list[dict]) -> list[dict]:
|
|
return [f for f in frames if f.get("type") == "tool"]
|
|
|
|
|
|
def _submit(page: Page, question: str) -> None:
|
|
page.evaluate("window.__sseFrames = []")
|
|
page.fill("#message-input", question)
|
|
page.click("#send-btn")
|
|
# The user bubble lands synchronously with the submit handler.
|
|
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
|
|
|
|
|
def _wait_settled(page: Page) -> None:
|
|
"""The turn is complete: answer text in the bubble, button recovered.
|
|
|
|
Phase 48: the label assertion carries the settle wait with an
|
|
explicit timeout — the in-flight button is the enabled Stop control
|
|
(never disabled), so ``to_be_enabled`` no longer blocks until the
|
|
turn settles, and Playwright expect's default (5s) does not inherit
|
|
the page default."""
|
|
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
|
|
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
|
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
|
|
|
|
|
def _assert_no_error_banner(page: Page) -> None:
|
|
"""The turn must settle WITHOUT the terminal error banner (the red
|
|
role=alert error banner — the KB-offline banner is a separate,
|
|
non-error state)."""
|
|
banner = page.locator("#kb-banner")
|
|
expect(banner).to_be_hidden()
|
|
expect(banner).not_to_have_attribute("role", "alert")
|
|
expect(banner).not_to_have_class(re.compile(r"is-error"))
|
|
|
|
|
|
def _tool_line(page: Page):
|
|
"""The turn's single tool line (the last .tool-call in the last
|
|
brain message)."""
|
|
return page.locator(".msg.brain .tool-call").last
|
|
|
|
|
|
def _assert_scoped_line(line, timeout: int = 30_000) -> None:
|
|
"""THE SUBJECT (shared by live + restore): the scoped grep's line
|
|
reads ``🔎 Searching for <pattern> in <source/path>`` — TWO
|
|
``<code>`` spans (pattern, then scope) with the `` in `` text node
|
|
between them (the textContent structure, not a regex over the
|
|
whole line — the phase-139 range-suffix pin style). State
|
|
assertions only (they retry until the boot load renders the line,
|
|
and they pass on the HIDDEN line of a folded disclosure —
|
|
phase 117: a restored turn renders folded)."""
|
|
expect(line).to_have_count(1, timeout=timeout)
|
|
expect(line).to_have_text(
|
|
f"🔎 Searching for {SEARCH_PATTERN} in {SEED_SP}", timeout=timeout
|
|
)
|
|
codes = line.locator("code")
|
|
expect(codes).to_have_count(2, timeout=timeout)
|
|
expect(codes.nth(0)).to_have_text(SEARCH_PATTERN, timeout=timeout)
|
|
expect(codes.nth(1)).to_have_text(SEED_SP, timeout=timeout)
|
|
scope_span = line.locator(".grep-scope")
|
|
expect(scope_span).to_have_count(1, timeout=timeout)
|
|
expect(scope_span).to_have_text(f" in {SEED_SP}", timeout=timeout)
|
|
|
|
|
|
def _assert_bare_line(line, timeout: int = 30_000) -> None:
|
|
"""The byte-identity pin: the bare grep's line is EXACTLY the
|
|
pre-phase line — ONE ``<code>`` span (the pattern), NO
|
|
``.grep-scope`` span, no extra bytes. (State assertions only —
|
|
same retry/hidden-line contract as ``_assert_scoped_line``.)"""
|
|
expect(line).to_have_count(1, timeout=timeout)
|
|
expect(line).to_have_text(f"🔎 Searching for {SEARCH_PATTERN}", timeout=timeout)
|
|
expect(line.locator("code")).to_have_count(1, timeout=timeout)
|
|
expect(line.locator("code")).to_have_text(SEARCH_PATTERN, timeout=timeout)
|
|
expect(line.locator(".grep-scope")).to_have_count(0, timeout=timeout)
|
|
|
|
|
|
def _admin_cookies(page: Page) -> dict[str, str]:
|
|
"""The signed session cookies the browser holds after a form
|
|
login — used to call the admin API with plain httpx."""
|
|
return {
|
|
c["name"]: c["value"]
|
|
for c in page.context.cookies()
|
|
if "name" in c and "value" in c
|
|
}
|
|
|
|
|
|
def _auto_title(question: str) -> str:
|
|
"""The phase-50 auto-title convention: the first question,
|
|
whitespace-collapsed, capped at 120 chars."""
|
|
return " ".join(question.split())[:120]
|
|
|
|
|
|
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
|
|
return next((c for c in rows if c["title"] == title), None)
|
|
|
|
|
|
def _wait_saved_row(
|
|
app_url: str, cookies: dict[str, str], title: str, messages: int = 2
|
|
) -> dict[str, Any]:
|
|
"""Wait for the auto-saved row (phase 55: auto-saves are SILENT —
|
|
so there is no status line to wait on). Poll the admin list until
|
|
the row with the conversation's auto-title appears with the
|
|
expected message count."""
|
|
deadline = time.monotonic() + 15
|
|
last: dict[str, Any] | None = None
|
|
while time.monotonic() < deadline:
|
|
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
|
|
assert r.status_code == 200
|
|
last = _find_row(r.json()["chats"], title)
|
|
if last is not None and last["message_count"] >= messages:
|
|
return last
|
|
time.sleep(0.2)
|
|
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
|
|
|
|
|
|
def _saved_tool_record(
|
|
app_url: str, cookies: dict[str, str], chat_id: str
|
|
) -> dict[str, Any]:
|
|
"""The last brain record's single tool record, as the API returns
|
|
it (the saved ``bor.chat.v1`` JSONB through the boundary)."""
|
|
r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
|
assert r.status_code == 200, r.text
|
|
messages = r.json()["messages"]
|
|
brain = [m for m in messages if m["who"] == "brain"]
|
|
assert len(brain) == 1, messages
|
|
tools = brain[0].get("tools")
|
|
assert isinstance(tools, list) and len(tools) == 1, messages
|
|
return tools[0]
|
|
|
|
|
|
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
|
|
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
|
|
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 1. The scoped grep: the tool line + status label show the scope
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def test_scoped_grep_line_and_status_show_the_scope(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_pin_fixture()
|
|
page.set_default_timeout(30_000)
|
|
_reset_db(mock_llm)
|
|
login(page, app_url, next="/")
|
|
_install_page_hooks(page)
|
|
|
|
_submit(page, SCOPED_QUESTION)
|
|
# The "calling tool" STATUS window is transient: the first `tool`
|
|
# frame sets #send-status and it holds until the FIRST answer delta
|
|
# (the agent loop completes before the answer stream) — the
|
|
# pre-submit MutationObserver records are the deterministic source
|
|
# of truth for the status transitions (the phase-44 flake fix).
|
|
_wait_settled(page)
|
|
|
|
# The scoped STATUS label (while the turn is live — recorded before
|
|
# the first delta replaced it): "… is searching for <pattern> in
|
|
# <source/path>", AFTER the "… is thinking" state.
|
|
statuses = page.evaluate("() => window.__statuses")
|
|
i_search = next(
|
|
(
|
|
i
|
|
for i, s in enumerate(statuses)
|
|
if s.endswith(f"is searching for {SEARCH_PATTERN} in {SEED_SP}")
|
|
),
|
|
None,
|
|
)
|
|
assert i_search is not None, statuses
|
|
i_think = next(
|
|
(i for i, s in enumerate(statuses) if "is thinking" in s), None
|
|
)
|
|
assert i_think is not None and i_think < i_search, statuses
|
|
|
|
# Wire level: exactly ONE `tool` frame — the grep carrying the
|
|
# PATTERN as its argument AND the scope as the additive `path`
|
|
# field (phase 146 task 01) — ahead of the first `delta` frame.
|
|
frames = _frames(page)
|
|
assert _tool_frames(frames) == [
|
|
{
|
|
"type": "tool",
|
|
"name": "grep",
|
|
"argument": SEARCH_PATTERN,
|
|
"path": SEED_SP,
|
|
}
|
|
]
|
|
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
|
|
assert all(
|
|
i < first_delta for i, f in enumerate(frames) if f.get("type") == "tool"
|
|
)
|
|
done = next(f for f in frames if f.get("type") == "done")
|
|
assert done["deflected"] is False
|
|
|
|
# THE SUBJECT: ONE tool line — the scope rendered after the
|
|
# pattern (two <code> spans + the " in " text node between them).
|
|
_assert_scoped_line(_tool_line(page))
|
|
|
|
# The answer quotes the MATCHED LINE — the agent's REAL scoped grep
|
|
# found the sentinel in the named document (the mock's echo).
|
|
bubble = page.locator(".msg.brain .bubble").last
|
|
expect(bubble).to_contain_text(FOUND_ANSWER)
|
|
|
|
# The turn settled to idle WITHOUT the error banner.
|
|
_assert_no_error_banner(page)
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 2. The byte-identity pin: a BARE grep renders exactly as pre-phase —
|
|
# on the wire, on the line, in the status, and in the saved record
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def test_bare_grep_is_byte_identical(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_pin_fixture()
|
|
page.set_default_timeout(30_000)
|
|
_reset_db(mock_llm)
|
|
login(page, app_url, next="/")
|
|
_install_page_hooks(page)
|
|
|
|
_submit(page, BARE_QUESTION)
|
|
_wait_settled(page)
|
|
|
|
# Wire level: the bare grep's frame is EXACTLY the pre-phase shape —
|
|
# the `path` key is ABSENT (never null).
|
|
frames = _frames(page)
|
|
tool_frames = _tool_frames(frames)
|
|
assert len(tool_frames) == 1, tool_frames
|
|
assert "path" not in tool_frames[0], tool_frames[0]
|
|
assert tool_frames[0] == {
|
|
"type": "tool",
|
|
"name": "grep",
|
|
"argument": SEARCH_PATTERN,
|
|
}, tool_frames[0]
|
|
done = next(f for f in frames if f.get("type") == "done")
|
|
assert done["deflected"] is False
|
|
|
|
# The tool line: byte-identical to pre-phase (one <code>, no scope
|
|
# span — the phase-70/87/117 shape).
|
|
_assert_bare_line(_tool_line(page))
|
|
|
|
# The status label: the EXACT pre-phase "… is searching for
|
|
# <pattern>" — no " in <scope>" extension for a bare grep.
|
|
statuses = page.evaluate("() => window.__statuses")
|
|
assert any(s.endswith(f"is searching for {SEARCH_PATTERN}") for s in statuses), (
|
|
statuses
|
|
)
|
|
assert not any(
|
|
f"is searching for {SEARCH_PATTERN} in" in s for s in statuses
|
|
), statuses
|
|
|
|
# The saved record: the auto-save (phase 55, silent) persists the
|
|
# toolAcc record — a bare grep keeps the EXACT pre-phase shape (NO
|
|
# `path` key — the additive key is stamped only when the frame
|
|
# carried one).
|
|
cookies = _admin_cookies(page)
|
|
row = _wait_saved_row(app_url, cookies, _auto_title(BARE_QUESTION))
|
|
try:
|
|
record = _saved_tool_record(app_url, cookies, row["id"])
|
|
assert "path" not in record, record
|
|
assert record["name"] == "grep", record
|
|
assert record["argument"] == SEARCH_PATTERN, record
|
|
|
|
# And the no-key record (== the pre-phase saved shape) restores
|
|
# byte-identically through /?chat=<id> (the phase-50
|
|
# backward-compat rule, end to end). The restored line rides the
|
|
# FOLDED disclosure (phase 117) — the state assertions pass on
|
|
# the hidden line and retry until the boot load renders it.
|
|
page.goto(app_url + f"/?chat={row['id']}")
|
|
_assert_bare_line(_tool_line(page))
|
|
finally:
|
|
_delete_chat(app_url, cookies, row["id"])
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 3. The scope survives save → restore: the saved record's additive
|
|
# `path` re-renders the same line on /?chat=<id> (the phase-117
|
|
# disclosure folded — the shape is unchanged)
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def test_scoped_scope_survives_save_restore(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_pin_fixture()
|
|
page.set_default_timeout(30_000)
|
|
_reset_db(mock_llm)
|
|
login(page, app_url, next="/")
|
|
_install_page_hooks(page)
|
|
|
|
_submit(page, SCOPED_QUESTION)
|
|
_wait_settled(page)
|
|
|
|
# The LIVE line carries the scope (the save point persists the
|
|
# toolAcc record — {name, argument, path}).
|
|
_assert_scoped_line(_tool_line(page))
|
|
|
|
# The conversation auto-saved (phase 55 — silent). Wait for the row
|
|
# by its auto-title (the first question), then pin the persisted
|
|
# record: the scope rides `path` ADDITIVELY on the tool record.
|
|
cookies = _admin_cookies(page)
|
|
row = _wait_saved_row(app_url, cookies, _auto_title(SCOPED_QUESTION))
|
|
try:
|
|
record = _saved_tool_record(app_url, cookies, row["id"])
|
|
assert record["name"] == "grep", record
|
|
assert record["argument"] == SEARCH_PATTERN, record
|
|
assert record["path"] == SEED_SP, record
|
|
|
|
# Restore: /?chat=<id> boots into the saved conversation
|
|
# through the SAME renderStoredMessage loop as the local
|
|
# restore — the saved record's additive `path` re-renders the
|
|
# scope through the SAME structure (the phase-139 save →
|
|
# restore pattern). The restored line rides the FOLDED
|
|
# disclosure (phase 117) — the state assertions pass on the
|
|
# hidden line and retry until the boot load renders it.
|
|
page.goto(app_url + f"/?chat={row['id']}")
|
|
_assert_scoped_line(_tool_line(page))
|
|
|
|
# Phase 117 (D3): a restored turn renders FOLDED — one compact
|
|
# "Tool call(s) (N)" line (the disclosure shape is unchanged by
|
|
# this phase: the `open` attribute is ABSENT).
|
|
disclosure = page.locator(".msg.brain .tool-calls-disclosure").last
|
|
expect(disclosure).not_to_have_attribute("open", timeout=30_000)
|
|
expect(disclosure.locator("summary")).to_have_text(
|
|
"Tool call (1)", timeout=30_000
|
|
)
|
|
|
|
# The answer text restored too (the record is intact).
|
|
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
|
|
FOUND_ANSWER, timeout=15_000
|
|
)
|
|
finally:
|
|
_delete_chat(app_url, cookies, row["id"])
|