**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**
- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
1429 lines
59 KiB
Python
1429 lines
59 KiB
Python
"""Phase 182 E2E (Playwright): the exact text an answer shows, byte for byte.
|
||
|
||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_verbatim_show_text.py -v --no-cov
|
||
|
||
The phase's incident (TODO.md, 2026-10-14): an answer that RE-TYPES a file
|
||
drifts from it by a character and the user cannot tell. The fix is that the
|
||
model only ever names a LOCATION — the ``show_text`` tool or a
|
||
`` ```bor <source/path> [range] `` locator fence in the answer — and the
|
||
SERVER resolves the bytes out of the index and displays them. This suite is
|
||
the end-to-end proof of that contract on the real user path, including the
|
||
clipboard truth (the mock LLM's scripted branches live in
|
||
``tests/e2e/mock_llm.py``; the KB is seeded straight into the shared e2e
|
||
Postgres so the fixtures' exact text is the assertion baseline):
|
||
|
||
* **App A (``BOR_ANSWER_TEXT=true``, ``BOR_ANSWER_IMAGES=true``)**
|
||
- *tool flow* — ``show me the exact text of <source/path>`` drives the
|
||
mock's scripted ``show_text`` call, then an answer whose fence carries
|
||
a DELIBERATELY CORRUPTED copy of the slice (one flipped character + a
|
||
trailing space + a sentinel line). The panel is the FILE: the caption's
|
||
extent is the server's, the body equals the stored text, the gutter runs
|
||
1…N, the VERBATIM badge and the viewer link are there — and NONE of the
|
||
model's bytes are anywhere in the DOM (no `` ```bor ``, no sentinel, no
|
||
flipped line). One ``.verbatim`` per locator (no second copy from the
|
||
append path), no placeholder left pending, the answer's ordinary
|
||
`` ```bash `` block wrapped as ``.code-block`` with its own Copy and NO
|
||
badge (locked A5). Wire pins: exactly FIVE tools offered (the three base
|
||
+ ``show_image`` + ``show_text``), exactly one executed call, and — from
|
||
an in-page fetch tap over the real ``/api/chat`` stream — the turn's SSE
|
||
``type`` set stays inside the pre-phase seven with ``answer_text``
|
||
riding INSIDE ``done`` (no eighth event type).
|
||
- *append path* — the same turn with ``describe it without a block`` (no
|
||
fence in the answer): the shown block renders as a panel APPENDED to the
|
||
bubble body, below the answer text and ABOVE the source clusters (the
|
||
``.msg-meta`` family — the phase-124/130 DOM order).
|
||
- *clipboard truth* — ``.verbatim-copy`` puts the server string on the
|
||
clipboard EXACTLY (read back through the async clipboard API), and the
|
||
ordinary block's Copy puts the model's line on it exactly.
|
||
- *standalone fence* — ``put the block inline`` answers with one locator
|
||
fence and ZERO tool calls; the panel resolves from the answer text
|
||
(lines 2–3 of the note file, the server's extent).
|
||
- *honest failures* — a locator with no document, and a range past the end
|
||
(the tool refuses with ``OFFSET_PAST_END``): the panel is the pinned
|
||
sentence, with no badge, no body, no line numbers and none of the
|
||
model's claimed numbers on screen; the record carries no ``answer_text``.
|
||
- *wrong range corrected* — a fence claiming ``2-999`` on a 9-line file:
|
||
the panel shows the SERVER's extent (``lines 2–9 of 9``) and ``999``
|
||
appears nowhere.
|
||
- *reload, history, share* — the panel survives the reload (the
|
||
``bor.chat.v1`` restore), re-renders from the SERVER's saved row when the
|
||
chat is opened off ``/history.html`` (``/?chat=<id>``, with local storage
|
||
cleared first so the pass cannot be vacuous), and renders identically on
|
||
the shared page for an anonymous reader (the one owner exception to that
|
||
page's zero-controls rule: the panel's Copy button).
|
||
- *image regression* — a ``show_image`` turn still shows exactly ONE
|
||
``.answer-image`` and NO panel, and ``show_text`` on an image document
|
||
is REFUSED by name (the refusal copy naming ``show_image`` reaches the
|
||
model; LOCKED A6) instead of silently converting it.
|
||
- *phone* — at 390 px the panel's long line scrolls INSIDE the panel (no
|
||
horizontal page scroll) and both Copy controls clear the 44 px touch
|
||
floor.
|
||
|
||
* **App B (``BOR_ANSWER_TEXT=false`` explicitly)** — the kill switch:
|
||
exactly FOUR tools offered (no ``show_text``), the show-me trigger never
|
||
fires (the clause is gone, so the mock stays pre-phase: one call, no tool,
|
||
no panel anywhere), and the `` ```bor `` fence the inline trigger still
|
||
lands never paints the model's bytes — with no ``answer_text`` on the wire
|
||
it settles to the same honest sentence (locked A7 is flag-independent: the
|
||
client never renders the model's re-typed body), while the ordinary code
|
||
block STILL gets its Copy button (A5 is flag-independent too). The saved
|
||
record carries no ``answer_text`` key.
|
||
|
||
DB isolation: the suites share one Postgres; ``saved_chats`` survives the
|
||
truncation (the house pattern), so every question text here is UNIQUE (its
|
||
auto-title is therefore unique) and the row lookups key on it.
|
||
:func:`_cleanup_saved_rows` deletes this suite's rows at both module
|
||
boundaries. App B re-seeds after App A's tests (file order) and never looks
|
||
back.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import base64
|
||
import hashlib
|
||
import json
|
||
import os
|
||
import re
|
||
import subprocess
|
||
import sys
|
||
import time
|
||
from collections.abc import Iterator
|
||
from datetime import UTC, datetime
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import httpx
|
||
import pytest
|
||
from playwright.sync_api import Browser, Page, expect
|
||
from sqlalchemy import text
|
||
|
||
from app.config import Settings
|
||
from app.db import SessionLocal
|
||
from app.models import Chunk, Document
|
||
from e2e.auth_helpers import login
|
||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||
from e2e.mock_llm import (
|
||
IMAGE_DESCRIPTION_ANSWER,
|
||
SHOW_TEXT_MISSING_TARGET,
|
||
SHOW_TEXT_MODEL_BODY_MARKER,
|
||
SHOW_TEXT_PLAIN_BODY,
|
||
corrupt_verbatim_line,
|
||
embed_text,
|
||
)
|
||
|
||
REPO = Path(__file__).resolve().parents[2]
|
||
|
||
#: The one source the suite seeds (a name no other suite uses).
|
||
SOURCE = "baselab"
|
||
DOC_A_PATH = "quadlets/gufo.container"
|
||
DOC_B_PATH = "notes/verbatim-b.md"
|
||
DOC_IMG_PATH = "diagrams/network-map.png"
|
||
|
||
#: Fixture A: a quadlet unit file — nine lines with an obvious per-line
|
||
#: identity, line 6 deliberately long so the panel MUST scroll sideways on a
|
||
#: phone instead of stretching the bubble (PLAN §7.1 / phase 116 A1). NO
|
||
#: trailing newline: the resolver's ``splitlines`` drops it, so the panel's
|
||
#: text equals this constant byte for byte.
|
||
DOC_A_CONTENT = "\n".join(
|
||
[
|
||
"# baselab quadlets — the gufo container's exact text, stored in the KB",
|
||
"[Container]",
|
||
"ContainerName=gufo",
|
||
"Image=ghcr.io/home-operations/gufo:2.4.1",
|
||
"Network=baselab",
|
||
(
|
||
'Environment=GUFO_MODE=exact GUFO_TEXT_SOURCE=baselab '
|
||
'GUFO_NOTE="the exact text of this baselab quadlet is what the '
|
||
'user copies — gufo container, line six, long enough that a phone '
|
||
'panel must scroll sideways instead of stretching the answer bubble"'
|
||
),
|
||
"Volume=/var/home/baselab/gufo:/data:z",
|
||
"RestartPolicy=always",
|
||
"# The baselab quadlet above is the gufo container's exact text.",
|
||
]
|
||
)
|
||
DOC_A_SUMMARY = (
|
||
"The exact text of the baselab quadlet for the gufo container: image, "
|
||
"network, ports, volume and restart policy."
|
||
)
|
||
|
||
#: Fixture B: a note whose text carries the shapes a markdown renderer likes
|
||
#: to eat — a URL, backticks and an em dash — so the panel's escaping is
|
||
#: proved against real content. Lines 2–3 are the standalone fence's target.
|
||
DOC_B_CONTENT = "\n".join(
|
||
[
|
||
"# Verbatim notes B — the baselab inline block reference",
|
||
(
|
||
"The inline block is quoted — exactly — from "
|
||
"https://lab.example/runbook#verbatim-b"
|
||
),
|
||
(
|
||
"Run `podman generate quadlet gufo`, then paste the notes block "
|
||
"inline for baselab."
|
||
),
|
||
"GUFO_BLOCK_MARKER_4 — the fourth line of the verbatim notes file.",
|
||
(
|
||
"GUFO_BLOCK_MARKER_5 — the fifth line, with `backticks`, a URL "
|
||
"https://lab.example/tokens and a long tail so the panel scrolls."
|
||
),
|
||
]
|
||
)
|
||
DOC_B_SUMMARY = (
|
||
"Baselab notes about quoting a block inline — the runbook URL, the "
|
||
"podman command and the block markers."
|
||
)
|
||
|
||
#: A real 1×1 transparent PNG (the phase-122/124 fixture bytes): the image
|
||
#: document is the LOCKED-A6 refusal subject and the ``show_image`` figure.
|
||
PNG_1X1 = base64.b64decode(
|
||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
|
||
"AAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
|
||
)
|
||
|
||
#: The mock's deterministic answer tail (the turn-settled wait signal).
|
||
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
||
#: The unresolved state's pinned sentence (``frontend/assets/verbatim.js``'s
|
||
#: ``verbatimUnresolvedCopy`` — the copy the reader gets for a locator the
|
||
#: server did not resolve; mirrored here on purpose: it is the contract).
|
||
UNRESOLVED_COPY = "I could not verify this against the knowledge base"
|
||
|
||
# --- questions (every text UNIQUE: the shared DB keeps saved_chats) -------
|
||
|
||
#: The happy path: the tool flow + one fence per locator + the escaping proof.
|
||
SHOW_QUESTION = "Show me the exact text of `baselab/quadlets/gufo.container`?"
|
||
#: The append path: the answer carries NO fence (``SHOW_TEXT_NO_FENCE_MARKER``).
|
||
APPEND_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container` and "
|
||
"describe it without a block?"
|
||
)
|
||
#: The clipboard turn (a unique question; the panel is the same file).
|
||
COPY_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container`? (clipboard)"
|
||
)
|
||
#: The wrong-range half: the fence CLAIMS ``2-999`` of a nine-line file.
|
||
WRONG_RANGE_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container` 2-999?"
|
||
)
|
||
#: The past-the-end half: the tool refuses (``OFFSET_PAST_END``) and the
|
||
#: fence resolves to nothing — the honest state, no claimed numbers.
|
||
PAST_END_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container` 40-60?"
|
||
)
|
||
#: The standalone fence: zero tool calls, one fence, lines 2–3 of fixture B.
|
||
INLINE_QUESTION = (
|
||
"Please put the block inline for `baselab/notes/verbatim-b.md` 2-3?"
|
||
)
|
||
#: The hallucinated locator (no document, no tool call).
|
||
MISSING_QUESTION = (
|
||
"Show me a block I cannot find in the notes — "
|
||
"`baselab/notes/the-file-that-is-not-there.md`?"
|
||
)
|
||
#: LOCKED A6: ``show_text`` pointed at the image document.
|
||
IMAGE_TARGET_QUESTION = (
|
||
"Show me the exact text of `baselab/diagrams/network-map.png`?"
|
||
)
|
||
#: The phase-124 image flow (one figure, no panel).
|
||
PICTURE_QUESTION = "Show me the picture of the homelab network diagram?"
|
||
#: The reload/share turn.
|
||
PERSIST_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container`? (persist)"
|
||
)
|
||
#: The phone turn.
|
||
MOBILE_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container`? (phone)"
|
||
)
|
||
#: App B: the trigger with the clause gone (no tool, no fence).
|
||
OFF_SHOW_QUESTION = (
|
||
"Show me the exact text of `baselab/quadlets/gufo.container`? (feature off)"
|
||
)
|
||
#: App B: the fence the client must still refuse to paint.
|
||
OFF_INLINE_QUESTION = (
|
||
"Please put the block inline for `baselab/notes/verbatim-b.md` 2-3? (feature off)"
|
||
)
|
||
|
||
DOC_A_LINES = DOC_A_CONTENT.split("\n")
|
||
DOC_B_LINES = DOC_B_CONTENT.split("\n")
|
||
|
||
#: The SSE ``type`` values the pre-phase stream already carried
|
||
#: (``app/schemas.py``'s chat events): the phase added NO eighth type.
|
||
PRE_PHASE_EVENT_TYPES = {
|
||
"delta",
|
||
"thinking",
|
||
"tool",
|
||
"tool_result",
|
||
"done",
|
||
"error",
|
||
"retry",
|
||
}
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Module apps: verbatim ON (App A) and the kill switch OFF (App B).
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def _app_env(
|
||
mock_llm: int,
|
||
*,
|
||
answer_text: str,
|
||
answer_images: str,
|
||
scratch: Path,
|
||
) -> dict[str, str]:
|
||
"""The conftest app env (leak guards included) with the phase-182 knobs
|
||
pinned EXPLICITLY — process env ranks above an operator's local
|
||
gitignored ``.env``, so neither direction of the contract can leak in
|
||
(the phase-122/124 default-app convention). ``BOR_IMAGES``/
|
||
``BOR_ANSWER_IMAGES`` stay ON on both apps so the FIVE-tool offer (App
|
||
A) and the FOUR-tool offer (App B) are the only difference between them,
|
||
and the image-document fixtures behave identically on either."""
|
||
env = dict(os.environ)
|
||
env.pop("DEBUGPY", None)
|
||
env["BOR_ENVIRONMENT"] = "e2e"
|
||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||
env["BOR_LLM_BASE_URL"] = (
|
||
"https://aipi.reeseapps.com/v1"
|
||
if USE_REAL_LLM
|
||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||
)
|
||
# Mock-calibrated gate (conftest pattern) — every question above grounds
|
||
# on these values against the two text fixtures.
|
||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
|
||
env.setdefault(
|
||
"BOR_DATABASE_URL",
|
||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||
)
|
||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||
# Leak guards (conftest pattern).
|
||
env["BOR_GIT_SOURCES"] = ""
|
||
env["BOR_DOCS_REPO"] = ""
|
||
env["BOR_SUGGESTIONS"] = json.dumps(Settings.model_fields["suggestions"].default)
|
||
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
|
||
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
|
||
# The phase under test — pinned EXPLICITLY per app, plus the caps at the
|
||
# CODE defaults (the suite's fixtures are far under both).
|
||
env["BOR_ANSWER_TEXT"] = answer_text
|
||
env["BOR_SHOW_TEXT_MAX_CHARS"] = str(
|
||
Settings.model_fields["show_text_max_chars"].default
|
||
)
|
||
env["BOR_SHOW_TEXT_MAX_BLOCKS"] = str(
|
||
Settings.model_fields["show_text_max_blocks"].default
|
||
)
|
||
env["BOR_IMAGES"] = "true"
|
||
env["BOR_ANSWER_IMAGES"] = answer_images
|
||
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
|
||
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
|
||
env["BOR_IMAGE_DIR"] = str(scratch / "images")
|
||
return env
|
||
|
||
|
||
def _serve(
|
||
mock_llm: int, port: int, env: dict[str, str]
|
||
) -> Iterator[str]:
|
||
proc = subprocess.Popen(
|
||
[
|
||
sys.executable,
|
||
"-m",
|
||
"uvicorn",
|
||
"app.main:app",
|
||
"--host",
|
||
"127.0.0.1",
|
||
"--port",
|
||
str(port),
|
||
"--log-level",
|
||
"warning",
|
||
],
|
||
cwd=REPO,
|
||
env=env,
|
||
)
|
||
try:
|
||
_wait_http(f"http://127.0.0.1:{port}/api/health")
|
||
yield f"http://127.0.0.1:{port}"
|
||
finally:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=10)
|
||
except subprocess.TimeoutExpired:
|
||
proc.kill()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_server(
|
||
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[str]:
|
||
"""App A — ``BOR_ANSWER_TEXT=true`` (explicit, the code default) with
|
||
images on, so the offer is the full FIVE tools."""
|
||
scratch = tmp_path_factory.mktemp("bor_verbatim_on")
|
||
port = int(os.environ.get("E2E_APP_PORT_VERBATIM_ON", "8197"))
|
||
yield from _serve(
|
||
mock_llm,
|
||
port,
|
||
_app_env(mock_llm, answer_text="true", answer_images="true", scratch=scratch),
|
||
)
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_url(app_server: str) -> str:
|
||
return app_server
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def off_app_server(
|
||
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[str]:
|
||
"""App B — ``BOR_ANSWER_TEXT=false`` pinned explicitly (the kill switch;
|
||
only the negative tests start it, and they run AFTER App A's)."""
|
||
scratch = tmp_path_factory.mktemp("bor_verbatim_off")
|
||
port = int(os.environ.get("E2E_APP_PORT_VERBATIM_OFF", "8198"))
|
||
yield from _serve(
|
||
mock_llm,
|
||
port,
|
||
_app_env(mock_llm, answer_text="false", answer_images="true", scratch=scratch),
|
||
)
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def off_app_url(off_app_server: str) -> str:
|
||
return off_app_server
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# KB seeding (straight into the shared e2e Postgres — the fixtures' exact
|
||
# text is the assertion baseline) + DB/UI/mock helpers
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def _truncate_all() -> None:
|
||
"""The E2E isolation pattern (``test_answer_images.py``): the suites
|
||
share one Postgres, so the KB tables reset per module. ``saved_chats``
|
||
is never truncated — the unique question texts keep the row lookups
|
||
unambiguous and :func:`_cleanup_saved_rows` owns this suite's rows."""
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, kb_overview, "
|
||
"git_sources, sources_meta, folder_summaries"
|
||
)
|
||
)
|
||
db.commit()
|
||
|
||
|
||
def _make_doc(
|
||
content: str, path: str, title: str, summary: str, *, is_image: bool = False
|
||
) -> Document:
|
||
"""An un-persisted document row for the fixture KB."""
|
||
return Document(
|
||
source=SOURCE,
|
||
path=path,
|
||
full_path=f"/tmp/{path}",
|
||
title=title,
|
||
content=content,
|
||
content_hash=hashlib.sha256(content.encode()).hexdigest(),
|
||
summary=summary,
|
||
is_image=is_image,
|
||
indexed_at=datetime.now(UTC),
|
||
# Byte-stable dates (the prompt seed's date attribute and the tool
|
||
# result's date line are both rendered from this).
|
||
created_at=datetime(2026, 1, 15, tzinfo=UTC),
|
||
)
|
||
|
||
|
||
def _seed_kb(scratch: Path) -> dict[str, Any]:
|
||
"""The three-document KB (the quadlet, the note, the standalone image).
|
||
|
||
Every document carries ONE chunk embedded with the MOCK's own
|
||
:func:`embed_text` — the only way a question's tokens overlap the fixture
|
||
so the turn grounds instead of deflecting. The image document gets a real
|
||
bytes file at :attr:`image_path` (the public bytes route serves it, so the
|
||
figure renders): the upload flow is phase 124's subject, not this phase's.
|
||
"""
|
||
_truncate_all()
|
||
image_file = scratch / "network-map.png"
|
||
image_file.write_bytes(PNG_1X1)
|
||
with SessionLocal() as db:
|
||
doc_a = _make_doc(DOC_A_CONTENT, DOC_A_PATH, "gufo.container", DOC_A_SUMMARY)
|
||
doc_b = _make_doc(DOC_B_CONTENT, DOC_B_PATH, "verbatim-b", DOC_B_SUMMARY)
|
||
doc_img = _make_doc(
|
||
IMAGE_DESCRIPTION_ANSWER,
|
||
DOC_IMG_PATH,
|
||
"network-map",
|
||
IMAGE_DESCRIPTION_ANSWER,
|
||
is_image=True,
|
||
)
|
||
doc_img.image_path = str(image_file)
|
||
for doc in (doc_a, doc_b, doc_img):
|
||
db.add(doc)
|
||
db.flush()
|
||
db.add(
|
||
Chunk(
|
||
document_id=doc.id,
|
||
position=0,
|
||
content=doc.content,
|
||
embedding=embed_text(doc.content),
|
||
)
|
||
)
|
||
db.commit()
|
||
ids = (str(doc_a.id), str(doc_b.id), str(doc_img.id))
|
||
return {
|
||
"a_id": ids[0],
|
||
"b_id": ids[1],
|
||
"img_id": ids[2],
|
||
"image_url": f"/api/documents/{ids[2]}/image",
|
||
"a_content": DOC_A_CONTENT,
|
||
"b_content": DOC_B_CONTENT,
|
||
"a_locator": f"{SOURCE}/{DOC_A_PATH}",
|
||
"b_locator": f"{SOURCE}/{DOC_B_PATH}",
|
||
}
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def seeded(
|
||
app_url: str, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[dict[str, Any]]:
|
||
"""App A's KB. Runs once for the module, after the health check."""
|
||
if not _health_db(app_url):
|
||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||
_cleanup_saved_rows()
|
||
yield _seed_kb(tmp_path_factory.mktemp("bor_verbatim_img_on"))
|
||
_cleanup_saved_rows()
|
||
_truncate_all()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def seeded_off(
|
||
off_app_url: str, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[dict[str, Any]]:
|
||
"""App B's KB — the SAME three documents (the kill switch changes the
|
||
app's behaviour, never the corpus). Runs after App A's tests."""
|
||
if not _health_db(off_app_url):
|
||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||
_cleanup_saved_rows()
|
||
yield _seed_kb(tmp_path_factory.mktemp("bor_verbatim_img_off"))
|
||
_cleanup_saved_rows()
|
||
_truncate_all()
|
||
|
||
|
||
def _health_db(url: str) -> bool:
|
||
try:
|
||
return httpx.get(f"{url}/api/health", timeout=5).json()["db"] == "up"
|
||
except Exception: # noqa: BLE001 — unreachable is the skip case
|
||
return False
|
||
|
||
|
||
def _mock_capture(mock_llm: int) -> list[dict[str, Any]]:
|
||
"""The mock's recent chat-completions request bodies (oldest first) —
|
||
the wire lens on the tool OFFER and the executed CALLS."""
|
||
r = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=5)
|
||
assert r.status_code == 200
|
||
return r.json()
|
||
|
||
|
||
def _mock_capture_reset(mock_llm: int) -> None:
|
||
r = httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=5)
|
||
assert r.status_code == 200
|
||
|
||
|
||
def _offered_tool_names(body: dict[str, Any]) -> list[str]:
|
||
return [t["function"]["name"] for t in (body.get("tools") or [])]
|
||
|
||
|
||
def _executed_tool_calls(captured: list[dict[str, Any]]) -> list[str]:
|
||
return [
|
||
tc["function"]["name"]
|
||
for body in captured
|
||
for m in body["messages"]
|
||
if m.get("role") == "assistant" and m.get("tool_calls")
|
||
for tc in m["tool_calls"]
|
||
]
|
||
|
||
|
||
def _tool_results(body: dict[str, Any]) -> list[str]:
|
||
return [
|
||
str(m.get("content") or "")
|
||
for m in body.get("messages", [])
|
||
if m.get("role") == "tool"
|
||
]
|
||
|
||
|
||
#: In-page tap over the app's own ``fetch("/api/chat")`` stream: the SSE
|
||
#: frames' ``type`` fields + the parsed ``done`` frame, collected on a second
|
||
#: tee of the body so the app's own reader is untouched. The ONLY lens on the
|
||
#: frame vocabulary from the browser (the phase added no eighth event type).
|
||
_SSE_TAP_JS = """
|
||
(() => {
|
||
const orig = window.fetch.bind(window);
|
||
window.__borSse = [];
|
||
window.fetch = async (...args) => {
|
||
const res = await orig(...args);
|
||
const url = String((args[0] && args[0].url) || args[0] || "");
|
||
if (!url.includes("/api/chat") || !res.body) return res;
|
||
let keep, tap;
|
||
try { [keep, tap] = res.body.tee(); } catch { return res; }
|
||
const reader = tap.getReader();
|
||
const decoder = new TextDecoder();
|
||
(async () => {
|
||
let buf = "";
|
||
for (;;) {
|
||
const { done, value } = await reader.read();
|
||
if (done) break;
|
||
buf += decoder.decode(value, { stream: true });
|
||
let idx;
|
||
while ((idx = buf.indexOf("\\n\\n")) >= 0) {
|
||
const chunk = buf.slice(0, idx);
|
||
buf = buf.slice(idx + 2);
|
||
for (const line of chunk.split("\\n")) {
|
||
if (!line.startsWith("data:")) continue;
|
||
try { window.__borSse.push(JSON.parse(line.slice(5))); } catch {}
|
||
}
|
||
}
|
||
}
|
||
})();
|
||
return new Response(keep, {
|
||
status: res.status,
|
||
statusText: res.statusText,
|
||
headers: res.headers,
|
||
});
|
||
};
|
||
})();
|
||
"""
|
||
|
||
|
||
def _sse_frames(page: Page) -> list[dict[str, Any]]:
|
||
return page.evaluate("() => window.__borSse || []")
|
||
|
||
|
||
def _ask(page: Page, question: str) -> None:
|
||
"""Send *question* and wait for the turn to SETTLE (the house signal:
|
||
the answer's marker text plus the re-enabled Send pill)."""
|
||
page.fill("#message-input", question)
|
||
page.click("#send-btn")
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=60_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=60_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
|
||
|
||
def _panel(page: Page, locator: str) -> Any:
|
||
"""The resolved panels carrying one ``data-bor-locator`` (the SERVER's
|
||
canonical key — the attribute the client hydrated against)."""
|
||
return page.locator(f".msg.brain .verbatim[data-bor-locator='{locator}']")
|
||
|
||
|
||
def _panel_texts(page: Page) -> list[str]:
|
||
"""Every panel's ``data-bor-locator`` in DOM order (the placement pin)."""
|
||
return page.evaluate(
|
||
"""() => [...document.querySelectorAll(".msg.brain .verbatim[data-bor-locator]")]
|
||
.map((el) => el.getAttribute("data-bor-locator"))"""
|
||
)
|
||
|
||
|
||
def _body_child_classes(page: Page) -> list[str]:
|
||
"""The LAST brain bubble's ``.msg-body`` direct children's classes, in
|
||
DOM order (the A4/append position pin)."""
|
||
return page.evaluate(
|
||
"""() => {
|
||
const body = [...document.querySelectorAll(".msg.brain")].pop()
|
||
.querySelector(".msg-body");
|
||
return [...body.children].map((el) => el.className);
|
||
}"""
|
||
)
|
||
|
||
|
||
def _admin_cookies(page: Page) -> dict[str, str]:
|
||
return {
|
||
c["name"]: c["value"]
|
||
for c in page.context.cookies()
|
||
if "name" in c and "value" in c
|
||
}
|
||
|
||
|
||
def _auto_title(question: str) -> str:
|
||
return " ".join(question.split())[:120]
|
||
|
||
|
||
_SUITE_TITLES = tuple(
|
||
_auto_title(q)
|
||
for q in (
|
||
SHOW_QUESTION,
|
||
APPEND_QUESTION,
|
||
COPY_QUESTION,
|
||
WRONG_RANGE_QUESTION,
|
||
PAST_END_QUESTION,
|
||
INLINE_QUESTION,
|
||
MISSING_QUESTION,
|
||
IMAGE_TARGET_QUESTION,
|
||
PICTURE_QUESTION,
|
||
PERSIST_QUESTION,
|
||
MOBILE_QUESTION,
|
||
OFF_SHOW_QUESTION,
|
||
OFF_INLINE_QUESTION,
|
||
)
|
||
)
|
||
|
||
|
||
def _cleanup_saved_rows() -> None:
|
||
"""Delete THIS suite's saved rows by title (the house pattern — a re-run
|
||
must not find the previous run's record with a stale document id)."""
|
||
try:
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text("DELETE FROM saved_chats WHERE title = ANY(:titles)"),
|
||
{"titles": list(_SUITE_TITLES)},
|
||
)
|
||
db.commit()
|
||
except Exception: # noqa: BLE001 — DB unreachable: the skip handles it
|
||
pass
|
||
|
||
|
||
def _wait_saved_row(url: str, cookies: dict[str, str], title: str) -> dict[str, Any]:
|
||
"""Poll the admin list until the turn's auto-saved row appears (auto-saves
|
||
are SILENT — phase 55 A2 — so there is no status line to wait on)."""
|
||
deadline = time.monotonic() + 20
|
||
last: Any = None
|
||
while time.monotonic() < deadline:
|
||
r = httpx.get(f"{url}/api/chats", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
rows = r.json()["chats"]
|
||
last = next((c for c in rows if c["title"] == title), None)
|
||
if last is not None and last["message_count"] >= 2:
|
||
return last
|
||
time.sleep(0.2)
|
||
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
|
||
|
||
|
||
def _brain_record(
|
||
url: str, cookies: dict[str, str], chat_id: str
|
||
) -> dict[str, Any]:
|
||
r = httpx.get(f"{url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
brains = [m for m in r.json()["messages"] if m.get("who") == "brain"]
|
||
assert brains, "no brain record in the saved conversation"
|
||
return brains[-1]
|
||
|
||
|
||
def _grant_clipboard(page: Page, url: str) -> None:
|
||
page.context.grant_permissions(
|
||
["clipboard-read", "clipboard-write"], origin=url
|
||
)
|
||
|
||
|
||
def _clipboard(page: Page) -> str:
|
||
return page.evaluate("async () => await navigator.clipboard.readText()")
|
||
|
||
|
||
def _assert_no_model_bytes(page: Page) -> None:
|
||
"""LOCKED A7 on the rendered surface: none of the mock's re-typed fence
|
||
body reached the DOM — no sentinel line, no `` ```bor `` fence, and no
|
||
flipped-character drift of a real fixture line (the corruption the phase
|
||
exists for, derived from the fixture with the mock's own function)."""
|
||
html = page.content()
|
||
assert SHOW_TEXT_MODEL_BODY_MARKER not in html, (
|
||
"the model's re-typed fence body painted"
|
||
)
|
||
assert "```bor" not in html, "a raw ```bor locator fence painted"
|
||
drift = corrupt_verbatim_line(DOC_A_LINES[0])
|
||
assert drift != DOC_A_LINES[0] + " ", (
|
||
"the mock's corruption helper stopped drifting — the assertion below "
|
||
"would prove nothing"
|
||
)
|
||
# The flipped character itself (``# b4selab …``): the one-character drift
|
||
# a retyping model produces, which is exactly what must never paint.
|
||
assert "b4selab" not in html, "the model's one-character drift painted"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 1. App A — the tool flow: the panel IS the file, the model's bytes are not
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_show_text_turn_paints_the_files_bytes(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""The incident, end to end: the mock's scripted ``show_text`` call shows
|
||
fixture A, and the answer's fence carries a DELIBERATELY CORRUPTED copy
|
||
of it. The panel is the file (caption extent, body bytes, gutter numbers
|
||
1…N, VERBATIM badge, viewer link, path-naming aria-label); the model's
|
||
bytes are nowhere; the ordinary code block is wrapped, badged nothing and
|
||
still the model's own; one panel per locator and nothing left pending.
|
||
The wire pins: FIVE tools offered, one executed ``show_text``, and the
|
||
turn's SSE ``type`` set inside the pre-phase seven with ``answer_text``
|
||
riding inside ``done``. Plus the persistence pin: the saved brain record
|
||
carries the same blocks."""
|
||
page.add_init_script(_SSE_TAP_JS)
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, SHOW_QUESTION)
|
||
|
||
# No placeholder survived the terminal frame, and none of the model's
|
||
# re-typed bytes reached the DOM.
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
|
||
# --- the fixture-A panel: the SERVER's bytes, caption and gutter -----
|
||
panel = _panel(page, seeded["a_locator"])
|
||
expect(panel).to_have_count(1)
|
||
expect(panel).to_have_attribute("role", "group")
|
||
expect(panel.locator(".verbatim-caption")).to_have_text(
|
||
f"{SOURCE}/{DOC_A_PATH} · all {len(DOC_A_LINES)} lines"
|
||
)
|
||
expect(panel.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
body = panel.locator(".verbatim-text")
|
||
expect(body).to_have_text(DOC_A_CONTENT)
|
||
# The panel's text really is the stored row (not the trimmed render).
|
||
assert body.text_content() == DOC_A_CONTENT
|
||
gutter = panel.locator(".verbatim-lines li")
|
||
expect(gutter).to_have_count(len(DOC_A_LINES))
|
||
expect(gutter.first).to_have_text("1")
|
||
expect(gutter.last).to_have_text(str(len(DOC_A_LINES)))
|
||
# The Copy control names the file and the SERVER's line count.
|
||
copy_btn = panel.locator(".verbatim-copy")
|
||
expect(copy_btn).to_have_count(1)
|
||
expect(copy_btn).to_have_attribute(
|
||
"aria-label",
|
||
f"Copy {len(DOC_A_LINES)} lines from {SOURCE}/{DOC_A_PATH}",
|
||
)
|
||
# The viewer link targets the document (where the git origin lives) —
|
||
# the chip's own URL builder, back → the chat page.
|
||
open_link = panel.locator(".verbatim-open")
|
||
expect(open_link).to_have_count(1)
|
||
expect(open_link).to_have_attribute(
|
||
"href",
|
||
f"/document.html?source={SOURCE}&path={DOC_A_PATH.replace('/', '%2F')}"
|
||
"&back=%2F",
|
||
)
|
||
|
||
# --- the second fence: the OTHER file, resolved the same way ---------
|
||
panel_b = _panel(page, seeded["b_locator"])
|
||
expect(panel_b).to_have_count(1)
|
||
expect(panel_b.locator(".verbatim-text")).to_have_text(DOC_B_CONTENT)
|
||
# The escaping proof: a URL, backticks and an em dash survive as TEXT
|
||
# (never markup, never eaten by the markdown pass).
|
||
text_b = panel_b.locator(".verbatim-text").text_content() or ""
|
||
assert "https://lab.example/runbook#verbatim-b" in text_b
|
||
assert "`backticks`" in text_b
|
||
assert "—" in text_b
|
||
|
||
# --- placement and the no-second-copy rule --------------------------
|
||
# Both fences live inside the answer, so both panels render AT THE FENCE
|
||
# (inside .bubble), in answer order; the append path adds nothing.
|
||
inside = page.locator(".msg.brain .bubble .verbatim")
|
||
expect(inside).to_have_count(2)
|
||
assert _panel_texts(page) == [seeded["a_locator"], seeded["b_locator"]]
|
||
locators = _panel_texts(page)
|
||
assert len(locators) == len(set(locators)), (
|
||
f"a locator rendered twice: {locators}"
|
||
)
|
||
|
||
# --- the ordinary code block: model's own, Copy, NO badge -----------
|
||
plain = page.locator(".msg.brain .bubble .code-block")
|
||
expect(plain).to_have_count(1)
|
||
expect(plain.locator("pre")).to_have_text(SHOW_TEXT_PLAIN_BODY)
|
||
expect(plain.locator(".code-block-copy")).to_have_count(1)
|
||
expect(plain.locator(".verbatim-badge")).to_have_count(0)
|
||
# …and the badge belongs to exactly the two resolved panels.
|
||
expect(page.locator(".msg.brain .verbatim-badge")).to_have_count(2)
|
||
|
||
# --- the wire --------------------------------------------------------
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 2, (
|
||
f"expected 2 model calls (show_text + answer), got {len(captured)}"
|
||
)
|
||
assert _offered_tool_names(captured[0]) == [
|
||
"ls",
|
||
"read",
|
||
"grep",
|
||
"show_image",
|
||
"show_text",
|
||
], "the feature-on offer must be exactly the five tools"
|
||
assert _executed_tool_calls(captured) == ["show_text"]
|
||
assert any(
|
||
"Text shown to the user" in r for r in _tool_results(captured[1])
|
||
), "the show_text confirming result never reached the model"
|
||
|
||
# --- the frame vocabulary (no eighth SSE type) ----------------------
|
||
frames = _sse_frames(page)
|
||
types: set[str] = {str(f.get("type")) for f in frames}
|
||
assert types <= PRE_PHASE_EVENT_TYPES, (
|
||
f"the stream invented an event type: {sorted(types)}"
|
||
)
|
||
assert len(types) <= 7
|
||
assert "answer_text" not in types, "answer_text leaked out as its own event"
|
||
done = [f for f in frames if f.get("type") == "done"]
|
||
assert len(done) == 1
|
||
blocks = done[0].get("answer_text")
|
||
assert isinstance(blocks, list) and [b["locator"] for b in blocks] == [
|
||
seeded["a_locator"],
|
||
seeded["b_locator"],
|
||
], f"the done frame's answer_text is wrong: {blocks!r}"
|
||
assert blocks[0]["text"] == DOC_A_CONTENT
|
||
assert blocks[1]["text"] == DOC_B_CONTENT
|
||
|
||
# --- the persistence pin --------------------------------------------
|
||
row = _wait_saved_row(
|
||
app_url, _admin_cookies(page), _auto_title(SHOW_QUESTION)
|
||
)
|
||
record = _brain_record(
|
||
app_url, _admin_cookies(page), str(row["id"])
|
||
)
|
||
assert [b["locator"] for b in record.get("answer_text") or []] == [
|
||
seeded["a_locator"],
|
||
seeded["b_locator"],
|
||
]
|
||
assert record["answer_text"][0]["text"] == DOC_A_CONTENT
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 2. App A — the block the answer never locates APPENDS above .sources
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_unlocated_block_appends_above_sources(
|
||
page: Page, app_url: str, seeded: dict[str, Any]
|
||
) -> None:
|
||
"""The same tool turn with the answer's fences REMOVED (``describe it
|
||
without a block``): the block the tool showed has no fence to live in, so
|
||
it renders as a panel APPENDED to the bubble body — below the answer text
|
||
and ABOVE the ``.sources`` clusters (the phase-124/130 order), still the
|
||
server's bytes and still badged."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_ask(page, APPEND_QUESTION)
|
||
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
# The panel is OUTSIDE the answer bubble (nothing located it there).
|
||
expect(page.locator(".msg.brain .bubble .verbatim")).to_have_count(0)
|
||
panel = _panel(page, seeded["a_locator"])
|
||
expect(panel).to_have_count(1)
|
||
expect(panel.locator(".verbatim-text")).to_have_text(DOC_A_CONTENT)
|
||
expect(panel.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
|
||
classes = _body_child_classes(page)
|
||
assert "bubble" in classes, f"no answer bubble in .msg-body: {classes}"
|
||
assert "verbatim" in classes, f"no appended panel in .msg-body: {classes}"
|
||
# The source clusters are the .msg-meta family (phase 130: the chips row
|
||
# is a .msg-meta, an image doc cites by figure in .msg-meta.msg-images —
|
||
# here the auto-cited PNG, since a show_text turn performs no read).
|
||
sources_idx = next(
|
||
(i for i, c in enumerate(classes) if c.split(" ")[0] == "msg-meta"), None
|
||
)
|
||
assert sources_idx is not None, f"no source cluster in .msg-body: {classes}"
|
||
assert (
|
||
classes.index("bubble") < classes.index("verbatim") < sources_idx
|
||
), f"append order violated (bubble < verbatim < sources): {classes}"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 3. App A — the clipboard equals the server's string, for both controls
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_copy_buttons_copy_the_panel_and_the_plain_block(
|
||
page: Page, app_url: str, seeded: dict[str, Any]
|
||
) -> None:
|
||
"""``.verbatim-copy`` puts the resolved block on the clipboard EXACTLY
|
||
(read back through the async clipboard API — the byte-for-byte equality
|
||
the phase exists for), the pill settles to ``Copied`` text, and the
|
||
ordinary answer code block's Copy puts the MODEL's line on it exactly —
|
||
the two controls, the two different sources, one shared builder."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
_grant_clipboard(page, app_url)
|
||
|
||
_ask(page, COPY_QUESTION)
|
||
|
||
panel = _panel(page, seeded["a_locator"])
|
||
expect(panel).to_have_count(1)
|
||
panel.locator(".verbatim-copy").click()
|
||
expect(panel.locator(".verbatim-copy")).to_have_text("Copied")
|
||
assert _clipboard(page) == DOC_A_CONTENT, (
|
||
"the panel's clipboard content is not the server's string exactly"
|
||
)
|
||
|
||
plain = page.locator(".msg.brain .bubble .code-block .code-block-copy")
|
||
expect(plain).to_have_count(1)
|
||
plain.click()
|
||
expect(plain).to_have_text("Copied")
|
||
assert _clipboard(page) == SHOW_TEXT_PLAIN_BODY, (
|
||
"the ordinary block's clipboard content is not its own text"
|
||
)
|
||
# The line-number gutter never rides into the clipboard (user-select:
|
||
# none): the panel's text has no leading digits on any line.
|
||
assert not re.search(r"(?m)^\d+\s", _clipboard(page))
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 4. App A — a standalone locator fence resolves with ZERO tool calls
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_standalone_fence_resolves_without_a_tool_call(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""The half of the contract the tool does not cover: an answer that only
|
||
WRITES a locator fence. Zero tool calls on the wire, one panel resolved
|
||
from the answer text, the server's extent in the caption (``lines 2–3 of
|
||
5``) and the model's claimed body nowhere."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, INLINE_QUESTION)
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1, (
|
||
f"a fence-only answer must make exactly 1 model call, got {len(captured)}"
|
||
)
|
||
assert _executed_tool_calls(captured) == [], "the fence-only turn called a tool"
|
||
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
locator = f"{seeded['b_locator']}#2-3"
|
||
panel = _panel(page, locator)
|
||
expect(panel).to_have_count(1)
|
||
expect(panel.locator(".verbatim-caption")).to_have_text(
|
||
f"{SOURCE}/{DOC_B_PATH} · lines 2–3 of {len(DOC_B_LINES)}"
|
||
)
|
||
expect(panel.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
expected = "\n".join(DOC_B_LINES[1:3])
|
||
assert panel.locator(".verbatim-text").text_content() == expected
|
||
# The panel is INSIDE the bubble at the fence position, and the answer's
|
||
# prose still rendered around it.
|
||
expect(page.locator(".msg.brain .bubble .verbatim")).to_have_count(1)
|
||
expect(page.locator(".msg.brain .bubble")).to_contain_text(
|
||
"the block itself"
|
||
)
|
||
# The gutter carries the SERVER's numbers (2, 3), not 1-based ones.
|
||
gutter = panel.locator(".verbatim-lines li")
|
||
expect(gutter).to_have_count(2)
|
||
expect(gutter.first).to_have_text("2")
|
||
expect(gutter.last).to_have_text("3")
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 5. App A — a locator the server cannot resolve says so
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_hallucinated_locator_renders_the_honest_state(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""A fence naming a file with no row: NO panel — the pinned sentence, with
|
||
no badge, no body, no gutter and none of the model's bytes; the viewer
|
||
link still targets the named file (the reader can go look). The saved
|
||
record carries no ``answer_text`` key at all (no block displayed)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, MISSING_QUESTION)
|
||
|
||
assert _executed_tool_calls(_mock_capture(mock_llm)) == []
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
|
||
honest = page.locator(".msg.brain .verbatim.verbatim-unresolved")
|
||
expect(honest).to_have_count(1)
|
||
expect(honest.locator(".verbatim-state")).to_have_text(UNRESOLVED_COPY)
|
||
expect(honest.locator(".verbatim-badge")).to_have_count(0)
|
||
expect(honest.locator(".verbatim-body")).to_have_count(0)
|
||
expect(honest.locator(".verbatim-lines")).to_have_count(0)
|
||
expect(page.locator(".msg.brain .verbatim-copy")).to_have_count(0)
|
||
# The honest state still names the file the model asked for, and still
|
||
# links to it (the reader can go look for themselves).
|
||
expect(honest).to_have_attribute(
|
||
"data-bor-locator", f"{SHOW_TEXT_MISSING_TARGET}#3-4"
|
||
)
|
||
expect(honest.locator(".verbatim-open")).to_have_attribute(
|
||
"href",
|
||
"/document.html?source=baselab&path=notes%2Fthe-file-that-is-not-there.md"
|
||
"&back=%2F",
|
||
)
|
||
# Nothing resolved → nothing displayed → no resolved panel anywhere.
|
||
expect(page.locator(".msg.brain .verbatim:not(.verbatim-unresolved)")).to_have_count(
|
||
0
|
||
)
|
||
|
||
row = _wait_saved_row(app_url, _admin_cookies(page), _auto_title(MISSING_QUESTION))
|
||
record = _brain_record(app_url, _admin_cookies(page), str(row["id"]))
|
||
assert "answer_text" not in record, (
|
||
f"a turn with no block must persist no answer_text: {record!r}"
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 6. App A — a wrong range: the server's extent, or the honest state
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_wrong_range_shows_the_servers_extent(
|
||
page: Page, app_url: str, seeded: dict[str, Any]
|
||
) -> None:
|
||
"""The fence claims ``2-999`` of a nine-line file. The tool clamps and
|
||
displays 2–9, so the panel shows the SERVER's extent (caption, gutter and
|
||
bytes all 2–9) under the model's own placeholder key — and the claimed
|
||
``999`` appears nowhere in the DOM."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_ask(page, WRONG_RANGE_QUESTION)
|
||
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
server_locator = f"{seeded['a_locator']}#2-{len(DOC_A_LINES)}"
|
||
panel = _panel(page, server_locator)
|
||
expect(panel).to_have_count(1)
|
||
expect(panel.locator(".verbatim-caption")).to_have_text(
|
||
f"{SOURCE}/{DOC_A_PATH} · lines 2–{len(DOC_A_LINES)} of {len(DOC_A_LINES)}"
|
||
)
|
||
assert (
|
||
panel.locator(".verbatim-text").text_content()
|
||
== "\n".join(DOC_A_LINES[1:])
|
||
)
|
||
expect(panel.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
expect(panel.locator(".verbatim-lines li").first).to_have_text("2")
|
||
expect(panel.locator(".verbatim-lines li").last).to_have_text(
|
||
str(len(DOC_A_LINES))
|
||
)
|
||
# The show SUCCEEDED (clamped), so the answer's second fence resolves too
|
||
# — the model's claimed ``999`` is still nowhere on screen.
|
||
expect(_panel(page, seeded["b_locator"])).to_have_count(1)
|
||
# VISIBLE text (the unresolved/pending node may carry the model's own
|
||
# locator as its identity attribute — that is how its link targets the
|
||
# file; what must never appear is the claim ON SCREEN).
|
||
visible = page.locator(".msg.brain").last.inner_text()
|
||
assert "999" not in visible, (
|
||
f"the model's claimed end line painted instead of the server's: {visible!r}"
|
||
)
|
||
|
||
|
||
def test_past_end_range_refuses_and_says_so(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""The other wrong-range half: ``offset`` past the last line. The tool
|
||
REFUSES (``OFFSET_PAST_END`` — the refusal names the true line count to
|
||
the model), nothing displays, and the client's honest state renders with
|
||
NONE of the model's claimed numbers on screen."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, PAST_END_QUESTION)
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert _executed_tool_calls(captured) == ["show_text"]
|
||
refusals = _tool_results(captured[1])
|
||
assert any(
|
||
"is past the end" in r
|
||
and f"is {len(DOC_A_LINES)} lines long" in r
|
||
and "`offset` 40" in r
|
||
for r in refusals
|
||
), f"the OFFSET_PAST_END teaching never reached the model: {refusals}"
|
||
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
honest = page.locator(".msg.brain .verbatim.verbatim-unresolved")
|
||
expect(honest).to_have_count(1)
|
||
expect(honest.locator(".verbatim-state")).to_have_text(UNRESOLVED_COPY)
|
||
expect(page.locator(".msg.brain .verbatim-body")).to_have_count(0)
|
||
expect(page.locator(".msg.brain .verbatim-badge")).to_have_count(0)
|
||
visible = page.locator(".msg.brain").last.inner_text()
|
||
assert "40-60" not in visible, f"the refused range painted: {visible!r}"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 7. App A — reload restores the panel; the shared page renders it too
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_reload_and_share_re_render_the_panel(
|
||
page: Page,
|
||
app_url: str,
|
||
seeded: dict[str, Any],
|
||
browser: Browser,
|
||
) -> None:
|
||
"""The same turn, four surfaces, one renderer: LIVE → RELOAD (the
|
||
``bor.chat.v1`` restore re-hydrates from the stored record) → the chat
|
||
OPENED FROM ``/history.html`` (the ``?chat=<id>`` boot load, which pulls
|
||
the row from ``GET /api/chats/{id}`` — the SERVER's record, not local
|
||
storage) → the SHARED PAGE for an anonymous reader. The panel's bytes,
|
||
caption and Copy button are identical on all four (the shared page's
|
||
zero-controls rule has exactly this exception)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_ask(page, PERSIST_QUESTION)
|
||
panel = _panel(page, seeded["a_locator"])
|
||
expect(panel).to_have_count(1)
|
||
assert panel.locator(".verbatim-text").text_content() == DOC_A_CONTENT
|
||
|
||
cookies = _admin_cookies(page)
|
||
saved = _wait_saved_row(app_url, cookies, _auto_title(PERSIST_QUESTION))
|
||
|
||
# RELOAD — the restore path renders the panel from stored state.
|
||
page.reload()
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
restored = _panel(page, seeded["a_locator"])
|
||
expect(restored).to_have_count(1, timeout=30_000)
|
||
assert restored.locator(".verbatim-text").text_content() == DOC_A_CONTENT
|
||
expect(restored.locator(".verbatim-caption")).to_have_text(
|
||
f"{SOURCE}/{DOC_A_PATH} · all {len(DOC_A_LINES)} lines"
|
||
)
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
|
||
# OPEN FROM HISTORY — the row's Open link (``/?chat=<id>``) boots the
|
||
# chat page from the SAVED record through the admin API, so this is the
|
||
# panel rendering off the persisted ``answer_text`` rather than local
|
||
# storage.
|
||
chat_id = str(saved["id"])
|
||
# Drop the local record FIRST: after this the panel can only come from
|
||
# the API's saved row (a vacuous pass through localStorage is impossible).
|
||
page.evaluate("() => window.localStorage.clear()")
|
||
page.goto(f"{app_url}/history.html")
|
||
open_link = page.locator(f".history-title-link[href='/?chat={chat_id}']")
|
||
expect(open_link).to_have_count(1, timeout=30_000)
|
||
open_link.click()
|
||
from_history = _panel(page, seeded["a_locator"])
|
||
expect(from_history).to_have_count(1, timeout=30_000)
|
||
assert from_history.locator(".verbatim-text").text_content() == DOC_A_CONTENT
|
||
expect(from_history.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
|
||
# SHARE — a fresh anonymous context, no cookies.
|
||
_grant_clipboard(page, app_url)
|
||
page.locator("#share-chat-btn").click()
|
||
if page.evaluate("() => !!navigator.clipboard"):
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link copied.", timeout=15_000
|
||
)
|
||
else:
|
||
expect(page.locator(".share-link-fallback")).to_be_visible()
|
||
rows = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies).json()[
|
||
"chats"
|
||
]
|
||
shared = next(
|
||
(c for c in rows if c["title"] == _auto_title(PERSIST_QUESTION) and c.get("share_url")),
|
||
None,
|
||
)
|
||
assert shared is not None and shared.get("share_url"), f"no share_url: {rows!r}"
|
||
|
||
anon = browser.new_page(viewport={"width": 1280, "height": 800})
|
||
try:
|
||
anon.goto(app_url + str(shared["share_url"]))
|
||
expect(anon.locator("#shared-title")).to_have_text(
|
||
_auto_title(PERSIST_QUESTION), timeout=30_000
|
||
)
|
||
shared_panel = anon.locator(
|
||
f".msg.brain .verbatim[data-bor-locator='{seeded['a_locator']}']"
|
||
)
|
||
expect(shared_panel).to_have_count(1, timeout=30_000)
|
||
assert shared_panel.locator(".verbatim-text").text_content() == DOC_A_CONTENT
|
||
expect(shared_panel.locator(".verbatim-badge")).to_have_text("VERBATIM")
|
||
# The one exception to the shared page's zero-controls rule: the
|
||
# panel's Copy button (and its plain viewer link — no modal there).
|
||
expect(shared_panel.locator(".verbatim-copy")).to_have_count(1)
|
||
expect(shared_panel.locator(".verbatim-open")).to_have_attribute(
|
||
"href",
|
||
"/document.html?source=baselab&path=quadlets%2Fgufo.container",
|
||
)
|
||
assert SHOW_TEXT_MODEL_BODY_MARKER not in anon.content()
|
||
finally:
|
||
anon.close()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 8. App A — the image surfaces: one figure, no panel; show_text refuses an
|
||
# image document by name (LOCKED A6)
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_answer_image_turn_shows_one_figure_and_no_panel(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""The phase-124 surface is untouched: the ``show me the picture of``
|
||
trigger still streams exactly ONE ``.answer-image`` figure (the bytes
|
||
route serves 200 with the fixture bytes) and NO verbatim panel — and no
|
||
``answer_text`` on the done frame, so nothing else renders."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, PICTURE_QUESTION)
|
||
|
||
figure = page.locator(".msg.brain .answer-image")
|
||
expect(figure).to_have_count(1)
|
||
img = figure.locator("img")
|
||
expect(img).to_have_attribute("src", seeded["image_url"])
|
||
expect(img).to_be_visible(timeout=15_000)
|
||
resp = httpx.get(app_url + seeded["image_url"], timeout=10)
|
||
assert resp.status_code == 200
|
||
assert resp.content == PNG_1X1
|
||
|
||
expect(page.locator(".msg.brain .verbatim")).to_have_count(0)
|
||
assert _executed_tool_calls(_mock_capture(mock_llm)) == ["show_image"]
|
||
|
||
|
||
def test_show_text_on_an_image_document_is_refused_by_name(
|
||
page: Page, app_url: str, seeded: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""LOCKED A6: ``show_text`` pointed at an image document is REFUSED,
|
||
never silently converted — the refusal copy naming ``show_image`` is what
|
||
reaches the model, and the surface shows the honest state (no panel, no
|
||
figure: the answer called no display tool)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, IMAGE_TARGET_QUESTION)
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert _executed_tool_calls(captured) == ["show_text"]
|
||
results = _tool_results(captured[1])
|
||
assert any(
|
||
"standalone image document" in r and "show_image" in r for r in results
|
||
), f"the A6 refusal naming show_image never reached the model: {results}"
|
||
|
||
expect(page.locator(".msg.brain .verbatim-body")).to_have_count(0)
|
||
expect(page.locator(".msg.brain .verbatim-badge")).to_have_count(0)
|
||
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
expect(page.locator(".msg.brain .verbatim.verbatim-unresolved")).to_have_count(1)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 9. App A — the phone: the panel scrolls inside itself, controls are 44px
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_phone_panel_scrolls_and_controls_clear_44px(
|
||
browser: Browser, app_url: str, seeded: dict[str, Any]
|
||
) -> None:
|
||
"""At 390 px the panel's long line (fixture A line 6) scrolls INSIDE the
|
||
bounded panel — the page itself never scrolls sideways — and both Copy
|
||
controls clear the 44 px WCAG touch floor."""
|
||
page = browser.new_page(viewport={"width": 390, "height": 844})
|
||
try:
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
_ask(page, MOBILE_QUESTION)
|
||
|
||
panel = _panel(page, seeded["a_locator"])
|
||
expect(panel).to_have_count(1, timeout=30_000)
|
||
metrics = panel.locator(".verbatim-body").evaluate(
|
||
"""(el) => ({
|
||
scrollWidth: el.scrollWidth,
|
||
clientWidth: el.clientWidth,
|
||
pageScroll: document.documentElement.scrollWidth,
|
||
viewport: window.innerWidth,
|
||
})"""
|
||
)
|
||
assert metrics["scrollWidth"] > metrics["clientWidth"], (
|
||
f"the long line does not scroll inside the panel: {metrics}"
|
||
)
|
||
assert metrics["pageScroll"] <= metrics["viewport"] + 1, (
|
||
f"the page itself scrolls sideways at 390px: {metrics}"
|
||
)
|
||
for selector in (".verbatim-copy", ".code-block-copy"):
|
||
box = page.locator(f".msg.brain {selector}").first.bounding_box()
|
||
assert box is not None, f"no {selector} to measure"
|
||
assert box["height"] >= 44, f"{selector} height {box['height']} < 44px"
|
||
assert box["width"] >= 44, f"{selector} width {box['width']} < 44px"
|
||
finally:
|
||
page.close()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 10. App B (BOR_ANSWER_TEXT=false) — the kill switch
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_answer_text_off_no_tool_no_panel(
|
||
page: Page, off_app_url: str, seeded_off: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""The escape hatch, end to end: the clause is gone, so the mock's
|
||
show-me flow never fires (ONE model call, zero tool calls) and the offer
|
||
is exactly FOUR tools with no ``show_text``. The answer renders as plain
|
||
prose — no ``.verbatim`` element of any kind, and the saved record
|
||
carries no ``answer_text`` key."""
|
||
login(page, off_app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, OFF_SHOW_QUESTION)
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1, (
|
||
f"the clause-gated flow must not fire, got {len(captured)} model calls"
|
||
)
|
||
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep", "show_image"], (
|
||
"show_text leaked into the kill-switch offer"
|
||
)
|
||
assert _executed_tool_calls(captured) == []
|
||
|
||
expect(page.locator(".msg.brain .verbatim")).to_have_count(0)
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
|
||
row = _wait_saved_row(
|
||
off_app_url, _admin_cookies(page), _auto_title(OFF_SHOW_QUESTION)
|
||
)
|
||
record = _brain_record(off_app_url, _admin_cookies(page), str(row["id"]))
|
||
assert "answer_text" not in record, (
|
||
f"the kill-switch record must carry no answer_text: {record!r}"
|
||
)
|
||
|
||
|
||
def test_answer_text_off_fence_never_paints_the_model(
|
||
page: Page, off_app_url: str, seeded_off: dict[str, Any], mock_llm: int
|
||
) -> None:
|
||
"""A `` ```bor `` fence on the kill-switch app: nothing on the wire
|
||
resolves it, so the client settles it to the honest sentence — the
|
||
model's re-typed body STILL never reaches the DOM (locked A7 is
|
||
flag-independent), there is no badge and no body, while the ordinary code
|
||
block keeps its working Copy button (A5 is flag-independent too)."""
|
||
login(page, off_app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
_grant_clipboard(page, off_app_url)
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
_ask(page, OFF_INLINE_QUESTION)
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1 and _executed_tool_calls(captured) == []
|
||
assert "show_text" not in _offered_tool_names(captured[0])
|
||
|
||
expect(page.locator(".verbatim-pending")).to_have_count(0)
|
||
_assert_no_model_bytes(page)
|
||
honest = page.locator(".msg.brain .verbatim.verbatim-unresolved")
|
||
expect(honest).to_have_count(1)
|
||
expect(honest.locator(".verbatim-state")).to_have_text(UNRESOLVED_COPY)
|
||
expect(honest.locator(".verbatim-badge")).to_have_count(0)
|
||
expect(honest.locator(".verbatim-body")).to_have_count(0)
|
||
|
||
plain = page.locator(".msg.brain .bubble .code-block")
|
||
expect(plain).to_have_count(1)
|
||
expect(plain.locator("pre")).to_have_text(SHOW_TEXT_PLAIN_BODY)
|
||
expect(plain.locator(".verbatim-badge")).to_have_count(0)
|
||
plain.locator(".code-block-copy").click()
|
||
expect(plain.locator(".code-block-copy")).to_have_text("Copied")
|
||
assert _clipboard(page) == SHOW_TEXT_PLAIN_BODY
|
||
|
||
row = _wait_saved_row(
|
||
off_app_url, _admin_cookies(page), _auto_title(OFF_INLINE_QUESTION)
|
||
)
|
||
record = _brain_record(off_app_url, _admin_cookies(page), str(row["id"]))
|
||
assert "answer_text" not in record
|
||
# The raw fence text is what persists (the answer is stored as typed) —
|
||
# it renders as the honest state, never as the model's claim.
|
||
assert "```bor" in (record["text"] or "")
|