Files
brain-of-reese/tests/e2e/test_answer_images.py
T
ducoterra c275735db4
Build and Push Containers / build-and-push-app (push) Successful in 2m26s
Build and Push Containers / build-and-push-db (push) Successful in 16s
phase: 154_sync_status_freshness
All 4 task files exist in `complete/`, and only `00_phase.md` remains in `todo/`. This is the final verification pass. I'll read through the task files and check the current state of the code.
2026-10-04 14:24:33 -04:00

1004 lines
44 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 124 E2E (Playwright): image source chips + LLM-chosen answer images.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_answer_images.py -v --no-cov
The suite proves BOTH halves of TODO.md L3 on the real user path, plus
the toggle-off contract — two module apps on dedicated ports (the
phase-122 ``test_image_documents.py`` pattern), sharing one deterministic
mock LLM and the shared e2e Postgres (truncate/re-import per app):
* **App A (feature ON)** — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=true``: the standalone PNG arrives through the
REAL admin upload flow + the RAG page's "Sync sources" (the mock's
vision model writes the byte-stable description), then:
- **Feature flow** — the ``show me the picture of …`` trigger (the
mock's phase-124 scripted trigger, ``tests/e2e/mock_llm.py``): the
mock scripts ONLY a ``show_image`` call (NO ``read`` — the ZERO-READ
turn, the shape that makes the chip assertion meaningful). The
brain bubble shows the ANSWER FIGURE (``.answer-image`` — ``<img>``
from the bytes route, ``loading="lazy"``, below the text and above
the source clusters per A4, alt/caption settling to the summary),
the answer TEXT is still streamed in full (the tail echo of the
``<documents>`` seed proves the description was in the prompt), the
sources block carries the image doc's citation as the compact
figure in the images row (``.msg-meta.msg-images`` — phase 130
task 01: NO chip for an image doc, the figure IS the citation;
the AUTO-CITE proof — the mock made zero ``read`` calls, asserted
on the mock's request capture), and the "Nearby docs" row does not
repeat the doc. The wire pin: the first model request offered
FOUR tools (the three base + ``show_image``) and the turn executed
exactly one ``show_image`` — the saved brain record persists the
``answer_image`` ref.
- **Restore + share** — a second feature turn, then a page RELOAD:
the restored brain bubble shows the figure again (the
``answer_image`` persistence through the ``bor.chat.v1`` restore
path), and the shared link (the existing chat-page Share flow, a
fresh anonymous context) renders the figure on the shared page.
- **Bug-fix-only flow** — the ``which diagram is in your notes``
trigger (the mock answers straight from the seed with ZERO tool
calls — no ``show_image``, no ``read``): the sources block STILL
carries the image doc's citation figure (no chip — phase 130;
the bug fix on the pure zero-read path — the assertion the owner's
report describes), the answer text quotes the seeded
description's tail, and NO answer figure renders (the model never
called the tool).
* **App B (default off)** — ``BOR_IMAGES=true``, ``BOR_ANSWER_IMAGES``
pinned to the code default ``false`` (EXPLICITLY — process env ranks
above an operator's local gitignored ``.env``, so the default-off
contract under test cannot leak in, the phase-122 default-app
convention; the production default is unset = the same ``False``):
the same feature trigger question → the clause is absent from the
prompt, so the mock's trigger NEVER fires — the mock never sees
``show_image`` among the offered tools (the request capture carries
exactly the THREE base tools), the brain bubble has NO answer
figure, and the saved brain record carries NO ``answer_image`` key
(the byte-identical-off contract). The bug-fix trigger STILL cites
the image doc (the figure, no chip — phase 130; the auto-cite fix is
NOT gated by ``BOR_ANSWER_IMAGES`` — it belongs to the
``BOR_IMAGES``-indexed docs, per the phase design).
DB isolation: the suites share one Postgres; ``saved_chats`` survives
the truncation (the house pattern), so every question text here is
UNIQUE (its auto-title is therefore unique) and the row lookups key on
it. App B's module seed truncates the KB tables App A's tests used —
App B's tests run AFTER App A's (file order) and never look back.
"""
from __future__ import annotations
import base64
import json
import os
import re
import subprocess
import sys
import time
import zipfile
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Browser, Page, expect
from sqlalchemy import select, text
from app.config import Settings
from app.db import SessionLocal
from app.models import Document
from e2e.auth_helpers import login
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
REPO = Path(__file__).resolve().parents[2]
GIT_SOURCES_URL = "/git-sources.html"
SOURCE_NAME = "e2e-answer-img" # the archive stem (archive_source_name)
DOC_PATH = "pic.png"
#: The mock's deterministic answer tail (every composed answer ends
#: with it — the turn-settled wait signal, the house convention).
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
#: The TAIL of the fixture description (``IMAGE_DESCRIPTION_ANSWER`` —
#: the image doc's entire content): both phase-124 mock answers quote
#: it (the feature flow's ``<documents>`` tail echo, the bug-fix flow's
#: block-content tail) — asserting it proves the SEED was in the
#: prompt and reached the rendered answer.
DESCRIPTION_TAIL = "Title: Homelab Network Map."
#: App A (feature ON) — the ``show_image`` trigger question (cosine
#: 0.4552 against the mock's token-overlap embeddings, well over the
#: mock-calibrated 0.30 gate). Unique auto-title (the DB is shared).
FEATURE_QUESTION = "Show me the picture of the homelab network diagram?"
#: The restore/share flow's variant (the trigger substring is kept —
#: cosine 0.4358, still over the gate).
RESTORE_QUESTION = "Show me the picture of the homelab network diagram? (restore)"
#: App B (default off) — the feature trigger's variant (the clause is
#: absent, so the mock answers with the plain composed answer — cosine
#: 0.4187, the turn still grounds).
FEATURE_OFF_QUESTION = (
"Show me the picture of the homelab network diagram? (toggle off)"
)
#: The bug-fix-only trigger question (cosine 0.3410): the mock answers
#: straight from the seed with ZERO tool calls and quotes the image
#: doc's block tail — the chip must appear anyway (the auto-cite).
DIAGRAM_QUESTION = "Which diagram is in your notes about the homelab network?"
#: App B's bug-fix variant (the chips fix is not toggle-gated — cosine
#: 0.3113, still over the mock-calibrated gate).
DIAGRAM_OFF_QUESTION = (
"Which diagram is in your notes about the homelab network? (default off)"
)
#: A real 1×1 transparent PNG (the unit/integration suites' fixture —
#: the pipeline is content-agnostic, the well-formed bytes keep the
#: upload + serve + render path honest).
PNG_1X1 = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
"AAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
)
UPLOAD_TIMEOUT_MS = 30_000
SYNC_TIMEOUT_MS = 60_000
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
# ---------------------------------------------------------------------------
# Module apps: feature ON (App A) and default-OFF (App B). Each owns
# its port + its scratch upload/source/image dirs.
# ---------------------------------------------------------------------------
def _app_env(
mock_llm: int, app_port: int, *, answer_images: str, scratch: Path
) -> dict[str, str]:
"""The conftest app env (leak guards included) with the phase-124
knobs: BOTH apps index images (``BOR_IMAGES=true`` — the fixture's
PNG must become a document on either), and ``BOR_ANSWER_IMAGES``
is pinned EXPLICITLY per app (``true`` for App A, ``false`` for
App B — process env ranks above an operator's local gitignored
``.env``, so the contract under test cannot leak in either
direction, the phase-122 default-app convention; App B's explicit
``false`` IS the production default — unset is ``False``). The
image/upload homes are in the suite's scratch dir (the app under
test must not write image copies into the owner's real
``~/bor-sources``)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern) — both trigger questions
# ground on these values (see the question constants above).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget.
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# Leak guards (conftest pattern).
env["BOR_GIT_SOURCES"] = ""
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
# Phase 124: both apps index images; only the ANSWER side differs.
env["BOR_IMAGES"] = "true"
env["BOR_ANSWER_IMAGES"] = answer_images
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
env["BOR_IMAGE_DIR"] = str(scratch / "images")
return env
@pytest.fixture(scope="module")
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
"""App A (feature ON) — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=true`` (the conftest session app is never
started in this isolated run, so no port clash)."""
scratch = tmp_path_factory.mktemp("bor_answer_img_on")
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_ON", "8162"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
cwd=REPO,
env=_app_env(mock_llm, port, answer_images="true", scratch=scratch),
)
try:
_wait_http(f"http://127.0.0.1:{port}/api/health")
yield f"http://127.0.0.1:{port}"
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
@pytest.fixture(scope="module")
def default_app_server(
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
) -> Iterator[str]:
"""App B (default off) — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=false`` pinned explicitly (the production
default — the toggle-off contract; only the negative tests start
it, and they run AFTER App A's — file order)."""
scratch = tmp_path_factory.mktemp("bor_answer_img_off")
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_OFF", "8163"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
cwd=REPO,
env=_app_env(mock_llm, port, answer_images="false", scratch=scratch),
)
try:
_wait_http(f"http://127.0.0.1:{port}/api/health")
yield f"http://127.0.0.1:{port}"
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def default_app_url(default_app_server: str) -> str:
return default_app_server
@pytest.fixture(scope="module")
def zip_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The fixture archive: ONE standalone PNG at the root — source
``e2e-answer-img``, document path ``pic.png`` (the mock's seeded-
image-doc pick is deterministic: the only image-extension block)."""
root = tmp_path_factory.mktemp("bor_answer_img_zip")
(root / DOC_PATH).write_bytes(PNG_1X1)
archive = root / f"{SOURCE_NAME}.zip"
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
zf.write(root / DOC_PATH, arcname=DOC_PATH)
return archive
# ---------------------------------------------------------------------------
# DB + UI + mock-capture helpers
# ---------------------------------------------------------------------------
def _truncate_all() -> None:
"""Fresh KB + registry (the E2E isolation pattern): the suites
share one Postgres, and a leftover row would corrupt the counts.
``sources_meta`` (the KB generation counter) resets with the KB —
the sync re-creates the row. ``saved_chats`` is never touched by
the truncation — the unique question texts keep the row lookups
unambiguous, and :func:`_cleanup_saved_rows` owns this suite's
rows (re-runnable on the shared DB)."""
with SessionLocal() as db:
db.execute(text(
"TRUNCATE chunks, documents, query_log, kb_overview, "
"git_sources, sources_meta, folder_summaries"
))
db.commit()
def _health_db(app_url: str) -> bool:
try:
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
except Exception: # noqa: BLE001 — unreachable is the skip case
return False
def _mock_capture(mock_llm: int) -> list[dict[str, Any]]:
"""The mock's recent chat-completions request bodies (oldest
first) — the wire lens on the app's tool OFFER and the turn's
tool CALLS (the mock is the only E2E lens on the LLM's context)."""
r = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=5)
assert r.status_code == 200
return r.json()
def _mock_capture_reset(mock_llm: int) -> None:
r = httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=5)
assert r.status_code == 200
def _executed_tool_calls(captured: list[dict[str, Any]]) -> list[str]:
"""Every tool call the MODEL made across *captured* requests (the
assistant ``tool_calls`` messages), in order — the executed-call
lens (a refusal still appears here; it is the model's call)."""
return [
tc["function"]["name"]
for body in captured
for m in body["messages"]
if m.get("role") == "assistant" and m.get("tool_calls")
for tc in m["tool_calls"]
]
def _offered_tool_names(body: dict[str, Any]) -> list[str]:
return [t["function"]["name"] for t in (body.get("tools") or [])]
def _ready_for_sync_line(source: str) -> re.Pattern[str]:
"""Phase 154 (task 02): the ready-for-sync line carrying the
completion stamp — "Uploaded <source> (<stamp>) — press Sync
sources to import it." (the fresh 202 settle). In E2E the stamp
is ALWAYS the same-day ``HH:MM`` form (the run settles within
seconds; the date-qualified forms are unit-pinned on a fixed
clock — E2E never depends on the wall clock)."""
return re.compile(
rf"^Uploaded {re.escape(source)} \(\d{{1,2}}:\d{{2}}\)"
r" — press Sync sources to import it\.$"
)
def _upload_zip(page: Page, app_url: str, archive: Path) -> None:
"""The real admin upload flow: form login → the git-sources page →
pick the archive → submit → the terminal result line (phase 90 A3,
phase 154 task 02: "Uploaded <source> (<stamp>) — press Sync
sources to import it." — the completion stamp)."""
login(page, app_url, next=GIT_SOURCES_URL)
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
page.set_input_files("#archive-upload-file", str(archive))
page.click("#archive-upload-btn")
result = page.locator("#archive-upload-result")
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
expect(result).to_have_text(
_ready_for_sync_line(SOURCE_NAME),
timeout=UPLOAD_TIMEOUT_MS,
)
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
"""The RAG page's "Sync sources" button (the scan — phase 90 A3):
click → "Syncing…" → the terminal "Synced HH:MM" label + the fresh
counts in #sync-progress (the never-stale contract)."""
page.goto(app_url + "/sources.html")
btn = page.locator("#sync-btn")
expect(btn).to_be_visible(timeout=30_000)
expect(page.locator("#sync-label")).to_have_text(
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
)
btn.click()
expect(btn).to_be_disabled()
expect(page.locator("#sync-label")).to_have_text("Syncing…")
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
expect(btn).to_be_enabled()
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
def _seed_image_doc(
app_url: str, browser: Browser, zip_path: Path
) -> dict[str, str]:
"""Upload the single-PNG zip through the admin UI, then scan it
with the RAG page's Sync button (the mock's vision model describes
the image in-process). Returns the seeded doc's identity (the
bytes route + the description + the title) for the render
assertions. The app's ``/api/config`` flag is asserted first — the
env override must have reached the app under test, or the whole
premise of the suite is void."""
cfg = httpx.get(f"{app_url}/api/config", timeout=5).json()
assert cfg["images"] is True, "the module app must run with BOR_IMAGES=true"
_truncate_all()
page = browser.new_page(viewport={"width": 1280, "height": 800})
try:
_upload_zip(page, app_url, zip_path)
_sync_via_ui(page, app_url, expected_result="1 added")
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == SOURCE_NAME, Document.path == DOC_PATH
)
)
assert doc is not None, "the uploaded image must become a document"
assert doc.is_image is True and doc.image_path is not None
# content == summary == the mock's description (the ONLY
# embedded text of the doc — the pipeline contract, phase 122).
assert doc.content == IMAGE_DESCRIPTION_ANSWER
assert doc.summary == IMAGE_DESCRIPTION_ANSWER
return {
"image_url": f"/api/documents/{doc.id}/image",
"description": IMAGE_DESCRIPTION_ANSWER,
"title": doc.title,
}
finally:
page.close()
@pytest.fixture(scope="module")
def seeded(
app_url: str,
browser: Browser,
zip_path: Path,
) -> Iterator[dict[str, str]]:
"""App A's KB: the fixture image doc through the real UI path."""
if not _health_db(app_url):
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
_cleanup_saved_rows() # a previous run's rows would be stale (new doc id)
yield _seed_image_doc(app_url, browser, zip_path)
_cleanup_saved_rows()
_truncate_all()
@pytest.fixture(scope="module")
def seeded_off(
default_app_url: str,
browser: Browser,
zip_path: Path,
) -> Iterator[dict[str, str]]:
"""App B's KB: the SAME fixture image doc through the SAME real
UI path (images indexed, the answer feature OFF). Runs AFTER all
of App A's tests (file order) — its truncation wipes App A's KB
tables, and App B's tests never look back."""
if not _health_db(default_app_url):
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
_cleanup_saved_rows()
yield _seed_image_doc(default_app_url, browser, zip_path)
_cleanup_saved_rows()
_truncate_all()
# ---------------------------------------------------------------------------
# Saved-chat + share helpers (the test_share_chat.py conventions)
# ---------------------------------------------------------------------------
def _admin_cookies(page: Page) -> dict[str, str]:
"""The signed session cookies the browser holds after a form login
— used to call the admin API with plain httpx (the test's API side
sees exactly what the signed-in browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
#: This suite's auto-titles (the shared e2e Postgres keeps
#: ``saved_chats`` across runs — the test_share_chat cleanup
#: concern): a re-run must not find the previous run's row (same
#: title, but the KB re-seed minted a NEW doc id — the row's
#: ``answer_image.image_url`` would no longer match the seed).
_SUITE_TITLES = tuple(
_auto_title(q)
for q in (
FEATURE_QUESTION,
RESTORE_QUESTION,
FEATURE_OFF_QUESTION,
DIAGRAM_QUESTION,
DIAGRAM_OFF_QUESTION,
)
)
def _cleanup_saved_rows() -> None:
"""Delete THIS suite's saved rows by title (the house pattern —
test_share_chat deletes its own rows; here the module-boundary
fixture calls it, so a re-run starts clean). Best-effort: a
failed delete (DB unreachable — the skip case) never masks the
skip; a failed delete with the DB up just re-creates the stale-
row premise, and the suite fails loudly on the row assertion."""
try:
with SessionLocal() as db:
db.execute(
text("DELETE FROM saved_chats WHERE title = ANY(:titles)"),
{"titles": list(_SUITE_TITLES)},
)
db.commit()
except Exception: # noqa: BLE001 — DB unreachable: the skip handles it
pass
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55: auto-saves are SILENT —
A2 — so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
rows = r.json()["chats"]
last = next((c for c in rows if c["title"] == title), None)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _chat_messages(
app_url: str, cookies: dict[str, str], chat_id: str
) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["messages"]
def _brain_record(messages: list[dict[str, Any]]) -> dict[str, Any]:
brains = [m for m in messages if m.get("who") == "brain"]
assert brains, f"no brain record in the saved conversation: {messages!r}"
return brains[-1]
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(test_share_chat.py — headless Chromium still requires the grant
before ``writeText`` resolves on a secure origin)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share(app_url: str, page: Page) -> None:
"""Press the chat page's Share pill and pin the owner-locked
outcome on the live region (the test_share_chat.py branch —
clipboard success vs the inline-link fallback)."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
expect(page.locator(".share-link-fallback")).to_be_visible()
def _assert_answer_figure(
page: Page,
app_url: str,
seeded: dict[str, str],
*,
caption: str,
last_brain: str = ".msg.brain",
) -> None:
"""The A4 answer-figure contract in one place (the live, restore,
and shared surfaces share the shape — the shared page's figure is
a plain ``<figure>`, the chat page's an anchor, but the inner
``img``/caption classes are the same): exactly ONE ``.answer-image``
in the brain bubble, its ``<img>`` from the bytes route
(``loading="lazy"``, visible — the bytes really serve), and the
caption/alt settled to *caption* (the summary on the admin
surfaces, the title on the guest one)."""
fig = page.locator(f"{last_brain} .answer-image")
expect(fig).to_have_count(1)
img = fig.locator("img")
expect(img).to_have_count(1)
expect(img).to_be_visible(timeout=15_000)
expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("loading", "lazy")
expect(fig.locator(".answer-image-caption")).to_have_text(
caption, timeout=15_000
)
# The served bytes are a real image (the public bytes route).
resp = httpx.get(app_url + seeded["image_url"], timeout=10)
assert resp.status_code == 200
assert resp.headers["content-type"].startswith("image/")
assert resp.content == PNG_1X1
def _assert_image_chip(
page: Page, seeded: dict[str, str], *, last_brain: str = ".msg.brain"
) -> None:
"""The AUTO-CITE (phase 124 task 05) in the phase-130 cluster
shape (task 01, TODO L6): an image doc gets NO chip — its figure
IS the citation — the compact figure (``<img>`` from the bytes
route, alt settling to the summary — the admin surface; the
shared page's plain-text chip is asserted separately) sits in the
images row (``.msg-meta.msg-images``)."""
chip = page.locator(f"{last_brain} .source-chip")
expect(chip).to_have_count(0)
row = page.locator(f"{last_brain} .msg-meta.msg-images")
expect(row).to_have_count(1)
expect(row).to_have_attribute("aria-label", "Source images")
fig = row.locator(".source-image")
expect(fig).to_have_count(1)
img = fig.locator(".source-image-img")
expect(img).to_be_visible(timeout=15_000)
expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("alt", seeded["description"], timeout=15_000)
def _body_child_classes(page: Page) -> list[str]:
"""The LAST brain bubble's ``.msg-body`` direct children's class
names, in DOM order — the A4 position pin (bubble < answer-image
< the first .msg-meta-family source row)."""
return page.evaluate(
"""() => {
const body = [...document.querySelectorAll(".msg.brain")].pop()
.querySelector(".msg-body");
return [...body.children].map((el) => el.className);
}"""
)
# ---------------------------------------------------------------------------
# 1. App A (feature ON) — the feature flow: the scripted show_image
# turn (ZERO reads) renders the answer figure + the auto-cited chip
# ---------------------------------------------------------------------------
def test_feature_flow_answer_figure_chips_and_zero_reads(
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
) -> None:
"""The ``show me the picture of …`` trigger: the mock scripts ONLY
a ``show_image`` call (NO ``read`` — the zero-read turn). Asserts:
(1) the brain bubble's ``.msg-body`` carries ``.answer-image`` with
an ``<img>`` from the bytes route (200, image Content-Type) and
``loading="lazy"``; (2) the figure sits BELOW the answer text and
ABOVE the source clusters (A4 — the image-only KB renders the
images row, ``.msg-meta.msg-images`` — phase 130 task 01: no chip
row); (3) the answer TEXT is non-empty and quotes the seed's tail
(the "summaries still provided" contract + the echo proving the
seed was in the prompt); (4) the images row carries the image
doc's figure — no chip (phase 130: the figure IS the citation) —
the AUTO-CITE proof (the mock made ZERO ``read`` calls: only
``show_image`` — asserted on the wire); (5) the "Nearby docs" row
does NOT repeat the doc. Plus the persistence pin: the saved
brain record carries the ``answer_image`` ref (the task-04 frame
field, round-tripped)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", FEATURE_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
# The answer TEXT is non-empty (the "summaries still provided"
# contract) — the tail echo lands LAST (the terminal signal): it
# quotes the <documents> seed's tail, i.e. the description.
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# (1) The answer figure: the bytes route, lazy, alt/caption the
# summary (the fetchContentSummary pattern settles both).
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
# (2) A4 position: below the answer text, above the source
# clusters — the image-only KB renders the images row
# (.msg-meta.msg-images, phase 130 task 01: no chip row).
classes = _body_child_classes(page)
assert "bubble" in classes, f"no answer bubble in .msg-body: {classes}"
assert "answer-image" in classes, f"no .answer-image in .msg-body: {classes}"
meta_idx = next(
(i for i, c in enumerate(classes) if c.startswith("msg-meta")), None
)
assert meta_idx is not None, f"no source row in .msg-body: {classes}"
assert (
classes.index("bubble")
< classes.index("answer-image")
< meta_idx
), f"A4 order violated (bubble < answer-image < source row): {classes}"
# (4) The images row carries the image doc's figure (no chip —
# phase 130) — the AUTO-CITE (the model never read it; the
# seed IS its entire content, so suggested == read for it).
_assert_image_chip(page, seeded)
# (5) The "Nearby docs" row does NOT repeat the image doc — the
# related tier excludes cited members (the single-doc KB has
# no related tier at all: the row is absent).
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
# The WIRE pin: exactly two model calls (the show_image round +
# the final answer), the FIRST request offered the FOUR tools
# (three base + show_image — the toggle-ON offer gate), and the
# turn executed EXACTLY one tool call: show_image — NO read (the
# zero-read turn that makes the chip assertion meaningful).
captured = _mock_capture(mock_llm)
assert len(captured) == 2, (
f"expected exactly 2 model calls (call + answer), got {len(captured)}"
)
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep", "show_image"]
assert _executed_tool_calls(captured) == ["show_image"]
tool_results = [
m.get("content") or ""
for m in captured[1]["messages"]
if m.get("role") == "tool"
]
assert any("Image shown to the user" in r for r in tool_results), (
f"the confirming tool result never reached the model: {tool_results}"
)
# The persistence pin: the auto-saved brain record carries the
# answer_image ref (source/path/title/image_url — the task-04
# frame field; the restore + shared flows assert on its render).
row = _wait_saved_row(app_url, _admin_cookies(page), _auto_title(FEATURE_QUESTION))
record = _brain_record(_chat_messages(app_url, _admin_cookies(page), str(row["id"])))
assert record.get("answer_image") == {
"source": SOURCE_NAME,
"path": DOC_PATH,
"title": seeded["title"],
"image_url": seeded["image_url"],
}, f"the brain record's answer_image is wrong: {record.get('answer_image')!r}"
# ---------------------------------------------------------------------------
# 2. App A — reload restores the figure; the shared page shows it
# ---------------------------------------------------------------------------
def test_reload_restores_figure_and_shared_page_shows_it(
page: Page,
app_url: str,
seeded: dict[str, str],
mock_llm: int,
browser: Browser,
) -> None:
"""A second feature turn (a unique question — unique auto-title on
the shared DB), then a page RELOAD: the restored brain bubble
shows the figure again (the ``answer_image`` persistence through
the ``bor.chat.v1`` restore path — pixel-parity with the live
render), and the existing Share flow's link — opened in a FRESH
anonymous context — renders the figure on the shared page (the
bytes route is public; the guest caption keeps the title — the
content endpoint is user-gated, so the summary fetch is skipped
there)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", RESTORE_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
_assert_image_chip(page, seeded)
# The auto-save settled (the row exists with both messages) before
# the reload — the Share click later shares the linked row, or
# re-creates it unlinked (either path is owner-locked; the row
# lookup keys on the unique auto-title).
cookies = _admin_cookies(page)
_wait_saved_row(app_url, cookies, _auto_title(RESTORE_QUESTION))
# RELOAD: the restored brain bubble shows the figure again — the
# localStorage restore (rememberBrainTurn's save point ran
# synchronously in the done handler, before the send button
# re-enabled, so the record is durable before we reload).
page.reload()
expect(page.locator("#kb-banner")).to_be_hidden()
restored = page.locator(".msg.brain .bubble").last
expect(restored).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(restored).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
_assert_image_chip(page, seeded)
# SHARE: the existing chat-page Share flow → a fresh anonymous
# context sees the figure (and the plain-text chip).
_grant_clipboard(page, app_url)
_click_share(app_url, page)
# The share landed on the pre-existing row (the linked branch —
# the idempotent share) or a fresh row (the unlinked save-and-
# share) — either carries the token; the unique auto-title keeps
# the lookup unambiguous on the shared DB.
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
rows = [
c for c in r.json()["chats"]
if c["title"] == _auto_title(RESTORE_QUESTION)
]
token_row = next((c for c in rows if c.get("share_url")), None)
assert token_row is not None and token_row.get("share_url"), (
f"no share_url on any shared row: {rows!r}"
)
share_link = str(token_row["share_url"])
assert share_link.startswith("/shared/")
anon = browser.new_page(viewport={"width": 1280, "height": 800})
try:
anon.goto(app_url + share_link)
expect(anon.locator("#shared-title")).to_have_text(
_auto_title(RESTORE_QUESTION), timeout=30_000
)
expect(anon.locator(".msg.brain .bubble").last).to_contain_text(
DESCRIPTION_TAIL, timeout=30_000
)
# The shared page renders the figure — the guest caption keeps
# the TITLE (no summary fetch — the content endpoint is
# user-gated), the img is from the public bytes route.
_assert_answer_figure(anon, app_url, seeded, caption=seeded["title"])
# The shared chip is PLAIN TEXT (guests cannot open documents):
chip = anon.locator(".msg.brain .source-chip")
expect(chip).to_have_count(1)
expect(chip).to_have_text(f"{SOURCE_NAME}/{DOC_PATH}")
# The shared page is zero-controls: the figure is a plain
# <figure>, never an anchor (the owner-locked contract).
assert anon.locator(".msg.brain a.answer-image").count() == 0
finally:
anon.close()
# ---------------------------------------------------------------------------
# 3. App A — the bug-fix-only flow: ZERO tool calls, the citation
# STILL renders (the pure zero-read path — the owner's report)
# ---------------------------------------------------------------------------
def test_bugfix_only_flow_zero_tool_calls_still_chips(
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
) -> None:
"""The ``which diagram is in your notes`` trigger: the mock
answers straight from the seed with ZERO tool calls (no
``show_image`` — the clause's flow is not triggered — and no
``read``): the sources block STILL carries the image doc's
citation figure (no chip — phase 130; the bug fix on the pure
zero-read path — the assertion the owner's report describes), the
answer text quotes the seeded description's tail (the description
WAS in the prompt), and NO answer figure renders (the model never
called the tool)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", DIAGRAM_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
# The answer quotes the seeded image doc's block tail (the
# description's end — the seed was in the prompt) and settles.
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# ZERO tool calls — on the wire (exactly ONE model call, no
# tool_calls message at all) and in the DOM (no tool record).
captured = _mock_capture(mock_llm)
assert len(captured) == 1, (
f"the bug-fix flow must make exactly 1 model call, got {len(captured)}"
)
assert _executed_tool_calls(captured) == [], (
f"the bug-fix flow must make ZERO tool calls: {_executed_tool_calls(captured)}"
)
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
# NO answer figure (the model never called show_image) — but the
# sources block STILL carries the image doc's citation figure
# (no chip — phase 130; the auto-cite: suggested == read for an
# image doc, no read performed).
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
_assert_image_chip(page, seeded)
# ---------------------------------------------------------------------------
# 4. App B (default off) — the toggle-off contract: no show_image
# offered, no answer figure, no answer_image key in the record
# ---------------------------------------------------------------------------
def test_toggle_off_flow_no_tool_no_answer_image_key(
page: Page,
default_app_url: str,
seeded_off: dict[str, str],
mock_llm: int,
) -> None:
"""The same feature trigger question on the default-OFF app: the
prompt clause is ABSENT, so the mock's trigger never fires (the
plain composed answer streams instead) — and the contract holds
at every surface: the mock NEVER sees ``show_image`` among the
offered tools (the request capture carries exactly the THREE base
tools), the brain bubble has NO ``.answer-image``, and the saved
brain record carries NO ``answer_image`` key (the byte-identical-
off contract — the key is absent, never null)."""
login(page, default_app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", FEATURE_OFF_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# No answer figure anywhere in the brain bubble.
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
# The WIRE pin: the offer list is EXACTLY the three base tools
# (no show_image — the toggle-OFF gate on the offer), and the
# turn made no tool calls at all (the mock's feature trigger is
# clause-gated — it never fired, so the plain answer streamed in
# one call).
captured = _mock_capture(mock_llm)
assert len(captured) == 1, (
f"expected exactly 1 model call (no tool round), got {len(captured)}"
)
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep"], (
f"show_image leaked into the default-off offer: "
f"{_offered_tool_names(captured[0])}"
)
assert _executed_tool_calls(captured) == []
# The saved brain record carries NO answer_image key (ABSENT — the
# byte-identical-off contract; the phase-113 422-silence pin is
# the positive half of this, in the App A flow + the integration
# suite). The chip, though, IS still auto-cited (the chips fix is
# not toggle-gated — asserted in the next test).
row = _wait_saved_row(
default_app_url,
_admin_cookies(page),
_auto_title(FEATURE_OFF_QUESTION),
)
record = _brain_record(
_chat_messages(default_app_url, _admin_cookies(page), str(row["id"]))
)
assert "answer_image" not in record, (
f"the default-off brain record must NOT carry answer_image: "
f"{record.get('answer_image')!r}"
)
# ---------------------------------------------------------------------------
# 5. App B (default off) — the bug-fix trigger STILL cites the image
# doc (the auto-cite fix belongs to BOR_IMAGES, not BOR_ANSWER_IMAGES)
# ---------------------------------------------------------------------------
def test_toggle_off_bugfix_trigger_still_chips(
page: Page, default_app_url: str, seeded_off: dict[str, str], mock_llm: int
) -> None:
"""The bug-fix trigger on the default-OFF app (the clause is
absent — the trigger does NOT require it): ZERO tool calls, the
answer quotes the seeded description's tail, and the sources
block STILL carries the image doc's citation figure (no chip —
phase 130) — the auto-cite fix is NOT gated by
``BOR_ANSWER_IMAGES`` (it belongs to the ``BOR_IMAGES``-indexed
docs, per the phase design)."""
login(page, default_app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", DIAGRAM_OFF_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
captured = _mock_capture(mock_llm)
assert len(captured) == 1
assert _executed_tool_calls(captured) == []
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
_assert_image_chip(page, seeded_off)