**Phase 130 — final verification pass: 100% green, complete.** - **Verified:** chips-only row (no chip for image docs) → images-only row (`.msg-images`) → related row → `.msg-actions` last; `getActionsRow` used by all 3 buttons + stopped/failed notes; `markLastRetryable` save-re-append hack gone; in-place chip fallback on image-load failure; CSS right-align with per-button auto margins removed — implementation matches the design in every detail. - **Completion criteria:** (1) mixed citation renders chips→images→actions-last, DOM-asserted ✅ (2) all 3 buttons in one right-aligned `.msg-actions`; non-admin sees Retry-only; restore path identical ✅ (3) 404-bytes figure → plain chip, unit-pinned ✅ (4) E2E isolated ✅ (5) no behavior change (modal affordances, related tier, ARIA names) ✅ - **Tests/lint/coverage:** `uv run pytest tests/e2e/test_chat_answer_clusters.py -v --no-cov` → 4 passed; `uv run pytest --cov=app --cov-report=term-missing` → 3061 passed, **99%** app coverage; the 4 E2E files this phase modified (test_answer_images, test_image_documents, test_pdf_support, test_responsive_polish) → 22 passed in isolation; `uv run ruff check .` clean, `uv run pyright` → 0 errors. - **Docs:** README.md — verified consistent in full (page table vs routes, env-var coverage vs `.env.example`, phase-130 cluster wording already in tree); no changes needed. AGENTS.md — fixed one pre-existing inconsistency: rule 5 still cited the retired "centered 46rem chat column"; now states the 72rem container contract (phase 100). - **Decisions:** PLAN.md §7.1 still says 46rem, but it's owner-managed (phase 100 records the supersession pending owner sign-off) and out of my editable scope — AGENTS.md now cross-references the supersession. - **Next pending phase:** `131_read_line_ranges` (todo/).
990 lines
43 KiB
Python
990 lines
43 KiB
Python
"""Phase 124 E2E (Playwright): image source chips + LLM-chosen answer images.
|
||
|
||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_answer_images.py -v --no-cov
|
||
|
||
The suite proves BOTH halves of TODO.md L3 on the real user path, plus
|
||
the toggle-off contract — two module apps on dedicated ports (the
|
||
phase-122 ``test_image_documents.py`` pattern), sharing one deterministic
|
||
mock LLM and the shared e2e Postgres (truncate/re-import per app):
|
||
|
||
* **App A (feature ON)** — ``BOR_IMAGES=true`` +
|
||
``BOR_ANSWER_IMAGES=true``: the standalone PNG arrives through the
|
||
REAL admin upload flow + the RAG page's "Sync sources" (the mock's
|
||
vision model writes the byte-stable description), then:
|
||
- **Feature flow** — the ``show me the picture of …`` trigger (the
|
||
mock's phase-124 scripted trigger, ``tests/e2e/mock_llm.py``): the
|
||
mock scripts ONLY a ``show_image`` call (NO ``read`` — the ZERO-READ
|
||
turn, the shape that makes the chip assertion meaningful). The
|
||
brain bubble shows the ANSWER FIGURE (``.answer-image`` — ``<img>``
|
||
from the bytes route, ``loading="lazy"``, below the text and above
|
||
the source clusters per A4, alt/caption settling to the summary),
|
||
the answer TEXT is still streamed in full (the tail echo of the
|
||
``<documents>`` seed proves the description was in the prompt), the
|
||
sources block carries the image doc's citation as the compact
|
||
figure in the images row (``.msg-meta.msg-images`` — phase 130
|
||
task 01: NO chip for an image doc, the figure IS the citation;
|
||
the AUTO-CITE proof — the mock made zero ``read`` calls, asserted
|
||
on the mock's request capture), and the "Nearby docs" row does not
|
||
repeat the doc. The wire pin: the first model request offered
|
||
FOUR tools (the three base + ``show_image``) and the turn executed
|
||
exactly one ``show_image`` — the saved brain record persists the
|
||
``answer_image`` ref.
|
||
- **Restore + share** — a second feature turn, then a page RELOAD:
|
||
the restored brain bubble shows the figure again (the
|
||
``answer_image`` persistence through the ``bor.chat.v1`` restore
|
||
path), and the shared link (the existing chat-page Share flow, a
|
||
fresh anonymous context) renders the figure on the shared page.
|
||
- **Bug-fix-only flow** — the ``which diagram is in your notes``
|
||
trigger (the mock answers straight from the seed with ZERO tool
|
||
calls — no ``show_image``, no ``read``): the sources block STILL
|
||
carries the image doc's citation figure (no chip — phase 130;
|
||
the bug fix on the pure zero-read path — the assertion the owner's
|
||
report describes), the answer text quotes the seeded
|
||
description's tail, and NO answer figure renders (the model never
|
||
called the tool).
|
||
* **App B (default off)** — ``BOR_IMAGES=true``, ``BOR_ANSWER_IMAGES``
|
||
pinned to the code default ``false`` (EXPLICITLY — process env ranks
|
||
above an operator's local gitignored ``.env``, so the default-off
|
||
contract under test cannot leak in, the phase-122 default-app
|
||
convention; the production default is unset = the same ``False``):
|
||
the same feature trigger question → the clause is absent from the
|
||
prompt, so the mock's trigger NEVER fires — the mock never sees
|
||
``show_image`` among the offered tools (the request capture carries
|
||
exactly the THREE base tools), the brain bubble has NO answer
|
||
figure, and the saved brain record carries NO ``answer_image`` key
|
||
(the byte-identical-off contract). The bug-fix trigger STILL cites
|
||
the image doc (the figure, no chip — phase 130; the auto-cite fix is
|
||
NOT gated by ``BOR_ANSWER_IMAGES`` — it belongs to the
|
||
``BOR_IMAGES``-indexed docs, per the phase design).
|
||
|
||
DB isolation: the suites share one Postgres; ``saved_chats`` survives
|
||
the truncation (the house pattern), so every question text here is
|
||
UNIQUE (its auto-title is therefore unique) and the row lookups key on
|
||
it. App B's module seed truncates the KB tables App A's tests used —
|
||
App B's tests run AFTER App A's (file order) and never look back.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import base64
|
||
import json
|
||
import os
|
||
import re
|
||
import subprocess
|
||
import sys
|
||
import time
|
||
import zipfile
|
||
from collections.abc import Iterator
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import httpx
|
||
import pytest
|
||
from playwright.sync_api import Browser, Page, expect
|
||
from sqlalchemy import select, text
|
||
|
||
from app.config import Settings
|
||
from app.db import SessionLocal
|
||
from app.models import Document
|
||
from e2e.auth_helpers import login
|
||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
|
||
|
||
REPO = Path(__file__).resolve().parents[2]
|
||
GIT_SOURCES_URL = "/git-sources.html"
|
||
SOURCE_NAME = "e2e-answer-img" # the archive stem (archive_source_name)
|
||
DOC_PATH = "pic.png"
|
||
|
||
#: The mock's deterministic answer tail (every composed answer ends
|
||
#: with it — the turn-settled wait signal, the house convention).
|
||
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
||
#: The TAIL of the fixture description (``IMAGE_DESCRIPTION_ANSWER`` —
|
||
#: the image doc's entire content): both phase-124 mock answers quote
|
||
#: it (the feature flow's ``<documents>`` tail echo, the bug-fix flow's
|
||
#: block-content tail) — asserting it proves the SEED was in the
|
||
#: prompt and reached the rendered answer.
|
||
DESCRIPTION_TAIL = "Title: Homelab Network Map."
|
||
|
||
#: App A (feature ON) — the ``show_image`` trigger question (cosine
|
||
#: 0.4552 against the mock's token-overlap embeddings, well over the
|
||
#: mock-calibrated 0.30 gate). Unique auto-title (the DB is shared).
|
||
FEATURE_QUESTION = "Show me the picture of the homelab network diagram?"
|
||
#: The restore/share flow's variant (the trigger substring is kept —
|
||
#: cosine 0.4358, still over the gate).
|
||
RESTORE_QUESTION = "Show me the picture of the homelab network diagram? (restore)"
|
||
#: App B (default off) — the feature trigger's variant (the clause is
|
||
#: absent, so the mock answers with the plain composed answer — cosine
|
||
#: 0.4187, the turn still grounds).
|
||
FEATURE_OFF_QUESTION = (
|
||
"Show me the picture of the homelab network diagram? (toggle off)"
|
||
)
|
||
#: The bug-fix-only trigger question (cosine 0.3410): the mock answers
|
||
#: straight from the seed with ZERO tool calls and quotes the image
|
||
#: doc's block tail — the chip must appear anyway (the auto-cite).
|
||
DIAGRAM_QUESTION = "Which diagram is in your notes about the homelab network?"
|
||
#: App B's bug-fix variant (the chips fix is not toggle-gated — cosine
|
||
#: 0.3113, still over the mock-calibrated gate).
|
||
DIAGRAM_OFF_QUESTION = (
|
||
"Which diagram is in your notes about the homelab network? (default off)"
|
||
)
|
||
|
||
#: A real 1×1 transparent PNG (the unit/integration suites' fixture —
|
||
#: the pipeline is content-agnostic, the well-formed bytes keep the
|
||
#: upload + serve + render path honest).
|
||
PNG_1X1 = base64.b64decode(
|
||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
|
||
"AAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
|
||
)
|
||
|
||
UPLOAD_TIMEOUT_MS = 30_000
|
||
SYNC_TIMEOUT_MS = 60_000
|
||
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Module apps: feature ON (App A) and default-OFF (App B). Each owns
|
||
# its port + its scratch upload/source/image dirs.
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def _app_env(
|
||
mock_llm: int, app_port: int, *, answer_images: str, scratch: Path
|
||
) -> dict[str, str]:
|
||
"""The conftest app env (leak guards included) with the phase-124
|
||
knobs: BOTH apps index images (``BOR_IMAGES=true`` — the fixture's
|
||
PNG must become a document on either), and ``BOR_ANSWER_IMAGES``
|
||
is pinned EXPLICITLY per app (``true`` for App A, ``false`` for
|
||
App B — process env ranks above an operator's local gitignored
|
||
``.env``, so the contract under test cannot leak in either
|
||
direction, the phase-122 default-app convention; App B's explicit
|
||
``false`` IS the production default — unset is ``False``). The
|
||
image/upload homes are in the suite's scratch dir (the app under
|
||
test must not write image copies into the owner's real
|
||
``~/bor-sources``)."""
|
||
env = dict(os.environ)
|
||
env.pop("DEBUGPY", None)
|
||
env["BOR_ENVIRONMENT"] = "e2e"
|
||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||
env["BOR_LLM_BASE_URL"] = (
|
||
"https://aipi.reeseapps.com/v1"
|
||
if USE_REAL_LLM
|
||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||
)
|
||
# Mock-calibrated gate (conftest pattern) — both trigger questions
|
||
# ground on these values (see the question constants above).
|
||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||
# Phase 67: instant retry waits + the code-default budget.
|
||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
|
||
env.setdefault(
|
||
"BOR_DATABASE_URL",
|
||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||
)
|
||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||
# Leak guards (conftest pattern).
|
||
env["BOR_GIT_SOURCES"] = ""
|
||
env["BOR_DOCS_REPO"] = ""
|
||
env["BOR_SUGGESTIONS"] = json.dumps(
|
||
Settings.model_fields["suggestions"].default
|
||
)
|
||
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
|
||
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
|
||
# Phase 124: both apps index images; only the ANSWER side differs.
|
||
env["BOR_IMAGES"] = "true"
|
||
env["BOR_ANSWER_IMAGES"] = answer_images
|
||
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
|
||
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
|
||
env["BOR_IMAGE_DIR"] = str(scratch / "images")
|
||
return env
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
|
||
"""App A (feature ON) — ``BOR_IMAGES=true`` +
|
||
``BOR_ANSWER_IMAGES=true`` (the conftest session app is never
|
||
started in this isolated run, so no port clash)."""
|
||
scratch = tmp_path_factory.mktemp("bor_answer_img_on")
|
||
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_ON", "8162"))
|
||
proc = subprocess.Popen(
|
||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
|
||
cwd=REPO,
|
||
env=_app_env(mock_llm, port, answer_images="true", scratch=scratch),
|
||
)
|
||
try:
|
||
_wait_http(f"http://127.0.0.1:{port}/api/health")
|
||
yield f"http://127.0.0.1:{port}"
|
||
finally:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=10)
|
||
except subprocess.TimeoutExpired:
|
||
proc.kill()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_url(app_server: str) -> str:
|
||
return app_server
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def default_app_server(
|
||
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[str]:
|
||
"""App B (default off) — ``BOR_IMAGES=true`` +
|
||
``BOR_ANSWER_IMAGES=false`` pinned explicitly (the production
|
||
default — the toggle-off contract; only the negative tests start
|
||
it, and they run AFTER App A's — file order)."""
|
||
scratch = tmp_path_factory.mktemp("bor_answer_img_off")
|
||
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_OFF", "8163"))
|
||
proc = subprocess.Popen(
|
||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
|
||
cwd=REPO,
|
||
env=_app_env(mock_llm, port, answer_images="false", scratch=scratch),
|
||
)
|
||
try:
|
||
_wait_http(f"http://127.0.0.1:{port}/api/health")
|
||
yield f"http://127.0.0.1:{port}"
|
||
finally:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=10)
|
||
except subprocess.TimeoutExpired:
|
||
proc.kill()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def default_app_url(default_app_server: str) -> str:
|
||
return default_app_server
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def zip_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||
"""The fixture archive: ONE standalone PNG at the root — source
|
||
``e2e-answer-img``, document path ``pic.png`` (the mock's seeded-
|
||
image-doc pick is deterministic: the only image-extension block)."""
|
||
root = tmp_path_factory.mktemp("bor_answer_img_zip")
|
||
(root / DOC_PATH).write_bytes(PNG_1X1)
|
||
archive = root / f"{SOURCE_NAME}.zip"
|
||
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
|
||
zf.write(root / DOC_PATH, arcname=DOC_PATH)
|
||
return archive
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# DB + UI + mock-capture helpers
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def _truncate_all() -> None:
|
||
"""Fresh KB + registry (the E2E isolation pattern): the suites
|
||
share one Postgres, and a leftover row would corrupt the counts.
|
||
``sources_meta`` (the KB generation counter) resets with the KB —
|
||
the sync re-creates the row. ``saved_chats`` is never touched by
|
||
the truncation — the unique question texts keep the row lookups
|
||
unambiguous, and :func:`_cleanup_saved_rows` owns this suite's
|
||
rows (re-runnable on the shared DB)."""
|
||
with SessionLocal() as db:
|
||
db.execute(text(
|
||
"TRUNCATE chunks, documents, query_log, kb_overview, "
|
||
"git_sources, sources_meta, folder_summaries"
|
||
))
|
||
db.commit()
|
||
|
||
|
||
def _health_db(app_url: str) -> bool:
|
||
try:
|
||
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
|
||
except Exception: # noqa: BLE001 — unreachable is the skip case
|
||
return False
|
||
|
||
|
||
def _mock_capture(mock_llm: int) -> list[dict[str, Any]]:
|
||
"""The mock's recent chat-completions request bodies (oldest
|
||
first) — the wire lens on the app's tool OFFER and the turn's
|
||
tool CALLS (the mock is the only E2E lens on the LLM's context)."""
|
||
r = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=5)
|
||
assert r.status_code == 200
|
||
return r.json()
|
||
|
||
|
||
def _mock_capture_reset(mock_llm: int) -> None:
|
||
r = httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=5)
|
||
assert r.status_code == 200
|
||
|
||
|
||
def _executed_tool_calls(captured: list[dict[str, Any]]) -> list[str]:
|
||
"""Every tool call the MODEL made across *captured* requests (the
|
||
assistant ``tool_calls`` messages), in order — the executed-call
|
||
lens (a refusal still appears here; it is the model's call)."""
|
||
return [
|
||
tc["function"]["name"]
|
||
for body in captured
|
||
for m in body["messages"]
|
||
if m.get("role") == "assistant" and m.get("tool_calls")
|
||
for tc in m["tool_calls"]
|
||
]
|
||
|
||
|
||
def _offered_tool_names(body: dict[str, Any]) -> list[str]:
|
||
return [t["function"]["name"] for t in (body.get("tools") or [])]
|
||
|
||
|
||
def _upload_zip(page: Page, app_url: str, archive: Path) -> None:
|
||
"""The real admin upload flow: form login → the git-sources page →
|
||
pick the archive → submit → the terminal result line (phase 90:
|
||
"Uploaded <source> — press Sync sources to import it.")."""
|
||
login(page, app_url, next=GIT_SOURCES_URL)
|
||
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
|
||
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
|
||
page.set_input_files("#archive-upload-file", str(archive))
|
||
page.click("#archive-upload-btn")
|
||
result = page.locator("#archive-upload-result")
|
||
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
|
||
expect(result).to_have_text(
|
||
f"Uploaded {SOURCE_NAME} — press Sync sources to import it.",
|
||
timeout=UPLOAD_TIMEOUT_MS,
|
||
)
|
||
|
||
|
||
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||
"""The RAG page's "Sync sources" button (the scan — phase 90 A3):
|
||
click → "Syncing…" → the terminal "Synced HH:MM" label + the fresh
|
||
counts in #sync-result (the never-stale contract)."""
|
||
page.goto(app_url + "/sources.html")
|
||
btn = page.locator("#sync-btn")
|
||
expect(btn).to_be_visible(timeout=30_000)
|
||
expect(page.locator("#sync-label")).to_have_text(
|
||
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
|
||
)
|
||
btn.click()
|
||
expect(btn).to_be_disabled()
|
||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||
expect(btn).to_be_enabled()
|
||
expect(page.locator("#sync-result")).to_have_text(expected_result)
|
||
|
||
|
||
def _seed_image_doc(
|
||
app_url: str, browser: Browser, zip_path: Path
|
||
) -> dict[str, str]:
|
||
"""Upload the single-PNG zip through the admin UI, then scan it
|
||
with the RAG page's Sync button (the mock's vision model describes
|
||
the image in-process). Returns the seeded doc's identity (the
|
||
bytes route + the description + the title) for the render
|
||
assertions. The app's ``/api/config`` flag is asserted first — the
|
||
env override must have reached the app under test, or the whole
|
||
premise of the suite is void."""
|
||
cfg = httpx.get(f"{app_url}/api/config", timeout=5).json()
|
||
assert cfg["images"] is True, "the module app must run with BOR_IMAGES=true"
|
||
_truncate_all()
|
||
page = browser.new_page(viewport={"width": 1280, "height": 800})
|
||
try:
|
||
_upload_zip(page, app_url, zip_path)
|
||
_sync_via_ui(page, app_url, expected_result="1 added")
|
||
with SessionLocal() as db:
|
||
doc = db.scalar(
|
||
select(Document).where(
|
||
Document.source == SOURCE_NAME, Document.path == DOC_PATH
|
||
)
|
||
)
|
||
assert doc is not None, "the uploaded image must become a document"
|
||
assert doc.is_image is True and doc.image_path is not None
|
||
# content == summary == the mock's description (the ONLY
|
||
# embedded text of the doc — the pipeline contract, phase 122).
|
||
assert doc.content == IMAGE_DESCRIPTION_ANSWER
|
||
assert doc.summary == IMAGE_DESCRIPTION_ANSWER
|
||
return {
|
||
"image_url": f"/api/documents/{doc.id}/image",
|
||
"description": IMAGE_DESCRIPTION_ANSWER,
|
||
"title": doc.title,
|
||
}
|
||
finally:
|
||
page.close()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def seeded(
|
||
app_url: str,
|
||
browser: Browser,
|
||
zip_path: Path,
|
||
) -> Iterator[dict[str, str]]:
|
||
"""App A's KB: the fixture image doc through the real UI path."""
|
||
if not _health_db(app_url):
|
||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||
_cleanup_saved_rows() # a previous run's rows would be stale (new doc id)
|
||
yield _seed_image_doc(app_url, browser, zip_path)
|
||
_cleanup_saved_rows()
|
||
_truncate_all()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def seeded_off(
|
||
default_app_url: str,
|
||
browser: Browser,
|
||
zip_path: Path,
|
||
) -> Iterator[dict[str, str]]:
|
||
"""App B's KB: the SAME fixture image doc through the SAME real
|
||
UI path (images indexed, the answer feature OFF). Runs AFTER all
|
||
of App A's tests (file order) — its truncation wipes App A's KB
|
||
tables, and App B's tests never look back."""
|
||
if not _health_db(default_app_url):
|
||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||
_cleanup_saved_rows()
|
||
yield _seed_image_doc(default_app_url, browser, zip_path)
|
||
_cleanup_saved_rows()
|
||
_truncate_all()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# Saved-chat + share helpers (the test_share_chat.py conventions)
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def _admin_cookies(page: Page) -> dict[str, str]:
|
||
"""The signed session cookies the browser holds after a form login
|
||
— used to call the admin API with plain httpx (the test's API side
|
||
sees exactly what the signed-in browser sees)."""
|
||
return {
|
||
c["name"]: c["value"]
|
||
for c in page.context.cookies()
|
||
if "name" in c and "value" in c
|
||
}
|
||
|
||
|
||
def _auto_title(question: str) -> str:
|
||
"""The phase-50 auto-title convention: the first question,
|
||
whitespace-collapsed, capped at 120 chars."""
|
||
return " ".join(question.split())[:120]
|
||
|
||
|
||
#: This suite's auto-titles (the shared e2e Postgres keeps
|
||
#: ``saved_chats`` across runs — the test_share_chat cleanup
|
||
#: concern): a re-run must not find the previous run's row (same
|
||
#: title, but the KB re-seed minted a NEW doc id — the row's
|
||
#: ``answer_image.image_url`` would no longer match the seed).
|
||
_SUITE_TITLES = tuple(
|
||
_auto_title(q)
|
||
for q in (
|
||
FEATURE_QUESTION,
|
||
RESTORE_QUESTION,
|
||
FEATURE_OFF_QUESTION,
|
||
DIAGRAM_QUESTION,
|
||
DIAGRAM_OFF_QUESTION,
|
||
)
|
||
)
|
||
|
||
|
||
def _cleanup_saved_rows() -> None:
|
||
"""Delete THIS suite's saved rows by title (the house pattern —
|
||
test_share_chat deletes its own rows; here the module-boundary
|
||
fixture calls it, so a re-run starts clean). Best-effort: a
|
||
failed delete (DB unreachable — the skip case) never masks the
|
||
skip; a failed delete with the DB up just re-creates the stale-
|
||
row premise, and the suite fails loudly on the row assertion."""
|
||
try:
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text("DELETE FROM saved_chats WHERE title = ANY(:titles)"),
|
||
{"titles": list(_SUITE_TITLES)},
|
||
)
|
||
db.commit()
|
||
except Exception: # noqa: BLE001 — DB unreachable: the skip handles it
|
||
pass
|
||
|
||
|
||
def _wait_saved_row(
|
||
app_url: str,
|
||
cookies: dict[str, str],
|
||
title: str,
|
||
messages: int = 2,
|
||
) -> dict[str, Any]:
|
||
"""Wait for the auto-saved row (phase 55: auto-saves are SILENT —
|
||
A2 — so there is no status line to wait on). The upsert is
|
||
fire-and-forget from the UI's point of view, so poll the admin
|
||
list until the row with the conversation's auto-title appears with
|
||
the expected message count."""
|
||
deadline = time.monotonic() + 15
|
||
last: dict[str, Any] | None = None
|
||
while time.monotonic() < deadline:
|
||
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
rows = r.json()["chats"]
|
||
last = next((c for c in rows if c["title"] == title), None)
|
||
if last is not None and last["message_count"] >= messages:
|
||
return last
|
||
time.sleep(0.2)
|
||
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
|
||
|
||
|
||
def _chat_messages(
|
||
app_url: str, cookies: dict[str, str], chat_id: str
|
||
) -> list[dict[str, Any]]:
|
||
r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
return r.json()["messages"]
|
||
|
||
|
||
def _brain_record(messages: list[dict[str, Any]]) -> dict[str, Any]:
|
||
brains = [m for m in messages if m.get("who") == "brain"]
|
||
assert brains, f"no brain record in the saved conversation: {messages!r}"
|
||
return brains[-1]
|
||
|
||
|
||
def _grant_clipboard(page: Page, app_url: str) -> None:
|
||
"""Grant the async-clipboard permissions on the admin context
|
||
(test_share_chat.py — headless Chromium still requires the grant
|
||
before ``writeText`` resolves on a secure origin)."""
|
||
page.context.grant_permissions(
|
||
["clipboard-read", "clipboard-write"], origin=app_url
|
||
)
|
||
|
||
|
||
def _click_share(app_url: str, page: Page) -> None:
|
||
"""Press the chat page's Share pill and pin the owner-locked
|
||
outcome on the live region (the test_share_chat.py branch —
|
||
clipboard success vs the inline-link fallback)."""
|
||
page.locator("#share-chat-btn").click()
|
||
if page.evaluate("() => !!navigator.clipboard"):
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link copied.", timeout=15_000
|
||
)
|
||
else:
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link ready — copy it from the field.", timeout=15_000
|
||
)
|
||
expect(page.locator(".share-link-fallback")).to_be_visible()
|
||
|
||
|
||
def _assert_answer_figure(
|
||
page: Page,
|
||
app_url: str,
|
||
seeded: dict[str, str],
|
||
*,
|
||
caption: str,
|
||
last_brain: str = ".msg.brain",
|
||
) -> None:
|
||
"""The A4 answer-figure contract in one place (the live, restore,
|
||
and shared surfaces share the shape — the shared page's figure is
|
||
a plain ``<figure>`, the chat page's an anchor, but the inner
|
||
``img``/caption classes are the same): exactly ONE ``.answer-image``
|
||
in the brain bubble, its ``<img>`` from the bytes route
|
||
(``loading="lazy"``, visible — the bytes really serve), and the
|
||
caption/alt settled to *caption* (the summary on the admin
|
||
surfaces, the title on the guest one)."""
|
||
fig = page.locator(f"{last_brain} .answer-image")
|
||
expect(fig).to_have_count(1)
|
||
img = fig.locator("img")
|
||
expect(img).to_have_count(1)
|
||
expect(img).to_be_visible(timeout=15_000)
|
||
expect(img).to_have_attribute("src", seeded["image_url"])
|
||
expect(img).to_have_attribute("loading", "lazy")
|
||
expect(fig.locator(".answer-image-caption")).to_have_text(
|
||
caption, timeout=15_000
|
||
)
|
||
# The served bytes are a real image (the public bytes route).
|
||
resp = httpx.get(app_url + seeded["image_url"], timeout=10)
|
||
assert resp.status_code == 200
|
||
assert resp.headers["content-type"].startswith("image/")
|
||
assert resp.content == PNG_1X1
|
||
|
||
|
||
def _assert_image_chip(
|
||
page: Page, seeded: dict[str, str], *, last_brain: str = ".msg.brain"
|
||
) -> None:
|
||
"""The AUTO-CITE (phase 124 task 05) in the phase-130 cluster
|
||
shape (task 01, TODO L6): an image doc gets NO chip — its figure
|
||
IS the citation — the compact figure (``<img>`` from the bytes
|
||
route, alt settling to the summary — the admin surface; the
|
||
shared page's plain-text chip is asserted separately) sits in the
|
||
images row (``.msg-meta.msg-images``)."""
|
||
chip = page.locator(f"{last_brain} .source-chip")
|
||
expect(chip).to_have_count(0)
|
||
row = page.locator(f"{last_brain} .msg-meta.msg-images")
|
||
expect(row).to_have_count(1)
|
||
expect(row).to_have_attribute("aria-label", "Source images")
|
||
fig = row.locator(".source-image")
|
||
expect(fig).to_have_count(1)
|
||
img = fig.locator(".source-image-img")
|
||
expect(img).to_be_visible(timeout=15_000)
|
||
expect(img).to_have_attribute("src", seeded["image_url"])
|
||
expect(img).to_have_attribute("alt", seeded["description"], timeout=15_000)
|
||
|
||
|
||
def _body_child_classes(page: Page) -> list[str]:
|
||
"""The LAST brain bubble's ``.msg-body`` direct children's class
|
||
names, in DOM order — the A4 position pin (bubble < answer-image
|
||
< the first .msg-meta-family source row)."""
|
||
return page.evaluate(
|
||
"""() => {
|
||
const body = [...document.querySelectorAll(".msg.brain")].pop()
|
||
.querySelector(".msg-body");
|
||
return [...body.children].map((el) => el.className);
|
||
}"""
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 1. App A (feature ON) — the feature flow: the scripted show_image
|
||
# turn (ZERO reads) renders the answer figure + the auto-cited chip
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_feature_flow_answer_figure_chips_and_zero_reads(
|
||
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
|
||
) -> None:
|
||
"""The ``show me the picture of …`` trigger: the mock scripts ONLY
|
||
a ``show_image`` call (NO ``read`` — the zero-read turn). Asserts:
|
||
(1) the brain bubble's ``.msg-body`` carries ``.answer-image`` with
|
||
an ``<img>`` from the bytes route (200, image Content-Type) and
|
||
``loading="lazy"``; (2) the figure sits BELOW the answer text and
|
||
ABOVE the source clusters (A4 — the image-only KB renders the
|
||
images row, ``.msg-meta.msg-images`` — phase 130 task 01: no chip
|
||
row); (3) the answer TEXT is non-empty and quotes the seed's tail
|
||
(the "summaries still provided" contract + the echo proving the
|
||
seed was in the prompt); (4) the images row carries the image
|
||
doc's figure — no chip (phase 130: the figure IS the citation) —
|
||
the AUTO-CITE proof (the mock made ZERO ``read`` calls: only
|
||
``show_image`` — asserted on the wire); (5) the "Nearby docs" row
|
||
does NOT repeat the doc. Plus the persistence pin: the saved
|
||
brain record carries the ``answer_image`` ref (the task-04 frame
|
||
field, round-tripped)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
page.fill("#message-input", FEATURE_QUESTION)
|
||
page.click("#send-btn")
|
||
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
# The answer TEXT is non-empty (the "summaries still provided"
|
||
# contract) — the tail echo lands LAST (the terminal signal): it
|
||
# quotes the <documents> seed's tail, i.e. the description.
|
||
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
|
||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
|
||
# (1) The answer figure: the bytes route, lazy, alt/caption the
|
||
# summary (the fetchContentSummary pattern settles both).
|
||
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
|
||
|
||
# (2) A4 position: below the answer text, above the source
|
||
# clusters — the image-only KB renders the images row
|
||
# (.msg-meta.msg-images, phase 130 task 01: no chip row).
|
||
classes = _body_child_classes(page)
|
||
assert "bubble" in classes, f"no answer bubble in .msg-body: {classes}"
|
||
assert "answer-image" in classes, f"no .answer-image in .msg-body: {classes}"
|
||
meta_idx = next(
|
||
(i for i, c in enumerate(classes) if c.startswith("msg-meta")), None
|
||
)
|
||
assert meta_idx is not None, f"no source row in .msg-body: {classes}"
|
||
assert (
|
||
classes.index("bubble")
|
||
< classes.index("answer-image")
|
||
< meta_idx
|
||
), f"A4 order violated (bubble < answer-image < source row): {classes}"
|
||
|
||
# (4) The images row carries the image doc's figure (no chip —
|
||
# phase 130) — the AUTO-CITE (the model never read it; the
|
||
# seed IS its entire content, so suggested == read for it).
|
||
_assert_image_chip(page, seeded)
|
||
|
||
# (5) The "Nearby docs" row does NOT repeat the image doc — the
|
||
# related tier excludes cited members (the single-doc KB has
|
||
# no related tier at all: the row is absent).
|
||
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
|
||
|
||
# The WIRE pin: exactly two model calls (the show_image round +
|
||
# the final answer), the FIRST request offered the FOUR tools
|
||
# (three base + show_image — the toggle-ON offer gate), and the
|
||
# turn executed EXACTLY one tool call: show_image — NO read (the
|
||
# zero-read turn that makes the chip assertion meaningful).
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 2, (
|
||
f"expected exactly 2 model calls (call + answer), got {len(captured)}"
|
||
)
|
||
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep", "show_image"]
|
||
assert _executed_tool_calls(captured) == ["show_image"]
|
||
tool_results = [
|
||
m.get("content") or ""
|
||
for m in captured[1]["messages"]
|
||
if m.get("role") == "tool"
|
||
]
|
||
assert any("Image shown to the user" in r for r in tool_results), (
|
||
f"the confirming tool result never reached the model: {tool_results}"
|
||
)
|
||
|
||
# The persistence pin: the auto-saved brain record carries the
|
||
# answer_image ref (source/path/title/image_url — the task-04
|
||
# frame field; the restore + shared flows assert on its render).
|
||
row = _wait_saved_row(app_url, _admin_cookies(page), _auto_title(FEATURE_QUESTION))
|
||
record = _brain_record(_chat_messages(app_url, _admin_cookies(page), str(row["id"])))
|
||
assert record.get("answer_image") == {
|
||
"source": SOURCE_NAME,
|
||
"path": DOC_PATH,
|
||
"title": seeded["title"],
|
||
"image_url": seeded["image_url"],
|
||
}, f"the brain record's answer_image is wrong: {record.get('answer_image')!r}"
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 2. App A — reload restores the figure; the shared page shows it
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_reload_restores_figure_and_shared_page_shows_it(
|
||
page: Page,
|
||
app_url: str,
|
||
seeded: dict[str, str],
|
||
mock_llm: int,
|
||
browser: Browser,
|
||
) -> None:
|
||
"""A second feature turn (a unique question — unique auto-title on
|
||
the shared DB), then a page RELOAD: the restored brain bubble
|
||
shows the figure again (the ``answer_image`` persistence through
|
||
the ``bor.chat.v1`` restore path — pixel-parity with the live
|
||
render), and the existing Share flow's link — opened in a FRESH
|
||
anonymous context — renders the figure on the shared page (the
|
||
bytes route is public; the guest caption keeps the title — the
|
||
content endpoint is user-gated, so the summary fetch is skipped
|
||
there)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
page.fill("#message-input", RESTORE_QUESTION)
|
||
page.click("#send-btn")
|
||
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
|
||
_assert_image_chip(page, seeded)
|
||
|
||
# The auto-save settled (the row exists with both messages) before
|
||
# the reload — the Share click later shares the linked row, or
|
||
# re-creates it unlinked (either path is owner-locked; the row
|
||
# lookup keys on the unique auto-title).
|
||
cookies = _admin_cookies(page)
|
||
_wait_saved_row(app_url, cookies, _auto_title(RESTORE_QUESTION))
|
||
|
||
# RELOAD: the restored brain bubble shows the figure again — the
|
||
# localStorage restore (rememberBrainTurn's save point ran
|
||
# synchronously in the done handler, before the send button
|
||
# re-enabled, so the record is durable before we reload).
|
||
page.reload()
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
restored = page.locator(".msg.brain .bubble").last
|
||
expect(restored).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
|
||
expect(restored).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
|
||
_assert_image_chip(page, seeded)
|
||
|
||
# SHARE: the existing chat-page Share flow → a fresh anonymous
|
||
# context sees the figure (and the plain-text chip).
|
||
_grant_clipboard(page, app_url)
|
||
_click_share(app_url, page)
|
||
# The share landed on the pre-existing row (the linked branch —
|
||
# the idempotent share) or a fresh row (the unlinked save-and-
|
||
# share) — either carries the token; the unique auto-title keeps
|
||
# the lookup unambiguous on the shared DB.
|
||
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
|
||
rows = [
|
||
c for c in r.json()["chats"]
|
||
if c["title"] == _auto_title(RESTORE_QUESTION)
|
||
]
|
||
token_row = next((c for c in rows if c.get("share_url")), None)
|
||
assert token_row is not None and token_row.get("share_url"), (
|
||
f"no share_url on any shared row: {rows!r}"
|
||
)
|
||
share_link = str(token_row["share_url"])
|
||
assert share_link.startswith("/shared/")
|
||
|
||
anon = browser.new_page(viewport={"width": 1280, "height": 800})
|
||
try:
|
||
anon.goto(app_url + share_link)
|
||
expect(anon.locator("#shared-title")).to_have_text(
|
||
_auto_title(RESTORE_QUESTION), timeout=30_000
|
||
)
|
||
expect(anon.locator(".msg.brain .bubble").last).to_contain_text(
|
||
DESCRIPTION_TAIL, timeout=30_000
|
||
)
|
||
# The shared page renders the figure — the guest caption keeps
|
||
# the TITLE (no summary fetch — the content endpoint is
|
||
# user-gated), the img is from the public bytes route.
|
||
_assert_answer_figure(anon, app_url, seeded, caption=seeded["title"])
|
||
# The shared chip is PLAIN TEXT (guests cannot open documents):
|
||
chip = anon.locator(".msg.brain .source-chip")
|
||
expect(chip).to_have_count(1)
|
||
expect(chip).to_have_text(f"{SOURCE_NAME}/{DOC_PATH}")
|
||
# The shared page is zero-controls: the figure is a plain
|
||
# <figure>, never an anchor (the owner-locked contract).
|
||
assert anon.locator(".msg.brain a.answer-image").count() == 0
|
||
finally:
|
||
anon.close()
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 3. App A — the bug-fix-only flow: ZERO tool calls, the citation
|
||
# STILL renders (the pure zero-read path — the owner's report)
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_bugfix_only_flow_zero_tool_calls_still_chips(
|
||
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
|
||
) -> None:
|
||
"""The ``which diagram is in your notes`` trigger: the mock
|
||
answers straight from the seed with ZERO tool calls (no
|
||
``show_image`` — the clause's flow is not triggered — and no
|
||
``read``): the sources block STILL carries the image doc's
|
||
citation figure (no chip — phase 130; the bug fix on the pure
|
||
zero-read path — the assertion the owner's report describes), the
|
||
answer text quotes the seeded description's tail (the description
|
||
WAS in the prompt), and NO answer figure renders (the model never
|
||
called the tool)."""
|
||
login(page, app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
page.fill("#message-input", DIAGRAM_QUESTION)
|
||
page.click("#send-btn")
|
||
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
# The answer quotes the seeded image doc's block tail (the
|
||
# description's end — the seed was in the prompt) and settles.
|
||
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
|
||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
|
||
# ZERO tool calls — on the wire (exactly ONE model call, no
|
||
# tool_calls message at all) and in the DOM (no tool record).
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1, (
|
||
f"the bug-fix flow must make exactly 1 model call, got {len(captured)}"
|
||
)
|
||
assert _executed_tool_calls(captured) == [], (
|
||
f"the bug-fix flow must make ZERO tool calls: {_executed_tool_calls(captured)}"
|
||
)
|
||
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
|
||
|
||
# NO answer figure (the model never called show_image) — but the
|
||
# sources block STILL carries the image doc's citation figure
|
||
# (no chip — phase 130; the auto-cite: suggested == read for an
|
||
# image doc, no read performed).
|
||
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
|
||
_assert_image_chip(page, seeded)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 4. App B (default off) — the toggle-off contract: no show_image
|
||
# offered, no answer figure, no answer_image key in the record
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_toggle_off_flow_no_tool_no_answer_image_key(
|
||
page: Page,
|
||
default_app_url: str,
|
||
seeded_off: dict[str, str],
|
||
mock_llm: int,
|
||
) -> None:
|
||
"""The same feature trigger question on the default-OFF app: the
|
||
prompt clause is ABSENT, so the mock's trigger never fires (the
|
||
plain composed answer streams instead) — and the contract holds
|
||
at every surface: the mock NEVER sees ``show_image`` among the
|
||
offered tools (the request capture carries exactly the THREE base
|
||
tools), the brain bubble has NO ``.answer-image``, and the saved
|
||
brain record carries NO ``answer_image`` key (the byte-identical-
|
||
off contract — the key is absent, never null)."""
|
||
login(page, default_app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
page.fill("#message-input", FEATURE_OFF_QUESTION)
|
||
page.click("#send-btn")
|
||
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
|
||
# No answer figure anywhere in the brain bubble.
|
||
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
|
||
|
||
# The WIRE pin: the offer list is EXACTLY the three base tools
|
||
# (no show_image — the toggle-OFF gate on the offer), and the
|
||
# turn made no tool calls at all (the mock's feature trigger is
|
||
# clause-gated — it never fired, so the plain answer streamed in
|
||
# one call).
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1, (
|
||
f"expected exactly 1 model call (no tool round), got {len(captured)}"
|
||
)
|
||
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep"], (
|
||
f"show_image leaked into the default-off offer: "
|
||
f"{_offered_tool_names(captured[0])}"
|
||
)
|
||
assert _executed_tool_calls(captured) == []
|
||
|
||
# The saved brain record carries NO answer_image key (ABSENT — the
|
||
# byte-identical-off contract; the phase-113 422-silence pin is
|
||
# the positive half of this, in the App A flow + the integration
|
||
# suite). The chip, though, IS still auto-cited (the chips fix is
|
||
# not toggle-gated — asserted in the next test).
|
||
row = _wait_saved_row(
|
||
default_app_url,
|
||
_admin_cookies(page),
|
||
_auto_title(FEATURE_OFF_QUESTION),
|
||
)
|
||
record = _brain_record(
|
||
_chat_messages(default_app_url, _admin_cookies(page), str(row["id"]))
|
||
)
|
||
assert "answer_image" not in record, (
|
||
f"the default-off brain record must NOT carry answer_image: "
|
||
f"{record.get('answer_image')!r}"
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 5. App B (default off) — the bug-fix trigger STILL cites the image
|
||
# doc (the auto-cite fix belongs to BOR_IMAGES, not BOR_ANSWER_IMAGES)
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_toggle_off_bugfix_trigger_still_chips(
|
||
page: Page, default_app_url: str, seeded_off: dict[str, str], mock_llm: int
|
||
) -> None:
|
||
"""The bug-fix trigger on the default-OFF app (the clause is
|
||
absent — the trigger does NOT require it): ZERO tool calls, the
|
||
answer quotes the seeded description's tail, and the sources
|
||
block STILL carries the image doc's citation figure (no chip —
|
||
phase 130) — the auto-cite fix is NOT gated by
|
||
``BOR_ANSWER_IMAGES`` (it belongs to the ``BOR_IMAGES``-indexed
|
||
docs, per the phase design)."""
|
||
login(page, default_app_url, next="/")
|
||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||
|
||
_mock_capture_reset(mock_llm)
|
||
page.fill("#message-input", DIAGRAM_OFF_QUESTION)
|
||
page.click("#send-btn")
|
||
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
|
||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send")
|
||
|
||
captured = _mock_capture(mock_llm)
|
||
assert len(captured) == 1
|
||
assert _executed_tool_calls(captured) == []
|
||
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
|
||
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
|
||
_assert_image_chip(page, seeded_off)
|