phase: 124_image_chips_and_answers
Build and Push Containers / build-and-push-app (push) Successful in 2m12s
Build and Push Containers / build-and-push-db (push) Failing after 1m39s

All green. Final report:

**Phase 124 final verification pass — all criteria verified, no defects found (nothing to fix)**
- Verified the implemented phase (all 7 tasks done, changes in working tree): `BOR_ANSWER_IMAGES` toggle, `show_image` tool + refusals + `holder.shown_images`, toggle-gated prompt clause, `answer_image` frame field (None-omitting serializer) + `ChatMessage` persistence, `merge_cited_docs` auto-cite bug fix, answer-figure UI (live/restore/shared/CSS), mock triggers + E2E suite. Also confirmed the bytes route → public change matches the phase-123 shared-chat precedent (task 06 design).
- `uv run pytest --cov=app --cov-report=term-missing` → **2848 passed**, TOTAL **99%** (>90% gate)
- `bash .agents/validate.sh` → **validation OK** (tests + coverage + ruff + pyright)
- `uv run ruff check . && uv run pyright` → clean, 0 errors
- `uv run pytest tests/e2e/test_answer_images.py -v --no-cov` (isolated) → **5/5 passed**
- Mock-inert spot checks: `tests/e2e/test_image_documents.py` 4/4, `tests/e2e/test_agent_document_tools.py` 4/4

**Completion criteria:** ① zero-read suggested-tier image doc chips with figure — verified (unit matrix, integration `test_zero_read_grounded_turn_auto_cites_suggested_image_doc`, E2E bug-fix flows); ② show_image → answer figure below text/above chips, second call refused, persists + restore + shared — verified (unit handler matrix, integration round-trip, E2E feature flow); ③ default-off byte-identical (3 tools, no clause, no key) — verified (unit + integration + E2E toggle-off); ④ full suite green / >90% / lint clean — verified above; ⑤ commit + phase move — harness responsibility (changes left uncommitted in working tree, no live-infrastructure changes made).

**Next pending phase:** none — `todo/` holds only this phase; the pipeline is clear once the harness moves it.
This commit is contained in:
2026-09-26 17:06:09 -04:00
parent 7bd6335929
commit 62e4f6b572
65 changed files with 4909 additions and 46 deletions
+7
View File
@@ -78,6 +78,13 @@ os.environ["BOR_IMAGES"] = str(_Settings.model_fields["images"].default)
os.environ["BOR_IMAGE_EXTENSIONS"] = str(_Settings.model_fields["image_extensions"].default)
os.environ["BOR_IMAGE_DIR"] = str(_Settings.model_fields["image_dir"].default)
# Phase 124 (task 04): the same leak class for the answer-image toggle
# — an operator's local ``.env`` may legitimately carry
# ``BOR_ANSWER_IMAGES`` (the dev environment runs the feature on), and
# the byte-identical-off pins (the ``show_image`` offer absent, the
# ``answer_image`` done-frame key absent) must see the code default.
os.environ["BOR_ANSWER_IMAGES"] = str(_Settings.model_fields["answer_images"].default)
from app.db import SessionLocal, db_available # noqa: E402
from app.main import app as fastapi_app # noqa: E402
+223
View File
@@ -467,6 +467,56 @@ Implements just enough of the aipi surface:
the trigger phrase is disjoint from every other trigger's; verified
2026-09-16: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``show me the picture of``
(``SHOW_IMAGE_TRIGGER``, phase 124 task 07 — the image-chips-and-
answers suite ``tests/e2e/test_answer_images.py``) **and** the
system prompt contains ``show_image`` (the tool clause's presence
— the ``BOR_ANSWER_IMAGES`` toggle's server-side marker: the
clause rides inside the HIGH prompt's ``<tools>`` section ONLY
when the toggle is on, so on a default-OFF app the trigger NEVER
fires and the mock behaves exactly as pre-phase for every other
suite) -> the deterministic SHOW_IMAGE flow, discriminated
statelessly from the tool results (streaming only): the mock
picks the IMAGE document seeded in the ``<documents>`` blocks —
the block whose ``path`` attribute ends in an image extension
(``app.rag.summarizer.IMAGE_MIMES``' family — the suite's
single-PNG fixture guarantees exactly one such block):
* request 1 (``tools`` offered, no confirmed show yet): stream
ONLY ``tool_calls`` deltas — ``show_image`` with
``{"path": <combined source/path>}`` (synthetic id
``call_show``, ``finish_reason: "tool_calls"``, no content) —
NO ``read`` (the ZERO-READ turn: the image doc's seed block
already carries its entire content, so a real model never
reads it — the zero-read shape is what makes the bug-fix's
chip assertion meaningful);
* the follow-up request (a ``tool``-role result carrying the
agent's confirming line ``Image shown to the user`` —
``_SHOW_CONFIRM_MARKER``) in the messages: the deterministic
FINAL answer — a short sentence ending with the ``<documents>``
block's tail echo (the phase-24 END_OF_NOTES echo convention —
it proves the seed was there), NO further tool calls.
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
phrase — the phase-71/72/94/118 ordering convention); verified
2026-09-26: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``which diagram is in your notes``
(``DIAGRAM_TRIGGER``, phase 124 task 07 — the BUG-FIX-ONLY
trigger, same suite) -> the deterministic ZERO-READ answer: the
mock answers straight from the seed with NO tool calls at all
(no ``show_image``, no ``read`` — unlike the ``SHOW_IMAGE_
TRIGGER`` flow it needs NO clause, so it also fires on a default-
OFF app: the chips fix is not gated by ``BOR_ANSWER_IMAGES``,
it belongs to the ``BOR_IMAGES``-indexed docs) and QUOTES THE
TAIL of the seeded image doc's ``<document>`` block (the same
image-doc pick as above): proving the description WAS in the
prompt (for an image doc the seed block IS its entire content)
and that its chip appears with NO tool calls at all (the phase-
124 auto-cite — the fix for the owner's report). Checked BEFORE
the ``DEFLECT_MODE`` branch (the marker lives in the user message
— the ``TABLE_TRIGGER``/``HISTORY_TRIGGER`` convention), so a
marker question always gets the echo whatever the honesty gate
says. Verified 2026-09-26: no existing E2E question or fixture
file contains the phrase, so every other suite is unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
@@ -580,6 +630,7 @@ from fastapi.responses import JSONResponse, StreamingResponse
from app.rag.agent import ALREADY_IN_CONTEXT, CORRECTION_INSTRUCTION
from app.rag.folder_summaries import FOLDER_HEADER_PREFIX # phase 94: the mock's key
from app.rag.summarizer import IMAGE_MIMES # phase 124: the image-doc extension family
app = FastAPI()
@@ -1756,6 +1807,117 @@ _SUMMARY_SEED_READ_CALL_RE = re.compile(
)
# ---------------------------------------------------------------------------
# Phase 124 (task 07, image chips and answers, TODO L3): the scripted
# SHOW_IMAGE flow + the zero-read bug-fix trigger — see the module
# docstring
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``show_image`` prompt clause in the system
#: prompt (the ``BOR_ANSWER_IMAGES`` toggle's server-side marker —
#: the clause rides inside the HIGH prompt's ``<tools>`` section ONLY
#: when the toggle is on) — drives the deterministic SHOW_IMAGE flow
#: (the mock shows the seeded image document and answers straight
#: from the seed — ZERO ``read`` calls). On a default-OFF app the
#: clause is absent, so the trigger NEVER fires there and the mock
#: behaves exactly as pre-phase for every other suite. Disjoint from
#: every other trigger phrase — the phase-71/72/94/118 ordering
#: convention; verified 2026-09-26: no existing E2E question or
#: fixture file contains the phrase, so every other suite is
#: unaffected.
SHOW_IMAGE_TRIGGER = "show me the picture of"
#: The phase-124 BUG-FIX-ONLY trigger (case-insensitive): the mock
#: answers straight from the seed with ZERO tool calls and QUOTES THE
#: TAIL of the seeded image doc's ``<document>`` block — proving the
#: description was in the prompt (for an image doc the seed block IS
#: its entire content) and that its chip appears with NO tool calls
#: at all (the phase-124 auto-cite). Unlike ``SHOW_IMAGE_TRIGGER``
#: it needs NO clause — the chips fix is not gated by
#: ``BOR_ANSWER_IMAGES`` (it belongs to the ``BOR_IMAGES``-indexed
#: docs), so it also fires on a default-OFF app. Checked BEFORE the
#: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker
#: lives in the user message — the TABLE/HISTORY convention), so a
#: marker question always gets the echo whatever the honesty gate
#: says.
DIAGRAM_TRIGGER = "which diagram is in your notes"
#: The agent's ``show_image`` SUCCESS result's confirming first line
#: (``app.rag.agent``'s ``show_image`` branch — the line ``read``'s
#: result never carries): the SHOW_IMAGE flow's follow-up request
#: keys on it in a ``tool``-role result (stateless discrimination —
#: the house marker-flow convention).
_SHOW_CONFIRM_MARKER = "Image shown to the user"
#: The image-doc extension family (``app.rag.summarizer.IMAGE_MIMES``
#: — phase 122's ext→mime map, imported so the mock can never drift
#: from the app's image family): a seeded ``<documents>`` block is
#: the IMAGE document when its ``path`` attribute ends in one of
#: these (the suite's single-PNG fixture guarantees exactly one such
#: block — the pick is deterministic).
_IMAGE_PATH_SUFFIXES = tuple(IMAGE_MIMES)
def _seeded_image_doc(system: str) -> tuple[str, str, str] | None:
"""The seeded image doc's ``(source, path, content)`` — or ``None``.
Phase 124 (task 07): the ``<documents>`` block (the
:data:`_DOCUMENT_BLOCK_RE` machinery) whose ``path`` attribute
ends in an image extension (:data:`_IMAGE_PATH_SUFFIXES`) is the
seeded image document — for an image doc the block's content IS
the document's entire text (the vision description — content ==
summary, phase 122 task 03). Both of this phase's triggers pick
the doc this way (the suite's single-PNG fixture guarantees
exactly one such block; ``None`` = a text-only KB, the premise
broke — the caller degrades loudly).
"""
for block in _DOCUMENT_BLOCK_RE.finditer(system):
if block.group("path").lower().endswith(_IMAGE_PATH_SUFFIXES):
return (block.group("source"), block.group("path"), block.group("content"))
return None
def _show_image_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-124 SHOW_IMAGE request (see the module
docstring). Stateless over the messages, like the other marker
flows:
* ``("call", combined)`` — the system prompt carries the
``show_image`` clause (the toggle-ON gate) and no ``tool``-role
result has confirmed the show yet: stream ONLY the
``show_image`` tool-call delta on the seeded image doc's
combined ``source/path`` (synthetic id ``call_show``,
``finish_reason: "tool_calls"``, no content) — NO ``read`` (the
zero-read turn).
* ``("answer",)`` — a ``tool``-role result carrying the agent's
confirming line (:data:`_SHOW_CONFIRM_MARKER`) is in the
messages: the deterministic FINAL answer — a short sentence
ending with the ``<documents>`` block's tail echo (the phase-24
END_OF_NOTES echo convention — it proves the seed was there),
NO further tool calls.
* ``None`` — not the flow: the trigger phrase is absent, the
clause is absent (a default-OFF app — the mock behaves exactly
as pre-phase), the turn is deflected (no ``<documents>`` seed
— a deflection prompt never carries one), or no seeded image
doc exists (a text-only KB — the premise broke; the suite fails
loudly on the missing figure).
"""
if SHOW_IMAGE_TRIGGER not in _user(body).lower():
return None
system = _system(body)
if "show_image" not in system:
return None
doc = _seeded_image_doc(system)
if doc is None:
return None
if any(_SHOW_CONFIRM_MARKER in r for r in _tool_results(body)):
return ("answer",)
if not body.get("tools"):
return None
return ("call", f"{doc[0]}/{doc[1]}")
def _summary_seed_read_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-118 scripted summary-read request (see the
module docstring). The question carries the scripted call (``read
@@ -2280,6 +2442,31 @@ def compose_answer(body: dict[str, Any]) -> str:
# message, a deflection prompt never carries it, so a marker
# question always gets the echo whatever the gate says.
answer = _history_echo(body)
elif DIAGRAM_TRIGGER in user.lower():
# Phase 124 (task 07): the BUG-FIX-ONLY trigger — the mock
# answers straight from the seed with ZERO tool calls (no
# ``show_image``, no ``read`` — the clause is NOT required,
# so the flow also fires on a default-OFF app: the chips fix
# belongs to the ``BOR_IMAGES``-indexed docs) and quotes the
# TAIL of the seeded image doc's ``<document>`` block (the
# same pick as the SHOW_IMAGE flow — the block's content is
# the image doc's entire text, the vision description):
# proving the description WAS in the prompt and that the chip
# appears with NO tool calls at all (the phase-124 auto-cite
# — the fix for the owner's report). Checked BEFORE the
# DEFLECT_MODE branch (the marker lives in the user message —
# the TABLE/HISTORY convention), so a marker question always
# gets the echo whatever the honesty gate says. A text-only KB
# (no seeded image doc) degrades to the plain line — the
# suite's premise broke; it fails loudly on the missing quote.
image_doc = _seeded_image_doc(system)
answer = (
f"Your notes carry that diagram — it reads: “{image_doc[2][-160:]}” "
"(Deterministic mock answer for E2E.)"
if image_doc
else "Your notes don't carry that diagram — there is nothing to show. "
"(Deterministic mock answer for E2E.)"
)
elif "DEFLECT_MODE" in system:
answer = (
"Ah — I haven't done anything like that, so I don't want to make stuff up! "
@@ -2911,6 +3098,42 @@ def chat_completions(body: dict[str, Any]) -> Any:
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 124 (task 07, image chips and answers, TODO L3): the
# scripted SHOW_IMAGE flow — the mock shows the seeded image
# document (the ``<documents>`` block with the image-
# extension path) and answers straight from the seed: ZERO
# ``read`` calls (the zero-read turn that makes the bug-fix's
# chip assertion meaningful). Clause-gated — the ``show_image``
# teaching line in the system prompt is the
# ``BOR_ANSWER_IMAGES`` toggle's presence marker, so on a
# default-OFF app the trigger never fires and the mock stays
# byte-identical to pre-phase for every other suite. Checked
# BEFORE the plain TOOLS_TRIGGER flow (disjoint trigger
# phrase — the phase-71/72/94/118 ordering convention; the
# trigger needs the clause, so deflected turns never hit it).
show_image = _show_image_flow(body)
if show_image is not None:
if show_image[0] == "call":
stream = _tool_call_stream(
"show_image", {"path": show_image[1]}, "call_show"
)
else: # "answer" — the final answer + the tail echo
block = _DOCUMENTS_BLOCK_RE.search(_system(body))
tail = (block.group(0) if block else _context(body))[-160:]
stream = _sse_stream(
_apply_max_tokens(
"Here is the picture you asked about — …and the very "
f"end of my notes reads: “{tail}” "
"(Deterministic mock answer for E2E.)",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 109 (task 03, never-frozen-turn story suite): the
# deterministic REPORTED-REPRO turn (delta → tool →
# thinking-after-delta — TODO.md L3): the scripted first answer,
+971
View File
@@ -0,0 +1,971 @@
"""Phase 124 E2E (Playwright): image source chips + LLM-chosen answer images.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_answer_images.py -v --no-cov
The suite proves BOTH halves of TODO.md L3 on the real user path, plus
the toggle-off contract — two module apps on dedicated ports (the
phase-122 ``test_image_documents.py`` pattern), sharing one deterministic
mock LLM and the shared e2e Postgres (truncate/re-import per app):
* **App A (feature ON)** — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=true``: the standalone PNG arrives through the
REAL admin upload flow + the RAG page's "Sync sources" (the mock's
vision model writes the byte-stable description), then:
- **Feature flow** — the ``show me the picture of …`` trigger (the
mock's phase-124 scripted trigger, ``tests/e2e/mock_llm.py``): the
mock scripts ONLY a ``show_image`` call (NO ``read`` — the ZERO-READ
turn, the shape that makes the chip assertion meaningful). The
brain bubble shows the ANSWER FIGURE (``.answer-image`` — ``<img>``
from the bytes route, ``loading="lazy"``, below the text and above
the chip row per A4, alt/caption settling to the summary), the
answer TEXT is still streamed in full (the tail echo of the
``<documents>`` seed proves the description was in the prompt), the
source-chip row carries the image doc's chip + its phase-122
figure (the AUTO-CITE proof — the mock made zero ``read`` calls,
asserted on the mock's request capture), and the "Nearby docs" row
does not repeat the doc. The wire pin: the first model request
offered FOUR tools (the three base + ``show_image``) and the turn
executed exactly one ``show_image`` — the saved brain record
persists the ``answer_image`` ref.
- **Restore + share** — a second feature turn, then a page RELOAD:
the restored brain bubble shows the figure again (the
``answer_image`` persistence through the ``bor.chat.v1`` restore
path), and the shared link (the existing chat-page Share flow, a
fresh anonymous context) renders the figure on the shared page.
- **Bug-fix-only flow** — the ``which diagram is in your notes``
trigger (the mock answers straight from the seed with ZERO tool
calls — no ``show_image``, no ``read``): the source-chip row STILL
carries the image doc's chip + figure (the bug fix on the pure
zero-read path — the assertion the owner's report describes), the
answer text quotes the seeded description's tail, and NO answer
figure renders (the model never called the tool).
* **App B (default off)** — ``BOR_IMAGES=true``, ``BOR_ANSWER_IMAGES``
pinned to the code default ``false`` (EXPLICITLY — process env ranks
above an operator's local gitignored ``.env``, so the default-off
contract under test cannot leak in, the phase-122 default-app
convention; the production default is unset = the same ``False``):
the same feature trigger question → the clause is absent from the
prompt, so the mock's trigger NEVER fires — the mock never sees
``show_image`` among the offered tools (the request capture carries
exactly the THREE base tools), the brain bubble has NO answer
figure, and the saved brain record carries NO ``answer_image`` key
(the byte-identical-off contract). The bug-fix trigger STILL chips
the image doc (the chips fix is NOT gated by ``BOR_ANSWER_IMAGES`` —
it belongs to the ``BOR_IMAGES``-indexed docs, per the phase design).
DB isolation: the suites share one Postgres; ``saved_chats`` survives
the truncation (the house pattern), so every question text here is
UNIQUE (its auto-title is therefore unique) and the row lookups key on
it. App B's module seed truncates the KB tables App A's tests used —
App B's tests run AFTER App A's (file order) and never look back.
"""
from __future__ import annotations
import base64
import json
import os
import re
import subprocess
import sys
import time
import zipfile
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Browser, Page, expect
from sqlalchemy import select, text
from app.config import Settings
from app.db import SessionLocal
from app.models import Document
from e2e.auth_helpers import login
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
REPO = Path(__file__).resolve().parents[2]
GIT_SOURCES_URL = "/git-sources.html"
SOURCE_NAME = "e2e-answer-img" # the archive stem (archive_source_name)
DOC_PATH = "pic.png"
#: The mock's deterministic answer tail (every composed answer ends
#: with it — the turn-settled wait signal, the house convention).
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
#: The TAIL of the fixture description (``IMAGE_DESCRIPTION_ANSWER`` —
#: the image doc's entire content): both phase-124 mock answers quote
#: it (the feature flow's ``<documents>`` tail echo, the bug-fix flow's
#: block-content tail) — asserting it proves the SEED was in the
#: prompt and reached the rendered answer.
DESCRIPTION_TAIL = "Title: Homelab Network Map."
#: App A (feature ON) — the ``show_image`` trigger question (cosine
#: 0.4552 against the mock's token-overlap embeddings, well over the
#: mock-calibrated 0.30 gate). Unique auto-title (the DB is shared).
FEATURE_QUESTION = "Show me the picture of the homelab network diagram?"
#: The restore/share flow's variant (the trigger substring is kept —
#: cosine 0.4358, still over the gate).
RESTORE_QUESTION = "Show me the picture of the homelab network diagram? (restore)"
#: App B (default off) — the feature trigger's variant (the clause is
#: absent, so the mock answers with the plain composed answer — cosine
#: 0.4187, the turn still grounds).
FEATURE_OFF_QUESTION = (
"Show me the picture of the homelab network diagram? (toggle off)"
)
#: The bug-fix-only trigger question (cosine 0.3410): the mock answers
#: straight from the seed with ZERO tool calls and quotes the image
#: doc's block tail — the chip must appear anyway (the auto-cite).
DIAGRAM_QUESTION = "Which diagram is in your notes about the homelab network?"
#: App B's bug-fix variant (the chips fix is not toggle-gated — cosine
#: 0.3113, still over the mock-calibrated gate).
DIAGRAM_OFF_QUESTION = (
"Which diagram is in your notes about the homelab network? (default off)"
)
#: A real 1×1 transparent PNG (the unit/integration suites' fixture —
#: the pipeline is content-agnostic, the well-formed bytes keep the
#: upload + serve + render path honest).
PNG_1X1 = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ"
"AAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
)
UPLOAD_TIMEOUT_MS = 30_000
SYNC_TIMEOUT_MS = 60_000
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
# ---------------------------------------------------------------------------
# Module apps: feature ON (App A) and default-OFF (App B). Each owns
# its port + its scratch upload/source/image dirs.
# ---------------------------------------------------------------------------
def _app_env(
mock_llm: int, app_port: int, *, answer_images: str, scratch: Path
) -> dict[str, str]:
"""The conftest app env (leak guards included) with the phase-124
knobs: BOTH apps index images (``BOR_IMAGES=true`` — the fixture's
PNG must become a document on either), and ``BOR_ANSWER_IMAGES``
is pinned EXPLICITLY per app (``true`` for App A, ``false`` for
App B — process env ranks above an operator's local gitignored
``.env``, so the contract under test cannot leak in either
direction, the phase-122 default-app convention; App B's explicit
``false`` IS the production default — unset is ``False``). The
image/upload homes are in the suite's scratch dir (the app under
test must not write image copies into the owner's real
``~/bor-sources``)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern) — both trigger questions
# ground on these values (see the question constants above).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget.
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# Leak guards (conftest pattern).
env["BOR_GIT_SOURCES"] = ""
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
# Phase 124: both apps index images; only the ANSWER side differs.
env["BOR_IMAGES"] = "true"
env["BOR_ANSWER_IMAGES"] = answer_images
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
env["BOR_IMAGE_DIR"] = str(scratch / "images")
return env
@pytest.fixture(scope="module")
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
"""App A (feature ON) — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=true`` (the conftest session app is never
started in this isolated run, so no port clash)."""
scratch = tmp_path_factory.mktemp("bor_answer_img_on")
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_ON", "8162"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
cwd=REPO,
env=_app_env(mock_llm, port, answer_images="true", scratch=scratch),
)
try:
_wait_http(f"http://127.0.0.1:{port}/api/health")
yield f"http://127.0.0.1:{port}"
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
@pytest.fixture(scope="module")
def default_app_server(
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
) -> Iterator[str]:
"""App B (default off) — ``BOR_IMAGES=true`` +
``BOR_ANSWER_IMAGES=false`` pinned explicitly (the production
default — the toggle-off contract; only the negative tests start
it, and they run AFTER App A's — file order)."""
scratch = tmp_path_factory.mktemp("bor_answer_img_off")
port = int(os.environ.get("E2E_APP_PORT_ANSWER_IMG_OFF", "8163"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
cwd=REPO,
env=_app_env(mock_llm, port, answer_images="false", scratch=scratch),
)
try:
_wait_http(f"http://127.0.0.1:{port}/api/health")
yield f"http://127.0.0.1:{port}"
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def default_app_url(default_app_server: str) -> str:
return default_app_server
@pytest.fixture(scope="module")
def zip_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The fixture archive: ONE standalone PNG at the root — source
``e2e-answer-img``, document path ``pic.png`` (the mock's seeded-
image-doc pick is deterministic: the only image-extension block)."""
root = tmp_path_factory.mktemp("bor_answer_img_zip")
(root / DOC_PATH).write_bytes(PNG_1X1)
archive = root / f"{SOURCE_NAME}.zip"
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
zf.write(root / DOC_PATH, arcname=DOC_PATH)
return archive
# ---------------------------------------------------------------------------
# DB + UI + mock-capture helpers
# ---------------------------------------------------------------------------
def _truncate_all() -> None:
"""Fresh KB + registry (the E2E isolation pattern): the suites
share one Postgres, and a leftover row would corrupt the counts.
``sources_meta`` (the KB generation counter) resets with the KB —
the sync re-creates the row. ``saved_chats`` is never touched by
the truncation — the unique question texts keep the row lookups
unambiguous, and :func:`_cleanup_saved_rows` owns this suite's
rows (re-runnable on the shared DB)."""
with SessionLocal() as db:
db.execute(text(
"TRUNCATE chunks, documents, query_log, kb_overview, "
"git_sources, sources_meta, folder_summaries"
))
db.commit()
def _health_db(app_url: str) -> bool:
try:
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
except Exception: # noqa: BLE001 — unreachable is the skip case
return False
def _mock_capture(mock_llm: int) -> list[dict[str, Any]]:
"""The mock's recent chat-completions request bodies (oldest
first) — the wire lens on the app's tool OFFER and the turn's
tool CALLS (the mock is the only E2E lens on the LLM's context)."""
r = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=5)
assert r.status_code == 200
return r.json()
def _mock_capture_reset(mock_llm: int) -> None:
r = httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=5)
assert r.status_code == 200
def _executed_tool_calls(captured: list[dict[str, Any]]) -> list[str]:
"""Every tool call the MODEL made across *captured* requests (the
assistant ``tool_calls`` messages), in order — the executed-call
lens (a refusal still appears here; it is the model's call)."""
return [
tc["function"]["name"]
for body in captured
for m in body["messages"]
if m.get("role") == "assistant" and m.get("tool_calls")
for tc in m["tool_calls"]
]
def _offered_tool_names(body: dict[str, Any]) -> list[str]:
return [t["function"]["name"] for t in (body.get("tools") or [])]
def _upload_zip(page: Page, app_url: str, archive: Path) -> None:
"""The real admin upload flow: form login → the git-sources page →
pick the archive → submit → the terminal result line (phase 90:
"Uploaded <source> — press Sync sources to import it.")."""
login(page, app_url, next=GIT_SOURCES_URL)
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
page.set_input_files("#archive-upload-file", str(archive))
page.click("#archive-upload-btn")
result = page.locator("#archive-upload-result")
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
expect(result).to_have_text(
f"Uploaded {SOURCE_NAME} — press Sync sources to import it.",
timeout=UPLOAD_TIMEOUT_MS,
)
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
"""The RAG page's "Sync sources" button (the scan — phase 90 A3):
click → "Syncing…" → the terminal "Synced HH:MM" label + the fresh
counts in #sync-result (the never-stale contract)."""
page.goto(app_url + "/sources.html")
btn = page.locator("#sync-btn")
expect(btn).to_be_visible(timeout=30_000)
expect(page.locator("#sync-label")).to_have_text(
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
)
btn.click()
expect(btn).to_be_disabled()
expect(page.locator("#sync-label")).to_have_text("Syncing…")
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
expect(btn).to_be_enabled()
expect(page.locator("#sync-result")).to_have_text(expected_result)
def _seed_image_doc(
app_url: str, browser: Browser, zip_path: Path
) -> dict[str, str]:
"""Upload the single-PNG zip through the admin UI, then scan it
with the RAG page's Sync button (the mock's vision model describes
the image in-process). Returns the seeded doc's identity (the
bytes route + the description + the title) for the render
assertions. The app's ``/api/config`` flag is asserted first — the
env override must have reached the app under test, or the whole
premise of the suite is void."""
cfg = httpx.get(f"{app_url}/api/config", timeout=5).json()
assert cfg["images"] is True, "the module app must run with BOR_IMAGES=true"
_truncate_all()
page = browser.new_page(viewport={"width": 1280, "height": 800})
try:
_upload_zip(page, app_url, zip_path)
_sync_via_ui(page, app_url, expected_result="1 added")
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == SOURCE_NAME, Document.path == DOC_PATH
)
)
assert doc is not None, "the uploaded image must become a document"
assert doc.is_image is True and doc.image_path is not None
# content == summary == the mock's description (the ONLY
# embedded text of the doc — the pipeline contract, phase 122).
assert doc.content == IMAGE_DESCRIPTION_ANSWER
assert doc.summary == IMAGE_DESCRIPTION_ANSWER
return {
"image_url": f"/api/documents/{doc.id}/image",
"description": IMAGE_DESCRIPTION_ANSWER,
"title": doc.title,
}
finally:
page.close()
@pytest.fixture(scope="module")
def seeded(
app_url: str,
browser: Browser,
zip_path: Path,
) -> Iterator[dict[str, str]]:
"""App A's KB: the fixture image doc through the real UI path."""
if not _health_db(app_url):
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
_cleanup_saved_rows() # a previous run's rows would be stale (new doc id)
yield _seed_image_doc(app_url, browser, zip_path)
_cleanup_saved_rows()
_truncate_all()
@pytest.fixture(scope="module")
def seeded_off(
default_app_url: str,
browser: Browser,
zip_path: Path,
) -> Iterator[dict[str, str]]:
"""App B's KB: the SAME fixture image doc through the SAME real
UI path (images indexed, the answer feature OFF). Runs AFTER all
of App A's tests (file order) — its truncation wipes App A's KB
tables, and App B's tests never look back."""
if not _health_db(default_app_url):
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
_cleanup_saved_rows()
yield _seed_image_doc(default_app_url, browser, zip_path)
_cleanup_saved_rows()
_truncate_all()
# ---------------------------------------------------------------------------
# Saved-chat + share helpers (the test_share_chat.py conventions)
# ---------------------------------------------------------------------------
def _admin_cookies(page: Page) -> dict[str, str]:
"""The signed session cookies the browser holds after a form login
— used to call the admin API with plain httpx (the test's API side
sees exactly what the signed-in browser sees)."""
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
#: This suite's auto-titles (the shared e2e Postgres keeps
#: ``saved_chats`` across runs — the test_share_chat cleanup
#: concern): a re-run must not find the previous run's row (same
#: title, but the KB re-seed minted a NEW doc id — the row's
#: ``answer_image.image_url`` would no longer match the seed).
_SUITE_TITLES = tuple(
_auto_title(q)
for q in (
FEATURE_QUESTION,
RESTORE_QUESTION,
FEATURE_OFF_QUESTION,
DIAGRAM_QUESTION,
DIAGRAM_OFF_QUESTION,
)
)
def _cleanup_saved_rows() -> None:
"""Delete THIS suite's saved rows by title (the house pattern —
test_share_chat deletes its own rows; here the module-boundary
fixture calls it, so a re-run starts clean). Best-effort: a
failed delete (DB unreachable — the skip case) never masks the
skip; a failed delete with the DB up just re-creates the stale-
row premise, and the suite fails loudly on the row assertion."""
try:
with SessionLocal() as db:
db.execute(
text("DELETE FROM saved_chats WHERE title = ANY(:titles)"),
{"titles": list(_SUITE_TITLES)},
)
db.commit()
except Exception: # noqa: BLE001 — DB unreachable: the skip handles it
pass
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55: auto-saves are SILENT —
A2 — so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
rows = r.json()["chats"]
last = next((c for c in rows if c["title"] == title), None)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _chat_messages(
app_url: str, cookies: dict[str, str], chat_id: str
) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["messages"]
def _brain_record(messages: list[dict[str, Any]]) -> dict[str, Any]:
brains = [m for m in messages if m.get("who") == "brain"]
assert brains, f"no brain record in the saved conversation: {messages!r}"
return brains[-1]
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(test_share_chat.py — headless Chromium still requires the grant
before ``writeText`` resolves on a secure origin)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share(app_url: str, page: Page) -> None:
"""Press the chat page's Share pill and pin the owner-locked
outcome on the live region (the test_share_chat.py branch —
clipboard success vs the inline-link fallback)."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
expect(page.locator(".share-link-fallback")).to_be_visible()
def _assert_answer_figure(
page: Page,
app_url: str,
seeded: dict[str, str],
*,
caption: str,
last_brain: str = ".msg.brain",
) -> None:
"""The A4 answer-figure contract in one place (the live, restore,
and shared surfaces share the shape — the shared page's figure is
a plain ``<figure>`, the chat page's an anchor, but the inner
``img``/caption classes are the same): exactly ONE ``.answer-image``
in the brain bubble, its ``<img>`` from the bytes route
(``loading="lazy"``, visible — the bytes really serve), and the
caption/alt settled to *caption* (the summary on the admin
surfaces, the title on the guest one)."""
fig = page.locator(f"{last_brain} .answer-image")
expect(fig).to_have_count(1)
img = fig.locator("img")
expect(img).to_have_count(1)
expect(img).to_be_visible(timeout=15_000)
expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("loading", "lazy")
expect(fig.locator(".answer-image-caption")).to_have_text(
caption, timeout=15_000
)
# The served bytes are a real image (the public bytes route).
resp = httpx.get(app_url + seeded["image_url"], timeout=10)
assert resp.status_code == 200
assert resp.headers["content-type"].startswith("image/")
assert resp.content == PNG_1X1
def _assert_image_chip(
page: Page, seeded: dict[str, str], *, last_brain: str = ".msg.brain"
) -> None:
"""The citation chip the AUTO-CITE earns (phase 124 task 05): the
image doc's text chip + its phase-122 compact figure (``<img>``
from the bytes route, alt settling to the summary — the admin
surface; the shared page's chip is plain text and asserted
separately)."""
chip = page.locator(f"{last_brain} .source-chip")
expect(chip).to_have_count(1)
expect(chip).to_have_text(f"{SOURCE_NAME}/{DOC_PATH}")
fig = page.locator(f"{last_brain} .source-image")
expect(fig).to_have_count(1)
img = fig.locator(".source-image-img")
expect(img).to_be_visible(timeout=15_000)
expect(img).to_have_attribute("src", seeded["image_url"])
expect(img).to_have_attribute("alt", seeded["description"], timeout=15_000)
def _body_child_classes(page: Page) -> list[str]:
"""The LAST brain bubble's ``.msg-body`` direct children's class
names, in DOM order — the A4 position pin (bubble < answer-image
< msg-meta)."""
return page.evaluate(
"""() => {
const body = [...document.querySelectorAll(".msg.brain")].pop()
.querySelector(".msg-body");
return [...body.children].map((el) => el.className);
}"""
)
# ---------------------------------------------------------------------------
# 1. App A (feature ON) — the feature flow: the scripted show_image
# turn (ZERO reads) renders the answer figure + the auto-cited chip
# ---------------------------------------------------------------------------
def test_feature_flow_answer_figure_chips_and_zero_reads(
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
) -> None:
"""The ``show me the picture of …`` trigger: the mock scripts ONLY
a ``show_image`` call (NO ``read`` — the zero-read turn). Asserts:
(1) the brain bubble's ``.msg-body`` carries ``.answer-image`` with
an ``<img>`` from the bytes route (200, image Content-Type) and
``loading="lazy"``; (2) the figure sits BEFORE the ``.msg-meta``
source-chip row (A4); (3) the answer TEXT is non-empty and quotes
the seed's tail (the "summaries still provided" contract + the
echo proving the seed was in the prompt); (4) the source-chip row
carries the image doc's chip + its figure (the AUTO-CITE proof —
the mock made ZERO ``read`` calls: only ``show_image`` — asserted
on the wire); (5) the "Nearby docs" row does NOT repeat the doc.
Plus the persistence pin: the saved brain record carries the
``answer_image`` ref (the task-04 frame field, round-tripped)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", FEATURE_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
# The answer TEXT is non-empty (the "summaries still provided"
# contract) — the tail echo lands LAST (the terminal signal): it
# quotes the <documents> seed's tail, i.e. the description.
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# (1) The answer figure: the bytes route, lazy, alt/caption the
# summary (the fetchContentSummary pattern settles both).
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
# (2) A4 position: below the answer text, above the chip row.
classes = _body_child_classes(page)
assert "bubble" in classes, f"no answer bubble in .msg-body: {classes}"
assert "answer-image" in classes, f"no .answer-image in .msg-body: {classes}"
assert "msg-meta" in classes, f"no chip row in .msg-body: {classes}"
assert (
classes.index("bubble")
< classes.index("answer-image")
< classes.index("msg-meta")
), f"A4 order violated (bubble < answer-image < msg-meta): {classes}"
# (4) The chip row carries the image doc's chip + its figure —
# the AUTO-CITE (the model never read it; the seed IS its
# entire content, so suggested == read for it).
_assert_image_chip(page, seeded)
# (5) The "Nearby docs" row does NOT repeat the image doc — the
# related tier excludes cited members (the single-doc KB has
# no related tier at all: the row is absent).
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
# The WIRE pin: exactly two model calls (the show_image round +
# the final answer), the FIRST request offered the FOUR tools
# (three base + show_image — the toggle-ON offer gate), and the
# turn executed EXACTLY one tool call: show_image — NO read (the
# zero-read turn that makes the chip assertion meaningful).
captured = _mock_capture(mock_llm)
assert len(captured) == 2, (
f"expected exactly 2 model calls (call + answer), got {len(captured)}"
)
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep", "show_image"]
assert _executed_tool_calls(captured) == ["show_image"]
tool_results = [
m.get("content") or ""
for m in captured[1]["messages"]
if m.get("role") == "tool"
]
assert any("Image shown to the user" in r for r in tool_results), (
f"the confirming tool result never reached the model: {tool_results}"
)
# The persistence pin: the auto-saved brain record carries the
# answer_image ref (source/path/title/image_url — the task-04
# frame field; the restore + shared flows assert on its render).
row = _wait_saved_row(app_url, _admin_cookies(page), _auto_title(FEATURE_QUESTION))
record = _brain_record(_chat_messages(app_url, _admin_cookies(page), str(row["id"])))
assert record.get("answer_image") == {
"source": SOURCE_NAME,
"path": DOC_PATH,
"title": seeded["title"],
"image_url": seeded["image_url"],
}, f"the brain record's answer_image is wrong: {record.get('answer_image')!r}"
# ---------------------------------------------------------------------------
# 2. App A — reload restores the figure; the shared page shows it
# ---------------------------------------------------------------------------
def test_reload_restores_figure_and_shared_page_shows_it(
page: Page,
app_url: str,
seeded: dict[str, str],
mock_llm: int,
browser: Browser,
) -> None:
"""A second feature turn (a unique question — unique auto-title on
the shared DB), then a page RELOAD: the restored brain bubble
shows the figure again (the ``answer_image`` persistence through
the ``bor.chat.v1`` restore path — pixel-parity with the live
render), and the existing Share flow's link — opened in a FRESH
anonymous context — renders the figure on the shared page (the
bytes route is public; the guest caption keeps the title — the
content endpoint is user-gated, so the summary fetch is skipped
there)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", RESTORE_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
_assert_image_chip(page, seeded)
# The auto-save settled (the row exists with both messages) before
# the reload — the Share click later shares the linked row, or
# re-creates it unlinked (either path is owner-locked; the row
# lookup keys on the unique auto-title).
cookies = _admin_cookies(page)
_wait_saved_row(app_url, cookies, _auto_title(RESTORE_QUESTION))
# RELOAD: the restored brain bubble shows the figure again — the
# localStorage restore (rememberBrainTurn's save point ran
# synchronously in the done handler, before the send button
# re-enabled, so the record is durable before we reload).
page.reload()
expect(page.locator("#kb-banner")).to_be_hidden()
restored = page.locator(".msg.brain .bubble").last
expect(restored).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(restored).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
_assert_answer_figure(page, app_url, seeded, caption=seeded["description"])
_assert_image_chip(page, seeded)
# SHARE: the existing chat-page Share flow → a fresh anonymous
# context sees the figure (and the plain-text chip).
_grant_clipboard(page, app_url)
_click_share(app_url, page)
# The share landed on the pre-existing row (the linked branch —
# the idempotent share) or a fresh row (the unlinked save-and-
# share) — either carries the token; the unique auto-title keeps
# the lookup unambiguous on the shared DB.
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
rows = [
c for c in r.json()["chats"]
if c["title"] == _auto_title(RESTORE_QUESTION)
]
token_row = next((c for c in rows if c.get("share_url")), None)
assert token_row is not None and token_row.get("share_url"), (
f"no share_url on any shared row: {rows!r}"
)
share_link = str(token_row["share_url"])
assert share_link.startswith("/shared/")
anon = browser.new_page(viewport={"width": 1280, "height": 800})
try:
anon.goto(app_url + share_link)
expect(anon.locator("#shared-title")).to_have_text(
_auto_title(RESTORE_QUESTION), timeout=30_000
)
expect(anon.locator(".msg.brain .bubble").last).to_contain_text(
DESCRIPTION_TAIL, timeout=30_000
)
# The shared page renders the figure — the guest caption keeps
# the TITLE (no summary fetch — the content endpoint is
# user-gated), the img is from the public bytes route.
_assert_answer_figure(anon, app_url, seeded, caption=seeded["title"])
# The shared chip is PLAIN TEXT (guests cannot open documents):
chip = anon.locator(".msg.brain .source-chip")
expect(chip).to_have_count(1)
expect(chip).to_have_text(f"{SOURCE_NAME}/{DOC_PATH}")
# The shared page is zero-controls: the figure is a plain
# <figure>, never an anchor (the owner-locked contract).
assert anon.locator(".msg.brain a.answer-image").count() == 0
finally:
anon.close()
# ---------------------------------------------------------------------------
# 3. App A — the bug-fix-only flow: ZERO tool calls, the chip STILL
# appears (the pure zero-read path — the owner's report)
# ---------------------------------------------------------------------------
def test_bugfix_only_flow_zero_tool_calls_still_chips(
page: Page, app_url: str, seeded: dict[str, str], mock_llm: int
) -> None:
"""The ``which diagram is in your notes`` trigger: the mock
answers straight from the seed with ZERO tool calls (no
``show_image`` — the clause's flow is not triggered — and no
``read``): the source-chip row STILL carries the image doc's chip
+ figure (the bug fix on the pure zero-read path — the assertion
the owner's report describes), the answer text quotes the seeded
description's tail (the description WAS in the prompt), and NO
answer figure renders (the model never called the tool)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", DIAGRAM_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
# The answer quotes the seeded image doc's block tail (the
# description's end — the seed was in the prompt) and settles.
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# ZERO tool calls — on the wire (exactly ONE model call, no
# tool_calls message at all) and in the DOM (no tool record).
captured = _mock_capture(mock_llm)
assert len(captured) == 1, (
f"the bug-fix flow must make exactly 1 model call, got {len(captured)}"
)
assert _executed_tool_calls(captured) == [], (
f"the bug-fix flow must make ZERO tool calls: {_executed_tool_calls(captured)}"
)
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
# NO answer figure (the model never called show_image) — but the
# chip row STILL carries the image doc's chip + figure (the auto-
# cite: suggested == read for an image doc, no read performed).
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
_assert_image_chip(page, seeded)
# ---------------------------------------------------------------------------
# 4. App B (default off) — the toggle-off contract: no show_image
# offered, no answer figure, no answer_image key in the record
# ---------------------------------------------------------------------------
def test_toggle_off_flow_no_tool_no_answer_image_key(
page: Page,
default_app_url: str,
seeded_off: dict[str, str],
mock_llm: int,
) -> None:
"""The same feature trigger question on the default-OFF app: the
prompt clause is ABSENT, so the mock's trigger never fires (the
plain composed answer streams instead) — and the contract holds
at every surface: the mock NEVER sees ``show_image`` among the
offered tools (the request capture carries exactly the THREE base
tools), the brain bubble has NO ``.answer-image``, and the saved
brain record carries NO ``answer_image`` key (the byte-identical-
off contract — the key is absent, never null)."""
login(page, default_app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", FEATURE_OFF_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
# No answer figure anywhere in the brain bubble.
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
# The WIRE pin: the offer list is EXACTLY the three base tools
# (no show_image — the toggle-OFF gate on the offer), and the
# turn made no tool calls at all (the mock's feature trigger is
# clause-gated — it never fired, so the plain answer streamed in
# one call).
captured = _mock_capture(mock_llm)
assert len(captured) == 1, (
f"expected exactly 1 model call (no tool round), got {len(captured)}"
)
assert _offered_tool_names(captured[0]) == ["ls", "read", "grep"], (
f"show_image leaked into the default-off offer: "
f"{_offered_tool_names(captured[0])}"
)
assert _executed_tool_calls(captured) == []
# The saved brain record carries NO answer_image key (ABSENT — the
# byte-identical-off contract; the phase-113 422-silence pin is
# the positive half of this, in the App A flow + the integration
# suite). The chip, though, IS still auto-cited (the chips fix is
# not toggle-gated — asserted in the next test).
row = _wait_saved_row(
default_app_url,
_admin_cookies(page),
_auto_title(FEATURE_OFF_QUESTION),
)
record = _brain_record(
_chat_messages(default_app_url, _admin_cookies(page), str(row["id"]))
)
assert "answer_image" not in record, (
f"the default-off brain record must NOT carry answer_image: "
f"{record.get('answer_image')!r}"
)
# ---------------------------------------------------------------------------
# 5. App B (default off) — the bug-fix trigger STILL chips the image
# doc (the chips fix belongs to BOR_IMAGES, not BOR_ANSWER_IMAGES)
# ---------------------------------------------------------------------------
def test_toggle_off_bugfix_trigger_still_chips(
page: Page, default_app_url: str, seeded_off: dict[str, str], mock_llm: int
) -> None:
"""The bug-fix trigger on the default-OFF app (the clause is
absent — the trigger does NOT require it): ZERO tool calls, the
answer quotes the seeded description's tail, and the source-chip
row STILL carries the image doc's chip + figure — the chips fix
is NOT gated by ``BOR_ANSWER_IMAGES`` (it belongs to the
``BOR_IMAGES``-indexed docs, per the phase design)."""
login(page, default_app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
_mock_capture_reset(mock_llm)
page.fill("#message-input", DIAGRAM_OFF_QUESTION)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(DESCRIPTION_TAIL, timeout=30_000)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send")
captured = _mock_capture(mock_llm)
assert len(captured) == 1
assert _executed_tool_calls(captured) == []
expect(page.locator(".msg.brain .tool-calls-disclosure")).to_have_count(0)
expect(page.locator(".msg.brain .answer-image")).to_have_count(0)
_assert_image_chip(page, seeded_off)
+283
View File
@@ -2507,3 +2507,286 @@ def test_shared_chat_serves_the_user_image_path(
finally:
db.execute(text("TRUNCATE saved_chats"))
db.commit()
# ---------------------------------------------------------------------------
# Phase 124, task 04 — the done frame's OPTIONAL answer_image (the
# shown document's ref: the LLM's ``show_image`` pick) + the byte-
# identical-off contract (every other turn: no key, no null).
# ---------------------------------------------------------------------------
@pytest.fixture()
def answer_images_on(monkeypatch: pytest.MonkeyPatch) -> None:
"""The chat endpoint's settings view with ``answer_images`` ON
(the ``BOR_ANSWER_IMAGES`` feature path; the monkeypatch fixture
reverts the patch, the conftest leak-guard pins the default OFF
for every other test in the module)."""
settings = Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
answer_images=True,
)
monkeypatch.setattr(chat_api, "get_settings", lambda: settings)
def test_show_image_turn_done_frame_carries_answer_image(
client, db, seeded_kb: FakeRagLLM, answer_images_on: None
) -> None:
"""A grounded turn whose mock client scripts a ``show_image`` tool
call on a seeded image doc (the scripted-tool mechanism — the
``use your tools`` precedent) → the final SSE ``done`` frame JSON
carries ``answer_image`` with ``image_url ==
/api/documents/<id>/image`` and the cited ``sources`` include the
same doc (its scripted read — the phase-119 citation tier)."""
doc = _seed_image_doc(db)
scripted = FakeRagLLM(
tool_script=[
[
ToolCallPiece(
id="call_1",
name="read",
arguments={"path": "docs/pic.png"},
)
],
[
ToolCallPiece(
id="call_2",
name="show_image",
arguments={"path": "docs/pic.png"},
)
],
]
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is False
assert done["answer_image"] == {
"source": "docs",
"path": "pic.png",
"title": "pic",
"image_url": f"/api/documents/{doc.id}/image",
}
# The cited tier includes the same doc (the scripted read).
assert ("docs", "pic.png") in [(s["source"], s["path"]) for s in done["sources"]]
# The toggle was honoured on the wire: the first (agent) request
# offered the 4th tool.
offered = scripted.seen_tools[0]
assert offered is not None
assert [t["function"]["name"] for t in offered] == [
"ls",
"read",
"grep",
"show_image",
]
def test_toggle_off_same_flow_offers_three_tools_and_no_answer_image_key(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""The SAME scripted flow with ``answer_images=False`` (the
default) → ``show_image`` is NOT in the offered tools (the mock's
recorded request carries exactly the three base tools) and the
``done`` frame has NO ``answer_image`` key — the feature's
server-side gate is the OFFER (a model that was never offered the
tool never calls it, so no such call occurs in the flow)."""
_seed_image_doc(db) # resolvable, but the tool is never offered
scripted = FakeRagLLM(
tool_script=[
[
ToolCallPiece(
id="call_1",
name="read",
arguments={"path": "docs/pic.png"},
)
]
]
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is False
assert "answer_image" not in done # ABSENT, not null
# The offer: exactly the three base tools — no ``show_image``.
offered = scripted.seen_tools[0]
assert offered is not None
assert [t["function"]["name"] for t in offered] == ["ls", "read", "grep"]
def test_zero_tool_grounded_turn_done_frame_has_no_answer_image_key(
client, db, seeded_kb: FakeRagLLM, answer_images_on: None
) -> None:
"""A zero-tool grounded turn (no scripted calls — the toggle is ON,
so the offer carries ``show_image``, but the model never calls it)
→ the ``done`` frame has NO ``answer_image`` key — byte-identical
to pre-phase, checked on the raw wire text (not a re-serialized
dict)."""
scripted = FakeRagLLM() # no tool_script: a zero-tool turn
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with client.stream("POST", "/api/chat", json={"message": QUESTION}) as r:
assert r.status_code == 200
buf = ""
raw_frames: list[str] = []
for part in r.iter_text():
buf += part
while "\n\n" in buf:
frame, buf = buf.split("\n\n", 1)
frame = frame.strip()
if frame.startswith("data:"):
raw_frames.append(frame)
finally:
fastapi_app.dependency_overrides.clear()
done_raw = [f for f in raw_frames if '"type": "done"' in f]
assert len(done_raw) == 1
# The BYTE check: the key is absent from the wire text itself.
assert "answer_image" not in done_raw[0]
done = json.loads(done_raw[0].removeprefix("data:").strip())
assert done["deflected"] is False
assert "answer_image" not in done
def test_deflected_turn_done_frame_has_no_answer_image_key_even_toggle_on(
client, db, seeded_kb: FakeRagLLM, answer_images_on: None
) -> None:
"""A deflected turn runs no agent and never shows an image (the
phase's locked design): even with the toggle ON, the ``done``
frame has NO ``answer_image`` key — byte-identical to pre-phase."""
scripted = FakeRagLLM()
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, OFF_TOPIC)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is True
assert "answer_image" not in done
# ---------------------------------------------------------------------------
# Phase 124, task 05 — bug fix: a suggested-tier IMAGE doc earns its
# source chip WITHOUT a ``read`` (its seed block IS its entire content —
# content == summary == the vision description, phase 122 task 03 — so
# "suggested" ≡ "read" for it and the phase-119 cite discipline is
# honored, not weakened). Text docs keep the read-docs-only rule.
# ---------------------------------------------------------------------------
def _seed_image_doc_with_chunk(db) -> Document:
"""An ``is_image`` documents row WITH a retrievable chunk — the
phase-122 pipeline embeds the vision description as the image doc's
content (the only embedded text of the document), so the fixture
chunk mirrors that: a description tuned to overlap ``QUESTION``
(the mock embeddings are token-overlap vectors, as in the E2E
mock), which lands the image doc in the SUGGESTED tier (top-5
fused rank, no floor — A3) where the auto-cite merge operates.
The ``tsv`` column is generated (``to_tsvector`` of the content),
so the chunk also hits the FTS half. The fixture's teardown
truncates the tables."""
doc = _seed_image_doc(db)
# The phase-122 invariant for image docs: content == summary == the
# vision description (the ONE embedded text of the document — no
# pointer line on the summary). The description is tuned to overlap
# ``QUESTION`` (the mock embeddings are token-overlap vectors, as in
# the E2E mock), which lands the image doc in the SUGGESTED tier
# (top-5 fused rank, no floor — A3) where the auto-cite merge
# operates.
description = "Diagram of how my Kubernetes cluster is set up"
doc.content = description
doc.summary = description
db.add(
Chunk(
id=uuid.uuid4(),
document_id=doc.id,
position=0,
content=description,
embedding=_token_vec(description),
)
)
db.commit()
return doc
def test_zero_read_grounded_turn_auto_cites_suggested_image_doc(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""The bug fix pinned end-to-end (TODO L3): a grounded turn with
NO scripted tool calls (the mock answers straight from the seed —
a real model never ``read``s an image doc: its seed block already
carries 100% of its content) whose suggested tier contains an
image doc → the done frame's ``sources`` include the image ref
WITH ``image_url`` set (the chip + figure data — pre-phase-124 a
zero-read turn chipped NOTHING, the phase-122 E2E had to script a
``read`` to earn the chip), and ``related`` does NOT contain the
same ``(source, path)`` (the auto-cited doc leaves the "Nearby
docs" row — no duplicate figure)."""
doc = _seed_image_doc_with_chunk(db)
scripted = FakeRagLLM() # no tool_script: a zero-read grounded turn
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is False
# The image doc IS a citation chip now — and its ref (and ONLY its
# ref) carries the bytes route the figure renders from.
sources = done["sources"]
img_refs = [s for s in sources if (s["source"], s["path"]) == ("docs", "pic.png")]
assert len(img_refs) == 1 # once — not duplicated by the read merge
assert img_refs[0]["image_url"] == f"/api/documents/{doc.id}/image"
for ref in sources:
if ref is not img_refs[0]:
assert "image_url" not in ref # the text refs stay pre-122-shaped
# Consequence (a): the related tier already excludes cited-seen
# members — the auto-cited image doc is NOT also a "nearby doc".
related_keys = {(s["source"], s["path"]) for s in done["related"]}
assert ("docs", "pic.png") not in related_keys
# Consequence (b): the observability fields are UNCHANGED — the
# durable record (query_log.sources) carried the suggested image
# doc before the phase (it was suggested, not read); auto-citation
# touches the citation tier only, not the record.
row = db.scalars(select(QueryLog)).one()
assert "docs/pic.png" in row.sources
assert "docs/homelab/kubernetes.md" in row.sources
# The seed contract held: the HIGH prompt seeded the image doc's
# summary — its vision description, the ENTIRE content (image docs
# store no ``Source:`` pointer line — the summary mirrors the
# description verbatim, phase 122 task 03) — as a ``<document>``
# block. This is WHY "suggested" ≡ "read" for the doc: a ``read``
# would add only the marker line.
system = scripted.seen_messages[0][0]["content"]
assert '<document source="docs" path="pic.png"' in system
assert "Diagram of how my Kubernetes cluster is set up" in system
def test_zero_read_text_only_turn_still_chips_nothing(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""Text docs are unaffected (consequence c): a zero-read grounded
turn on the plain text-only KB (no image docs anywhere in the
retrieval) → ``sources`` is ``[]`` — byte-identical to pre-phase
(no auto-cite without image docs) and no ``image_url`` key on any
ref (the omission rule at the frame level)."""
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is False
assert done["sources"] == [] # phase-119 read-docs-only rule intact
for ref in done["sources"] + done["related"]:
assert "image_url" not in ref
+106
View File
@@ -899,6 +899,112 @@ def test_shared_serve_includes_the_user_image_path(
assert "image" not in messages[2]
# ---------------------------------------------------------------------------
# Phase 124 (task 04): the brain answer's first-class image — the
# shown document's ref (the LLM's ``show_image`` pick) rides the BRAIN
# record's ``answer_image`` key (distinct from the user record's
# phase-123 ``image`` attachment path — the attachment belongs to the
# question, the answer image to the brain's reply). The phase-113
# lesson applied up front: the done-time auto-save (``rememberBrainTurn``)
# carries the key, and the ``extra="forbid"`` boundary silently 422'd
# any key the model was missing (the A2 quiet-failure path swallowed
# it — grounded turns silently unsaved). These round-trips are that
# regression pin.
# ---------------------------------------------------------------------------
#: The shown image doc's full ref — the done frame's ``answer_image``
#: shape (source/path/title + the ``image_url`` bytes route).
ANSWER_IMAGE_REF: dict[str, Any] = {
"source": "Homelab",
"path": "kubernetes-cluster.png",
"title": "Kubernetes Cluster Diagram",
"image_url": "/api/documents/0e8a6e5d-7b6c-4f2a-9c1d-3e5f7a9b1c2d/image",
}
def test_create_round_trips_the_brain_answer_image_ref(
admin_client: TestClient, db: Session
) -> None:
"""The phase-113 regression pin (the A2 quiet-failure path): a
brain record carrying ``answer_image`` → ``POST /api/chats`` is
201 (NOT a silent 422) and the served chat round-trips the ref
BYTE-FOR-BYTE (the restore path re-renders the answer figure from
it). The user record carries NO ``answer_image`` key, and the
user's phase-123 ``image`` attachment key stays distinct."""
brain = {**FULL_BRAIN, "answer_image": ANSWER_IMAGE_REF}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, f"expected 201, got {r.status_code}: {r.text}"
body = r.json()
assert body["messages"][1]["answer_image"] == ANSWER_IMAGE_REF
assert "answer_image" not in body["messages"][0]
assert "image" not in body["messages"][1] # the brain's image is a REF
# The RAW stored JSONB (the DB is the durable boundary): the full
# ref, byte-for-byte.
raw = db.execute(
text("SELECT messages FROM saved_chats WHERE id = :id"),
{"id": body["id"]},
).scalar_one()
assert raw[1]["answer_image"] == ANSWER_IMAGE_REF
assert "answer_image" not in raw[0]
# The admin GET round-trips the ref losslessly.
got = admin_client.get(f"/api/chats/{body['id']}")
assert got.status_code == 200
assert got.json()["messages"][1]["answer_image"] == ANSWER_IMAGE_REF
def test_create_without_answer_image_round_trips_unchanged(
admin_client: TestClient,
) -> None:
"""A record WITHOUT the key (every pre-phase-124 chat, every
image-less turn) round-trips UNCHANGED: the response carries no
``answer_image`` key at all — ABSENT, never ``null`` (the
byte-identical-off contract; the phase-50 round-trip contract).
``_expect`` (the stored shape) is unchanged by phase 124 — the
omission serializer inherits it on every surface."""
brain = {**FULL_BRAIN}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, r.text
body = r.json()
assert body["messages"] == _expect([_user(FIRST_QUESTION), brain])
assert "answer_image" not in body["messages"][0]
assert "answer_image" not in body["messages"][1]
got = admin_client.get(f"/api/chats/{body['id']}")
assert got.status_code == 200
assert "answer_image" not in got.json()["messages"][1]
def test_shared_serve_includes_the_brain_answer_image_ref(
admin_client: TestClient,
) -> None:
"""The shared view is faithful (task 06's render depends on it):
the PUBLIC snapshot's brain record carries the answer ref (the
bytes route is public — a shared chat is faithful, the phase-123
user-image precedent), and the user record stays WITHOUT the key."""
brain = {**FULL_BRAIN, "answer_image": ANSWER_IMAGE_REF}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, r.text
share_url = _share(admin_client, r.json()["id"])["share_url"]
anon = TestClient(fastapi_app) # fresh jar: truly anonymous
got = anon.get(f"/api{share_url}")
assert got.status_code == 200
messages = got.json()["messages"]
assert messages[1]["answer_image"] == ANSWER_IMAGE_REF
assert "answer_image" not in messages[0]
def test_public_read_wrong_and_revoked_tokens_404_with_one_detail(
admin_client: TestClient,
) -> None:
+17 -11
View File
@@ -1305,24 +1305,30 @@ def test_image_route_404_matrix(admin_client: TestClient, db, tmp_path: Path) ->
_cleanup_kb(db)
def test_image_route_requires_user_like_the_content_endpoint(
def test_image_route_is_public_like_the_shared_chat_content(
admin_client: TestClient, db, tmp_path: Path
) -> None:
"""Phase 79 posture (the task's "PUBLIC, like the document content
endpoint" — the content endpoint has been user-gated since phase
79; the ONLY anonymous surface is the shared chats, PLAN A10): an
anonymous caller gets 401 ``authentication required`` before any
row is read (a FRESH client — the module's fixture ``client``
stays unsigned here), a signed-in caller gets the bytes."""
"""Phase 124 (LOCKED A4 — the answer figure is "visible … on the
shared page"): the BYTES route is PUBLIC (the phase-123
chat-image precedent — the image is part of a shared chat's
rendered content, and the shared chat is the app's ONLY anonymous
surface, PLAN A10; the unguessable ``uuid4`` document id IS the
credential, no enumeration value): an ANONYMOUS caller (a FRESH
client — no session) gets the bytes with the image Content-Type,
and a signed-in caller gets the same. The document's TEXT (the
content endpoint) stays user-gated — only the bytes are public."""
_cleanup_kb(db)
try:
doc = _seed_image_doc(db, tmp_path)
anonymous = TestClient(fastapi_app)
r = anonymous.get(f"/api/documents/{doc.id}/image")
assert r.status_code == 401
assert r.json() == {"detail": "authentication required"}
# The signed-in client (admin) passes.
assert admin_client.get(f"/api/documents/{doc.id}/image").status_code == 200
assert r.status_code == 200
assert r.headers["content-type"].startswith("image/")
assert r.content == PNG_1X1
# The signed-in client (admin) gets the same bytes.
r_admin = admin_client.get(f"/api/documents/{doc.id}/image")
assert r_admin.status_code == 200
assert r_admin.content == PNG_1X1
finally:
_cleanup_kb(db)
+971
View File
@@ -0,0 +1,971 @@
"""Unit tests: phase 124 — image chips and answers.
Task 01 covers the ``BOR_ANSWER_IMAGES`` feature toggle
(``Settings.answer_images``). Later tasks in this phase extend this
file (the ``show_image`` tool, the prompt clause, the ``answer_image``
frame key, the auto-cite merge).
"""
from __future__ import annotations
from typing import Any
import pytest
from app.config import Settings
def _settings(**kwargs: Any) -> Settings:
"""Build Settings without reading a .env file (deterministic tests).
Mirrors ``tests/unit/test_config.py::_settings``.
"""
kwargs.setdefault("_env_file", None)
return Settings(**kwargs) # pyright: ignore[reportCallIssue] (kwarg exists at runtime)
def test_answer_images_defaults_off(monkeypatch: pytest.MonkeyPatch) -> None:
"""LOCKED A2: ``BOR_ANSWER_IMAGES`` gates the whole answer-image
feature and is OFF by default — unset, ``=0``, and ``=false`` all
parse to ``False`` (off = byte-identical behavior everywhere)."""
monkeypatch.delenv("BOR_ANSWER_IMAGES", raising=False)
assert _settings().answer_images is False
monkeypatch.setenv("BOR_ANSWER_IMAGES", "0")
assert _settings().answer_images is False
monkeypatch.setenv("BOR_ANSWER_IMAGES", "false")
assert _settings().answer_images is False
def test_answer_images_truthy_env_parse(monkeypatch: pytest.MonkeyPatch) -> None:
"""The settings loader's accepted truthy forms (the
``stream_thinking``/``images`` bool-test precedent): ``1`` and
``true`` enable the toggle."""
monkeypatch.setenv("BOR_ANSWER_IMAGES", "1")
assert _settings().answer_images is True
monkeypatch.setenv("BOR_ANSWER_IMAGES", "true")
assert _settings().answer_images is True
def test_answer_images_independent_of_images(monkeypatch: pytest.MonkeyPatch) -> None:
"""LOCKED A2: the answer-side toggle is a SEPARATE knob from
``images``/``BOR_IMAGES`` — indexing image docs and showing them in
answers are independent owner decisions. Setting one never moves
the other, in either direction."""
monkeypatch.delenv("BOR_ANSWER_IMAGES", raising=False)
monkeypatch.delenv("BOR_IMAGES", raising=False)
# Default: both off.
s = _settings()
assert s.answer_images is False
assert s.images is False
# Enabling the answer toggle alone leaves indexing off — a
# coherent (if image-less) state: the tool is offered, finds no
# image docs, and always refuses.
monkeypatch.setenv("BOR_ANSWER_IMAGES", "1")
s = _settings()
assert s.answer_images is True
assert s.images is False
# Enabling indexing alone leaves the answer side off (byte-
# identical pre-phase-124 behavior).
monkeypatch.delenv("BOR_ANSWER_IMAGES")
monkeypatch.setenv("BOR_IMAGES", "1")
s = _settings()
assert s.answer_images is False
assert s.images is True
# Both on — the full feature state.
monkeypatch.setenv("BOR_ANSWER_IMAGES", "true")
s = _settings()
assert s.answer_images is True
assert s.images is True
# ---------------------------------------------------------------------------
# Task 02 — the ``show_image`` agent tool (toggle-gated)
import asyncio # noqa: E402
import uuid # noqa: E402
from collections.abc import AsyncIterator # noqa: E402
from copy import deepcopy # noqa: E402
from datetime import UTC, datetime # noqa: E402
from typing import cast # noqa: E402
from unittest.mock import MagicMock # noqa: E402
from sqlalchemy.orm import Session # noqa: E402
from app.models import Document # noqa: E402
from app.rag.agent import ( # noqa: E402
AGENT_TOOLS,
ALREADY_SHOWING,
IMAGE_DOC_MARKER,
MISSING_SHOW_ARGS,
NOT_AN_IMAGE_DOC,
SHOW_IMAGE_TOOL,
AgentHolder,
_execute_tool,
_no_document_refusal,
agent_tools,
run_agent,
)
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece # noqa: E402
#: The fixture documents' fixed creation date (phase 106, D5): the
#: ``show_image`` result formats ``doc.created_at`` on its date line —
#: a pinned value keeps the result-string assertions exact.
_SHOW_FIXTURE_CREATED_AT = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
# --- the offer list: toggle-gated, the base three byte-identical ---
def test_agent_tools_off_is_the_three_base_tools() -> None:
"""LOCKED A2: toggle off (the default) → exactly the three base
tools, element-for-element equal to ``AGENT_TOOLS`` — and a FRESH
list (not the constant), so a caller mutating the offer can never
mutate the constant."""
off = agent_tools(_settings())
assert [t["function"]["name"] for t in off] == ["ls", "read", "grep"]
assert off == AGENT_TOOLS # element-for-element
assert off is not AGENT_TOOLS # a fresh list, never the constant itself
off.append({"junk": True}) # mutating the offer must not touch the constant
assert len(AGENT_TOOLS) == 3
def test_agent_tools_on_appends_show_image() -> None:
"""Toggle on → the three base tools (byte-identical) plus the 4th
tool, ``show_image`` — the offer list the grounded turn passes to
the model request (LOCKED A2: ``BOR_ANSWER_IMAGES`` gates ONLY the
offer; the base three are unchanged)."""
on = agent_tools(_settings(answer_images=True))
assert [t["function"]["name"] for t in on] == ["ls", "read", "grep", "show_image"]
assert on[:3] == AGENT_TOOLS # the base three byte-identical
assert on[3] == SHOW_IMAGE_TOOL
def test_show_image_tool_definition_carries_the_relevance_gate() -> None:
"""The tool definition is the same OpenAI function shape as the
other three (one required string arg ``path``, the combined
``source/path`` identity — the bare-path-will-not-resolve teaching
mirrored from ``read``), and its description IS the relevance gate
(LOCKED A5: never a merely related image, at most ONE image, the
full text answer always stands)."""
fn = SHOW_IMAGE_TOOL["function"]
assert SHOW_IMAGE_TOOL["type"] == "function"
assert fn["name"] == "show_image"
assert fn["parameters"]["required"] == ["path"]
assert fn["parameters"]["properties"]["path"]["type"] == "string"
desc = fn["description"]
assert "at most one image" in desc
assert "never for a merely related" in desc
assert "answer fully in text either way" in desc
assert "bare document path will not resolve" in desc
def test_run_agent_gates_the_show_image_offer() -> None:
"""The wiring pin (completion criteria): a grounded turn's offer
(the ``tools`` on the model request, ``max_rounds`` > 0) is the
three base tools when the toggle is off — element-for-element the
pre-phase-124 list — and gains ``show_image`` only when it is on."""
class _ScriptedLLM:
"""One canned content-only round; records the ``tools`` every
request carried (the DB is never touched — no tool calls)."""
def __init__(self) -> None:
self.requests: list[
tuple[list[dict[str, Any]], list[dict[str, Any]] | None]
] = []
async def chat_stream(
self,
messages: list[dict[str, str]],
tools: list[dict[str, Any]] | None = None,
scaffolding: object | None = None,
) -> AsyncIterator[StreamPiece | ToolCallPiece]:
self.requests.append((deepcopy(messages), tools))
yield StreamPiece("content", "Done!")
def _mock_session() -> Session:
mock = cast("Session", MagicMock())
mock.__enter__ = MagicMock(return_value=mock)
mock.__exit__ = MagicMock(return_value=False)
mock.scalar = MagicMock(return_value=None)
mock.execute = MagicMock(return_value=MagicMock(scalars=MagicMock(return_value=[])))
return mock
async def _consume(llm: _ScriptedLLM, settings: Settings) -> None:
async for _piece in run_agent(
cast("LLMClient", llm),
_mock_session,
system_prompt="SYSTEM_PROMPT",
user_message="QUESTION",
seed_docs=[],
settings=settings,
holder=AgentHolder(),
):
pass
llm_off = _ScriptedLLM()
asyncio.run(_consume(llm_off, _settings()))
assert len(llm_off.requests) == 1
assert llm_off.requests[0][1] == AGENT_TOOLS # off: element-for-element pre-124
llm_on = _ScriptedLLM()
asyncio.run(_consume(llm_on, _settings(answer_images=True)))
sent = llm_on.requests[0][1]
assert sent is not None # the grounded turn (max_rounds > 0) offers tools
assert [t["function"]["name"] for t in sent] == [
"ls",
"read",
"grep",
"show_image",
]
# --- the handler: the refusal matrix + the confirming success ---
def test_show_image_refusal_texts_are_pinned() -> None:
"""The refusal texts (the ``MISSING_READ_ARGS`` /
``ALREADY_IN_CONTEXT`` shapes) are pinned — the model's
self-correction relies on them being stable."""
assert MISSING_SHOW_ARGS == "show_image requires a string argument 'path'."
assert ALREADY_SHOWING == (
"Already showing an image this answer — do not show another one. "
"Answer from what you have."
)
assert NOT_AN_IMAGE_DOC.format(arg="s/p.png") == (
"'s/p.png' is not an image document — only standalone image "
"documents can be shown as answers."
)
def _show_call(arguments: dict[str, Any]) -> ToolCallPiece:
return ToolCallPiece(id="call_show", name="show_image", arguments=arguments)
def _seed_show_doc(
db: Session, *, is_image: bool, content: str, path: str
) -> Document:
"""One real documents row ``show_image`` can resolve (source
``showsrc`` — no fixture collision); cleaned up by the caller."""
doc = Document(
id=uuid.uuid4(),
source="showsrc",
path=path,
full_path=f"/showsrc/{path}",
title="diagram" if is_image else "Notes",
content=content,
content_hash="0" * 64,
created_at=_SHOW_FIXTURE_CREATED_AT,
is_image=is_image,
)
if is_image:
doc.image_path = f"/tmp/{doc.id}.png"
db.add(doc)
db.commit()
return doc
def _show(db: Session, settings: Settings, holder: AgentHolder, arguments: dict[str, Any]) -> str:
return _execute_tool(db, _show_call(arguments), [], holder, settings)
def test_show_image_missing_or_empty_path_is_refused(db: Session) -> None:
"""(a) missing/empty/whitespace/non-string ``path`` →
``MISSING_SHOW_ARGS`` — counts in nothing, nothing shown."""
holder = AgentHolder()
for arguments in ({}, {"path": " "}, {"path": None}):
assert _show(db, _settings(), holder, arguments) == MISSING_SHOW_ARGS
assert holder.tool_calls == 0
assert holder.shown_images == []
def test_show_image_unresolvable_arg_gets_the_no_document_teaching(
db: Session,
) -> None:
"""(b) an unresolvable argument → the phase-72
``_no_document_refusal`` text (the bare-argument line, and the
did-you-mean combined-identity for a path-like argument) — counts
in nothing, nothing shown."""
holder = AgentHolder()
# A bare argument (no ``/``) can never be a document — the
# no-DB-lookup refusal, byte-identical to the read branch's line.
assert _show(db, _settings(), holder, {"path": "nosuchdoc"}) == (
"No document at 'nosuchdoc' — check the ls output."
)
assert holder.tool_calls == 0
assert holder.shown_images == []
# A path-like argument matching exactly ONE indexed document's path
# gets the combined-identity teaching.
doc = _seed_show_doc(db, is_image=True, content="A red square.", path="phase124/diagram.png")
try:
result = _show(db, _settings(), holder, {"path": "phase124/diagram.png"})
assert result == _no_document_refusal(db, "phase124/diagram.png")
assert "showsrc/phase124/diagram.png" in result # the combined identity named
assert holder.tool_calls == 0
assert holder.shown_images == []
finally:
db.delete(doc)
db.commit()
def test_show_image_of_a_text_doc_is_refused_not_converted(db: Session) -> None:
"""(c) a resolved TEXT document → ``NOT_AN_IMAGE_DOC`` (the argument
echoed) — a text doc is refused, NEVER silently converted to a
read; counts in nothing, nothing shown or read."""
doc = _seed_show_doc(
db, is_image=False, content="just text.", path="phase124/notes.md"
)
try:
holder = AgentHolder()
result = _show(db, _settings(), holder, {"path": "showsrc/phase124/notes.md"})
assert result == NOT_AN_IMAGE_DOC.format(arg="showsrc/phase124/notes.md")
assert holder.tool_calls == 0
assert holder.shown_images == []
assert holder.read_docs == [] # never converted to a read
finally:
db.delete(doc)
db.commit()
def test_show_image_of_an_image_doc_confirms_and_records(db: Session) -> None:
"""(d) a resolved IMAGE document → the confirming result — the NEW
first line (absent from ``read``) followed by the exact
read-style document block (the ``Document …`` header + date line +
:data:`IMAGE_DOC_MARKER` + the description), ``shown_images ==
[doc]``, and ``tool_calls`` bumped (an executed call — it feeds the
per-turn ``tool_calls=N``)."""
doc = _seed_show_doc(
db,
is_image=True,
content="A red square on a white background.",
path="phase124/diagram.png",
)
try:
holder = AgentHolder()
result = _show(db, _settings(), holder, {"path": "showsrc/phase124/diagram.png"})
assert result == (
"Image shown to the user — it is now part of your answer.\n"
"Document showsrc/phase124/diagram.png:\n"
"date: 2024-06-15\n"
f"{IMAGE_DOC_MARKER}\n"
"A red square on a white background."
)
assert holder.shown_images == [doc]
assert holder.tool_calls == 1
finally:
db.delete(doc)
db.commit()
def test_show_image_second_call_is_refused(db: Session) -> None:
"""(e) a second ``show_image`` in the same turn →
``ALREADY_SHOWING`` (LOCKED A3, server-enforced) — appends nothing,
counts in nothing; the first document stays the one shown."""
doc = _seed_show_doc(
db, is_image=True, content="A red square.", path="phase124/diagram2.png"
)
try:
holder = AgentHolder()
first = _show(db, _settings(), holder, {"path": "showsrc/phase124/diagram2.png"})
assert first.startswith("Image shown to the user — it is now part of your answer.")
second = _show(db, _settings(), holder, {"path": "showsrc/phase124/diagram2.png"})
assert second == ALREADY_SHOWING
assert holder.shown_images == [doc] # still exactly one shown
assert holder.tool_calls == 1 # the refusal counts in nothing
finally:
db.delete(doc)
db.commit()
# ---------------------------------------------------------------------------
# Task 03 — the ``show_image`` prompt clause (toggle-gated)
#
# The clause body itself is pinned in ``tests/unit/test_prompts.py``
# (``SHOW_IMAGE_CLAUSE`` + the ``build_high_prompt`` builds). This
# section pins the chat-side wiring: the grounded plan carries the
# clause EXACTLY when ``settings.answer_images`` is true, and a
# deflected plan never does.
from app.api.chat import plan_turn # noqa: E402
from app.rag.prompts import SHOW_IMAGE_CLAUSE, build_high_prompt # noqa: E402
from app.rag.retriever import RetrievedChunk # noqa: E402
#: A detached document row + its chunk — ``plan_turn`` is pure (no DB),
#: so the wiring is pinned without a database (the cosine 0.9 clears
#: the 0.62 default ``relevance_threshold`` → HIGH/grounded; the
#: 0.2 / no-fts chunk stays LOW/deflected — the gate is strict).
_P3_DOC = Document(
id=uuid.uuid4(),
source="Homelab",
path="kubernetes.md",
full_path="/tmp/kubernetes.md",
title="Kubernetes Homelab Cluster",
content="Talos Linux on three nodes.",
content_hash="0" * 64,
created_at=_SHOW_FIXTURE_CREATED_AT,
summary="A Talos Linux cluster on three nodes.",
)
def _plan3(cosine: float, settings: Settings):
chunk = RetrievedChunk(
chunk_id=uuid.uuid4(),
position=0,
content="kubernetes",
score=1.0,
document=_P3_DOC,
cosine=cosine,
)
return plan_turn([chunk], settings)
def test_plan_turn_grounded_carries_clause_exactly_when_toggle_on() -> None:
"""Completion pin: the grounded plan's system prompt is BYTE-
IDENTICAL to the pre-task build when ``answer_images`` is off (the
default — the pre-phase pins pass untouched) and gains the clause
(inside ``<tools>``) exactly when it is on."""
off = _plan3(0.9, _settings())
assert off.deflected is False
assert off.system_prompt == build_high_prompt([_P3_DOC])
assert SHOW_IMAGE_CLAUSE not in off.system_prompt
on = _plan3(0.9, _settings(answer_images=True))
assert on.deflected is False
assert on.system_prompt == build_high_prompt([_P3_DOC], show_image=True)
assert SHOW_IMAGE_CLAUSE in on.system_prompt
assert on.system_prompt.index("<tools>") < on.system_prompt.index(SHOW_IMAGE_CLAUSE)
assert on.system_prompt.index(SHOW_IMAGE_CLAUSE) < on.system_prompt.index("</tools>")
def test_plan_turn_deflected_never_carries_the_clause() -> None:
"""A deflected turn has no tools and never shows an image (the
phase's locked design): the deflection prompt takes no such flag,
so its build is identical — and clause-free — whether the toggle
is off or on."""
off = _plan3(0.2, _settings())
assert off.deflected is True
assert SHOW_IMAGE_CLAUSE not in off.system_prompt
assert "<tools>" not in off.system_prompt
on = _plan3(0.2, _settings(answer_images=True))
assert on.deflected is True
assert on.system_prompt == off.system_prompt # toggle has no effect at all
assert SHOW_IMAGE_CLAUSE not in on.system_prompt
# ---------------------------------------------------------------------------
# Task 04 — the ``answer_image`` done frame + saved-chat persistence
#
# The wire contract: ``ChatDoneEvent.answer_image`` (the shown doc's
# :class:`SourceRef`, OMITTED from the frame when absent — every
# toggle-off / image-less turn stays byte-identical) and
# ``ChatMessage.answer_image`` (the brain-record key the done-time
# auto-save persists).
#
from json import dumps as _json_dumps # noqa: E402
from pydantic import ValidationError # noqa: E402
from app.schemas import ChatDoneEvent, ChatMessage, SourceRef # noqa: E402
#: The shown image doc's full ref (all four fields — a shown doc is an
#: image doc, so its ref always carries the ``image_url`` bytes route).
_SHOWN_REF = {
"source": "docs",
"path": "pic.png",
"title": "pic",
"image_url": "/api/documents/0e8a6e5d-7b6c-4f2a-9c1d-3e5f7a9b1c2d/image",
}
def test_done_event_without_answer_image_drops_the_key() -> None:
"""``answer_image=None`` (every deflected / image-less / pre-phase
turn) → ``model_dump()`` has NO ``answer_image`` key — ABSENT, never
``null`` — and the dump is byte-identical to the pre-phase frame
shape (the byte-identical-off contract, task 01). Checked on the
dict AND on the raw JSON (no ``"answer_image"`` token, no
``null`` field)."""
event = ChatDoneEvent(
deflected=False,
sources=[SourceRef(source="docs", path="a.md", title="A")],
related=[SourceRef(source="docs", path="b.md", title="B")],
suggestions=["Maybe try X?"],
)
dump = event.model_dump()
assert "answer_image" not in dump
# The pre-phase-124 key set, exactly.
assert set(dump) == {"type", "deflected", "sources", "related", "suggestions"}
assert _json_dumps(dump, sort_keys=True) == _json_dumps(
{
"type": "done",
"deflected": False,
"sources": [{"source": "docs", "path": "a.md", "title": "A"}],
"related": [{"source": "docs", "path": "b.md", "title": "B"}],
"suggestions": ["Maybe try X?"],
},
sort_keys=True,
)
raw = _json_dumps(dump)
assert "answer_image" not in raw
def test_done_event_without_answer_image_parses_a_pre_phase_frame() -> None:
"""Back-compat: a pre-phase-124 done frame (no ``answer_image``
key) parses with the default ``None`` and re-dumps byte-identically
(PLAN §4 — the field is additive in both directions)."""
old_payload = {
"type": "done",
"deflected": True,
"sources": [],
"related": [],
"suggestions": [],
}
event = ChatDoneEvent(**old_payload)
assert event.answer_image is None
assert event.model_dump() == old_payload
def test_done_event_with_answer_image_carries_the_full_ref() -> None:
"""``answer_image`` set (the shown doc) → the key is present with
ALL FOUR ref fields (source/path/title + the ``image_url`` bytes
route), and the existing ``sources``/``related``/``suggestions``
serialization is untouched."""
event = ChatDoneEvent(
deflected=False,
sources=[SourceRef(source="docs", path="pic.png", title="pic")],
related=[],
suggestions=[],
answer_image=SourceRef(**_SHOWN_REF),
)
dump = event.model_dump()
assert dump["answer_image"] == _SHOWN_REF
assert dump["sources"] == [{"source": "docs", "path": "pic.png", "title": "pic"}]
assert dump["related"] == []
assert dump["suggestions"] == []
assert dump["type"] == "done"
assert dump["deflected"] is False
def test_done_event_answer_image_ref_drops_its_own_null_image_url() -> None:
"""The ref's OWN serializer still applies inside the frame: a
text-shaped ref (``image_url=None`` — a corner that cannot occur
server-side, since only image docs are shown, but a client-built
frame could carry it) keeps the key and drops the null field
(the :class:`SourceRef.image_url` omission rule)."""
event = ChatDoneEvent(
deflected=False,
sources=[],
answer_image=SourceRef(source="s", path="p.png", title="t"),
)
dump = event.model_dump()
assert dump["answer_image"] == {"source": "s", "path": "p.png", "title": "t"}
assert "image_url" not in dump["answer_image"]
# --- the ``ChatMessage`` boundary (the saved-chat record key) ---
def test_chat_message_brain_record_with_answer_image_parses() -> None:
"""A brain record carrying ``answer_image`` (the shown doc's full
ref shape) parses — the phase-113 regression: without the field,
the ``extra="forbid"`` boundary would 422 every done-time auto-save
that carries the key (the A2 quiet-failure path)."""
msg = ChatMessage.model_validate(
{"who": "brain", "text": "Here's the diagram…", "answer_image": _SHOWN_REF}
)
assert msg.answer_image is not None
assert msg.answer_image.source == "docs"
assert msg.answer_image.path == "pic.png"
assert msg.answer_image.title == "pic"
assert msg.answer_image.image_url == _SHOWN_REF["image_url"]
def test_chat_message_without_or_null_answer_image_parses() -> None:
"""``answer_image`` ABSENT (every pre-phase-124 record) and explicit
``None`` both parse (the default) — and neither serializes the key
(ABSENT, never ``null`` — pre-phase and image-less records
round-trip byte-identically, the phase-50 contract)."""
absent = ChatMessage.model_validate({"who": "brain", "text": "hi"})
assert absent.answer_image is None
assert "answer_image" not in absent.model_dump()
explicit_null = ChatMessage.model_validate(
{"who": "brain", "text": "hi", "answer_image": None}
)
assert explicit_null.answer_image is None
assert "answer_image" not in explicit_null.model_dump()
def test_chat_message_with_answer_image_serializes_the_ref() -> None:
"""A record WITH a shown ref keeps it in the dump — the restore
path re-renders the answer figure from it (the round-trip)."""
msg = ChatMessage.model_validate(
{"who": "brain", "text": "Here's the diagram…", "answer_image": _SHOWN_REF}
)
dump = msg.model_dump()
assert dump["answer_image"] == _SHOWN_REF
def test_chat_message_user_image_field_still_parses_independently() -> None:
"""The user record's phase-123 ``image`` (the stored PATH string of
the question's attachment) is untouched and semantically distinct:
it still parses on a user record, still serializes the path, and is
independent of ``answer_image`` (a brain record carries the shown
ref, never the attachment path)."""
user = ChatMessage.model_validate(
{
"who": "user",
"text": "What is in this image?",
"image": "/api/chat-images/" + "a" * 32 + ".png",
}
)
assert user.image == "/api/chat-images/" + "a" * 32 + ".png"
assert user.answer_image is None
dump = user.model_dump()
assert dump["image"] == user.image
assert "answer_image" not in dump
def test_chat_message_answer_image_without_image_url_parses() -> None:
"""``answer_image`` missing ``image_url`` parses — the field is
optional on :class:`SourceRef` (its own serializer then drops the
null on the way out)."""
msg = ChatMessage.model_validate(
{
"who": "brain",
"text": "hi",
"answer_image": {"source": "s", "path": "p.png", "title": "t"},
}
)
assert msg.answer_image is not None
assert msg.answer_image.image_url is None
assert msg.model_dump()["answer_image"] == {"source": "s", "path": "p.png", "title": "t"}
def test_chat_message_phase83_bounds_still_bind_the_answer_image_ref() -> None:
"""The phase-83 value bounds bind the ref's fields EXACTLY like
every other :class:`SourceRef` (``source`` ≤ 120, ``path`` ≤ 1000,
``title`` ≤ 500 — the ``documents`` column lengths): one over any
cap is a 422 at the boundary."""
for bad in (
{"source": "s" * 121, "path": "p", "title": "t"},
{"source": "s", "path": "p" * 1001, "title": "t"},
{"source": "s", "path": "p", "title": "t" * 501},
):
with pytest.raises(ValidationError):
ChatMessage.model_validate(
{"who": "brain", "text": "hi", "answer_image": bad}
)
# Exactly AT the caps: parses (a server-built ref from a full-length
# row fits by construction — the same pin as the done-event ref).
at_caps = ChatMessage.model_validate(
{
"who": "brain",
"text": "hi",
"answer_image": {"source": "s" * 120, "path": "p" * 1000, "title": "t" * 500},
}
)
assert at_caps.answer_image is not None
# ---------------------------------------------------------------------------
# Task 05 — bug fix: suggested-tier image docs earn their source chips
#
# ``merge_cited_docs`` (``app/api/chat.py``) composes the citation tier
# (the done frame's ``sources``): the phase-119 read-docs-only rule
# plus phase 124's single documented exception — a suggested-tier image
# doc's seed block IS its entire content (content == summary == the
# vision description, phase 122 task 03), so "suggested" ≡ "read" for
# it and it is auto-cited even on a zero-read grounded turn (a ``read``
# would add only the marker line — the bug: the chip never appeared).
from app.api.chat import merge_cited_docs # noqa: E402
def _uc_doc(source: str, path: str, *, is_image: bool = False) -> Document:
"""A detached documents row (no DB) — ``merge_cited_docs`` touches
only the ``(source, path)`` identity and ``is_image``."""
return Document(
id=uuid.uuid4(),
source=source,
path=path,
full_path=f"/{source}/{path}",
title=path.rsplit("/", 1)[-1],
content="A red square." if is_image else "just text.",
content_hash="0" * 64,
created_at=_SHOW_FIXTURE_CREATED_AT,
is_image=is_image,
)
def test_merge_cited_docs_zero_read_suggested_image_doc_is_cited() -> None:
"""(a) The bug fix: a suggested-tier IMAGE doc + zero reads → it
IS cited (the zero-read grounded turn — the phase-119 "chips
nothing" fast path — chips it; pre-phase-124 this returned
``[]``)."""
img = _uc_doc("docs", "pic.png", is_image=True)
assert merge_cited_docs([], [img], deflected=False) == [img]
def test_merge_cited_docs_read_image_doc_appears_once_at_read_position() -> None:
"""(b) Dedup: an image doc the agent ALSO explicitly read appears
exactly ONCE — at its READ position (the read loop sees it first;
the suggestion merge must not re-add or relocate it)."""
img = _uc_doc("docs", "pic.png", is_image=True)
text = _uc_doc("docs", "notes.md")
merged = merge_cited_docs([img, text], [img, text], deflected=False)
assert merged == [img, text] # once each, read order
assert merged.count(img) == 1
# Read position preserved even when the suggestion order differs:
# the image doc read SECOND stays second (the suggested tier does
# not move read docs).
merged = merge_cited_docs([text, img], [img], deflected=False)
assert merged == [text, img]
def test_merge_cited_docs_zero_read_suggested_text_doc_not_cited() -> None:
"""(c) The phase-119 contract intact: a suggested TEXT doc + zero
reads → NOT cited (auto-citation is image docs only — a zero-read
text turn still chips nothing)."""
text = _uc_doc("docs", "notes.md")
assert merge_cited_docs([], [text], deflected=False) == []
def test_merge_cited_docs_deflected_cites_nothing() -> None:
"""(d) A deflected turn cites NOTHING (phase 112) — even when the
suggested tier (the weak hits, A3) holds image docs and even when
``read_docs`` is non-empty by construction (deflected turns run no
agent — the guard makes the contract explicit)."""
img = _uc_doc("docs", "pic.png", is_image=True)
assert merge_cited_docs([], [img], deflected=True) == []
assert merge_cited_docs([img], [img], deflected=True) == []
def test_merge_cited_docs_two_suggested_image_docs_in_suggestion_order() -> None:
"""(e) Two suggested image docs (zero reads, text docs interleaved
in the tier) → BOTH are cited, in SUGGESTION order (the retrieval
rank is the chip order) — the text members are skipped."""
text1 = _uc_doc("docs", "a.md")
second = _uc_doc("docs", "b.png", is_image=True)
text2 = _uc_doc("docs", "c.md")
first = _uc_doc("docs", "a.png", is_image=True)
merged = merge_cited_docs([], [text1, second, text2, first], deflected=False)
assert merged == [second, first] # suggestion order, images only
# With a read doc present, the reads come first, then the image
# docs in suggestion order.
read = _uc_doc("docs", "read.md")
merged = merge_cited_docs([read], [second, first], deflected=False)
assert merged == [read, second, first]
def test_merge_cited_docs_read_order_preserved_for_mixed_read_docs() -> None:
"""(f) Read order is preserved for mixed read text docs and
duplicate reads collapse (dedupe by ``(source, path)``, first
occurrence wins) — with the suggested image doc appended AFTER
the reads."""
a = _uc_doc("docs", "a.md")
b = _uc_doc("docs", "b.md")
img = _uc_doc("docs", "pic.png", is_image=True)
merged = merge_cited_docs([b, a, b], [img, a], deflected=False)
assert merged == [b, a, img]
# A duplicate image read collapses too (once, at its read
# position) — the suggestion merge adds nothing for it.
merged = merge_cited_docs([img, img, a], [img], deflected=False)
assert merged == [img, a]
# ---------------------------------------------------------------------------
# Task 06 — the answer figure: live render, persistence, restore, shared
#
# House-style frontend source contracts (the phase-122/123 pattern —
# the browser-level scenario is task 07's isolated E2E): the chat
# page's ``appendAnswerImage`` helper (lazy img, the public bytes
# route, the visible caption, the collapse-on-error rule, the chip
# navigation affordance), its three wiring points (the ``done``
# handler BEFORE the chip row, the ``rememberBrainTurn`` meta key, the
# ``renderStoredMessage`` restore BEFORE the chip row), the shared
# page's NON-INTERACTIVE local copy (zero controls), and the
# answer-scale themed CSS (token-only, the 320px cap).
import re # noqa: E402
from pathlib import Path # noqa: E402
FRONTEND = Path(__file__).resolve().parents[2] / "frontend" # noqa: E402
APP_JS = FRONTEND / "assets" / "app.js" # noqa: E402
SHARED_JS = FRONTEND / "assets" / "shared.js" # noqa: E402
STYLES_CSS = FRONTEND / "assets" / "styles.css" # noqa: E402
def _function_body(js: str, header: str) -> str:
"""The full text of the function whose header is ``header`` — from
the header to its brace-matched closing ``}`` (the
``test_image_documents.py`` house helper; the pinned functions' code
carries balanced braces)."""
start = js.index(header)
body_open = js.index("{", start)
depth = 0
for j in range(body_open, len(js)):
if js[j] == "{":
depth += 1
elif js[j] == "}":
depth -= 1
if depth == 0:
return js[start : j + 1]
raise AssertionError(f"unbalanced braces for {header!r}")
def _js() -> str:
return APP_JS.read_text(encoding="utf-8")
def _shared_js() -> str:
return SHARED_JS.read_text(encoding="utf-8")
def _css() -> str:
return STYLES_CSS.read_text(encoding="utf-8")
def test_answer_image_helper_builds_the_lazy_capturing_figure() -> None:
"""The chat page's ``appendAnswerImage`` (the visual sibling of
``appendSourceImageFigure`` — the house createElement style, no
HTML strings): the figure anchors on the chip's navigation
affordance (the ``documentUrl`` href + the phase-26 same-page
modal), the img is LAZY on the public bytes route, alt + the
visible caption start at the TITLE and swap to the document's
SUMMARY via the SAME ``fetchContentSummary`` pattern (fails soft —
the title stands), and a FAILED IMAGE LOAD removes the whole
figure (the collapse rule — never a broken icon)."""
body = _function_body(_js(), "function appendAnswerImage(wrap, ref) {")
assert 'fig.className = "answer-image";' in body
assert 'fig.href = documentUrl(ref.source, ref.path, "/");' in body
assert "openDocumentModal(ref.source, ref.path, fig);" in body
assert "e.preventDefault();" in body # no new tab (phase 26)
assert 'img.className = "answer-image-img";' in body
assert "img.src = ref.image_url;" in body # the public bytes route
assert 'img.loading = "lazy";' in body # A4: lazy-loaded
assert "img.alt = ref.title || label;" in body # title first (always visible)
assert 'caption.className = "answer-image-caption";' in body
assert "caption.textContent = ref.title || label;" in body # visible caption
assert "fetchContentSummary(ref.source, ref.path)" in body
assert "img.alt = summary;" in body # the swap once it settles
assert "caption.textContent = summary;" in body
assert "if (!fig.isConnected) return;" in body # collapsed/cleared → no swap
assert "fig.remove();" in body # the collapse-on-error rule
# The guard: no ref (or a malformed one) → no figure DOM at all.
assert "!ref" in body
assert 'typeof ref.image_url !== "string"' in body
def test_done_handler_renders_the_figure_before_the_chips() -> None:
"""A4 position, live path: the ``done`` handler appends the figure
to ``.msg-body`` right BEFORE ``appendSources`` creates the chip
row (below the text, above the chips) — and only when the frame
carries the key (a frame without it renders byte-identically,
no figure DOM)."""
js = _js()
call = "if (ev.answer_image) appendAnswerImage(wrap, ev.answer_image);"
assert call in js
done = js.index('} else if (ev.type === "done") {')
next_branch = js.index('} else if (ev.type === "tool_result") {')
handler = js[done:next_branch]
assert call in handler
assert handler.index(call) < handler.index("appendSources(wrap, ev.sources);")
def test_done_handler_persists_the_answer_image_meta() -> None:
"""The done-time save point carries the key (the house
undefined-drops-the-key pattern — the ``related`` line is the
precedent): image-less / pre-phase turns store byte-identical
records; a record WITH the key round-trips through the task-04
boundary (``ChatMessage.answer_image``)."""
js = _js()
done = js.index('} else if (ev.type === "done") {')
next_branch = js.index('} else if (ev.type === "tool_result") {')
handler = js[done:next_branch]
assert "answer_image: ev.answer_image || undefined," in handler
def test_stored_message_restore_renders_the_figure_before_the_chips() -> None:
"""The refresh landing: ``renderStoredMessage``'s brain branch
renders ``m.answer_image`` through the SAME helper at the SAME A4
position (before the chip row) — a record without the key renders
byte-identically (pre-phase chats restore unchanged)."""
body = _function_body(_js(), "function renderStoredMessage(m) {")
call = "if (m.answer_image) appendAnswerImage(wrap, m.answer_image);"
assert call in body
assert body.index(call) < body.index("appendSources(wrap, m.sources);")
def test_shared_page_figure_is_non_interactive() -> None:
"""The shared page's local copy (the per-page duplication house
style): a PLAIN non-interactive ``<figure>`` (owner-locked zero
controls — no anchor, no href, no click handler), the LAZY img on
the PUBLIC bytes route (the guest view is faithful — the
phase-123 user-image contract), the title as the always-visible
caption (the content endpoint is user-gated — a guest fetch would
401; no fetch, the title stands), and the same collapse-on-error
rule. Wired where the brain branch appends sources (figure
BEFORE the chips)."""
shared = _shared_js()
body = _function_body(shared, "function addAnswerImage(wrap, ref) {")
assert 'document.createElement("figure")' in body # plain figure, no anchor
assert 'fig.className = "answer-image";' in body
assert 'document.createElement("a")' not in body # zero controls
assert ".href" not in body
assert 'addEventListener("click"' not in body
assert 'img.className = "answer-image-img";' in body
assert "img.src = ref.image_url;" in body # the public bytes route
assert 'img.loading = "lazy";' in body
assert "img.alt = title;" in body
assert 'caption.className = "answer-image-caption";' in body
assert "caption.textContent = title;" in body
assert "fig.remove()" in body # the collapse-on-error rule
assert "fetchContentSummary" not in body # user-gated endpoint — no guest fetch
# The brain branch wiring: figure BEFORE the chips.
msg = _function_body(shared, "function renderSharedMessage(m) {")
assert "addAnswerImage(wrap, m.answer_image);" in msg
assert msg.index("addAnswerImage(wrap, m.answer_image);") < msg.index(
"addSources(wrap, m.sources);"
)
def test_answer_image_css_is_answer_scale_and_themed() -> None:
"""The ``.answer-image`` family (the ``.source-image`` language at
answer scale, the zero-literal invariant so the monochrome themes
carry it): the img is block, full-column-width safe, CAPPED at
320px, contain on the theme's surface with the bordered-image
treatment; the caption is the AA-safe muted ink (8.6:1 on the
row's page background); the hover underline is scoped to the chat
page's ANCHOR figure only (the shared page's plain ``<figure>`` is
out of scope by element type)."""
css = _css()
for selector in (".answer-image", ".answer-image-img", ".answer-image-caption"):
assert re.search(rf"^{re.escape(selector)} \{{", css, re.M), f"missing rule: {selector}"
img = re.search(r"^\.answer-image-img \{\n([\s\S]*?)\n\}", css, re.M)
assert img, "missing .answer-image-img rule"
img_body = img.group(1)
assert "max-height: 320px" in img_body # the answer-scale cap
assert "object-fit: contain" in img_body
assert "display: block" in img_body
assert "background: var(--surface)" in img_body # the theme's surface
assert "border: 1px solid var(--line)" in img_body
assert "max-width: 100%" in img_body
cap = re.search(r"^\.answer-image-caption \{\n([\s\S]*?)\n\}", css, re.M)
assert cap, "missing .answer-image-caption rule"
assert "color: var(--ink-soft)" in cap.group(1) # AA: 8.6:1 on --bg
base = re.search(r"^\.answer-image \{\n([\s\S]*?)\n\}", css, re.M)
assert base and "max-width: 100%" in base.group(1) # full bubble width
hover = re.search(r"^a\.answer-image:hover \{([^}]*)\}", css, re.M)
assert hover and "text-decoration: underline" in hover.group(1), (
"the chat-page anchor figure keeps the flat-underline hover"
)
# Zero literals: the whole phase-124 block is token-only (no hex
# colors — the monochrome theme grays it automatically).
start = css.index("/* Phase 124 (task 06")
end = css.index("/* Phase 113 (task 02): the related-docs row")
assert start != -1 and end > start
assert re.search(r"#[0-9a-fA-F]{3,6}\b", css[start:end]) is None
+126
View File
@@ -34,6 +34,7 @@ the LOW prompt and ``PERSONA`` / ``TOOLS_SECTION`` stay byte-identical
"""
from __future__ import annotations
import inspect
import uuid
from datetime import UTC, datetime
@@ -43,6 +44,7 @@ from app.config import Settings
from app.models import Document
from app.rag.prompts import (
PERSONA,
SHOW_IMAGE_CLAUSE,
SUGGEST_INTRO,
TOOLS_SECTION,
_base,
@@ -1015,3 +1017,127 @@ def test_plain_text_line_never_leaks_into_high_prompt() -> None:
high = build_high_prompt([doc], notes=notes, kb_overview=kb)
assert PLAIN_TEXT_ONLY_LINE not in high
assert "you have no tools" not in high
# ---------- phase 124 (task 03, LOCKED A5): the show_image prompt clause ----------
def test_show_image_clause_states_the_four_gate_elements() -> None:
"""LOCKED A5: the clause states ALL FOUR gate elements, pinned on
stable phrases, and references the tool by its backticked name (the
``TOOLS_SECTION`` voice). (1) the image-doc nature — its text is a
description GENERATED from the image; (2) the "contains the answer"
bar — never a merely related image, never padding; (3) at most one
image per answer; (4) the text answer is still provided in full."""
assert "`show_image`" in SHOW_IMAGE_CLAUSE # backticked, like the other tools
assert "description GENERATED from the image" in SHOW_IMAGE_CLAUSE # (1)
assert "what the user actually wants to see" in SHOW_IMAGE_CLAUSE # (2)
assert "never for a merely related image" in SHOW_IMAGE_CLAUSE # (2)
assert "never to pad an answer" in SHOW_IMAGE_CLAUSE # (2)
assert "at most one image per answer" in SHOW_IMAGE_CLAUSE # (3)
assert "answer fully in text" in SHOW_IMAGE_CLAUSE # (4)
# It is a clause, not a section: no ``<tools>`` markup of its own
# (the ``<documents>`` mention is plain text, as in the section copy).
assert "<tools>" not in SHOW_IMAGE_CLAUSE
assert "</tools>" not in SHOW_IMAGE_CLAUSE
def test_high_prompt_default_stays_byte_identical_to_pre_task() -> None:
"""Completion pin: ``build_high_prompt(docs)`` (no new arg) and the
explicit ``show_image=False`` are byte-identical to each other and
end with the untouched ``TOOLS_SECTION`` — the pre-task pins above
(and the sha pins in :mod:`tests.unit.test_prompt_lock`) pass
UNMODIFIED, with or without documents."""
doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes Homelab Cluster")
plain = build_high_prompt([doc])
assert build_high_prompt([doc], show_image=False) == plain
assert plain.endswith(TOOLS_SECTION)
assert SHOW_IMAGE_CLAUSE not in plain
empty = build_high_prompt([])
assert build_high_prompt([], show_image=False) == empty
assert empty.endswith(TOOLS_SECTION)
assert SHOW_IMAGE_CLAUSE not in empty
def test_high_prompt_show_image_clause_lands_inside_tools_after_the_copy() -> None:
"""Phase 124 (task 03): ``show_image=True`` builds EXACTLY the
toggle-off prompt with its final ``</tools>`` replaced by the clause
(on its own line) + the closing tag — the prefix up to the clause
is byte-identical to the toggle-off output (the three-tool copy
untouched), the clause lands AFTER the existing copy and BEFORE the
closing tag, and appears exactly once."""
doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes Homelab Cluster")
off = build_high_prompt([doc])
on = build_high_prompt([doc], show_image=True)
# The full structural equation (the clause is the ONLY difference).
assert on == off[: -len("</tools>")] + SHOW_IMAGE_CLAUSE + "\n</tools>"
clause_start = on.index(SHOW_IMAGE_CLAUSE)
# The prefix up to the clause is byte-identical to the toggle-off
# build minus its closing tag.
assert on[:clause_start] == off[: -len("</tools>")]
# From the <tools> opener to the clause is TOOLS_SECTION's own text
# minus its final </tools> — the constant itself is unedited (sha-
# pinned in test_prompt_lock) and the three-tool copy is intact.
assert on[on.index("<tools>") : clause_start] == TOOLS_SECTION[: -len("</tools>")]
assert "Answer as soon as you have what you need." in on[:clause_start]
# The clause (then the closing tag) is the tail of the prompt, after
# the existing copy and before the closing tag — exactly once.
assert on.endswith(SHOW_IMAGE_CLAUSE + "\n</tools>")
assert on.index("Answer as soon as you have what you need.") < clause_start
assert clause_start < on.index("</tools>")
assert on.count(SHOW_IMAGE_CLAUSE) == 1
assert on.count("<tools>") == 1 and on.count("</tools>") == 1
def test_show_image_clause_rides_only_the_high_prompt() -> None:
"""The clause appears ONLY in the HIGH prompt: the deflection prompt
takes NO ``show_image`` flag at all and its output never carries the
clause (byte-identical builds with/without notes/overview — a
deflected turn has no tools and never shows an image, the phase's
locked design)."""
assert "show_image" not in inspect.signature(build_deflect_prompt).parameters
for prompt in (
build_deflect_prompt(["T1", "T2"]),
build_deflect_prompt(["T1"], notes=["be concise"], kb_overview=OVERVIEW),
build_deflect_prompt([]),
):
assert SHOW_IMAGE_CLAUSE not in prompt
assert "`show_image`" not in prompt
assert "at most one image per answer" not in prompt
def test_high_prompt_empty_documents_show_image_clause_still_in_tools() -> None:
"""The "(no documents matched…)" body + ``show_image=True``: the
clause still lands inside ``<tools>`` — the grounded turn may still
``ls``/``grep`` its way to an image doc and show it."""
off = build_high_prompt([])
on = build_high_prompt([], show_image=True)
assert on == off[: -len("</tools>")] + SHOW_IMAGE_CLAUSE + "\n</tools>"
assert "(no documents matched — do not invent specifics)" in on
assert on.index("<tools>") < on.index(SHOW_IMAGE_CLAUSE) < on.index("</tools>")
assert on.endswith(SHOW_IMAGE_CLAUSE + "\n</tools>")
def test_show_image_clause_is_last_for_every_high_shape() -> None:
"""Steering + KB overview + folder lines + documents all present:
the section order is UNCHANGED (``<relevance>`` →
``<knowledge_base>`` → ``<tuning>`` → ``<documents>`` → ``<tools>``)
and the clause is still the last thing the model reads, inside
``<tools>`` after the three-tool copy."""
doc = _doc("deploy/k8s.md", "CONTENT_SENTINEL", "K8s Deploy", summary="K8s deploy notes.")
lines = ["Homelab/deploy/: k8s.md (1 doc)"]
off = build_high_prompt([doc], notes=["be concise"], kb_overview=OVERVIEW, folder_lines=lines)
on = build_high_prompt(
[doc],
notes=["be concise"],
kb_overview=OVERVIEW,
folder_lines=lines,
show_image=True,
)
assert on == off[: -len("</tools>")] + SHOW_IMAGE_CLAUSE + "\n</tools>"
clause_start = on.index(SHOW_IMAGE_CLAUSE)
assert on[on.index("<tools>") : clause_start] == TOOLS_SECTION[: -len("</tools>")]
for section in ("<relevance>", "<knowledge_base>", "<tuning>", "<documents>"):
assert on.index(section) < on.index("<tools>")
assert on.index("K8s deploy notes.") < on.index("<tools>") # the summary still seeds
assert SHOW_IMAGE_CLAUSE not in off # the toggle-off twin carries none of it