ducoterra
c275735db4
All 4 task files exist in `complete/`, and only `00_phase.md` remains in `todo/`. This is the final verification pass. I'll read through the task files and check the current state of the code.
781 lines
36 KiB
Python
781 lines
36 KiB
Python
"""Phase 140 task 04 E2E (Playwright, mock-only): the pictorial PDF —
|
|
sparse pages rendered + vision-described into the indexed content.
|
|
|
|
Run in isolation (DB must be up: ``podman compose up -d db``):
|
|
|
|
uv run pytest tests/e2e/test_pictorial_pdf.py -v --no-cov
|
|
|
|
The dedicated suite for ``140_pictorial_pdf_pages`` (A16 — one Playwright
|
|
file per phase, run in isolation): it proves the owner's TODO L5 scenario
|
|
end to end on a SYNTHETIC fixture (the owner's real
|
|
``ikea_instructions.tar`` is pinned at the integration level, task 03 —
|
|
the E2E keeps the deterministic canned-reply premise): a 2-page PDF
|
|
built with PyMuPDF (the ``test_pdf_support.py`` recipe) —
|
|
|
|
* page 1 = a 700+ char prose paragraph (over the 100-char sparseness
|
|
bar — NOT a pictorial candidate),
|
|
* page 2 = the 14-char ``Illustration 2`` marker + a drawn VECTOR figure
|
|
(a rectangle + an arrow — the IKEA class: vector-drawn pictograms,
|
|
ZERO embedded raster images, so the phase-122/137 embedded-image
|
|
pipeline sees nothing; the thin text IS a candidate),
|
|
|
|
tarred as a single-root-member ``.tar`` (the owner's fixture's exact
|
|
format — the upload gate admits it) and arriving through the REAL admin
|
|
upload flow → the RAG page's Sync button:
|
|
|
|
* (a) the document appears on the Sources page and the document viewer
|
|
shows the PDF's ACTUAL pages (phase 148 — a described sparse page
|
|
makes the doc page-QUALIFYING: the page strip REPLACES the text
|
|
content AND the Summary panel, the owner's TODO L5 contract): two
|
|
``.doc-page`` figures with loaded lazy images + ``Page N`` captions,
|
|
NO ``.doc-raw``, NO ``.doc-summary``; the described content itself
|
|
(page 1's prose WITHOUT an illustration block, page 2 WITH
|
|
``[Page 2 illustration: <the mock's canned PICTORIAL_PAGE_ANSWER>]``
|
|
— the task-02 ``PICTORIAL_PAGE_MODE`` branch, deterministic) stays in
|
|
the DB: the seed asserts it BYTE-EXACT against the fixture's own
|
|
extraction (the candidates computed from
|
|
``extract_pdf_page_texts`` + ``pictorial_pages`` IN the test, no
|
|
hardcoded page list) and the chat test below proves it still reaches
|
|
the model;
|
|
* (b) a chat question that ONLY the illustration answers ("how are the
|
|
two parts joined on page two?" — the phrase lives NOWHERE in the text
|
|
layer) grounds (the described page's embedding retrieves — the mock's
|
|
deterministic token-overlap cosine clears the mock-calibrated 0.30
|
|
gate; FTS corroborates) and, scripted through the house seeded-read
|
|
convention (phase 119 A1 — a zero-read grounded turn chips NOTHING, so
|
|
the PDF must be READ to land in ``done.sources``), the SSE completes
|
|
without deflecting: the ``done`` event's ``sources`` carries the PDF
|
|
and the answer echoes the read result VERBATIM — the
|
|
``[Page 2 illustration: …]`` block and its phrase reach the model and
|
|
land in the answer;
|
|
* (c) the sync's ``detail`` JSON keeps its pre-phase keys EXACTLY (no
|
|
new field — the ``pdfs_pictorial`` / ``pdf_pictorial_pages`` counters
|
|
are log-only, PLAN §9, the phase-133 convention).
|
|
|
|
House pattern: one module app over the shared E2E Postgres (its own
|
|
port, module env — the conftest session app is never started in this
|
|
isolated run, so no port clash). The three tests SHARE the module seed
|
|
(one source, one document), so the shared registry + KB is truncated
|
|
ONCE, after the last test (a module-granularity autouse — a per-test
|
|
truncate would wipe the seed from under its siblings), with a
|
|
no-running-sync wait so a leaked run cannot race it.
|
|
|
|
App env: ``BOR_IMPORT_EXTENSIONS`` pinned WITH the ``pdf`` token (the
|
|
phase-126 convention — without the token the walk is blind to the file)
|
|
and ``BOR_PDF_PICTORIAL`` pinned EXPLICITLY to ``1``: the conftest pins
|
|
it OFF in ``os.environ`` for the pre-phase-140 story suites (their
|
|
synthetic PDFs carry sparse pages — a default-ON app would move their
|
|
byte-exact pins), and this suite is the documented opt-in (the
|
|
conftest's phase-140 comment). ``BOR_IMAGES`` is pinned ``false`` — the
|
|
embedded-image pipeline is out of scope (the fixture's figure is
|
|
vector-drawn; it yields no image doc either way). The byte-exact
|
|
content assertion in the seed is the toggle's premise witness: without
|
|
the opt-in, page 2 would lack its illustration block and the seed
|
|
fails loudly.
|
|
|
|
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the described
|
|
block is the mock's deterministic ``PICTORIAL_PAGE_MODE`` canned reply
|
|
(``PICTORIAL_PAGE_ANSWER``), so the byte-exact content + echo
|
|
assertions are exact only against it.
|
|
|
|
Test → observable mapping (Playwright Mapping Rule):
|
|
1. ``test_sources_page_and_viewer_show_the_described_page`` — the
|
|
Sources page reads 1 doc / 3 chunks (2 page-anchored content chunks +
|
|
the summary chunk — no image doc), the drill-down table lists the PDF
|
|
as a plain text row (LOCKED A4 — a text document, no thumbnail), and
|
|
the same-page document viewer (the row's path link) shows the
|
|
PHASE-148 PAGE STRIP under the METADATA title (the fixture is
|
|
page-qualifying — the pictorial pass described its sparse page):
|
|
exactly two ``.doc-page`` figures, each a LOADED lazy ``<img>``
|
|
(naturalWidth > 0) with the ``Page N of <title>`` alt + the
|
|
``Page N`` caption, NO ``.doc-raw`` / NO ``.doc-summary`` (the
|
|
page descriptions stay in the DB for Q&A, not displayed — the
|
|
owner's contract).
|
|
2. ``test_chat_question_grounds_and_echoes_the_illustration`` — the raw
|
|
SSE over the scripted seeded-read question (httpx — the ``done``
|
|
event is the wire lens the task pins; the UI flow's premise is the
|
|
module seed): the stream settles ``done`` with ``deflected`` false
|
|
and NO ``error`` frame, EXACTLY one ``tool`` frame (the scripted
|
|
``read`` on the PDF; the doc is far under the read cap — no
|
|
truncation, so no ``tool_result`` frame), the ``done`` event's
|
|
``sources`` is EXACTLY the PDF (the citation surface — read docs
|
|
only, phase 119 A1), and the joined deltas echo the read result
|
|
VERBATIM — the ``Document <source>/<path>:`` header and the
|
|
``[Page 2 illustration: …]`` block with the canned phrase (the
|
|
described page's content reaches the model and lands in the answer).
|
|
3. ``test_sync_detail_keeps_the_pre_phase_keys`` — the module run's
|
|
terminal ``detail`` (``GET /api/sync/status`` — the same object the
|
|
UI's "1 added" line renders from) carries the pre-phase key set
|
|
EXACTLY: the ``pdfs_pictorial`` / ``pdf_pictorial_pages`` counters
|
|
never ride the wire (the stats log-only convention), and the run's
|
|
own file counters read 1 file / 1 added / 0 unchanged / 0 pruned /
|
|
0 errors.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
import re
|
|
import subprocess
|
|
import sys
|
|
import tarfile
|
|
import time
|
|
from collections.abc import Iterator
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
import httpx
|
|
import pymupdf as fitz # PyMuPDF — the synthetic-fixture builder (the house recipe)
|
|
import pytest
|
|
from playwright.sync_api import Browser, Page, expect
|
|
from sqlalchemy import select, text
|
|
|
|
from app.config import Settings
|
|
from app.db import SessionLocal
|
|
from app.models import Document
|
|
from app.rag.pdf import (
|
|
PAGE_SEPARATOR,
|
|
extract_pdf_page_texts,
|
|
pictorial_pages,
|
|
)
|
|
from e2e.auth_helpers import login
|
|
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
|
from e2e.mock_llm import PICTORIAL_PAGE_ANSWER
|
|
|
|
REPO = Path(__file__).resolve().parents[2]
|
|
SOURCE_NAME = "e2e-pictorial" # the archive stem (archive_source_name)
|
|
DOC_PATH = "instructions.pdf"
|
|
PDF_TITLE = "Assembly Instructions" # the metadata title (LOCKED A2)
|
|
|
|
#: Page 1 — a 700+ char prose paragraph (over the 100-char sparseness
|
|
#: bar — NOT a candidate). Deliberately carries NO "two parts joined"
|
|
#: content: the phrase the question targets lives ONLY in the
|
|
#: description the vision pass appends to page 2 (the owner's L5
|
|
#: scenario — the illustration answers what the text layer cannot).
|
|
PAGE_ONE = (
|
|
"This manual covers the complete flat-pack furniture frame, its "
|
|
"shipped hardware kit, and the safety checks a first-time assembler "
|
|
"should run before starting. The box holds the two side panels, the "
|
|
"top board, the adjustable leg set, eight cam locks, fourteen wood "
|
|
"screws, two bolts, and the wall-anchoring strap. Work on a flat "
|
|
"surface with soft flooring, keep the hardware sorted by size, and "
|
|
"read every step through before you begin so you know which panel "
|
|
"is which. The frame stands roughly a metre tall once assembled, "
|
|
"weights under twelve kilograms, and needs no further tools beyond "
|
|
"the included Allen key. Keep the small parts away from children "
|
|
"until the final step is done and the cam locks are fully seated."
|
|
)
|
|
assert len(PAGE_ONE) > 500, "page 1 must be a 500+ char paragraph (the task's recipe)"
|
|
|
|
#: Page 2 — the sparse page: the marker line is ALL the text layer
|
|
#: carries (14 stripped chars, under the 100-char bar — the candidate);
|
|
#: the figure is drawn vector (no raster image).
|
|
PAGE_TWO = "Illustration 2"
|
|
|
|
#: The stored illustration block (LOCKED A4) — the mock's canned
|
|
#: ``PICTORIAL_PAGE_MODE`` reply, deterministic.
|
|
ILLUSTRATION_BLOCK = f"[Page 2 illustration: {PICTORIAL_PAGE_ANSWER}]"
|
|
|
|
#: The grounded question (b): the house seeded-read call (phase 119 A1 —
|
|
#: a zero-read grounded turn chips NOTHING, so the PDF must be READ to
|
|
#: earn its ``done.sources`` citation) + the natural question, whose
|
|
#: tokens ("two parts joined page") overlap ONLY the illustrated page's
|
|
#: content (the mock's token-overlap cosine grounds; the text layer
|
|
#: alone would answer nothing).
|
|
QUESTION = (
|
|
f"Read the suggested document: read {SOURCE_NAME}/{DOC_PATH} — "
|
|
"how are the two parts joined on page two?"
|
|
)
|
|
|
|
#: The sync ``detail`` key set EXACTLY as pre-phase 140 (the
|
|
#: ``app/api/sync.py`` terminal detail — phase 126/133 shape). The
|
|
#: phase-140 counters are log-only and must NOT appear (c).
|
|
PRE_PHASE_DETAIL_KEYS = frozenset(
|
|
{
|
|
"files",
|
|
"added",
|
|
"updated",
|
|
"unchanged",
|
|
"pruned",
|
|
"errors",
|
|
"chunks",
|
|
"summaries",
|
|
"summary_errors",
|
|
"dates_updated",
|
|
"overview",
|
|
"sources_version",
|
|
}
|
|
)
|
|
|
|
#: Phase 79 (task 04, full inventory): the conftest session app owns its
|
|
#: port in a combined run — this module app binds its own port instead
|
|
#: (a same-port second uvicorn dies on bind and would drive the wrong
|
|
#: server). Env-overridable.
|
|
APP_PORT = int(os.environ.get("E2E_APP_PORT_PICTPDF", "8155"))
|
|
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
|
|
|
UPLOAD_TIMEOUT_MS = 30_000
|
|
SYNC_TIMEOUT_MS = 60_000
|
|
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Fixture builder (the ``test_pdf_support.py`` PyMuPDF recipe, the
|
|
# owner's class: a dense prose page + a sparse vector-illustrated page)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _build_pdf(path: Path) -> None:
|
|
"""ONE 2-page PDF (the task's recipe): page 1 = the 700+ char prose
|
|
paragraph (``insert_textbox`` — word-wrapped, no mid-word breaks in
|
|
the extraction), page 2 = the ``Illustration 2`` marker + a drawn
|
|
VECTOR figure (a rectangle + a line arrow with a two-stroke head —
|
|
the IKEA pictogram class: ``get_image_info()`` sees ZERO embedded
|
|
raster images, the phase-122/137 pipeline is blind to it)."""
|
|
doc = fitz.open()
|
|
page1 = doc.new_page()
|
|
rc = page1.insert_textbox(fitz.Rect(72, 72, 523, 772), PAGE_ONE, fontsize=11)
|
|
assert rc >= 0, "the page-1 paragraph must fit its text box"
|
|
page2 = doc.new_page()
|
|
page2.insert_text((72, 72), PAGE_TWO)
|
|
# The vector figure: a rectangle (a part) + an arrow (the join).
|
|
page2.draw_rect(fitz.Rect(100, 120, 300, 220))
|
|
page2.draw_line(fitz.Point(320, 170), fitz.Point(470, 170))
|
|
page2.draw_line(fitz.Point(470, 170), fitz.Point(450, 158))
|
|
page2.draw_line(fitz.Point(470, 170), fitz.Point(450, 182))
|
|
doc.set_metadata({"title": PDF_TITLE})
|
|
doc.save(path)
|
|
doc.close()
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def pdf_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
|
"""The fixture PDF (module-scoped, built once) — the local copy the
|
|
byte-exact content assertion computes against (the app indexes the
|
|
unpacked copy of the SAME bytes)."""
|
|
root = tmp_path_factory.mktemp("bor_pict_pdf")
|
|
pdf = root / DOC_PATH
|
|
_build_pdf(pdf)
|
|
# The premise pins (deterministic — the pass's candidate set and the
|
|
# gate's own contract, computed from the fixture itself): page 1 is
|
|
# prose (no candidate), page 2 is sparse (the candidate).
|
|
page_texts = extract_pdf_page_texts(pdf)
|
|
candidates = pictorial_pages(
|
|
page_texts,
|
|
min_chars=Settings.model_fields["pdf_pictorial_min_chars"].default,
|
|
)
|
|
assert candidates == [2], (
|
|
f"the fixture's sparse-page premise broke — candidates={candidates}"
|
|
)
|
|
return pdf
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def tar_path(pdf_path: Path) -> Path:
|
|
"""The fixture archive: ONE PDF at the root (the ``.tar`` format —
|
|
the owner's ``ikea_instructions.tar`` class; the upload gate admits
|
|
it) — source ``e2e-pictorial``, document path ``instructions.pdf``."""
|
|
archive = pdf_path.parent / f"{SOURCE_NAME}.tar"
|
|
with tarfile.open(archive, "w") as tf:
|
|
tf.add(pdf_path, arcname=DOC_PATH)
|
|
return archive
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Module app (the conftest env pattern + the phase-140 knobs)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _app_env(mock_llm: int, app_port: int, scratch: Path) -> dict[str, str]:
|
|
"""The conftest app env (leak guards included) with the phase-140
|
|
knobs: ``BOR_IMPORT_EXTENSIONS`` pinned to a set carrying the
|
|
``pdf`` token (the phase-126 gate — an operator's local gitignored
|
|
``.env`` cannot leak the scope either way), ``BOR_PDF_PICTORIAL``
|
|
pinned EXPLICITLY on (the conftest pins it off in ``os.environ``
|
|
for the pre-phase suites — this suite is the documented opt-in;
|
|
without it the whole premise is void), and ``BOR_IMAGES`` pinned
|
|
off (out of scope — the vector figure yields no image doc anyway)."""
|
|
env = dict(os.environ)
|
|
env.pop("DEBUGPY", None)
|
|
env["BOR_ENVIRONMENT"] = "e2e"
|
|
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
|
env["BOR_LLM_BASE_URL"] = (
|
|
"https://aipi.reeseapps.com/v1"
|
|
if USE_REAL_LLM
|
|
else f"http://127.0.0.1:{mock_llm}/v1"
|
|
)
|
|
# Mock-calibrated gate (conftest pattern) — the story's question
|
|
# grounds on these values (see QUESTION).
|
|
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
|
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
|
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
|
# Phase 67: instant retry waits + the code-default budget.
|
|
env["BOR_LLM_RETRY_DELAY"] = "0"
|
|
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
|
|
env.setdefault(
|
|
"BOR_DATABASE_URL",
|
|
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
|
)
|
|
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
|
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
|
# Leak guards (conftest pattern).
|
|
env["BOR_GIT_SOURCES"] = ""
|
|
env["BOR_DOCS_REPO"] = ""
|
|
env["BOR_SUGGESTIONS"] = json.dumps(
|
|
Settings.model_fields["suggestions"].default
|
|
)
|
|
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
|
|
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
|
|
# Phase 140: the import scope WITH the ``pdf`` token + the toggle
|
|
# under test (the EXPLICIT opt-in — see the module docstring) + the
|
|
# suite-private homes.
|
|
env["BOR_IMPORT_EXTENSIONS"] = "md,markdown,txt,pdf"
|
|
env["BOR_PDF_PICTORIAL"] = "1"
|
|
env["BOR_IMAGES"] = "false"
|
|
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
|
|
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
|
|
env["BOR_IMAGE_DIR"] = str(scratch / "images")
|
|
return env
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def app_server(
|
|
mock_llm: int, tmp_path_factory: pytest.TempPathFactory
|
|
) -> Iterator[str]:
|
|
"""The app under test (all three tests): one module app with the
|
|
phase-140 env (see :func:`_app_env`)."""
|
|
scratch = tmp_path_factory.mktemp("bor_pict_scratch")
|
|
proc = subprocess.Popen(
|
|
[sys.executable, "-m", "uvicorn", "app.main:app",
|
|
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
|
cwd=REPO,
|
|
env=_app_env(mock_llm, APP_PORT, scratch=scratch),
|
|
)
|
|
try:
|
|
_wait_http(f"{APP_URL}/api/health")
|
|
yield APP_URL
|
|
finally:
|
|
proc.terminate()
|
|
try:
|
|
proc.wait(timeout=10)
|
|
except subprocess.TimeoutExpired:
|
|
proc.kill()
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def app_url(app_server: str) -> str:
|
|
return app_server
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# DB + UI helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _truncate_all() -> None:
|
|
"""Fresh KB + registry (the E2E isolation pattern): the suites share
|
|
one Postgres, and a leftover row would corrupt the counts.
|
|
``sources_meta`` (the KB generation counter) resets with the KB —
|
|
the sync re-creates the row."""
|
|
with SessionLocal() as db:
|
|
db.execute(
|
|
text(
|
|
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
|
"kb_overview, git_sources, folder_summaries, sources_meta"
|
|
)
|
|
)
|
|
db.commit() # without the commit the TRUNCATE rolls back (the house pattern)
|
|
|
|
|
|
def _health_db(app_url: str) -> bool:
|
|
try:
|
|
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
|
|
except Exception: # noqa: BLE001 — unreachable is the skip case
|
|
return False
|
|
|
|
|
|
def _wait_sync_settled(base_url: str, timeout_s: float = 60.0) -> None:
|
|
"""No background sync may leak across tests (the run state lives in
|
|
the app's memory, and a still-running run would keep importing into
|
|
the NEXT test's truncated KB): wait for the app's sync status to be
|
|
non-running BEFORE the truncate (the phase-96 pattern)."""
|
|
with httpx.Client(base_url=base_url, timeout=5.0) as client:
|
|
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
|
assert r.status_code == 204, r.text
|
|
deadline = time.monotonic() + timeout_s
|
|
while time.monotonic() < deadline:
|
|
r = client.get("/api/sync/status")
|
|
assert r.status_code == 200, r.text
|
|
if r.json()["state"] != "running":
|
|
return
|
|
time.sleep(0.2)
|
|
raise AssertionError(f"a sync was still running at a test boundary: {base_url}")
|
|
|
|
|
|
@pytest.fixture(scope="module", autouse=True)
|
|
def _clean_kb(app_url: str) -> Iterator[None]:
|
|
"""Module-end KB cleanup (the phase-96 pattern, module granularity):
|
|
the three tests SHARE the module seed's single source + document,
|
|
so there is NO per-test truncate (it would wipe the seed from under
|
|
the next test — the per-test pattern belongs to the suites where
|
|
every test imports its own source); instead the shared KB is
|
|
flushed ONCE, after the last test — no rows leak into the next
|
|
suite's run. The pre-wait keeps a leaked sync (the module app is
|
|
still up here) from racing the truncate."""
|
|
yield
|
|
_wait_sync_settled(app_url)
|
|
_truncate_all()
|
|
|
|
|
|
def _ready_for_sync_line(source: str) -> re.Pattern[str]:
|
|
"""Phase 154 (task 02): the ready-for-sync line carrying the
|
|
completion stamp — "Uploaded <source> (<stamp>) — press Sync
|
|
sources to import it." (the fresh 202 settle). In E2E the stamp
|
|
is ALWAYS the same-day ``HH:MM`` form (the run settles within
|
|
seconds; the date-qualified forms are unit-pinned on a fixed
|
|
clock — E2E never depends on the wall clock)."""
|
|
return re.compile(
|
|
rf"^Uploaded {re.escape(source)} \(\d{{1,2}}:\d{{2}}\)"
|
|
r" — press Sync sources to import it\.$"
|
|
)
|
|
|
|
|
|
def _upload_tar(page: Page, app_url: str, archive: Path) -> None:
|
|
"""The real admin upload flow: form login → the git-sources page →
|
|
pick the archive → submit → the terminal result line (phase 90 A3,
|
|
phase 154 task 02: "Uploaded <source> (<stamp>) — press Sync
|
|
sources to import it." — the completion stamp)."""
|
|
login(page, app_url, next="/git-sources.html")
|
|
expect(page).to_have_url(app_url + "/git-sources.html", timeout=30_000)
|
|
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
|
|
page.set_input_files("#archive-upload-file", str(archive))
|
|
page.click("#archive-upload-btn")
|
|
result = page.locator("#archive-upload-result")
|
|
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
|
|
expect(result).to_have_text(
|
|
_ready_for_sync_line(SOURCE_NAME),
|
|
timeout=UPLOAD_TIMEOUT_MS,
|
|
)
|
|
|
|
|
|
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
|
"""The RAG page's "Sync sources" button (the scan): click →
|
|
"Syncing…" → the terminal "Synced HH:MM" label + the fresh counts
|
|
in #sync-progress (the never-stale contract)."""
|
|
page.goto(app_url + "/sources.html")
|
|
btn = page.locator("#sync-btn")
|
|
expect(btn).to_be_visible(timeout=30_000)
|
|
expect(page.locator("#sync-label")).to_have_text(
|
|
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
|
|
)
|
|
btn.click()
|
|
expect(btn).to_be_disabled()
|
|
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
|
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
|
expect(btn).to_be_enabled()
|
|
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
|
|
|
|
|
def _drill_to_source(page: Page, name: str) -> None:
|
|
"""Top level → the source (the file table then holds its direct
|
|
files — ours is at the source root)."""
|
|
page.locator("#folders-tbody .folder-link").first.wait_for(state="visible")
|
|
page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
|
|
expect(page.locator("#docs-tbody tr").first).to_be_visible(timeout=30_000)
|
|
|
|
|
|
def _chat_frames(app_url: str, message: str) -> list[dict[str, Any]]:
|
|
"""One ``POST /api/chat`` turn as RAW SSE frames (the wire lens —
|
|
the house pattern from ``test_honest_deflection.py``): the signed
|
|
admin client streams the turn and parses the ``data: <json>``
|
|
frames (PLAN §4: ``data: <json>\\n\\n``)."""
|
|
with httpx.Client(base_url=app_url, timeout=120.0) as client:
|
|
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
|
assert r.status_code == 204, r.text
|
|
frames: list[dict[str, Any]] = []
|
|
with client.stream(
|
|
"POST", "/api/chat", json={"message": message}, timeout=120.0
|
|
) as r:
|
|
assert r.status_code == 200, r.read().decode()
|
|
buf = ""
|
|
for part in r.iter_text():
|
|
buf += part
|
|
while "\n\n" in buf:
|
|
frame, buf = buf.split("\n\n", 1)
|
|
if frame.strip().startswith("data:"):
|
|
frames.append(
|
|
json.loads(frame.strip().removeprefix("data:").strip())
|
|
)
|
|
assert buf.strip() == "" # the stream ends cleanly on a frame boundary
|
|
return frames
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The module seed: upload + sync the fixture PDF through the real UI,
|
|
# once (the story E2E truncate/re-import-per-module pattern)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def seeded(
|
|
app_url: str,
|
|
browser: Browser,
|
|
tar_path: Path,
|
|
pdf_path: Path,
|
|
) -> Iterator[dict[str, str]]:
|
|
"""Upload the single-PDF tar through the admin UI, then scan it with
|
|
the RAG page's Sync button (the mock's vision model describes the
|
|
sparse page in-process — the ``PICTORIAL_PAGE_MODE`` canned reply).
|
|
The seed asserts the stored document BYTE-EXACT: the fixture's own
|
|
extraction (page blocks, the page-2 candidate carrying the canned
|
|
illustration block) — the candidates computed from
|
|
``extract_pdf_page_texts`` + ``pictorial_pages`` IN the fixture (no
|
|
hardcoded page list). This is the module app's toggle premise
|
|
witness too: without the ``BOR_PDF_PICTORIAL`` opt-in, page 2 would
|
|
lack its block and this fails loudly. Yields the doc's identity."""
|
|
if not _health_db(app_url):
|
|
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
|
|
|
_truncate_all()
|
|
page = browser.new_page(viewport={"width": 1280, "height": 800})
|
|
try:
|
|
_upload_tar(page, app_url, tar_path)
|
|
# The PDF alone (no embedded raster images — the vector figure
|
|
# yields no image doc): one file, one added document.
|
|
_sync_via_ui(page, app_url, expected_result="1 added")
|
|
with SessionLocal() as db:
|
|
doc = db.scalar(
|
|
select(Document).where(
|
|
Document.source == SOURCE_NAME, Document.path == DOC_PATH
|
|
)
|
|
)
|
|
assert doc is not None, "the uploaded PDF must become a document"
|
|
# LOCKED A2: one document per PDF, the metadata title; a TEXT
|
|
# document (no image doc, no image path).
|
|
assert doc.title == PDF_TITLE
|
|
assert doc.is_image is False and doc.image_path is None
|
|
# The candidates (computed from the fixture itself — the
|
|
# deterministic gate, LOCKED A1) + the byte-exact stored
|
|
# content: the extraction's page blocks, the described
|
|
# candidate carrying ``[Page N illustration: <canned>]``
|
|
# (LOCKED A4).
|
|
page_texts = extract_pdf_page_texts(pdf_path)
|
|
candidates = pictorial_pages(
|
|
page_texts,
|
|
min_chars=Settings.model_fields["pdf_pictorial_min_chars"].default,
|
|
)
|
|
assert candidates == [2]
|
|
parts = []
|
|
for p, t in page_texts.items():
|
|
block = t
|
|
if p in candidates:
|
|
block += f"\n\n[Page {p} illustration: {PICTORIAL_PAGE_ANSWER}]"
|
|
parts.append(f"{PAGE_SEPARATOR.format(page=p)}\n{block}")
|
|
expected = "\n\n".join(parts)
|
|
assert doc.content == expected, (
|
|
"the stored content is not the extraction with the "
|
|
"described sparse page — the pictorial pass misfired"
|
|
)
|
|
# The non-markdown doc got its lite summary (the mock's
|
|
# SUMMARY_MODE digest) — the summary chunk exists (the
|
|
# viewer's Summary panel is suppressed by the phase-148
|
|
# strip — the summary itself stays in the DB).
|
|
assert doc.summary
|
|
# Phase 148: the described sparse page makes this doc
|
|
# page-QUALIFYING — the strip's payload (page_urls) rides
|
|
# on page_count (the 2-page fixture's count).
|
|
assert doc.page_count == 2
|
|
finally:
|
|
page.close()
|
|
yield {"title": PDF_TITLE}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. Sources page + document viewer (a): the described page renders
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_sources_page_and_viewer_show_the_described_page(
|
|
page: Page, app_url: str, seeded: dict[str, str]
|
|
) -> None:
|
|
"""The KB total is 1 doc / 3 chunks (the PDF: 2 page-anchored
|
|
content chunks + the phase-30 summary chunk — no image doc; the
|
|
vector figure is invisible to the embedded-image pipeline). The
|
|
drill-down table lists the PDF as a plain text row (LOCKED A4), and
|
|
the same-page document viewer (the row's path link, phase 26) shows
|
|
the PHASE-148 PAGE STRIP under the METADATA title: the fixture is
|
|
page-qualifying (the pictorial pass described its sparse page), so
|
|
the strip REPLACES the text content AND the Summary panel — two
|
|
``.doc-page`` figures, each a loaded lazy ``<img>`` with the
|
|
``Page N of <title>`` alt + the ``Page N`` caption, NO
|
|
``.doc-raw``, NO ``.doc-summary``. The described content (page 1's
|
|
prose WITHOUT an illustration block, page 2 WITH
|
|
``[Page 2 illustration: <canned>]`` — LOCKED A4) stays in the DB
|
|
for Q&A: the seed asserted it byte-exact, the chat test proves it
|
|
reaches the model — it is simply not displayed (the owner's
|
|
contract)."""
|
|
login(page, app_url) # lands on /sources.html
|
|
expect(page.locator("#stat-docs")).to_have_text("1")
|
|
expect(page.locator("#stat-chunks")).to_have_text("3")
|
|
expect(page.locator("#sources-empty")).to_be_hidden()
|
|
|
|
_drill_to_source(page, SOURCE_NAME)
|
|
expect(page.locator("#docs-tbody tr")).to_have_count(1)
|
|
row = page.locator("#docs-tbody tr").first
|
|
# A plain text row — no thumbnail box (it is a text document, A4).
|
|
expect(row.locator(".kb-doc-thumb")).to_have_count(0)
|
|
|
|
page.locator(f'#docs-tbody a.doc-link:text-is("{DOC_PATH}")').click()
|
|
modal = page.locator("#doc-modal")
|
|
expect(modal).to_be_visible(timeout=30_000)
|
|
expect(page.locator("#doc-modal-title")).to_have_text(PDF_TITLE)
|
|
|
|
# Phase 148: the page-qualifying fixture shows its ACTUAL pages —
|
|
# a .doc-pages strip with exactly two .doc-page figures (the
|
|
# 2-page fixture), each a LOADED lazy <img> (naturalWidth > 0 —
|
|
# the route served a real PNG) with the "Page N of <title>" alt +
|
|
# the "Page N" caption. The text content AND the Summary panel are
|
|
# SUPPRESSED (the owner's contract — the descriptions stay in the
|
|
# DB for Q&A, not displayed).
|
|
strip = modal.locator(".doc-pages")
|
|
expect(strip).to_have_count(1)
|
|
figures = strip.locator(".doc-page")
|
|
expect(figures).to_have_count(2)
|
|
for i in range(2):
|
|
n = i + 1
|
|
img = figures.nth(i).locator("img")
|
|
# The figure still carries its <img> (a lost copy would have
|
|
# been swapped for the "Page N unavailable" note instead).
|
|
expect(img).to_have_count(1)
|
|
expect(img).to_have_attribute("loading", "lazy")
|
|
expect(img).to_have_attribute("alt", f"Page {n} of {PDF_TITLE}")
|
|
img.scroll_into_view_if_needed()
|
|
img.evaluate("el => el.decode()")
|
|
assert img.evaluate("el => el.naturalWidth") > 0, (
|
|
f"page {n} image did not load (the route must serve a real PNG)"
|
|
)
|
|
expect(figures.nth(i).locator("figcaption.doc-page-num")).to_have_text(
|
|
f"Page {n}"
|
|
)
|
|
# The suppression contract: no text content block, no Summary
|
|
# panel, no image block (the PDF is not an image document).
|
|
expect(modal.locator("pre.doc-raw")).to_have_count(0)
|
|
expect(modal.locator(".doc-summary")).to_have_count(0)
|
|
expect(modal.locator(".doc-image-img")).to_have_count(0)
|
|
# The canned description is in the DB (the seed asserted it)
|
|
# but nowhere in the rendered modal.
|
|
assert PICTORIAL_PAGE_ANSWER not in modal.inner_text(), (
|
|
"the described page's text is in the DB for Q&A — it must not "
|
|
"be shown to the user (the owner's contract)"
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. Chat (b): the question only the illustration answers grounds, and
|
|
# the answer echoes the described phrase
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_chat_question_grounds_and_echoes_the_illustration(
|
|
app_url: str, seeded: dict[str, str]
|
|
) -> None:
|
|
"""The raw SSE over the scripted seeded-read question (phase 119 A1
|
|
— a zero-read grounded turn chips NOTHING, so the PDF must be READ
|
|
to earn its ``done.sources`` citation): the turn GROUNDS (no
|
|
deflection — the described page's embedding retrieves: the mock's
|
|
deterministic token-overlap cosine clears the mock-calibrated 0.30
|
|
gate, FTS corroborates), the mock's scripted ``read`` runs (EXACTLY
|
|
one ``tool`` frame — the doc is far under the read cap, so no
|
|
truncation and no ``tool_result`` frame), and the answer ECHOES the
|
|
read result verbatim — the ``[Page 2 illustration: …]`` block and
|
|
its phrase reach the model and land in the rendered answer (the
|
|
described page's content is what a chat turn can ground on — the
|
|
owner's L5 rationale end to end)."""
|
|
frames = _chat_frames(app_url, QUESTION)
|
|
assert frames, "the SSE stream settled with no frames at all"
|
|
assert not any(f.get("type") == "error" for f in frames), (
|
|
f"the turn errored: {[f for f in frames if f.get('type') == 'error']}"
|
|
)
|
|
|
|
# The scripted read — exactly one tool call, on the PDF.
|
|
tools = [f for f in frames if f.get("type") == "tool"]
|
|
assert len(tools) == 1, f"expected exactly one tool frame: {tools}"
|
|
assert tools[0]["name"] == "read"
|
|
assert tools[0]["argument"] == f"{SOURCE_NAME}/{DOC_PATH}"
|
|
# The doc is far under the read cap — no truncation frame (the echo
|
|
# below therefore carries the FULL content, the block included).
|
|
assert not any(f.get("type") == "tool_result" for f in frames)
|
|
|
|
# The terminal frame: ``done``, NOT deflected.
|
|
done = [f for f in frames if f.get("type") == "done"]
|
|
assert len(done) == 1 and frames[-1] is done[0], (
|
|
f"the stream must settle with exactly one terminal done frame: "
|
|
f"{[f.get('type') for f in frames]}"
|
|
)
|
|
assert done[0]["deflected"] is False, (
|
|
"the question grounded on the described page — a deflection means "
|
|
"the illustrated content did not retrieve"
|
|
)
|
|
# The ``done`` event's ``sources`` — the citation surface (read docs
|
|
# only, phase 119 A1) — carries EXACTLY the PDF.
|
|
refs = done[0]["sources"]
|
|
assert [(r["source"], r["path"]) for r in refs] == [(SOURCE_NAME, DOC_PATH)], (
|
|
f"the done event's sources must be exactly the read PDF: {refs}"
|
|
)
|
|
assert refs[0]["title"] == PDF_TITLE
|
|
|
|
# The answer (the joined deltas) echoes the read result VERBATIM —
|
|
# the header line and the described block with the canned phrase.
|
|
answer = "".join(f["text"] for f in frames if f.get("type") == "delta")
|
|
assert "Here's what the read returned:" in answer
|
|
assert f"Document {SOURCE_NAME}/{DOC_PATH}:" in answer
|
|
assert ILLUSTRATION_BLOCK in answer, (
|
|
"the answer does not echo the described illustration block — "
|
|
"the vision pass's content never reached the model"
|
|
)
|
|
assert PICTORIAL_PAGE_ANSWER in answer # the phrase itself (the task's pin)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. Sync detail (c): the pre-phase key set is untouched (stats are
|
|
# log-only)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_sync_detail_keeps_the_pre_phase_keys(
|
|
app_url: str, seeded: dict[str, str]
|
|
) -> None:
|
|
"""The module run's terminal ``detail`` (``GET /api/sync/status`` —
|
|
the same object the UI's "1 added" line renders from) keeps the
|
|
pre-phase key set EXACTLY: the phase-140 counters
|
|
(``pdfs_pictorial`` / ``pdf_pictorial_pages``) are log-only (PLAN
|
|
§9, the phase-133 convention) and must NOT ride the wire, and the
|
|
run's own file counters read 1 file / 1 added / 0 unchanged /
|
|
0 pruned / 0 errors (the single PDF, a fresh KB)."""
|
|
with httpx.Client(base_url=app_url, timeout=30.0) as client:
|
|
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
|
assert r.status_code == 204, r.text
|
|
r = client.get("/api/sync/status")
|
|
assert r.status_code == 200, r.text
|
|
body = r.json()
|
|
assert body["state"] == "success", body
|
|
detail = body["detail"]
|
|
# The stats log-only convention — the pictorial counters never
|
|
# reach the wire.
|
|
assert "pdfs_pictorial" not in detail, (
|
|
"the log-only pdfs_pictorial counter leaked into the sync detail"
|
|
)
|
|
assert "pdf_pictorial_pages" not in detail, (
|
|
"the log-only pdf_pictorial_pages counter leaked into the sync detail"
|
|
)
|
|
# The shape itself — the pre-phase key set EXACTLY (no new field).
|
|
assert set(detail) == set(PRE_PHASE_DETAIL_KEYS), (
|
|
f"the sync detail shape changed — got {sorted(detail)}"
|
|
)
|
|
# The run's own file counters (a fresh KB: one file, one added).
|
|
assert detail["files"] == 1, detail
|
|
assert detail["added"] == 1, detail
|
|
assert detail["unchanged"] == 0, detail
|
|
assert detail["pruned"] == 0, detail
|
|
assert detail["errors"] == 0, detail
|