brain-of-reese/tests/e2e/test_garbage_pdf.py

"""Phase 133 task 04 E2E (Playwright, mock-only): the real guide stays
clean, a synthetic garbage PDF takes the page-screenshot path.

Run in isolation (DB must be up: ``podman compose up -d db``):

    uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov

The dedicated suite for ``133_pdf_screenshot_fallback`` (A16 — one
Playwright file per phase, run in isolation): it pins both legs of the
phase's guarantee at the USER SURFACE. (a) The owner's REAL guide
(``test_sources/hnb_2026_guide.tar.gz`` — the file whose PRE-phase-126
import stored raw Illustrator XMP lines, the TODO.md L9 complaint)
imports through the admin sync and PREVIEWs CLEAN text — the exact
complaint, negated on the served content AND the rendered viewer page.
(b) A synthetic PDF whose text layer is pure XML garbage
(``<illustrator:StartupProfile>Web</…>`` — the owner's exact line
shape) takes the page-screenshot fallback end to end — its stored
content is the mock's CANNED vision descriptions, one per rendered
page, and the garbage text layer is NOT what got stored. (c) A clean
PDF never burns the fallback — its stored content is byte-faithful to
the text layer (the gate's no-false-negative pin at the E2E level).

House pattern: one module app over the shared E2E Postgres (its own
port, module env — the conftest session app is never started in this
isolated run, so no port clash), each test registers its HOST TEMP DIR
as a ``kind=local`` source through the admin git-sources API (the
phase-49/phase-127 API-registration pattern — adding a plain directory
is an API-only operation), runs the real admin sync
(``POST /api/sync`` + the status poll, the surface the UI button
wraps), and an autouse truncate keeps the shared registry + KB fresh
per test (the phase-96 cleanup pattern, with the no-running-sync wait
so a leaked run cannot race the truncate).

App env: ``BOR_IMPORT_EXTENSIONS`` pinned WITH the ``pdf`` token (the
owner's live config — the phase-126/127 suites' pattern: without the
token the walk is blind to the file, and an operator's local
gitignored ``.env`` cannot leak the scope) and ``BOR_IMAGES=1`` (the
owner's live toggle — the embedded-image pipeline is live, so the
fallback PDF's extracted images join the KB exactly as a clean PDF's
do).

MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the fallback's
vision surface is the mock's deterministic ``IMAGE_DESCRIPTION_MODE``
canned reply: the BARE ``IMAGE_DESCRIPTION_ANSWER`` for the page
screenshots (the fallback's describe calls carry no context —
byte-identical to pre-phase-137) and the CONTEXT-AWARE
``IMAGE_DESCRIPTION_CONTEXT_ANSWER`` for the fallback PDF's EMBEDDED
images (phase 137 task 05 — their describe calls carry the
``Extracted from `` provenance block, so the mock's trigger fires),
which makes the fallback's stored content a PURE FUNCTION of the
fixture — the byte-exact content assertions below are exact only
against those canned texts.

Fixture provenance:
* ``hnb/`` — the REAL guide: ``test_sources/hnb_2026_guide.tar.gz``
  (UNTRACKED owner fixture — the stable reference is the tarball,
  never the extracted PDF) extracted into a module temp dir named
  ``hnb`` (the directory's basename is the ``kind=local`` source name,
  phase 38; one 6.2 MB / 48-page Illustrator PDF — the extraction is
  seconds, module-scoped, once).
* ``garbagedoc/`` — a synthetic garbage PDF built IN the test with
  PyMuPDF (the fixture pattern from phase 133 task 03's unit test,
  lifted here): 2 pages, each page's ONLY text = the owner's exact
  garbage line repeated 20× (``<illustrator:StartupProfile>Web</…>``
  — 100% of the lines look like tags, so the task-02 gate fails
  deterministically), plus ONE small embedded raster image per page
  (a colorful noise PNG via ``insert_image`` — the fallback's
  embedded-image leg runs too: the screenshots are the CONTENT, the
  extracted images are the citation figures — both, no conflict).
* ``cleandoc/`` — ONE small clean PDF (PyMuPDF, real prose text, no
  images) — the gate's no-false-negative subject.

Test → observable mapping (Playwright Mapping Rule):
1. ``test_real_guide_previews_clean_text`` — the admin sync over
   ``hnb/`` settles ``success`` (the terminal status body);
   ``GET /api/documents/content?source=hnb&path=<the guide's pdf>``
   (the PREVIEW surface — the endpoint the document viewer's content
   uses) carries the real first-page text ("2026 U.S. Benefits
   Guide") and NO line matching the XML-ish regex (the owner's
   complaint, negated on the SERVED content, not the DB); the
   document viewer page (``/document.html?source=hnb&path=…``)
   renders that content — real text, page markers, the stem title,
   never ``<illustrator:…>`` lines.
2. ``test_garbage_pdf_takes_the_screenshot_path`` — the admin sync
   over ``garbagedoc/`` settles ``success``; the stored content (the
   same content endpoint) starts with ``---PAGE-1---`` and is the
   mock's CANNED vision description for BOTH pages (2 separators, 2
   descriptions — the fallback fired end to end, mock-deterministic)
   and contains NO ``<illustrator:StartupProfile>`` line (the garbage
   text layer is NOT what got stored); the two embedded images
   (``#page=N#img=0.png``) indexed with the canned description too,
   and the sync's ``detail`` is the UNCHANGED shape (``added`` counts
   the PDF + its image docs per the phase-126 A3 convention — the
   fallback is invisible in the status; its counter is log-only,
   PLAN §9 — ``pdfs_screenshotted`` is NOT a ``detail`` key).
3. ``test_clean_pdf_never_burns_the_fallback`` — the admin sync over
   ``cleandoc/`` settles ``success``; the stored content is the
   PROSE, byte-faithful to the extraction (``---PAGE-1---\n<prose>``
   exactly) and NOT the canned vision description (the fallback did
   not fire on a clean layer).
"""
from __future__ import annotations

import json
import os
import re
import subprocess
import sys
import tarfile
import time
from collections.abc import Iterator
from datetime import datetime
from pathlib import Path
from typing import Any

import httpx
import pymupdf as fitz  # PyMuPDF — the synthetic-fixture builder (LOCKED A1)
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import select, text

from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document
from e2e.auth_helpers import login
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER, IMAGE_DESCRIPTION_CONTEXT_ANSWER

REPO = Path(__file__).resolve().parents[2]

#: The owner's untracked fixture (the stable reference — the tarball,
#: never the extracted PDF, is what this suite starts from).
TARBALL = REPO / "test_sources" / "hnb_2026_guide.tar.gz"

# Phase 79 (task 04, full inventory): the conftest session app owns its
# port in a combined run — this module app binds its own port instead
# (a same-port second uvicorn dies on bind and would drive the wrong
# server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_GARBPDF", "8149"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"

# ---------------------------------------------------------------------------
# Fixture constants (deterministic, pinned)
# ---------------------------------------------------------------------------

#: The real guide — the source dir's basename is the source name
#: (``kind=local``, phase 38) and the inner file keeps its name.
HNB_SOURCE = "hnb"
GUIDE_PDF = "HBI-2026-Guide-Year-Round_260704_131221.pdf"

#: The pinned fixture text (the real first page — phase 133 task 01's
#: pin). If the owner ever re-cuts the fixture, the failure names the
#: regression — the point.
FIRST_PAGE_TEXT = "2026 U.S. Benefits Guide"

#: The title: the stem fallback — the guide's metadata carries no
#: title (only the PDFium creator/producer), so the non-markdown rule
#: applies (phase 126 A2 — one document per PDF, metadata-or-stem).
GUIDE_TITLE = GUIDE_PDF[:-4]

#: The phase-133 XML-ish line heuristic (the task-02 gate's regex):
#: ``<tag>``, ``</tag>``, ``<tag attr=``, and the PDF-structure
#: ``<</Key…`` noise all match. The preview is clean when NO line
#: matches (the real extraction is 0/2,409 XML-ish lines).
XMLISH_RE = re.compile(r"</?[A-Za-z_][\w:.-]*[\s/>=]")

#: The owner's exact garbage line (the TODO.md L9 preview shape).
GARBAGE_SOURCE = "garbagedoc"
GARBAGE_PDF = "garbage.pdf"
GARBAGE_LINE = "<illustrator:StartupProfile>Web</illustrator:StartupProfile>"
GARBAGE_LINES_PER_PAGE = 20  # the page's ONLY text (100% XML-ish)
N_GARBAGE_PAGES = 2

#: The clean PDF — real prose (no tag-shaped lines, no images): the
#: gate's no-false-negative subject (the stored content must be
#: byte-faithful to the extraction).
CLEAN_SOURCE = "cleandoc"
CLEAN_PDF = "clean.pdf"
CLEAN_PROSE = (
    "The clean reference page covers the homelab firewall rule audit."
)

def _embedded_png(seed: int) -> bytes:
    """A small deterministic colorful noise PNG (channel means ~97 —
    the "real image" stand-in that CLEARS the phase-151 blank/
    decorative gate's near-white floor with room to spare). The old
    1×1 transparent fixture decomposed in PyMuPDF to a near-white
    1-sample base — blank by the owner-locked gate — and is now
    (correctly) skipped as a decorative underlay, so the suite's
    pipeline pins ride on a real non-blank image instead (the
    pipeline is content-agnostic; the well-formed bytes keep the
    extraction + copy + serve path honest). *seed* offsets the pixel
    formula — the two pages carry DISTINCT images (the same stream
    twice would be one placement pair: the phase-151 dedupe keeps the
    first and drops the repeat, and this suite pins one image DOC PER
    PAGE)."""
    pix = fitz.Pixmap(fitz.csRGB, fitz.IRect(0, 0, 24, 18))
    for y in range(18):
        for x in range(24):
            pix.set_pixel(
                x, y,
                ((x * 7 + seed) % 256, (y * 13 + seed) % 256, ((x + y) * 5 + seed) % 256),
            )
    return pix.tobytes("png")


#: The two embedded fixture images (24×18 colorful noise — non-blank
#: by the phase-151 gate, distinct per page — see :func:`_embedded_png`).
EMBED_PNGS = (_embedded_png(0), _embedded_png(1))


def _build_garbage_pdf(path: Path) -> None:
    """A REAL tiny garbage PDF (the fixture pattern from phase 133
    task 03's unit test, lifted here): 2 pages, each page's ONLY text
    = the owner's exact garbage line × 20 (the gate's XML signal fails
    deterministically — 100% of the non-empty lines look like tags;
    the ``---PAGE-N---`` separators are not tag-shaped), plus ONE small
    embedded raster image per page (a DISTINCT noise PNG per page —
    the fallback's embedded-image leg runs too). The text layer is real
    EXTRACTABLE text — this is the FINDING's class (the pre-126 raw-XMP
    soup), not the empty-layer class."""
    doc = fitz.open()
    for page_num in range(1, N_GARBAGE_PAGES + 1):
        page = doc.new_page()
        for i in range(GARBAGE_LINES_PER_PAGE):
            page.insert_text((72, 72 + i * 14), GARBAGE_LINE)
        page.insert_image(
            fitz.Rect(72, 500, 96, 518), stream=EMBED_PNGS[page_num - 1]
        )
    doc.save(path)
    doc.close()


def _build_clean_pdf(path: Path) -> None:
    """ONE small clean PDF: a single page of real prose (no
    tag-shaped lines, no embedded images) — the extracted text layer
    is exactly :data:`CLEAN_PROSE`, so the byte-faithful content
    assertion is exact."""
    doc = fitz.open()
    doc.new_page().insert_text((72, 72), CLEAN_PROSE)
    doc.save(path)
    doc.close()


# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------


@pytest.fixture(scope="module")
def hnb_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The REAL guide (module-scoped, once — the 6 MB extraction is
    seconds): the tarball is extracted into a temp source dir named
    ``hnb`` (the basename is the source name; the inner file keeps its
    name). ``test_sources/`` is untracked — the extraction never
    touches the repo, so ``git status`` stays clean of binaries."""
    if not TARBALL.is_file():
        pytest.skip(
            "test_sources/hnb_2026_guide.tar.gz is missing — the "
            "untracked owner fixture (2.6 MB, one 48-page Illustrator PDF)"
        )
    src = tmp_path_factory.mktemp("bor_garb_hnb") / HNB_SOURCE
    src.mkdir()
    with tarfile.open(TARBALL, "r:gz") as tar:
        tar.extractall(src, filter="data")
    assert (src / GUIDE_PDF).is_file()
    return src


@pytest.fixture(scope="module")
def garbage_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The synthetic garbage source (module-scoped): ``garbagedoc/``
    with the 2-page garbage PDF (see :func:`_build_garbage_pdf`)."""
    src = tmp_path_factory.mktemp("bor_garb_garbage") / GARBAGE_SOURCE
    src.mkdir()
    _build_garbage_pdf(src / GARBAGE_PDF)
    assert (src / GARBAGE_PDF).is_file()
    return src


@pytest.fixture(scope="module")
def clean_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The clean-PDF source (module-scoped): ``cleandoc/`` with the
    one-page prose PDF (see :func:`_build_clean_pdf`)."""
    src = tmp_path_factory.mktemp("bor_garb_clean") / CLEAN_SOURCE
    src.mkdir()
    _build_clean_pdf(src / CLEAN_PDF)
    assert (src / CLEAN_PDF).is_file()
    return src


def _app_env(
    base_url: str, tmp_path_factory: pytest.TempPathFactory
) -> dict[str, str]:
    """The per-module app env (the conftest pattern, cf.
    ``test_pdf_support.py`` / ``test_incremental_folder_summaries.py``):
    the ``base_url`` is the mock LLM directly (the deterministic
    canned replies are what the byte-exact assertions key on),
    ``BOR_GIT_SOURCES`` forced empty (the sync sources are this
    suite's own DB-registered local rows — an operator's local
    gitignored ``.env`` cannot leak the owner's corpus), the
    leak-guarded code defaults, and the phase-133 knobs:
    ``BOR_IMPORT_EXTENSIONS`` pinned WITH the ``pdf`` token (the
    owner's live config — the feature's gate; without it the walk is
    blind to the PDFs) and ``BOR_IMAGES=1`` (the owner's live toggle —
    the embedded-image pipeline is live on BOTH the clean and the
    fallback PDFs), with the image/upload homes in the suite's scratch
    dir (the app under test must not write copies into the owner's
    real ``~/bor-sources``)."""
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["BOR_ENVIRONMENT"] = "e2e"
    env["BOR_STATIC_DIR"] = str(REPO / "frontend")
    env["BOR_LLM_BASE_URL"] = (
        "https://aipi.reeseapps.com/v1" if USE_REAL_LLM else base_url
    )
    # Mock-calibrated gate (conftest pattern) — no chat turn is ever
    # sent in this suite, but the app boots with the same env shape.
    env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
    env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
    env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
    # Phase 67: instant retry waits + the code-default budget.
    env["BOR_LLM_RETRY_DELAY"] = "0"
    env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
    env.setdefault(
        "BOR_DATABASE_URL",
        "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
    )
    # Phase 16: admin auth must be set or create_app() refuses to boot.
    env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
    env["BOR_SESSION_SECRET"] = SESSION_SECRET
    # The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
    # reads it from cwd) — override it with an EMPTY value (the env var
    # beats the .env file): the registry must hold EXACTLY the local
    # directories this suite registers.
    env["BOR_GIT_SOURCES"] = ""
    # Leak guards (conftest pattern): an operator's local (gitignored)
    # .env cannot leak corpus-specific settings into the app under test.
    env["BOR_DOCS_REPO"] = ""
    env["BOR_SUGGESTIONS"] = json.dumps(
        _Settings.model_fields["suggestions"].default
    )
    env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
    env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
    # Phase 133: the owner's live import scope WITH the ``pdf`` token +
    # the images toggle under test + the suite-private homes.
    env["BOR_IMPORT_EXTENSIONS"] = "md,markdown,txt,pdf"
    env["BOR_IMAGES"] = "1"
    scratch = tmp_path_factory.mktemp("bor_garb_scratch")
    env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
    env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
    env["BOR_IMAGE_DIR"] = str(scratch / "images")
    return env


@pytest.fixture(scope="module")
def app_server(
    mock_llm: int, tmp_path_factory: pytest.TempPathFactory
) -> Iterator[str]:
    """The app under test (all three tests): one module app with the
    phase-133 env (see :func:`_app_env`). The temp source dirs are NOT
    dependencies — the app reads them only at sync time, and the
    ``hnb_dir`` fixture skips (not the whole module) when the untracked
    owner fixture is missing."""
    env = _app_env(f"http://127.0.0.1:{mock_llm}/v1", tmp_path_factory)
    proc = subprocess.Popen(
        [sys.executable, "-m", "uvicorn", "app.main:app",
         "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{APP_URL}/api/health")
        yield APP_URL
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
    return app_server


def _truncate_all() -> None:
    """Fresh registry + KB per test (the E2E isolation pattern): the
    E2E suites share one Postgres, so a leftover git_sources row or
    document would pollute the sync's counts and the content the
    assertions pin. ``sources_meta`` (the KB generation counter)
    resets with the KB — the sync re-creates the row."""
    with SessionLocal() as db:
        db.execute(
            text(
                "TRUNCATE chunks, documents, query_log, steering_notes, "
                "kb_overview, git_sources, folder_summaries, sources_meta"
            )
        )
        db.commit()  # without the commit the TRUNCATE rolls back (the house pattern)


def _wait_sync_settled(base_url: str, timeout_s: float = 180.0) -> None:
    """No background sync may leak across tests (the run state lives in
    the app's memory, and a still-running run would keep importing into
    the NEXT test's truncated KB): wait for the app's sync status to
    be non-running BEFORE the truncate. Own admin session (the
    endpoint is admin-only) — usually a no-op: the tests settle only
    after their run's terminal state."""
    with httpx.Client(base_url=base_url, timeout=5.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        deadline = time.monotonic() + timeout_s
        while time.monotonic() < deadline:
            r = client.get("/api/sync/status")
            assert r.status_code == 200, r.text
            if r.json()["state"] != "running":
                return
            time.sleep(0.2)
    raise AssertionError(f"a sync was still running at a test boundary: {base_url}")


@pytest.fixture(autouse=True)
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
    """Per-test KB isolation (the phase-96 cleanup pattern): the
    shared E2E Postgres is truncated before AND after every test —
    each test registers its own source and runs its own sync, so the
    KB rows are exactly this test's own. The pre-wait keeps a leaked
    run from the previous test (the module app is shared) from racing
    the truncate."""
    _wait_sync_settled(app_url)
    _truncate_all()
    yield
    _truncate_all()


# ---------------------------------------------------------------------------
# Helpers (the phase-127 house pattern: API-driven source registration
# + sync over plain httpx — the admin cookie rides the client jar)
# ---------------------------------------------------------------------------


def _register_source(base_url: str, path: str) -> None:
    """Register the temp directory through the authenticated API (the
    ``test_local_directory_sources.py`` / phase-94 pattern — a plain
    directory is an API/DB-only operation; the page form is gone)."""
    with httpx.Client(base_url=base_url, timeout=30.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        r = client.post("/api/git-sources", json={"kind": "local", "path": path})
        assert r.status_code == 201, r.text


def _started_at(body: dict[str, Any]) -> float:
    """The status body's ``started_at`` as a wall-clock epoch (the
    server stamps it in UTC; this process runs on the same host —
    same clock, the phase-64 recorder's freshness convention)."""
    started = body.get("started_at")
    if not started:
        return 0.0
    try:
        return datetime.fromisoformat(str(started)).timestamp()
    except ValueError:
        return 0.0


def _run_sync(base_url: str, timeout_s: float = 120.0) -> dict[str, Any]:
    """Login + ``POST /api/sync`` + poll the status endpoint until THIS
    run reaches a terminal state (the ``test_local_directory_sources``
    pattern, over plain httpx — with the stale-terminal freshness gate:
    the module app is shared, so the PREVIOUS run's terminal lives in
    the app's memory until this run supersedes it; a terminal is
    accepted only after a running tick was observed or its
    ``started_at`` post-dates this POST)."""
    with httpx.Client(base_url=base_url, timeout=30.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        t0 = time.time()
        r = client.post("/api/sync")
        assert r.status_code == 202, r.text
        saw_running = False
        deadline = time.monotonic() + timeout_s
        body: dict[str, Any] = {}
        while time.monotonic() < deadline:
            r = client.get("/api/sync/status")
            assert r.status_code == 200, r.text
            body = r.json()
            state = body["state"]
            if state == "running":
                saw_running = True
            elif state in ("success", "failed") and (
                saw_running
                or _started_at(body) >= t0 - 1.0  # same-host clock; 1 s tolerance
            ):
                return body
            time.sleep(0.1)
    raise AssertionError(f"sync did not reach a terminal state: {body}")


def _doc_content(base_url: str, source: str, path: str) -> dict[str, Any]:
    """``GET /api/documents/content`` (the PREVIEW surface — the
    endpoint the document viewer's content uses) as the signed-in
    admin. The wire shape (``DocContent``): ``content`` is the stored
    document content verbatim; ``is_image`` is always present."""
    with httpx.Client(base_url=base_url, timeout=30.0) as client:
        r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
        assert r.status_code == 204, r.text
        r = client.get(
            "/api/documents/content", params={"source": source, "path": path}
        )
        assert r.status_code == 200, f"content lookup failed: {r.text}"
        return r.json()


def _image_docs(source: str) -> list[Document]:
    """The source's stored ``is_image`` documents, in path order (the
    test process's direct DB access — the E2E's other established
    lens)."""
    with SessionLocal() as db:
        rows = db.scalars(
            select(Document)
            .where(Document.source == source, Document.is_image.is_(True))
            .order_by(Document.path)
        ).all()
        return list(rows)


def _xmlish_lines(content: str) -> list[str]:
    """The content's non-empty lines matching the phase-133 XML-ish
    regex (the owner's complaint — the pre-126 raw-XMP lines)."""
    return [
        line
        for line in content.splitlines()
        if line.strip() and XMLISH_RE.search(line)
    ]


# ---------------------------------------------------------------------------
# 1. The REAL guide (the owner's exact file) previews CLEAN text
# ---------------------------------------------------------------------------


def test_real_guide_previews_clean_text(
    app_url: str, hnb_dir: Path, page: Page, db_ready: None
) -> None:
    """The owner's exact file, through the real admin sync: the
    preview (``GET /api/documents/content`` — the endpoint the
    document viewer's content uses) carries the real first-page text
    and NO XML-ish line (the TODO.md L9 complaint, negated on the
    SERVED content), and the document viewer page renders that content
    — real text, never ``<illustrator:…>`` lines. The pre-126
    binary-text import (the raw-XMP soup) can never come back: if the
    PDF were ever routed through the text branch again, the preview
    would be ~46% XML-ish and both assertions break."""
    _register_source(app_url, str(hnb_dir))
    # 48 pages + the embedded-image pass against the mock — the
    # generous budget (the in-process regression pin's 120 s bar plus
    # the HTTP round-trips over the mock).
    body = _run_sync(app_url, timeout_s=180.0)
    assert body["state"] == "success", body
    detail = body["detail"]
    # The walk admitted exactly the one file (the ``pdf`` token — the
    # PDF branch, never the text branch), no errors, nothing pruned.
    assert detail["files"] == 1, detail
    assert detail["errors"] == 0, detail
    assert detail["pruned"] == 0, detail

    # The preview surface (the endpoint the document viewer's content
    # uses): the body is the CLEAN extracted text.
    doc = _doc_content(app_url, HNB_SOURCE, GUIDE_PDF)
    assert doc["is_image"] is False  # it is a PDF document, not an image doc
    assert doc["title"] == GUIDE_TITLE  # the stem fallback (no metadata title)
    content = doc["content"]
    assert FIRST_PAGE_TEXT in content, (
        "the real first-page text is missing from the served content — "
        "the stored content is not the extraction"
    )
    xml_lines = _xmlish_lines(content)
    assert xml_lines == [], (
        f"the owner's complaint returned on the served content — "
        f"{len(xml_lines)} XML-ish lines, e.g. {xml_lines[:3]!r}"
    )

    # The phase-126 A3 leg: the guide's embedded images joined the KB
    # with the canned descriptions — the ``added`` accounting is the
    # one file PLUS its image docs (the phase-126 convention the
    # task-01 regression pin documents).
    imgs = _image_docs(HNB_SOURCE)
    assert imgs, "the guide's embedded images did not index"
    assert detail["added"] == 1 + len(imgs), detail

    # The document viewer PAGE renders that content (the user surface):
    # the real text + the page-1 marker, the stem title, and the owner's
    # line is ABSENT from the rendered preview.
    login(page, app_url)
    page.goto(
        app_url + f"/document.html?source={HNB_SOURCE}&path={GUIDE_PDF}"
    )
    expect(page.locator("#doc-not-found")).to_be_hidden(timeout=30_000)
    expect(page.locator("#doc-title")).to_have_text(GUIDE_TITLE)
    raw = page.locator("pre.doc-raw")
    expect(raw).to_be_visible(timeout=30_000)
    expect(raw).to_contain_text(FIRST_PAGE_TEXT)
    expect(raw).to_contain_text("---PAGE-1---")
    rendered = raw.inner_text()
    assert GARBAGE_LINE not in rendered, (
        "the viewer renders the owner's <illustrator:StartupProfile> line — "
        "the pre-126 garbage preview came back"
    )
    assert _xmlish_lines(rendered) == [], (
        "the viewer renders XML-ish lines — the clean extraction is not what "
        "the preview shows"
    )
    # The non-image doc with a non-empty summary renders the Summary
    # panel (the phase-36 shape, above the content).
    expect(page.locator(".doc-summary")).to_have_count(1)


# ---------------------------------------------------------------------------
# 2. The synthetic garbage PDF takes the screenshot fallback end to end
# ---------------------------------------------------------------------------


def test_garbage_pdf_takes_the_screenshot_path(
    app_url: str, garbage_dir: Path, db_ready: None
) -> None:
    """The owner's specified path, end to end: a PDF whose text layer
    is pure XML garbage fails the task-02 gate, every page is rendered
    to a screenshot and read by the (mock) vision model, and the stored
    content is the CANNED page descriptions — 2 separators, 2
    descriptions, in page order — with the garbage text layer NOT what
    got stored. The fallback is invisible in the sync's status shape
    (``detail`` is the phase-126 shape — the counter is log-only,
    PLAN §9), and the extracted embedded images join the KB exactly as
    a clean PDF's do (the screenshots are the CONTENT; the extracted
    images are the citation figures — both, no conflict)."""
    _register_source(app_url, str(garbage_dir))
    body = _run_sync(app_url, timeout_s=120.0)
    assert body["state"] == "success", body
    detail = body["detail"]
    assert detail["files"] == 1, detail
    assert detail["errors"] == 0, detail

    # The stored content (the same content endpoint the preview uses):
    # the fallback's page-anchored canned descriptions, byte-exact.
    doc = _doc_content(app_url, GARBAGE_SOURCE, GARBAGE_PDF)
    assert doc["is_image"] is False  # a PDF whose content is vision text
    content = doc["content"]
    assert content.startswith("---PAGE-1---")
    assert content.count("---PAGE-1---") == 1
    assert content.count("---PAGE-2---") == 1
    # The mock's CANNED vision description for BOTH pages (the
    # ``IMAGE_DESCRIPTION_MODE`` reply — deterministic). The first-40
    # char pin is the task's lens; the count + the byte-exact shape
    # below make it airtight.
    assert IMAGE_DESCRIPTION_ANSWER[:40] in content
    assert content.count(IMAGE_DESCRIPTION_ANSWER) == 2
    expected = (
        f"---PAGE-1---\n{IMAGE_DESCRIPTION_ANSWER}\n\n"
        f"---PAGE-2---\n{IMAGE_DESCRIPTION_ANSWER}"
    )
    assert content == expected, (
        "the stored content is not the fallback's canned page descriptions"
    )
    # The garbage text layer is NOT what got stored — the screenshot
    # path replaced it (the owner's exact line, negated).
    assert GARBAGE_LINE not in content
    assert "<illustrator" not in content

    # The fallback's embedded-image leg: both extracted images (one per
    # page, the noise PNG — ``#page=N#img=0`` paths) indexed — and phase
    # 137 (task 02, A7) gives each the ORIGIN-LINE-ONLY document
    # context: the garbage text layer failed the gate, so the "page
    # text" is dropped (the soup never reaches the embedder), while the
    # provenance line still names the document and the page.
    imgs = _image_docs(GARBAGE_SOURCE)
    assert [i.path for i in imgs] == [
        f"{GARBAGE_PDF}#page=1#img=0.png",
        f"{GARBAGE_PDF}#page=2#img=0.png",
    ]
    for img in imgs:
        page = int(img.path.split("#page=")[1].split("#")[0])
        origin = f"Extracted from {GARBAGE_SOURCE}/{GARBAGE_PDF} (page {page}):"
        assert img.image_context == origin
        # Phase 137 task 05: the origin-line context still names the
        # document (the mock's ``Extracted from `` trigger fires on the
        # header alone — the page text was dropped by the gate), so the
        # canned reply is the CONTEXT-AWARE one, not the bare pre-137
        # answer.
        assert img.content == f"{origin}\n\n{IMAGE_DESCRIPTION_CONTEXT_ANSWER}"
        assert img.summary == img.content  # the image convention

    # The sync's ``detail`` is the UNCHANGED shape: the garbage PDF is
    # a NORMAL import (the one file + its two image docs per the
    # phase-126 A3 convention) and the fallback's counter is
    # log-only — ``pdfs_screenshotted`` is NOT a ``detail`` key.
    assert detail["added"] == 1 + len(imgs), detail
    assert detail["pruned"] == 0, detail
    assert "pdfs_screenshotted" not in detail


# ---------------------------------------------------------------------------
# 3. A clean PDF never burns the fallback (the gate's no-false-
#    negative pin at the E2E level)
# ---------------------------------------------------------------------------


def test_clean_pdf_never_burns_the_fallback(
    app_url: str, clean_dir: Path, db_ready: None
) -> None:
    """The gate is the ONLY gate: a clean text layer never reaches the
    fallback — the stored content is the PROSE, byte-faithful to the
    extraction (``---PAGE-1---\n<prose>`` exactly — the phase-126
    happy path, unchanged) and NOT the canned vision description (no
    screenshot, no vision burn on the happy path)."""
    _register_source(app_url, str(clean_dir))
    body = _run_sync(app_url, timeout_s=60.0)
    assert body["state"] == "success", body
    detail = body["detail"]
    # One file, one added document — no embedded images in the clean
    # fixture, so the ``added`` count is the PDF alone.
    assert detail["files"] == 1, detail
    assert detail["added"] == 1, detail
    assert detail["errors"] == 0, detail

    doc = _doc_content(app_url, CLEAN_SOURCE, CLEAN_PDF)
    assert doc["is_image"] is False
    # Byte-faithful to the text layer (the extraction's exact shape —
    # the page marker + the stripped page text).
    assert doc["content"] == f"---PAGE-1---\n{CLEAN_PROSE}"
    # The fallback did NOT fire: no canned vision text anywhere in the
    # stored content.
    assert IMAGE_DESCRIPTION_ANSWER[:40] not in doc["content"]
    assert _image_docs(CLEAN_SOURCE) == []