phase: 133_pdf_screenshot_fallback

All verification is complete — everything is green. No repo-side changes were needed in this pass (the phase's work in the working tree is correct and complete).

## Phase 133 final verification report

- **Verified (already implemented in working tree):** `is_usable_pdf_text` gate (0.30 XML-ish / 0.05 non-printable / empty), `render_pdf_pages`, `_fallback_page_screenshots` + `pdfs_screenshotted` counter in `app/rag/{pdf,importer}.py`; regression pin, unit/integration/E2E suites.
- **Completion criteria:**
  1. Real guide clean through real pipeline — ✅ `test_pdf_hnb_guide_regression` + E2E preview/viewer checks (0 XML-ish lines, first-page text present).
  2. Garbage PDF → vision fallback end to end — ✅ unit + E2E (stored content = canned per-page descriptions, garbage not stored, sync `detail` shape unchanged).
  3. Clean PDF never burns a fallback vision call — ✅ unit + E2E (byte-faithful content).
  4. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3129 passed, app/ 99%** (>90%); `uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov`: **3 passed** in isolation.
  5. No behavior change to completed phases — ✅ entire suite green (all prior-phase tests included); `ruff check .` + `pyright` clean.
  6. Production stale row — post-deploy re-sync verification (live infra, no repo work; no live changes made — deploy of `main` is the owner's step).
- **Documentation:** README.md reviewed in full — every env default cross-checked against `app/config.py`, page routes against `app/main.py`, scripts/Containerfile/compose flags, `.env.example`; the phase-133 PDF bullet added in the working tree is accurate. AGENTS.md verified consistent (all referenced files/commands exist). **No changes needed.**
- **Next pending phase:** `134_sync_cancel`.

tests/unit/test_importer.py

@@ -14,6 +14,8 @@ for the fail-soft path.
 from __future__ import annotations
 
 import asyncio
+import base64
+import hashlib
 import logging
 import os
 from collections.abc import Callable
@@ -985,14 +987,16 @@ def test_import_summary_log_line_includes_summary_counters(
     """PLAN §9 summary line: the phase-30 counters sit between
     ``embed_batches`` and ``formats``; the phase-118 backfill counter
     sits between ``summary_errors`` and ``dates_updated``; the
-    phase-106 date-refresh counter and the phase-122 ``images_failed``
-    counter sit after ``dates_updated``, before ``formats``."""
+    phase-106 date-refresh counter, the phase-122 ``images_failed``
+    counter, and the phase-133 ``pdfs_screenshotted`` counter sit after
+    ``dates_updated``, before ``formats``."""
     s = ImportSummary()
     s.files, s.added, s.chunks, s.embed_batches = 3, 3, 5, 4
     s.summaries, s.summary_errors = 2, 1
     s.summary_backfilled = 1
     s.dates_updated = 0
     s.images_failed = 0
+    s.pdfs_screenshotted = 1
     s.formats = {"md": 1, "yaml": 2}
     with caplog.at_level(logging.INFO, logger="app.importer"):
         s.log()
@@ -1000,7 +1004,7 @@ def test_import_summary_log_line_includes_summary_counters(
     assert line == (
         "import: summary files=3 added=3 updated=0 unchanged=0 pruned=0 errors=0 "
         "chunks=5 embed_batches=4 summaries=2 summary_errors=1 summary_backfilled=1 "
-        "dates_updated=0 images_failed=0 formats=yaml:2,md:1"
+        "dates_updated=0 images_failed=0 pdfs_screenshotted=1 formats=yaml:2,md:1"
     )
 
 
@@ -1644,3 +1648,418 @@ def test_on_chunk_progress_pdf_branch_reports_content_chunks(
     # The content embed (the 2 page chunks, one batch) + the excluded
     # summary-chunk embed — the hook fired for the first only.
     assert [len(c) for c in llm.calls] == [2, 1]
+
+
+# ---------- phase 133, task 03: the page-screenshot fallback ----------
+
+
+class _PageVisionLLM(FakeEmbedder):
+    """The phase-133 mock VISION client — a STABLE vision model: the
+    MULTIMODAL describe call (``user`` content = a part list — the
+    shape ``describe_image`` sends) gets a canned reply DETERMINED BY
+    THE IMAGE'S BYTES (a short sha256 of the data URL's base64 — the
+    same rendered page always gets the same reply, ACROSS RUNS; the
+    pages' bytes differ, so the replies differ and the page order
+    stays checkable, and the digest-stability test's "stable vision
+    model" premise holds). Text (``lite``) calls keep the
+    ``FakeEmbedder`` behaviour. Every vision call's model is recorded
+    (``vision_models``)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.vision_calls = 0
+        self.vision_models: list[str | None] = []
+
+    @staticmethod
+    def reply_for(image_bytes: bytes) -> str:
+        """The canned reply a vision call for *image_bytes* gets —
+        deterministic in the bytes (tests build the expected content
+        with the same helper)."""
+        return f"Page {hashlib.sha256(image_bytes).hexdigest()[:8]} described."
+
+    async def chat(self, messages, model=None):  # pyright: ignore[reportIncompatibleMethodOverride]
+        self.chat_calls.append(list(messages))
+        user = next((m["content"] for m in messages if m.get("role") == "user"), "")
+        if isinstance(user, list):
+            self.vision_calls += 1
+            self.vision_models.append(model)
+            for part in user:
+                if part.get("type") == "image_url":
+                    b64 = part["image_url"]["url"].rsplit(",", 1)[1]
+                    return self.reply_for(base64.b64decode(b64))
+            raise LLMError("no image part in the multimodal message (test bug)")
+        first = user.split()
+        return "Summary of " + (first[0] if first else "<empty>")
+
+
+class _FirstPageFailsVisionLLM(FakeEmbedder):
+    """A vision client that REJECTS the first multimodal call (a dead
+    page's vision request) and then describes — drives the per-page
+    fail-soft (the dead page becomes the placeholder, the next page
+    still gets its description)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.vision_calls = 0
+
+    async def chat(self, messages, model=None):  # pyright: ignore[reportIncompatibleMethodOverride]
+        self.chat_calls.append(list(messages))
+        user = next((m["content"] for m in messages if m.get("role") == "user"), "")
+        if isinstance(user, list):
+            self.vision_calls += 1
+            if self.vision_calls == 1:
+                raise LLMError("simulated dead vision call (test sentinel)")
+            return "Page two described fine."
+        first = user.split()
+        return "Summary of " + (first[0] if first else "<empty>")
+
+
+class _AlwaysFailsVisionLLM(FakeEmbedder):
+    """A vision client that REJECTS every multimodal call (the dead
+    vision endpoint — the per-page fail-soft's total-failure case:
+    every page becomes the placeholder and the doc lands empty)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.vision_calls = 0
+
+    async def chat(self, messages, model=None):  # pyright: ignore[reportIncompatibleMethodOverride]
+        self.chat_calls.append(list(messages))
+        user = next((m["content"] for m in messages if m.get("role") == "user"), "")
+        if isinstance(user, list):
+            self.vision_calls += 1
+            raise LLMError("simulated dead vision endpoint (test sentinel)")
+        first = user.split()
+        return "Summary of " + (first[0] if first else "<empty>")
+
+
+def _pdf_settings(tmp_path: Path, images: bool = False) -> Settings:
+    """Settings admitting the ``pdf`` import token (the code DEFAULT set
+    has no ``pdf`` — the phase-133 task-01 trap: a default fake would
+    silently skip the PDF) with the image knobs at their test values
+    (``images`` off unless pinned otherwise — the fallback's own vision
+    calls need no image copy, only the phase-122 embedded-image pass
+    does)."""
+    return Settings(
+        _env_file=None,  # pyright: ignore[reportCallIssue] (kwarg exists at runtime)
+        import_extensions="md,txt,pdf",
+        images=images,
+        image_dir=str(tmp_path / "images"),
+    )
+
+
+def _build_garbage_pdf(path: Path, pages: int = 2) -> None:
+    """A REAL tiny garbage PDF: every page carries ONLY XML-ish lines
+    (``<foo:pN>i</foo:pN>`` — page-specific tags, so the rendered page
+    PNGs differ and the stable fake's replies do too) — the gate's XML
+    signal fails deterministically (100% of the text lines look like
+    tags; the ``---PAGE-N---`` separators are not tag-shaped). The text
+    layer is real EXTRACTABLE text — this is the FINDING's class (the
+    pre-126 raw-XMP soup), not the empty-layer class."""
+    doc = fitz.open()
+    for p in range(1, pages + 1):
+        page = doc.new_page()
+        for i in range(10):
+            page.insert_text((72, 72 + i * 14), f"<foo:p{p}>{i}</foo:p{p}>")
+    doc.save(path)
+    doc.close()
+
+
+def _pdf_doc(db, source: str, rel: str) -> Document:
+    doc = db.scalar(
+        select(Document).where(Document.source == source, Document.path == rel)
+    )
+    assert doc is not None, f"no documents row for ({source!r}, {rel!r})"
+    return doc
+
+
+def test_garbage_pdf_takes_the_page_screenshot_fallback(
+    db, tmp_path: Path, caplog: pytest.LogCaptureFixture
+) -> None:
+    """The owner's specified path end to end: a PDF whose text layer is
+    XML garbage fails the gate, every page is rendered to a PNG and
+    read by the vision model, and the doc lands with the
+    ``---PAGE-N---``-anchored page descriptions (PAGE ORDER pinned by
+    the differing canned replies) — the page-aware chunker works
+    unchanged on the separators, the digest is over the FINAL content,
+    and the counter + the loud warning (the FINDING's signature) fire.
+    ``is_image`` stays False (it is a PDF document whose content
+    happens to be vision text)."""
+    root = tmp_path / "garbage"
+    root.mkdir()
+    _build_garbage_pdf(root / "garbage.pdf")
+    llm = _PageVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    # The stable fake's expected replies (rendered page bytes → reply;
+    # page order pinned: page i's bytes carry page i's description).
+    pngs = importer.render_pdf_pages(root / "garbage.pdf")
+    d1, d2 = _PageVisionLLM.reply_for(pngs[0]), _PageVisionLLM.reply_for(pngs[1])
+    try:
+        with caplog.at_level(logging.INFO, logger="app.importer"):
+            summary = asyncio.run(import_sources([root], llm, session=db))
+        assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
+        assert summary.pdfs_screenshotted == 1
+        assert llm.vision_calls == 2  # one vision call per page
+
+        doc = _pdf_doc(db, "garbage", "garbage.pdf")
+        assert doc.is_image is False and doc.image_path is None
+        assert doc.content == f"---PAGE-1---\n{d1}\n\n---PAGE-2---\n{d2}"
+        assert d1 != d2, "the pages' bytes differ — so do their replies"
+        # The digest runs over the FINAL (fallback) content — the
+        # phase-126 digest rule, unchanged.
+        assert doc.content_hash == hashlib.sha256(
+            doc.content.encode("utf-8")
+        ).hexdigest()
+        chunks = db.scalars(
+            select(Chunk).where(Chunk.document_id == doc.id).order_by(Chunk.position)
+        ).all()
+        content_chunks = [c for c in chunks if not c.is_summary]
+        assert [c.content for c in content_chunks] == [
+            f"Page 1:\n{d1}",
+            f"Page 2:\n{d2}",
+        ]
+        for c in chunks:
+            assert c.embedding is not None and len(c.embedding) == 768
+        summary_chunks = [c for c in chunks if c.is_summary]
+        assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
+        assert doc.summary is not None
+
+        # The loud warning (PLAN §9, the FINDING's signature) — the
+        # required line, exact.
+        warnings = [
+            r for r in caplog.records if "text layer unusable" in r.getMessage()
+        ]
+        assert len(warnings) == 1 and warnings[0].levelno == logging.WARNING
+        assert warnings[0].getMessage() == (
+            "pdf: text layer unusable for garbage/garbage.pdf — "
+            "falling back to 2 page screenshots"
+        )
+        # The vision calls went to the CHAT model (the phase-122 seam).
+        assert llm.vision_models == [llm.settings.llm_chat_model] * 2
+    finally:
+        _cleanup_source(db, "garbage")
+
+
+def test_clean_pdf_never_burns_a_fallback_vision_call(
+    db, tmp_path: Path, caplog: pytest.LogCaptureFixture
+) -> None:
+    """The gate is the ONLY gate: a clean text layer never reaches the
+    fallback — ``pdfs_screenshotted`` stays 0 and the multimodal chat
+    is NEVER called (the images toggle is off, so the only vision seam
+    in play is the fallback itself; the phase-126 happy path is
+    byte-identical — the extracted text is the content, no fallback
+    warning, no ``Page N —`` placeholders)."""
+    root = tmp_path / "clean"
+    root.mkdir()
+    _build_one_page_pdf(root / "clean.pdf", "Clean prose about the homelab.")
+    llm = _PageVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    try:
+        with caplog.at_level(logging.INFO, logger="app.importer"):
+            summary = asyncio.run(import_sources([root], llm, session=db))
+        assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
+        assert summary.pdfs_screenshotted == 0
+        assert llm.vision_calls == 0
+        doc = _pdf_doc(db, "clean", "clean.pdf")
+        assert doc.content == "---PAGE-1---\nClean prose about the homelab."
+        assert not [
+            r for r in caplog.records if "text layer unusable" in r.getMessage()
+        ]
+    finally:
+        _cleanup_source(db, "clean")
+
+
+def test_fallback_is_fail_soft_per_page(
+    db, tmp_path: Path, caplog: pytest.LogCaptureFixture
+) -> None:
+    """A dead vision call on ONE page → the deterministic placeholder
+    line (``page N — description unavailable``) for that page, the
+    other page still gets its description — the doc still lands (a
+    dead vision endpoint must not sink the doc, the phase-122
+    fail-soft spirit). The vision seam logged its failure (the
+    phase-122 warning); the fallback counted once."""
+    root = tmp_path / "partfail"
+    root.mkdir()
+    _build_garbage_pdf(root / "partfail.pdf", pages=2)
+    llm = _FirstPageFailsVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    try:
+        with caplog.at_level(logging.INFO, logger="app.summarizer"):
+            summary = asyncio.run(import_sources([root], llm, session=db))
+        assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
+        assert summary.pdfs_screenshotted == 1
+        assert llm.vision_calls == 2  # both pages were attempted
+        doc = _pdf_doc(db, "partfail", "partfail.pdf")
+        assert doc.content == (
+            "---PAGE-1---\npage 1 — description unavailable\n\n"
+            "---PAGE-2---\nPage two described fine."
+        )
+        # The phase-122 seam's failure warning (the dead page's signal).
+        assert [
+            r for r in caplog.records if "image description failed" in r.getMessage()
+        ]
+    finally:
+        _cleanup_source(db, "partfail")
+
+
+def test_fallback_all_pages_failed_lands_empty_content(
+    db, tmp_path: Path, caplog: pytest.LogCaptureFixture
+) -> None:
+    """A garbage PDF whose vision calls ALL fail (the dead vision
+    endpoint) → the existing EMPTY-CONTENT path: the doc lands with
+    ``""`` (the empty byte string's digest, no content chunks), the
+    fallback was still taken (``pdfs_screenshotted == 1`` — the gate
+    fired), and the phase-126 "no extractable text" warning fires —
+    the doc really is empty. A dead endpoint must not sink the sync."""
+    root = tmp_path / "allfail"
+    root.mkdir()
+    _build_garbage_pdf(root / "allfail.pdf", pages=1)
+    llm = _AlwaysFailsVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    try:
+        with caplog.at_level(logging.INFO, logger="app.importer"):
+            summary = asyncio.run(import_sources([root], llm, session=db))
+        assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
+        assert summary.pdfs_screenshotted == 1
+        assert llm.vision_calls == 1
+        doc = _pdf_doc(db, "allfail", "allfail.pdf")
+        assert doc.content == ""
+        assert doc.content_hash == hashlib.sha256(b"").hexdigest()
+        chunks = db.scalars(
+            select(Chunk).where(Chunk.document_id == doc.id).order_by(Chunk.position)
+        ).all()
+        assert [c for c in chunks if not c.is_summary] == []
+        warnings = [
+            r for r in caplog.records if "no extractable text" in r.getMessage()
+        ]
+        assert len(warnings) == 1 and warnings[0].levelno == logging.WARNING
+    finally:
+        _cleanup_source(db, "allfail")
+
+
+def test_fallback_unrenderable_page_gets_the_placeholder(
+    db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """A page that cannot be RENDERED (``b""`` at its index — the
+    :func:`render_pdf_pages` fail-soft slot) gets the deterministic
+    placeholder while the good pages keep their descriptions — the
+    page index never shifts (``---PAGE-N---`` stays truthful), and the
+    unrenderable page burns no vision call."""
+    root = tmp_path / "halfblind"
+    root.mkdir()
+    _build_garbage_pdf(root / "halfblind.pdf", pages=3)
+    monkeypatch.setattr(
+        importer,
+        "render_pdf_pages",
+        lambda _path, dpi=150: [b"\x89PNG-page-1", b"", b"\x89PNG-page-3"],
+    )
+    llm = _PageVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    d1 = _PageVisionLLM.reply_for(b"\x89PNG-page-1")
+    d3 = _PageVisionLLM.reply_for(b"\x89PNG-page-3")
+    try:
+        summary = asyncio.run(import_sources([root], llm, session=db))
+        assert summary.pdfs_screenshotted == 1
+        assert llm.vision_calls == 2  # the blank page burned no vision call
+        doc = _pdf_doc(db, "halfblind", "halfblind.pdf")
+        assert doc.content == (
+            f"---PAGE-1---\n{d1}\n\n"
+            "---PAGE-2---\npage 2 — description unavailable\n\n"
+            f"---PAGE-3---\n{d3}"
+        )
+    finally:
+        _cleanup_source(db, "halfblind")
+
+
+def test_fallback_rerun_lands_unchanged_no_reembed(
+    db, tmp_path: Path, caplog: pytest.LogCaptureFixture
+) -> None:
+    """Digest stability (the task's re-burn guard): the digest runs
+    over the constructed fallback content — a re-sync of an unchanged
+    garbage PDF (same pages + the deterministic fake's same replies)
+    re-imports as UNCHANGED: the fallback re-runs (the gate + the
+    render are cheap; the vision calls are re-issued — the documented
+    no-cache cost), but the digest MATCHES, so the row is skipped —
+    no re-chunk, no re-embed, no re-summary, no KB change (the row is
+    not ``updated`` and records no changed path). The doc's content
+    and digest are byte-stable across the re-sync."""
+    root = tmp_path / "stable"
+    root.mkdir()
+    _build_garbage_pdf(root / "stable.pdf")
+    llm = _PageVisionLLM()
+    llm.settings = _pdf_settings(tmp_path)
+    try:
+        s1 = asyncio.run(import_sources([root], llm, session=db))
+        assert (s1.added, s1.pdfs_screenshotted) == (1, 1)
+        assert llm.vision_calls == 2
+        embeds1 = len(llm.calls)
+        doc1 = _pdf_doc(db, "stable", "stable.pdf")
+        content1, digest1 = doc1.content, doc1.content_hash
+
+        s2 = asyncio.run(import_sources([root], llm, session=db))
+        # The fallback re-ran (gate re-fired, vision re-issued) — but
+        # the row itself re-imported as unchanged (no re-embed, no
+        # re-chunk, no KB change: the stable replies give the same
+        # digest, so the upsert is skipped entirely).
+        assert (s2.added, s2.updated, s2.unchanged) == (0, 0, 1)
+        assert s2.pdfs_screenshotted == 1  # the gate re-fired
+        assert llm.vision_calls == 4  # the vision calls were re-issued
+        assert len(llm.calls) == embeds1, "unchanged: no re-embed"
+        assert s2.changed_paths == [], "unchanged: no KB change recorded"
+
+        doc2 = _pdf_doc(db, "stable", "stable.pdf")
+        assert doc2.content == content1  # byte-stable content
+        assert doc2.content_hash == digest1  # byte-stable digest
+    finally:
+        _cleanup_source(db, "stable")
+
+
+def test_fallback_embedded_images_still_join_the_pipeline(
+    db, tmp_path: Path
+) -> None:
+    """Both, no conflict (the phase design): a fallback PDF with the
+    images toggle ON ALSO runs the phase-126 embedded-image pass — the
+    screenshots are the CONTENT (the fallback's vision calls: one per
+    page), the extracted images are the citation figures (the
+    phase-122 pass's vision calls: one per image). The content keeps
+    the page descriptions; the image doc is separate."""
+    root = tmp_path / "both"
+    root.mkdir()
+    # Garbage text (the gate fires) + an embedded image on page 2
+    # (the phase-126 A3 pass indexes it).
+    doc = fitz.open()
+    page1 = doc.new_page()
+    for i in range(10):
+        page1.insert_text((72, 72 + i * 14), f"<foo:bar>{i}</foo:bar>")
+    page2 = doc.new_page()
+    for i in range(10):
+        page2.insert_text((72, 72 + i * 14), f"<foo:baz>{i}</foo:baz>")
+    pix = fitz.Pixmap(fitz.csRGB, fitz.IRect(0, 0, 40, 30), 0)
+    pix.clear_with(200)
+    page2.insert_image(fitz.Rect(72, 220, 112, 250), stream=pix.tobytes("png"))
+    doc.save(root / "both.pdf")
+    doc.close()
+    llm = _PageVisionLLM()
+    llm.settings = _pdf_settings(tmp_path, images=True)
+    try:
+        summary = asyncio.run(import_sources([root], llm, session=db))
+        # The PDF (screenshot content) + its one embedded image doc.
+        assert (summary.files, summary.added, summary.errors) == (1, 2, 0)
+        assert summary.pdfs_screenshotted == 1
+        assert summary.images_failed == 0
+        # 2 fallback page descriptions + 1 embedded-image description.
+        assert llm.vision_calls == 3
+
+        pdf_doc = _pdf_doc(db, "both", "both.pdf")
+        pngs = importer.render_pdf_pages(root / "both.pdf")
+        d1 = _PageVisionLLM.reply_for(pngs[0])
+        d2 = _PageVisionLLM.reply_for(pngs[1])
+        assert d1 != d2
+        assert pdf_doc.content == f"---PAGE-1---\n{d1}\n\n---PAGE-2---\n{d2}"
+        # The embedded image (the clear_with(200) 40x30 PNG) gets its
+        # OWN stable reply — the third vision call, the citation figure.
+        image_doc = _pdf_doc(db, "both", "both.pdf#page=2#img=0.png")
+        assert image_doc.is_image is True
+        assert image_doc.content != d1 and image_doc.content != d2
+    finally:
+        _cleanup_source(db, "both")