phase: 133_pdf_screenshot_fallback

All verification is complete — everything is green. No repo-side changes were needed in this pass (the phase's work in the working tree is correct and complete).

## Phase 133 final verification report

- **Verified (already implemented in working tree):** `is_usable_pdf_text` gate (0.30 XML-ish / 0.05 non-printable / empty), `render_pdf_pages`, `_fallback_page_screenshots` + `pdfs_screenshotted` counter in `app/rag/{pdf,importer}.py`; regression pin, unit/integration/E2E suites.
- **Completion criteria:**
  1. Real guide clean through real pipeline — ✅ `test_pdf_hnb_guide_regression` + E2E preview/viewer checks (0 XML-ish lines, first-page text present).
  2. Garbage PDF → vision fallback end to end — ✅ unit + E2E (stored content = canned per-page descriptions, garbage not stored, sync `detail` shape unchanged).
  3. Clean PDF never burns a fallback vision call — ✅ unit + E2E (byte-faithful content).
  4. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3129 passed, app/ 99%** (>90%); `uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov`: **3 passed** in isolation.
  5. No behavior change to completed phases — ✅ entire suite green (all prior-phase tests included); `ruff check .` + `pyright` clean.
  6. Production stale row — post-deploy re-sync verification (live infra, no repo work; no live changes made — deploy of `main` is the owner's step).
- **Documentation:** README.md reviewed in full — every env default cross-checked against `app/config.py`, page routes against `app/main.py`, scripts/Containerfile/compose flags, `.env.example`; the phase-133 PDF bullet added in the working tree is accurate. AGENTS.md verified consistent (all referenced files/commands exist). **No changes needed.**
- **Next pending phase:** `134_sync_cancel`.

app/rag/importer.py

@@ -109,6 +109,26 @@ any existing image docs; an unsupported image mime (outside
 ``BOR_IMAGE_EXTENSIONS``) is skipped with a warning. A PDF's viewer
 presentation is its extracted text (A4 — nothing binary is served).
 
+Text-layer garbage gate + page-screenshot fallback (phase 133,
+task 03): AFTER extraction and BEFORE the digest/chunk/summary
+pipeline, the extracted text passes the deterministic garbage gate
+(:func:`app.rag.pdf.is_usable_pdf_text` — empty text layer or
+XML/binary soup, the pre-126 raw-bytes class). On failure the content
+becomes the page-screenshot fallback (:func:
+`_fallback_page_screenshots` — every page rendered to a PNG via
+:func:`app.rag.pdf.render_pdf_pages` and read by the phase-122 vision
+call, the SAME ``---PAGE-N---`` separators, the loud
+``pdf: text layer unusable for …`` warning, the
+``pdfs_screenshotted`` counter) — the owner's specified path for the
+class of PDF whose text layer is garbage. A CLEAN text layer NEVER
+burns a fallback vision call (the gate is the only gate — the
+phase-126 happy path is byte-identical), and the digest runs over the
+FINAL content exactly once — for a stable file + stable vision model
+the digest is byte-stable, so an unchanged re-sync re-imports the row
+as unchanged (no re-chunk/re-embed/re-summary, no KB change; the
+vision calls are re-issued each sync — the no-cache cost, documented
+in :func:`_fallback_page_screenshots`).
+
 Image-anchor context (phase 125, task 03, LOCKED A1/A2/A3): BEFORE the
 description, the same source's importable markdown files are scanned for
 ``![alt](path)`` references that name the image (the task-02
@@ -173,7 +193,14 @@ from app.rag.chunker import chunk_document, extract_title
 from app.rag.doc_dates import file_mtime_datetime, normalize_doc_date
 from app.rag.llm import EmbeddingError, LLMError
 from app.rag.md_images import ImageRef, extract_image_refs
-from app.rag.pdf import PdfImage, extract_pdf_images, extract_pdf_text
+from app.rag.pdf import (
+    PAGE_SEPARATOR,
+    PdfImage,
+    extract_pdf_images,
+    extract_pdf_text,
+    is_usable_pdf_text,
+    render_pdf_pages,
+)
 from app.rag.summarizer import (
     IMAGE_FALLBACK_MIME,
     IMAGE_MIMES,
@@ -254,6 +281,14 @@ class ImportSummary:
     #: Fail-soft: the sync continues, this counter + the warning line
     #: are the signal.
     images_failed: int = 0
+    #: PDFs (phase 133, task 03) whose text layer failed the garbage
+    #: gate (:func:`app.rag.pdf.is_usable_pdf_text` — empty layer or
+    #: XML/binary soup) and took the page-screenshot vision fallback
+    #: (:func:`_fallback_page_screenshots` — every page rendered to a
+    #: PNG and read by the phase-122 vision call; the content is the
+    #: ``---PAGE-N---``-anchored descriptions, or ``""`` when the
+    #: fallback produced nothing — the phase-126 empty-content path).
+    pdfs_screenshotted: int = 0
     #: Documents this run ADDED, UPDATED, or PRUNED, as (source, path)
     #: tuples in encounter order (added/updated in walk order, pruned in
     #: _prune order). Drives the sync's incremental folder-summary
@@ -274,7 +309,8 @@ class ImportSummary:
         logger.info(
             "import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
             "errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
-            "summary_backfilled=%d dates_updated=%d images_failed=%d formats=%s",
+            "summary_backfilled=%d dates_updated=%d images_failed=%d "
+            "pdfs_screenshotted=%d formats=%s",
             self.files,
             self.added,
             self.updated,
@@ -288,6 +324,7 @@ class ImportSummary:
             self.summary_backfilled,
             self.dates_updated,
             self.images_failed,
+            self.pdfs_screenshotted,
             self.format_counts(),
         )
 
@@ -1034,12 +1071,24 @@ async def _index_pdf_file(
     * ``content`` is the parser's text — every text-bearing page
       prefixed with the ``---PAGE-N---`` marker (A2 — one document per
       PDF, the markers are the page anchors the page-aware chunker
-      (task 03) splits on); an unopenable/empty PDF yields ``""``
-      (still a document — it may carry embedded images — with no
-      content chunks; a warning is the signal).
-    * the sha256 digest is over the extracted text (the digest rule is
+      (task 03) splits on). Phase 133 (task 03): if that text fails
+      the garbage gate (:func:`is_usable_pdf_text` — empty layer or
+      XML/binary soup), the content becomes the page-screenshot
+      fallback (:func:`_fallback_page_screenshots` — the rendered
+      pages read by the vision model, the same separators) BEFORE the
+      digest/chunk/summary pipeline runs; the gate never fires for a
+      clean layer (no fallback vision calls on the happy path).
+      An unopenable PDF — or one whose fallback also produced nothing
+      — yields ``""`` (still a document — it may carry embedded images
+      — with no content chunks; a warning is the signal).
+    * the sha256 digest is over the FINAL content (the digest rule is
       content identity — the same text is the same document version;
-      the empty case digests the empty byte string).
+      the fallback's constructed page descriptions digest exactly the
+      same way, so an unchanged fallback doc re-syncs as unchanged —
+      no re-chunk/re-embed/re-summary, no KB change — while the vision
+      model's replies stay stable; a non-deterministic vision model
+      re-describes the row each sync, a cost, not a bug; the empty
+      case digests the empty byte string).
     * the title is the metadata title or the file stem
       (:func:`_extract_pdf_title`); the date follows the text path
       exactly (the phase-106 D2 mtime fallback — PDFs carry no
@@ -1071,12 +1120,22 @@ async def _index_pdf_file(
     # Extract the text (fail-soft: an unopenable PDF yields "" — the
     # document may still carry embedded images worth indexing).
     content = extract_pdf_text(full_path)
+    # Phase 133 (task 03): the garbage gate — BEFORE the digest/chunk/
+    # summary pipeline, so it runs on the FINAL content exactly once.
+    # A clean layer never reaches the fallback (the gate is the only
+    # gate — no fallback vision calls on the happy path).
+    if not is_usable_pdf_text(content):
+        summary.pdfs_screenshotted += 1
+        content = await _fallback_page_screenshots(
+            session, source=source, rel=rel, full_path=full_path, llm=llm
+        )
     if not content:
         logger.warning(
             "import: PDF has no extractable text source=%s path=%s", source, rel
         )
-    # Content identity: the digest is over the extracted text (the
-    # empty case is the empty byte string's digest).
+    # Content identity: the digest is over the FINAL content — the
+    # extracted text, or the fallback's constructed page descriptions
+    # (the empty case is the empty byte string's digest).
     digest = (
         hashlib.sha256(content.encode("utf-8")).hexdigest()
         if content
@@ -1174,13 +1233,95 @@ async def _index_pdf_file(
     # Phase 126 (A3): the embedded images join the phase-122 pipeline
     # (gated by the ``images`` toggle — skipped, debug-logged, while it
     # is off). Runs after the document commit, so a per-image failure
-    # never loses the PDF's text index.
+    # never loses the PDF's text index. Runs on BOTH the clean and the
+    # fallback content (phase 133: the screenshots are the CONTENT; the
+    # extracted images are the citation figures — both, no conflict).
     await _index_pdf_embedded_images(
         session, source=source, rel=rel, full_path=full_path,
         llm=llm, summary=summary, raw_date=raw_date,
     )
 
 
+async def _fallback_page_screenshots(
+    session: Session,
+    *,
+    source: str,
+    rel: str,
+    full_path: Path,
+    llm: Embedder,
+) -> str:
+    """The phase-133 (task 03) page-screenshot fallback — the owner's
+    specified path ("screenshotting the pages and reading each one as
+    an image") for a PDF whose text layer failed
+    :func:`is_usable_pdf_text` (XML/binary garbage, or an empty text
+    layer): every page is rendered to a PNG screenshot and read by the
+    vision model, so tables/images/whatever the text layer lost are
+    preserved as the model saw them on the page.
+
+    * ``content`` is the :data:`PAGE_SEPARATOR`-anchored page
+      descriptions (``---PAGE-N---\n<description>`` — the SAME
+      separator convention as :func:`extract_pdf_text`, so the
+      page-aware chunker's page-boundary handling works unchanged);
+      the doc stays a PDF document (``is_image`` untouched — it is a
+      PDF whose content happens to be vision text).
+    * one vision call per page (the phase-122
+      :func:`describe_image` — ONE CHAT-model call, the PNG bytes as a
+      data URL, ``context`` None); FAIL-SOFT PER PAGE: a page that
+      could not be rendered (``b""`` from :func:`render_pdf_pages` —
+      the index never shifts) or whose vision call failed/came back
+      blank gets the deterministic placeholder
+      ``page N — description unavailable`` (a dead vision endpoint
+      must not sink the doc — the phase-122 fail-soft spirit).
+    * an ALL-failed render (no pages rendered — an unopenable file —
+      or every page a placeholder) returns ``""``: the existing
+      empty-content path (the doc lands as a zero-text doc, the
+      phase-126 no-extractable-text behavior).
+    * the digest the caller computes is over the constructed content —
+      for a stable file + stable vision model the content (and thus
+      the digest) is byte-stable across re-syncs, so the row
+      re-imports as UNCHANGED: no re-chunk, no re-embed, no
+      re-summary, no KB change. The fallback itself RE-RUNS on every
+      sync of a garbage PDF (the gate is a cheap pure predicate and
+      the render needs no LLM; the vision calls are re-issued before
+      the unchanged check can skip them — a documented cost of the
+      no-cache design, bounded by the page count of the garbage
+      PDFs). If the production vision model's reply is
+      non-deterministic, the row is re-described each sync (updated —
+      a cost, not a bug; the doc is still correct).
+
+    *session* is part of the signature for symmetry with the other
+    per-file index helpers (the fallback itself needs no DB access —
+    the caller's upsert pipeline owns the row).
+    """
+    pages = render_pdf_pages(full_path)
+    # The loud one-line log (PLAN §9, the FINDING's signature).
+    logger.warning(
+        "pdf: text layer unusable for %s/%s — falling back to %d page screenshots",
+        source, rel, len(pages),
+    )
+    if not pages:
+        return ""
+    blocks: list[str] = []
+    got_any = False
+    for i, png in enumerate(pages):
+        page = i + 1
+        if not png:
+            blocks.append(f"page {page} — description unavailable")
+            continue
+        desc = await describe_image(llm, data=png, mime="image/png")
+        if desc is None or not desc.strip():
+            blocks.append(f"page {page} — description unavailable")
+        else:
+            blocks.append(desc.strip())
+            got_any = True
+    if not got_any:
+        return ""
+    return "\n\n".join(
+        f"{PAGE_SEPARATOR.format(page=page)}\n{block}"
+        for page, block in enumerate(blocks, start=1)
+    )
+
+
 async def _index_pdf_embedded_images(
     session: Session,
     *,