phase: 133_pdf_screenshot_fallback
Build and Push Containers / build-and-push-app (push) Successful in 2m25s
Build and Push Containers / build-and-push-db (push) Successful in 14s

All verification is complete — everything is green. No repo-side changes were needed in this pass (the phase's work in the working tree is correct and complete).

## Phase 133 final verification report

- **Verified (already implemented in working tree):** `is_usable_pdf_text` gate (0.30 XML-ish / 0.05 non-printable / empty), `render_pdf_pages`, `_fallback_page_screenshots` + `pdfs_screenshotted` counter in `app/rag/{pdf,importer}.py`; regression pin, unit/integration/E2E suites.
- **Completion criteria:**
  1. Real guide clean through real pipeline — ✅ `test_pdf_hnb_guide_regression` + E2E preview/viewer checks (0 XML-ish lines, first-page text present).
  2. Garbage PDF → vision fallback end to end — ✅ unit + E2E (stored content = canned per-page descriptions, garbage not stored, sync `detail` shape unchanged).
  3. Clean PDF never burns a fallback vision call — ✅ unit + E2E (byte-faithful content).
  4. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3129 passed, app/ 99%** (>90%); `uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov`: **3 passed** in isolation.
  5. No behavior change to completed phases — ✅ entire suite green (all prior-phase tests included); `ruff check .` + `pyright` clean.
  6. Production stale row — post-deploy re-sync verification (live infra, no repo work; no live changes made — deploy of `main` is the owner's step).
- **Documentation:** README.md reviewed in full — every env default cross-checked against `app/config.py`, page routes against `app/main.py`, scripts/Containerfile/compose flags, `.env.example`; the phase-133 PDF bullet added in the working tree is accurate. AGENTS.md verified consistent (all referenced files/commands exist). **No changes needed.**
- **Next pending phase:** `134_sync_cancel`.
This commit is contained in:
2026-09-28 15:39:52 -04:00
parent 4cb85e5baa
commit 25de518dc9
28 changed files with 2701 additions and 44 deletions
+10
View File
@@ -556,6 +556,16 @@ uploaded archives):
with the toggle off the pass is skipped and the PDF text indexes
normally. Embedded images in a format outside `BOR_IMAGE_EXTENSIONS`
are skipped with a warning.
- **Garbage text layer → page-screenshot fallback (phase 133).** Before
indexing, the extracted text passes a deterministic garbage gate
(empty text layer, or ≥ 30% tag-looking lines, or ≥ 5% non-printable
chars — the class of PDF that reads as raw XML/binary soup). On
failure every page is rendered to a screenshot and read by the vision
model; the stored content is the `---PAGE-N---`-anchored page
descriptions (a failed page gets a `page N — description unavailable`
placeholder — fail-soft, the doc still lands). Clean text layers
never burn a fallback vision call. The run counts the affected PDFs
on the import summary line (`pdfs_screenshotted=N`).
- **Served as text.** The document viewer renders the extracted text —
never the binary PDF.