phase: 133_pdf_screenshot_fallback
All verification is complete — everything is green. No repo-side changes were needed in this pass (the phase's work in the working tree is correct and complete).
## Phase 133 final verification report
- **Verified (already implemented in working tree):** `is_usable_pdf_text` gate (0.30 XML-ish / 0.05 non-printable / empty), `render_pdf_pages`, `_fallback_page_screenshots` + `pdfs_screenshotted` counter in `app/rag/{pdf,importer}.py`; regression pin, unit/integration/E2E suites.
- **Completion criteria:**
1. Real guide clean through real pipeline — ✅ `test_pdf_hnb_guide_regression` + E2E preview/viewer checks (0 XML-ish lines, first-page text present).
2. Garbage PDF → vision fallback end to end — ✅ unit + E2E (stored content = canned per-page descriptions, garbage not stored, sync `detail` shape unchanged).
3. Clean PDF never burns a fallback vision call — ✅ unit + E2E (byte-faithful content).
4. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3129 passed, app/ 99%** (>90%); `uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov`: **3 passed** in isolation.
5. No behavior change to completed phases — ✅ entire suite green (all prior-phase tests included); `ruff check .` + `pyright` clean.
6. Production stale row — post-deploy re-sync verification (live infra, no repo work; no live changes made — deploy of `main` is the owner's step).
- **Documentation:** README.md reviewed in full — every env default cross-checked against `app/config.py`, page routes against `app/main.py`, scripts/Containerfile/compose flags, `.env.example`; the phase-133 PDF bullet added in the working tree is accurate. AGENTS.md verified consistent (all referenced files/commands exist). **No changes needed.**
- **Next pending phase:** `134_sync_cancel`.
This commit is contained in:
@@ -556,6 +556,16 @@ uploaded archives):
|
||||
with the toggle off the pass is skipped and the PDF text indexes
|
||||
normally. Embedded images in a format outside `BOR_IMAGE_EXTENSIONS`
|
||||
are skipped with a warning.
|
||||
- **Garbage text layer → page-screenshot fallback (phase 133).** Before
|
||||
indexing, the extracted text passes a deterministic garbage gate
|
||||
(empty text layer, or ≥ 30% tag-looking lines, or ≥ 5% non-printable
|
||||
chars — the class of PDF that reads as raw XML/binary soup). On
|
||||
failure every page is rendered to a screenshot and read by the vision
|
||||
model; the stored content is the `---PAGE-N---`-anchored page
|
||||
descriptions (a failed page gets a `page N — description unavailable`
|
||||
placeholder — fail-soft, the doc still lands). Clean text layers
|
||||
never burn a fallback vision call. The run counts the affected PDFs
|
||||
on the import summary line (`pdfs_screenshotted=N`).
|
||||
- **Served as text.** The document viewer renders the extracted text —
|
||||
never the binary PDF.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user