phase: 133_pdf_screenshot_fallback
Build and Push Containers / build-and-push-app (push) Successful in 2m25s
Build and Push Containers / build-and-push-db (push) Successful in 14s

All verification is complete — everything is green. No repo-side changes were needed in this pass (the phase's work in the working tree is correct and complete).

## Phase 133 final verification report

- **Verified (already implemented in working tree):** `is_usable_pdf_text` gate (0.30 XML-ish / 0.05 non-printable / empty), `render_pdf_pages`, `_fallback_page_screenshots` + `pdfs_screenshotted` counter in `app/rag/{pdf,importer}.py`; regression pin, unit/integration/E2E suites.
- **Completion criteria:**
  1. Real guide clean through real pipeline — ✅ `test_pdf_hnb_guide_regression` + E2E preview/viewer checks (0 XML-ish lines, first-page text present).
  2. Garbage PDF → vision fallback end to end — ✅ unit + E2E (stored content = canned per-page descriptions, garbage not stored, sync `detail` shape unchanged).
  3. Clean PDF never burns a fallback vision call — ✅ unit + E2E (byte-faithful content).
  4. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3129 passed, app/ 99%** (>90%); `uv run pytest tests/e2e/test_garbage_pdf.py -v --no-cov`: **3 passed** in isolation.
  5. No behavior change to completed phases — ✅ entire suite green (all prior-phase tests included); `ruff check .` + `pyright` clean.
  6. Production stale row — post-deploy re-sync verification (live infra, no repo work; no live changes made — deploy of `main` is the owner's step).
- **Documentation:** README.md reviewed in full — every env default cross-checked against `app/config.py`, page routes against `app/main.py`, scripts/Containerfile/compose flags, `.env.example`; the phase-133 PDF bullet added in the working tree is accurate. AGENTS.md verified consistent (all referenced files/commands exist). **No changes needed.**
- **Next pending phase:** `134_sync_cancel`.
This commit is contained in:
2026-09-28 15:39:52 -04:00
parent 4cb85e5baa
commit 25de518dc9
28 changed files with 2701 additions and 44 deletions
+22 -15
View File
@@ -636,13 +636,18 @@ def test_pdf_without_embedded_images_and_unchanged_rerun(db, tmp_path) -> None:
_cleanup_source(db, source_root.name)
def test_pdf_without_text_becomes_an_empty_document(db, tmp_path) -> None:
"""A PDF with no extractable text (image-only, ``images`` OFF — the
embedded image is invisible) → the document is still created, with
EMPTY content (the empty byte string's digest), NO content chunks
and NO content embedding (there is no text to embed — the only
chunk is the best-effort ``lite`` summary's ``is_summary`` row), and
the warning line is the signal."""
def test_pdf_without_text_takes_the_screenshot_fallback(db, tmp_path) -> None:
"""Phase 133 (task 03): a PDF with no extractable text layer
(image-only, ``images`` OFF — the embedded image is invisible to
the phase-126 A3 pass, but the FALLBACK is not gated by the
toggle) → the gate's empty-layer target fires: the single page is
rendered to a PNG and vision-described (the mock's fixed
:data:`DESCRIPTION`), and the document lands with the
``---PAGE-N---``-anchored description as content (one page-anchored
content chunk + the ``lite`` summary chunk, both embedded). The
phase-126 empty-content behavior is retired for layers that CAN be
screenshotted — it survives only for an unopenable file, where the
render fails too (the unit suite pins it)."""
source_root = tmp_path / "pdfemptykb"
source_root.mkdir()
_build_pdf(source_root / "pics.pdf", [("", [_png_bytes(40, 30)])])
@@ -650,6 +655,7 @@ def test_pdf_without_text_becomes_an_empty_document(db, tmp_path) -> None:
try:
summary = asyncio.run(import_sources([source_root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (1, 1, 0)
assert summary.pdfs_screenshotted == 1
doc = db.scalar(
select(Document).where(
@@ -657,18 +663,19 @@ def test_pdf_without_text_becomes_an_empty_document(db, tmp_path) -> None:
)
)
assert doc is not None and doc.is_image is False
assert doc.content == ""
assert doc.content_hash == hashlib.sha256(b"").hexdigest()
assert doc.content == f"---PAGE-1---\n{DESCRIPTION}"
assert doc.content_hash == hashlib.sha256(doc.content.encode("utf-8")).hexdigest()
chunks = db.scalars(
select(Chunk).where(Chunk.document_id == doc.id).order_by(Chunk.position)
).all()
assert [c for c in chunks if not c.is_summary] == [], (
"no text → no content chunks (and no content embedding)"
content_chunks = [c for c in chunks if not c.is_summary]
assert [c.content for c in content_chunks] == [f"Page 1:\n{DESCRIPTION}"], (
"the page-anchored fallback content is chunked like any PDF"
)
# The phase-30/118 path still runs best-effort on the empty
# content — the summary is the only chunk, and it is embedded.
assert len(chunks) == 1 and chunks[0].is_summary
assert chunks[0].embedding is not None and len(chunks[0].embedding) == 768
# The phase-30/118 path runs on the fallback content — the
# summary chunk mirrors the stored summary, both embedded.
assert len(chunks) == 2
assert all(c.embedding is not None and len(c.embedding) == 768 for c in chunks)
assert doc.summary is not None
finally:
_cleanup_source(db, source_root.name)