Everything is verified green. Final report: ## Phase 140 final verification pass — report - Verified all 4 tasks already complete (code in working tree, task files in `complete/`): `pictorial_pages` gate + 2 settings (`BOR_PDF_PICTORIAL`, `BOR_PDF_PICTORIAL_MIN_CHARS` + validator), `PICTORIAL_PAGE_MODE`/`describe_pdf_page` seam + `render_pdf_pages` page selector + mock branch, `_pictorial_pdf_pass` in `_index_pdf_file` with log-only counters, dedicated E2E. No code changes needed. - Test results (exact commands): - `uv run pytest --cov=app --cov-report=term-missing` → **3320 passed, 11 skipped, 0 failed; TOTAL 99%** (>90% gate) - `uv run pytest tests/e2e/test_pictorial_pdf.py -v --no-cov` → **3 passed in isolation** (Playwright, mock-only) - Pre-existing suites unedited: `test_pdf_support.py` 6 passed; `test_garbage_pdf.py` 2 passed + 1 skipped (owner fixture); `test_pdf_image_context.py` 4 passed - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors - Completion criteria: 1. IKEA tar import repro — pinned by 2 integration tests; **skipped loudly here** (`test_sources/ikea_instructions.tar` absent on this machine — verified repo-wide; same house convention as completed phase-133 HNB tests; runs real on the owner's machine). Equivalent logic fully covered by unit pins on synthetic IKEA-class (vector-drawn, zero-raster) PDFs. 2. Chat grounded on described page — **green** (E2E test b: phrase-only-in-illustration question, PDF in `done.sources`, answer echoes the block) 3. Clean-text no-op (zero vision calls) + garbage precedence pin — **green** (`test_zero_candidates_is_a_noop` unit; `test_garbage_pdf_with_pictorial_on_takes_the_fallback_not_the_pass` integration, verified alone) 4. `BOR_PDF_PICTORIAL=0` byte-identical — **green** (unit pin on synthetic PDF; IKEA variant runs on owner's machine) 5. Full gate + E2E isolation + pre-existing suites — **green** (above) 6. ruff + pyright — **clean** 7. Commit/phase-move — delegated to harness per run rules (left in working tree: 16 modified + 8 new files, uncommitted) - Deviations: none beyond the expected owner-fixture skips (11, all pre-existing fixture-missing skips, including 2 IKEA). - Next pending phase: **none** — `todo/` holds only 140; 139 is already complete.
608 lines
26 KiB
Python
608 lines
26 KiB
Python
"""Phase 140, task 03 — the importer's pictorial pass (unit).
|
||
|
||
The pass itself (``app.rag.importer._pictorial_pdf_pass`` — the
|
||
IKEA assembly-instruction class, TODO L5): a PDF whose text layer
|
||
PASSES the phase-133 garbage gate but is SPARSE (mostly pictorial
|
||
pages) gets every candidate page (``pictorial_pages`` — the
|
||
deterministic gate, LOCKED A1) rendered and vision-described, and the
|
||
description appends to the page's content block as
|
||
``[Page N illustration: …]`` (LOCKED A2/A4 — no LLM judge; the digest
|
||
runs over the FINAL content).
|
||
|
||
Pinned here (the synthetic IKEA-class recipe — PyMuPDF, no binary
|
||
fixtures; the real tar is pinned in
|
||
``tests/integration/test_importer.py``):
|
||
|
||
* the happy path — every candidate page (computed IN the test from
|
||
``extract_pdf_page_texts`` + ``pictorial_pages`` — no hardcoded page
|
||
list) carries its illustration block, the text-heavy pages do NOT,
|
||
the chunks carry the canned description, the digest is over the
|
||
FINAL content, the counters + the ONE loud log line fire (PLAN §9).
|
||
* the assembly BYTE-IDENTITY pin — with NO descriptions (every vision
|
||
call fails) the stored content is ``extract_pdf_text``'s output
|
||
byte-for-byte (the page's text is never lost, never reworded).
|
||
* the branch matrix: the kill switch off (``pdf_pictorial=False`` →
|
||
byte-identical to pre-phase, zero vision calls), zero candidates
|
||
(a clean-text PDF — no render, no vision call, no log line), the
|
||
fail-soft ``None`` page (one dead page keeps its text only, the
|
||
others are described), the unreadable-page placeholder skip (the
|
||
zero-byte ``render_pdf_pages`` slot burns no vision call, the page
|
||
keeps its text only).
|
||
* the digest-stability rerun — a stable file + a stable vision reply
|
||
re-syncs as ``unchanged`` (no re-embed, byte-stable content +
|
||
digest); the vision calls are re-issued before the unchanged check
|
||
can skip them — the documented no-cache cost (the phase-133
|
||
convention, A21 endorses it).
|
||
|
||
The E2E surface (the real UI + the mock LLM's ``PICTORIAL_PAGE_MODE``
|
||
branch) is pinned in ``tests/e2e/test_pictorial_pdf.py`` (task 04).
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import asyncio
|
||
import base64
|
||
import hashlib
|
||
import logging
|
||
import textwrap
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import pymupdf as fitz
|
||
import pytest
|
||
from sqlalchemy import select
|
||
|
||
from app.config import Settings
|
||
from app.models import Chunk, Document
|
||
from app.rag import importer
|
||
from app.rag.importer import import_sources
|
||
from app.rag.llm import LLMError
|
||
from app.rag.pdf import (
|
||
PAGE_SEPARATOR,
|
||
extract_pdf_page_texts,
|
||
extract_pdf_text,
|
||
pictorial_pages,
|
||
)
|
||
from app.rag.summarizer import PICTORIAL_PAGE_MODE, PICTORIAL_PROMPT
|
||
from tests.fakes import FakeEmbedder
|
||
|
||
#: The IKEA fixture's metadata title (the phase-140 fixture, TODO L5 —
|
||
#: Adobe InDesign's "Assembly Instructions").
|
||
TITLE = "Assembly Instructions"
|
||
|
||
#: The text-heavy pages (IKEA pages 2–5 — the multilingual warnings,
|
||
#: 3,091–7,288 chars in the real fixture): a fixed warning block well
|
||
#: over the 100-char sparseness bar, so it is NEVER a candidate.
|
||
DENSE_TEXT = (
|
||
"WARNING: Adult assembly required. Read all instructions before "
|
||
"starting. Keep this package away from children. The furniture "
|
||
"must be secured to the wall using the supplied anti-tip device."
|
||
)
|
||
assert len(DENSE_TEXT.strip()) >= 100 # the class property, not a typo
|
||
|
||
|
||
def _insert_dense(page) -> None:
|
||
"""One dense-warning page's text, inserted in 60-char wrapped
|
||
lines: a single 188-char line would be CLIPPED at the page edge
|
||
on extraction (verified — only ~104 chars survive), while the
|
||
multi-line read round-trips byte-exactly (the fixture's class
|
||
property — over the 100-char bar — must hold on the EXTRACTED
|
||
text, which the gate sees)."""
|
||
for i, line in enumerate(textwrap.wrap(DENSE_TEXT, 60)):
|
||
page.insert_text((72, 72 + i * 14), line)
|
||
|
||
|
||
def _build_ikea_like_pdf(path: Path) -> None:
|
||
"""A synthetic IKEA-class PDF (TODO L5 — the vector-drawn
|
||
assembly-instruction class): page 1 is the title page (``BESTA``
|
||
— 5 chars, as in the real fixture's ``BESTÅ``), pages 2–5 the
|
||
text-heavy warnings (≥100 chars — never candidates), pages 6–10
|
||
sparse ``Illustration N`` markers (15–16 chars — candidates)
|
||
standing in for the vector pictograms (no embedded raster images
|
||
— the phase-122 image pipeline sees nothing, as on the real
|
||
fixture). The metadata title is the fixture's."""
|
||
doc = fitz.open()
|
||
doc.new_page().insert_text((72, 72), "BESTA")
|
||
for _ in range(4):
|
||
_insert_dense(doc.new_page())
|
||
for n in range(6, 11):
|
||
doc.new_page().insert_text((72, 72), f"Illustration {n}")
|
||
doc.set_metadata({"title": TITLE})
|
||
doc.save(path)
|
||
doc.close()
|
||
|
||
|
||
def _build_dense_pdf(path: Path) -> None:
|
||
"""A clean-text PDF: every page over the 100-char bar — ZERO
|
||
pictorial candidates (the pass's no-op branch)."""
|
||
doc = fitz.open()
|
||
for _ in range(2):
|
||
_insert_dense(doc.new_page())
|
||
doc.set_metadata({"title": "Clean Reference"})
|
||
doc.save(path)
|
||
doc.close()
|
||
|
||
|
||
class _PictorialLLM(FakeEmbedder):
|
||
"""The capturing fake LLM (task 03): the MULTIMODAL describe call
|
||
(``user`` content = a part list) gets a canned reply DETERMINED BY
|
||
THE RENDERED PAGE'S BYTES (a short sha256 of the data-URL's
|
||
base64 — the same rendered page always gets the same reply,
|
||
ACROSS RUNS: the digest-stability premise holds), and the TEXT
|
||
part is recorded (``pictorial_texts`` — the ``PICTORIAL_PROMPT``
|
||
head + the ``Page N of the PDF`` block are the per-page context
|
||
witness). String (``lite``) calls keep the ``FakeEmbedder``
|
||
behaviour. The REAL ``import_sources`` → ``_pictorial_pdf_pass``
|
||
→ ``describe_pdf_page`` chain runs against it (no seam patch)."""
|
||
|
||
def __init__(self) -> None:
|
||
super().__init__()
|
||
self.pictorial_texts: list[str] = []
|
||
self.pictorial_calls = 0
|
||
|
||
@staticmethod
|
||
def reply_for(png: bytes) -> str:
|
||
"""The canned reply a vision call for *png* gets —
|
||
deterministic in the bytes (the tests build the expected
|
||
content with the same helper)."""
|
||
return (
|
||
f"Page {hashlib.sha256(png).hexdigest()[:8]} illustrated: "
|
||
"two parts are joined with a blue arrow."
|
||
)
|
||
|
||
async def chat(
|
||
self, messages: list[dict[str, Any]], model: str | None = None
|
||
) -> str:
|
||
self.chat_calls.append(list(messages))
|
||
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
|
||
if isinstance(user, list):
|
||
text = next((p["text"] for p in user if p.get("type") == "text"), "")
|
||
assert PICTORIAL_PAGE_MODE in text, (
|
||
"a multimodal call reached the fake that is not the "
|
||
f"pictorial pass's seam: {text[:80]!r}"
|
||
)
|
||
self.pictorial_calls += 1
|
||
self.pictorial_texts.append(text)
|
||
b64 = next(p for p in user if p.get("type") == "image_url")[
|
||
"image_url"
|
||
]["url"].rsplit(",", 1)[1]
|
||
return self.reply_for(base64.b64decode(b64))
|
||
first = user.split()
|
||
return "Summary of " + (first[0] if first else "<empty>")
|
||
|
||
|
||
class _FailPagePictorialLLM(_PictorialLLM):
|
||
"""Rejects the vision call for page *fail_page* (parsed from the
|
||
text part's ``Page N of the PDF.`` block) and describes the rest —
|
||
drives the per-page fail-soft (the dead page keeps its text only,
|
||
the other candidates still get their descriptions)."""
|
||
|
||
def __init__(self, fail_page: int) -> None:
|
||
super().__init__()
|
||
self.fail_page = fail_page
|
||
|
||
async def chat(
|
||
self, messages: list[dict[str, Any]], model: str | None = None
|
||
) -> str:
|
||
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
|
||
if isinstance(user, list):
|
||
text = next((p["text"] for p in user if p.get("type") == "text"), "")
|
||
if f"Page {self.fail_page} of the PDF." in text:
|
||
self.pictorial_calls += 1
|
||
self.pictorial_texts.append(text)
|
||
raise LLMError(
|
||
f"simulated dead vision call for page {self.fail_page} "
|
||
"(test sentinel)"
|
||
)
|
||
return await super().chat(messages, model=model)
|
||
|
||
|
||
class _DeadVisionLLM(FakeEmbedder):
|
||
"""A vision client that REJECTS every multimodal call (the dead
|
||
vision endpoint — the fail-soft's total-failure case: EVERY
|
||
candidate page keeps its text only, and the stored content is the
|
||
extraction byte-identical)."""
|
||
|
||
def __init__(self) -> None:
|
||
super().__init__()
|
||
self.pictorial_calls = 0
|
||
|
||
async def chat(
|
||
self, messages: list[dict[str, Any]], model: str | None = None
|
||
) -> str:
|
||
self.chat_calls.append(list(messages))
|
||
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
|
||
if isinstance(user, list):
|
||
self.pictorial_calls += 1
|
||
raise LLMError("simulated dead vision endpoint (test sentinel)")
|
||
first = user.split()
|
||
return "Summary of " + (first[0] if first else "<empty>")
|
||
|
||
|
||
def _settings(tmp_path: Path, **kwargs: Any) -> Settings:
|
||
"""Settings admitting the ``pdf`` import token with the phase-140
|
||
knobs at their CODE DEFAULTS (``pdf_pictorial`` ON — LOCKED A3 —
|
||
and ``pdf_pictorial_min_chars`` 100 — LOCKED A1) unless pinned
|
||
otherwise; ``images`` off (the embedded-image pass is not in play
|
||
— the fixture class is vector-drawn)."""
|
||
kwargs.setdefault("_env_file", None)
|
||
kwargs.setdefault("import_extensions", "md,txt,pdf")
|
||
kwargs.setdefault("images", False)
|
||
kwargs.setdefault("image_dir", str(tmp_path / "images"))
|
||
return Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
||
|
||
|
||
def _pdf_doc(db, source: str, rel: str) -> Document:
|
||
doc = db.scalar(
|
||
select(Document).where(Document.source == source, Document.path == rel)
|
||
)
|
||
assert doc is not None, f"no documents row for ({source!r}, {rel!r})"
|
||
return doc
|
||
|
||
|
||
def _cleanup_source(db, source: str) -> None:
|
||
for doc in db.scalars(select(Document).where(Document.source == source)).all():
|
||
db.delete(doc)
|
||
db.commit()
|
||
|
||
|
||
def _candidates_and_pngs(pdf_path: Path) -> tuple[list[int], list[bytes]]:
|
||
"""The pass's own inputs, computed IN the test (no hardcoded page
|
||
list — the fixture is stable but the test must not rot): the
|
||
deterministic gate over the per-page text map, and the candidate
|
||
render (aligned with the candidates — ascending, one slot each)."""
|
||
page_texts = extract_pdf_page_texts(pdf_path)
|
||
candidates = pictorial_pages(page_texts, min_chars=100)
|
||
pngs = importer.render_pdf_pages(pdf_path, pages=candidates)
|
||
return candidates, pngs
|
||
|
||
|
||
def _expected_content(
|
||
pdf_path: Path, fail_pages: frozenset[int] = frozenset()
|
||
) -> tuple[str, list[int]]:
|
||
"""The FINAL content the pass produces for *pdf_path* under the
|
||
stable fake — derived from the same inputs the pass uses: every
|
||
page of the extraction map as its ``PAGE_SEPARATOR`` + the page
|
||
text, the described candidates (the candidates MINUS
|
||
*fail_pages*) carrying ``[Page N illustration: <stable reply>]``
|
||
AFTER their text (LOCKED A4 — the text stays)."""
|
||
page_texts = extract_pdf_page_texts(pdf_path)
|
||
candidates, pngs = _candidates_and_pngs(pdf_path)
|
||
parts: list[str] = []
|
||
for page, text in page_texts.items():
|
||
block = text
|
||
if page in candidates and page not in fail_pages:
|
||
png = pngs[candidates.index(page)]
|
||
block += f"\n\n[Page {page} illustration: {_PictorialLLM.reply_for(png)}]"
|
||
parts.append(f"{PAGE_SEPARATOR.format(page=page)}\n{block}")
|
||
return "\n\n".join(parts), candidates
|
||
|
||
|
||
def test_sparse_pages_get_their_illustration_blocks(
|
||
db, tmp_path: Path, caplog: pytest.LogCaptureFixture
|
||
) -> None:
|
||
"""The happy path end to end (the owner's L5 rationale): every
|
||
candidate page (computed from the gate — the fixture's title page
|
||
+ its six illustration pages) carries its illustration block, the
|
||
text-heavy pages (2–5) do NOT, the chunks carry the canned
|
||
description (the embeddings hit the illustrations), the title is
|
||
the metadata title, the digest is over the FINAL content, and the
|
||
counters + the ONE loud log line fire (PLAN §9)."""
|
||
root = tmp_path / "pik"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
llm = _PictorialLLM()
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
with caplog.at_level(logging.INFO, logger="app.importer"):
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
|
||
content, candidates = _expected_content(pdf)
|
||
assert len(candidates) == 6 # the title page + the six markers
|
||
assert set(candidates) == {1, 6, 7, 8, 9, 10}
|
||
|
||
doc = _pdf_doc(db, "pik", "ikea.pdf")
|
||
assert doc.title == TITLE # the metadata title (the fixture's)
|
||
assert doc.is_image is False
|
||
# The assembly is byte-exact: the page text STAYS, the
|
||
# description appends after it (LOCKED A4).
|
||
assert doc.content == content
|
||
# The text-heavy pages (2–5 — the multilingual warnings) carry
|
||
# NO illustration block (the gate is the ONLY selector).
|
||
for n in (2, 3, 4, 5):
|
||
assert f"[Page {n} illustration" not in doc.content
|
||
# The digest runs over the FINAL content (the phase-126/133
|
||
# convention — A4).
|
||
assert doc.content_hash == hashlib.sha256(
|
||
doc.content.encode("utf-8")
|
||
).hexdigest()
|
||
|
||
# The counters (log-only — the sync detail shape is untouched):
|
||
# one PDF with six described sparse pages.
|
||
assert summary.pdfs_pictorial == 1
|
||
assert summary.pdf_pictorial_pages == 6
|
||
assert summary.pdfs_screenshotted == 0 # the gate passed (usable)
|
||
|
||
# The chunks carry the canned description — the joined chunk
|
||
# text hits every candidate's reply (each page is its own
|
||
# ``Page N:`` chunk, the page-aware chunker).
|
||
chunks = db.scalars(
|
||
select(Chunk).where(Chunk.document_id == doc.id)
|
||
.order_by(Chunk.position)
|
||
).all()
|
||
content_chunks = [c for c in chunks if not c.is_summary]
|
||
joined = "\n".join(c.content for c in content_chunks)
|
||
_, pngs = _candidates_and_pngs(pdf)
|
||
for page in candidates:
|
||
reply = _PictorialLLM.reply_for(pngs[candidates.index(page)])
|
||
assert reply in joined, f"page {page}'s description is not indexed"
|
||
for c in chunks:
|
||
assert c.embedding is not None and len(c.embedding) == 768
|
||
summary_chunks = [c for c in chunks if c.is_summary]
|
||
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
|
||
assert doc.summary is not None
|
||
|
||
# ONE vision call per candidate, in candidate order — the text
|
||
# part heads with the locked prompt and carries the page's OWN
|
||
# text as context (the page block, task 02's contract).
|
||
assert llm.pictorial_calls == len(candidates)
|
||
assert len(llm.pictorial_texts) == len(candidates)
|
||
page_texts = extract_pdf_page_texts(pdf)
|
||
for page, text_part in zip(candidates, llm.pictorial_texts, strict=True):
|
||
assert text_part.startswith(PICTORIAL_PROMPT)
|
||
assert f"Page {page} of the PDF." in text_part
|
||
assert page_texts[page] in text_part # the page's own text
|
||
|
||
# The ONE loud log line (PLAN §9) — exact, with the computed N.
|
||
lines = [
|
||
r.getMessage()
|
||
for r in caplog.records
|
||
if "pictorial pass" in r.getMessage()
|
||
]
|
||
assert lines == [
|
||
"pdf: pictorial pass source=pik path=ikea.pdf — "
|
||
"6 candidate pages, 6 described (min_chars=100)"
|
||
]
|
||
finally:
|
||
_cleanup_source(db, "pik")
|
||
|
||
|
||
def test_assembly_byte_identity_without_descriptions(
|
||
db, tmp_path: Path, caplog: pytest.LogCaptureFixture
|
||
) -> None:
|
||
"""THE byte-identity pin: with NO descriptions (the dead vision
|
||
endpoint — every candidate's call fails) the stored content is
|
||
``extract_pdf_text``'s output BYTE-FOR-BYTE (the page's text is
|
||
never lost, never reworded, the separators are untouched), the
|
||
pass still counted its candidates in the log line (0 described),
|
||
and no PDF is counted pictorial (a description is what counts)."""
|
||
root = tmp_path / "dead"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
llm = _DeadVisionLLM()
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
with caplog.at_level(logging.INFO, logger="app.importer"):
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
assert llm.pictorial_calls == 6 # every candidate was attempted
|
||
|
||
doc = _pdf_doc(db, "dead", "ikea.pdf")
|
||
assert doc.content == extract_pdf_text(pdf), (
|
||
"no descriptions → the stored content is the extraction "
|
||
"byte-identical (the assembly pin)"
|
||
)
|
||
assert summary.pdfs_pictorial == 0 # nothing was described
|
||
assert summary.pdf_pictorial_pages == 0
|
||
lines = [
|
||
r.getMessage()
|
||
for r in caplog.records
|
||
if "pictorial pass" in r.getMessage()
|
||
]
|
||
assert lines == [
|
||
"pdf: pictorial pass source=dead path=ikea.pdf — "
|
||
"6 candidate pages, 0 described (min_chars=100)"
|
||
]
|
||
# The dead pages' warnings (the fail-soft's signal, per page).
|
||
assert [
|
||
r
|
||
for r in caplog.records
|
||
if r.levelno == logging.WARNING
|
||
and "has no description" in r.getMessage()
|
||
]
|
||
finally:
|
||
_cleanup_source(db, "dead")
|
||
|
||
|
||
def test_feature_off_is_byte_identical_to_pre_phase(db, tmp_path: Path) -> None:
|
||
"""The kill switch (LOCKED A3, ``BOR_PDF_PICTORIAL=0``): the pass
|
||
never runs — no render, no vision call, no log line, no counters —
|
||
and the stored content is ``extract_pdf_text``'s output
|
||
byte-identical to the pre-phase-140 path."""
|
||
root = tmp_path / "off"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
llm = _PictorialLLM()
|
||
llm.settings = _settings(tmp_path, pdf_pictorial=False)
|
||
try:
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
doc = _pdf_doc(db, "off", "ikea.pdf")
|
||
assert doc.content == extract_pdf_text(pdf)
|
||
assert llm.pictorial_calls == 0
|
||
assert (summary.pdfs_pictorial, summary.pdf_pictorial_pages) == (0, 0)
|
||
finally:
|
||
_cleanup_source(db, "off")
|
||
|
||
|
||
def test_zero_candidates_is_a_noop(
|
||
db, tmp_path: Path, caplog: pytest.LogCaptureFixture
|
||
) -> None:
|
||
"""A clean-text PDF (every page over the 100-char bar) has ZERO
|
||
candidates — the pass is a no-op: no render, NO vision call
|
||
(the pin — a clean-text PDF never burns a vision call), no log
|
||
line, no counters, and the content is the extraction untouched."""
|
||
root = tmp_path / "dense"
|
||
root.mkdir()
|
||
pdf = root / "clean.pdf"
|
||
_build_dense_pdf(pdf)
|
||
llm = _PictorialLLM()
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
with caplog.at_level(logging.INFO, logger="app.importer"):
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
doc = _pdf_doc(db, "dense", "clean.pdf")
|
||
assert doc.content == extract_pdf_text(pdf)
|
||
assert llm.pictorial_calls == 0
|
||
assert (summary.pdfs_pictorial, summary.pdf_pictorial_pages) == (0, 0)
|
||
assert not [
|
||
r for r in caplog.records if "pictorial pass" in r.getMessage()
|
||
]
|
||
finally:
|
||
_cleanup_source(db, "dense")
|
||
|
||
|
||
def test_fail_soft_page_keeps_its_text(
|
||
db, tmp_path: Path, caplog: pytest.LogCaptureFixture
|
||
) -> None:
|
||
"""One dead candidate page (the vision call rejects it) keeps its
|
||
text only — a warning is the signal, the OTHER candidates still
|
||
get their descriptions, one dead page never kills the pass
|
||
(LOCKED A2 fail-soft). The described count is five of six; the
|
||
PDF still counts pictorial (≥1 described page)."""
|
||
root = tmp_path / "partfail"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
llm = _FailPagePictorialLLM(fail_page=7)
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
with caplog.at_level(logging.INFO, logger="app.importer"):
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
assert llm.pictorial_calls == 6 # every candidate was attempted
|
||
|
||
content, candidates = _expected_content(pdf, fail_pages=frozenset({7}))
|
||
doc = _pdf_doc(db, "partfail", "ikea.pdf")
|
||
assert doc.content == content
|
||
assert "[Page 7 illustration" not in doc.content # the dead page
|
||
assert "---PAGE-7---\nIllustration 7" in doc.content # its text stays
|
||
assert summary.pdfs_pictorial == 1
|
||
assert summary.pdf_pictorial_pages == 5
|
||
lines = [
|
||
r.getMessage()
|
||
for r in caplog.records
|
||
if "pictorial pass" in r.getMessage()
|
||
]
|
||
assert lines == [
|
||
"pdf: pictorial pass source=partfail path=ikea.pdf — "
|
||
"6 candidate pages, 5 described (min_chars=100)"
|
||
]
|
||
# The importer's per-page warning (the dead page's signal).
|
||
warnings = [
|
||
r
|
||
for r in caplog.records
|
||
if r.levelno == logging.WARNING
|
||
and "pictorial page 7" in r.getMessage()
|
||
]
|
||
assert len(warnings) == 1
|
||
finally:
|
||
_cleanup_source(db, "partfail")
|
||
|
||
|
||
def test_unreadable_page_placeholder_skips_the_vision_call(
|
||
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch,
|
||
caplog: pytest.LogCaptureFixture,
|
||
) -> None:
|
||
"""A candidate page that could not be RENDERED (the zero-byte
|
||
``render_pdf_pages`` placeholder — the phase-133 handling) is
|
||
skipped with a warning: it burns NO vision call, keeps its text
|
||
only, and the OTHER candidates keep their positions +
|
||
descriptions (the result's order never shifts)."""
|
||
root = tmp_path / "halfblind"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
|
||
real_render = importer.render_pdf_pages
|
||
|
||
def _render(path: Path, dpi: int = 150, pages: Any = None) -> list[bytes]:
|
||
assert pages is not None # only the pass calls it (with a selector)
|
||
pngs = real_render(path, dpi=dpi, pages=pages)
|
||
return [
|
||
b"" if n == 6 else png
|
||
for n, png in zip(pages, pngs, strict=True)
|
||
]
|
||
|
||
monkeypatch.setattr(importer, "render_pdf_pages", _render)
|
||
llm = _PictorialLLM()
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
with caplog.at_level(logging.WARNING, logger="app.importer"):
|
||
summary = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (summary.files, summary.added, summary.errors) == (1, 1, 0)
|
||
assert llm.pictorial_calls == 5 # the blank page burned no call
|
||
|
||
content, candidates = _expected_content(pdf, fail_pages=frozenset({6}))
|
||
doc = _pdf_doc(db, "halfblind", "ikea.pdf")
|
||
assert doc.content == content
|
||
assert "[Page 6 illustration" not in doc.content
|
||
assert "---PAGE-6---\nIllustration 6" in doc.content
|
||
assert summary.pdfs_pictorial == 1
|
||
assert summary.pdf_pictorial_pages == 5
|
||
warnings = [
|
||
r
|
||
for r in caplog.records
|
||
if r.levelno == logging.WARNING
|
||
and "skipping pictorial page 6" in r.getMessage()
|
||
]
|
||
assert len(warnings) == 1
|
||
finally:
|
||
_cleanup_source(db, "halfblind")
|
||
|
||
|
||
def test_rerun_lands_unchanged_with_byte_stable_content(
|
||
db, tmp_path: Path
|
||
) -> None:
|
||
"""Digest stability (A4 — the phase-126/133 convention): the
|
||
digest runs over the FINAL content, so a re-sync of an unchanged
|
||
IKEA-class PDF (same pages + the stable fake's same replies)
|
||
re-imports as UNCHANGED: no re-chunk, no re-embed, no re-summary,
|
||
no KB change (not ``updated``, no changed path), and the content +
|
||
digest are byte-stable. The vision calls ARE re-issued before the
|
||
unchanged check can skip them — the documented no-cache cost (the
|
||
phase-133 convention; a non-deterministic vision model re-describes
|
||
the row each sync — a cost, not a bug; A21 endorses it)."""
|
||
root = tmp_path / "stable"
|
||
root.mkdir()
|
||
pdf = root / "ikea.pdf"
|
||
_build_ikea_like_pdf(pdf)
|
||
llm = _PictorialLLM()
|
||
llm.settings = _settings(tmp_path)
|
||
try:
|
||
s1 = asyncio.run(import_sources([root], llm, session=db))
|
||
assert (s1.added, s1.pdfs_pictorial, s1.pdf_pictorial_pages) == (1, 1, 6)
|
||
assert llm.pictorial_calls == 6
|
||
embeds1 = len(llm.calls)
|
||
doc1 = _pdf_doc(db, "stable", "ikea.pdf")
|
||
content1, digest1 = doc1.content, doc1.content_hash
|
||
|
||
s2 = asyncio.run(import_sources([root], llm, session=db))
|
||
# The pass re-ran (gate re-fired, vision re-issued — the
|
||
# documented cost) — but the row itself re-imported unchanged.
|
||
assert (s2.added, s2.updated, s2.unchanged) == (0, 0, 1)
|
||
assert s2.pdfs_pictorial == 1 # the pass re-fired
|
||
assert llm.pictorial_calls == 12 # the vision calls were re-issued
|
||
assert len(llm.calls) == embeds1, "unchanged: no re-embed"
|
||
assert s2.changed_paths == [], "unchanged: no KB change recorded"
|
||
|
||
doc2 = _pdf_doc(db, "stable", "ikea.pdf")
|
||
assert doc2.content == content1 # byte-stable content
|
||
assert doc2.content_hash == digest1 # byte-stable digest
|
||
finally:
|
||
_cleanup_source(db, "stable")
|