phase: 140_pictorial_pdf_pages
Everything is verified green. Final report: ## Phase 140 final verification pass — report - Verified all 4 tasks already complete (code in working tree, task files in `complete/`): `pictorial_pages` gate + 2 settings (`BOR_PDF_PICTORIAL`, `BOR_PDF_PICTORIAL_MIN_CHARS` + validator), `PICTORIAL_PAGE_MODE`/`describe_pdf_page` seam + `render_pdf_pages` page selector + mock branch, `_pictorial_pdf_pass` in `_index_pdf_file` with log-only counters, dedicated E2E. No code changes needed. - Test results (exact commands): - `uv run pytest --cov=app --cov-report=term-missing` → **3320 passed, 11 skipped, 0 failed; TOTAL 99%** (>90% gate) - `uv run pytest tests/e2e/test_pictorial_pdf.py -v --no-cov` → **3 passed in isolation** (Playwright, mock-only) - Pre-existing suites unedited: `test_pdf_support.py` 6 passed; `test_garbage_pdf.py` 2 passed + 1 skipped (owner fixture); `test_pdf_image_context.py` 4 passed - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors - Completion criteria: 1. IKEA tar import repro — pinned by 2 integration tests; **skipped loudly here** (`test_sources/ikea_instructions.tar` absent on this machine — verified repo-wide; same house convention as completed phase-133 HNB tests; runs real on the owner's machine). Equivalent logic fully covered by unit pins on synthetic IKEA-class (vector-drawn, zero-raster) PDFs. 2. Chat grounded on described page — **green** (E2E test b: phrase-only-in-illustration question, PDF in `done.sources`, answer echoes the block) 3. Clean-text no-op (zero vision calls) + garbage precedence pin — **green** (`test_zero_candidates_is_a_noop` unit; `test_garbage_pdf_with_pictorial_on_takes_the_fallback_not_the_pass` integration, verified alone) 4. `BOR_PDF_PICTORIAL=0` byte-identical — **green** (unit pin on synthetic PDF; IKEA variant runs on owner's machine) 5. Full gate + E2E isolation + pre-existing suites — **green** (above) 6. ruff + pyright — **clean** 7. Commit/phase-move — delegated to harness per run rules (left in working tree: 16 modified + 8 new files, uncommitted) - Deviations: none beyond the expected owner-fixture skips (11, all pre-existing fixture-missing skips, including 2 IKEA). - Next pending phase: **none** — `todo/` holds only 140; 139 is already complete.
This commit is contained in:
+191
-1
@@ -129,6 +129,28 @@ as unchanged (no re-chunk/re-embed/re-summary, no KB change; the
|
||||
vision calls are re-issued each sync — the no-cache cost, documented
|
||||
in :func:`_fallback_page_screenshots`).
|
||||
|
||||
Pictorial sparse pages (phase 140, task 03, LOCKED A1–A4): AFTER the
|
||||
phase-133 gate, a PDF whose text layer is USABLE gets the sparse-page
|
||||
pass (:func:`_pictorial_pdf_pass`, gated by the ``pdf_pictorial``
|
||||
setting — ``BOR_PDF_PICTORIAL``, the house kill-switch): the
|
||||
deterministic gate (:func:`app.rag.pdf.pictorial_pages` — NO LLM
|
||||
call, LOCKED A1) selects the pages whose stripped text is under
|
||||
``pdf_pictorial_min_chars``; EVERY candidate is rendered (PLAN A21 —
|
||||
no page cap, no call budget) and vision-described through the phase-
|
||||
140 seam (:func:`app.rag.summarizer.describe_pdf_page` — the
|
||||
``PICTORIAL_PAGE_MODE`` one-shot, fail-soft per page), and the
|
||||
description appends to the page's content block as
|
||||
``[Page N illustration: …]`` (LOCKED A2 — no LLM judge: every
|
||||
described page joins the content; the owner's L5 rationale: more
|
||||
information is better for the embeddings). The precedence is locked:
|
||||
the pass runs on a USABLE layer ONLY — an unusable layer takes the
|
||||
phase-133 full-page screenshot fallback and never burns a second
|
||||
vision pass. Zero candidates → a no-op (no render, no vision call —
|
||||
a clean-text PDF's path is byte-identical to pre-phase); the
|
||||
counters ``pdfs_pictorial`` / ``pdf_pictorial_pages`` are log-only
|
||||
(the sync ``detail`` API shape is untouched, the stats log-only
|
||||
convention).
|
||||
|
||||
Image-anchor context (phase 125, task 03, LOCKED A1/A2/A3): BEFORE the
|
||||
description, the same source's importable markdown files are scanned for
|
||||
```` references that name the image (the task-02
|
||||
@@ -250,6 +272,7 @@ from app.rag.pdf import (
|
||||
extract_pdf_page_texts,
|
||||
extract_pdf_text,
|
||||
is_usable_pdf_text,
|
||||
pictorial_pages,
|
||||
render_pdf_pages,
|
||||
)
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
@@ -257,6 +280,7 @@ from app.rag.summarizer import (
|
||||
IMAGE_FALLBACK_MIME,
|
||||
IMAGE_MIMES,
|
||||
describe_image,
|
||||
describe_pdf_page,
|
||||
generate_summary,
|
||||
)
|
||||
|
||||
@@ -341,6 +365,20 @@ class ImportSummary:
|
||||
#: ``---PAGE-N---``-anchored descriptions, or ``""`` when the
|
||||
#: fallback produced nothing — the phase-126 empty-content path).
|
||||
pdfs_screenshotted: int = 0
|
||||
#: PDFs (phase 140, task 03, LOCKED A2/A4) whose USABLE text layer
|
||||
#: carried ≥1 pictorial candidate page (``app.rag.pdf.
|
||||
#: pictorial_pages`` — the deterministic sparse-page gate, LOCKED
|
||||
#: A1) whose vision description was appended to the page block
|
||||
#: (``[Page N illustration: …]``) — the sparse-page pass
|
||||
#: (:func:`_pictorial_pdf_pass`). Log-only (the sync ``detail``
|
||||
#: API shape is untouched — the stats log-only convention,
|
||||
#: phase 133); ``pdf_pictorial_pages`` counts the described pages.
|
||||
pdfs_pictorial: int = 0
|
||||
#: Pictorial candidate pages described across the run (phase 140,
|
||||
#: task 03) — one PDF with three described sparse pages counts
|
||||
#: ``pdfs_pictorial=1`` + ``pdf_pictorial_pages=3`` (A21: no page
|
||||
#: cap — the count may be large, that is the point).
|
||||
pdf_pictorial_pages: int = 0
|
||||
#: PDF-embedded image docs whose stored ``image_context`` was
|
||||
#: backfilled with the document-context block (phase 137, task 03)
|
||||
#: — the one-time best-effort pass; a success never repeats, a
|
||||
@@ -367,7 +405,8 @@ class ImportSummary:
|
||||
"import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
|
||||
"errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
|
||||
"summary_backfilled=%d dates_updated=%d images_failed=%d "
|
||||
"image_context_backfilled=%d pdfs_screenshotted=%d formats=%s",
|
||||
"image_context_backfilled=%d pdfs_screenshotted=%d "
|
||||
"pdfs_pictorial=%d pdf_pictorial_pages=%d formats=%s",
|
||||
self.files,
|
||||
self.added,
|
||||
self.updated,
|
||||
@@ -383,6 +422,8 @@ class ImportSummary:
|
||||
self.images_failed,
|
||||
self.image_context_backfilled,
|
||||
self.pdfs_screenshotted,
|
||||
self.pdfs_pictorial,
|
||||
self.pdf_pictorial_pages,
|
||||
self.format_counts(),
|
||||
)
|
||||
|
||||
@@ -1165,6 +1206,19 @@ async def _index_pdf_file(
|
||||
An unopenable PDF — or one whose fallback also produced nothing
|
||||
— yields ``""`` (still a document — it may carry embedded images
|
||||
— with no content chunks; a warning is the signal).
|
||||
* Phase 140 (task 03, LOCKED A1–A4): when the layer is USABLE and
|
||||
``settings.pdf_pictorial`` is on, the sparse-page pass
|
||||
(:func:`_pictorial_pdf_pass`) renders + vision-describes every
|
||||
page whose stripped text is under
|
||||
``settings.pdf_pictorial_min_chars`` (the deterministic gate —
|
||||
NO LLM call, A1; every candidate, no page cap — A21/A3) and
|
||||
appends the description to the page block as
|
||||
``[Page N illustration: …]`` (A2/A4). The precedence is locked:
|
||||
the pass sits AFTER the phase-133 gate, so an unusable layer
|
||||
(the fallback's full-page screenshots) never burns a second
|
||||
vision pass; zero candidates → a no-op (the extracted text
|
||||
stays byte-identical to the pre-phase-140 path — no render, no
|
||||
vision call).
|
||||
* the sha256 digest is over the FINAL content (the digest rule is
|
||||
content identity — the same text is the same document version;
|
||||
the fallback's constructed page descriptions digest exactly the
|
||||
@@ -1213,6 +1267,20 @@ async def _index_pdf_file(
|
||||
content = await _fallback_page_screenshots(
|
||||
session, source=source, rel=rel, full_path=full_path, llm=llm
|
||||
)
|
||||
elif llm.settings.pdf_pictorial:
|
||||
# Phase 140 (task 03, LOCKED A1–A4): the pictorial pass —
|
||||
# USABLE layers only (the precedence is locked: the phase-133
|
||||
# fallback above already screenshotted an unusable layer, no
|
||||
# double vision). Zero candidates → a no-op (the pass returns
|
||||
# None — the extracted text stays byte-identical to the
|
||||
# pre-phase-140 path); otherwise the FINAL content carries the
|
||||
# described candidates' illustration blocks (A4: the digest
|
||||
# below runs over it, the phase-126/133 convention).
|
||||
passed = await _pictorial_pdf_pass(
|
||||
source=source, rel=rel, full_path=full_path, llm=llm, summary=summary
|
||||
)
|
||||
if passed is not None:
|
||||
content = passed
|
||||
if not content:
|
||||
logger.warning(
|
||||
"import: PDF has no extractable text source=%s path=%s", source, rel
|
||||
@@ -1406,6 +1474,128 @@ async def _fallback_page_screenshots(
|
||||
)
|
||||
|
||||
|
||||
async def _pictorial_pdf_pass(
|
||||
*,
|
||||
source: str,
|
||||
rel: str,
|
||||
full_path: Path,
|
||||
llm: Embedder,
|
||||
summary: ImportSummary,
|
||||
) -> str | None:
|
||||
"""The phase-140 (task 03) pictorial pass (LOCKED A1–A4) — the
|
||||
IKEA assembly-instruction class (TODO L5): a PDF whose text layer
|
||||
is USABLE but SPARSE (mostly pictorial pages — 4–73 chars of
|
||||
markers around vector-drawn pictograms) gets every sparse page
|
||||
rendered to a PNG and vision-described, and the description
|
||||
APPENDS to that page's indexed content block
|
||||
(``[Page N illustration: …]``) — the page's text stays, so the
|
||||
embeddings carry what the thin text layer couldn't (the owner's
|
||||
L5 rationale: "More information is better when generating the
|
||||
text embeddings").
|
||||
|
||||
* the gate is :func:`app.rag.pdf.pictorial_pages` over
|
||||
:func:`app.rag.pdf.extract_pdf_page_texts` — deterministic, NO
|
||||
LLM call (LOCKED A1): the 1-based page numbers whose stripped
|
||||
text is under ``settings.pdf_pictorial_min_chars``, ascending.
|
||||
(The kill-switch itself — ``settings.pdf_pictorial`` — gates the
|
||||
whole pass in the caller: while off, this function is never
|
||||
called and the content is byte-identical to pre-phase.)
|
||||
* ZERO candidates → a NO-OP (returns ``None``): no render, no
|
||||
vision call — the caller keeps its :func:`extract_pdf_text`
|
||||
output untouched, so a clean-text PDF's path is byte-identical
|
||||
to the pre-phase-140 path (the pin).
|
||||
* a page that could not be rendered (the zero-byte
|
||||
:func:`render_pdf_pages` placeholder — the index never shifts,
|
||||
so it stays aligned with the candidates) is skipped with a
|
||||
warning (the phase-133 handling — the page keeps its text only,
|
||||
no vision call is burned for it); a page whose vision call
|
||||
failed or came back blank (:func:`describe_pdf_page` →
|
||||
``None``) is skipped with a warning and keeps its text only —
|
||||
one dead page never kills the pass (LOCKED A2 fail-soft).
|
||||
* the assembly (LOCKED A4): every page of the
|
||||
:func:`extract_pdf_page_texts` map (the SAME pages, order and
|
||||
stripped text as :func:`extract_pdf_text`) as its
|
||||
:data:`PAGE_SEPARATOR` + the page text + — for the described
|
||||
candidates — ``"\\n\\n[Page {n} illustration: {description}]"``.
|
||||
The assembly with NO descriptions is byte-identical to
|
||||
:func:`extract_pdf_text`'s output (the unit pin).
|
||||
* the caller's digest runs over this FINAL content (the
|
||||
phase-126/133 convention — A4): a stable file + a stable vision
|
||||
reply re-syncs as ``unchanged``; a non-deterministic re-describe
|
||||
re-updates the row — a cost, not a bug (the phase-133 docstring
|
||||
documents it, A21 endorses it).
|
||||
* stats (log-only — the sync ``detail`` API shape is untouched,
|
||||
the phase-133 convention): ``summary.pdfs_pictorial`` (+1 when
|
||||
≥1 page was described) and ``summary.pdf_pictorial_pages``
|
||||
(+the described page count), plus the ONE loud log line (PLAN
|
||||
§9): ``pdf: pictorial pass source=… path=… — N candidate pages,
|
||||
M described (min_chars=…)``.
|
||||
"""
|
||||
settings = llm.settings
|
||||
page_texts = extract_pdf_page_texts(full_path)
|
||||
candidates = pictorial_pages(
|
||||
page_texts, min_chars=settings.pdf_pictorial_min_chars
|
||||
)
|
||||
if not candidates:
|
||||
return None
|
||||
# ONE render for the whole candidate set (the phase-140 selector —
|
||||
# a 500-page PDF with 30 sparse pages renders 30 PNGs, not 500);
|
||||
# the result is aligned with *candidates* (ascending, one slot
|
||||
# each — a zero-byte placeholder never shifts the positions).
|
||||
pngs = render_pdf_pages(full_path, pages=candidates)
|
||||
descriptions: dict[int, str] = {}
|
||||
# strict: the render's result is aligned with the candidate list by
|
||||
# contract (one slot per named page, ascending) — a length drift is
|
||||
# a parser regression, fail loud.
|
||||
for page, png in zip(candidates, pngs, strict=True):
|
||||
if not png:
|
||||
# The phase-133 zero-byte placeholder: an unreadable page.
|
||||
logger.warning(
|
||||
"pdf: skipping pictorial page %d of %s/%s — the render is "
|
||||
"empty (unreadable page)",
|
||||
page, source, rel,
|
||||
)
|
||||
continue
|
||||
desc = await describe_pdf_page(
|
||||
llm,
|
||||
data=png,
|
||||
mime="image/png",
|
||||
page=page,
|
||||
page_text=page_texts[page],
|
||||
)
|
||||
if desc is None:
|
||||
# Fail-soft (LOCKED A2): one dead page never kills the
|
||||
# pass — the page keeps its text only, the sync continues.
|
||||
logger.warning(
|
||||
"pdf: pictorial page %d of %s/%s has no description — "
|
||||
"the page keeps its text only",
|
||||
page, source, rel,
|
||||
)
|
||||
continue
|
||||
descriptions[page] = desc
|
||||
# The ONE loud log line (PLAN §9): N candidate pages, M described.
|
||||
logger.info(
|
||||
"pdf: pictorial pass source=%s path=%s — %d candidate pages, %d "
|
||||
"described (min_chars=%d)",
|
||||
source, rel, len(candidates), len(descriptions),
|
||||
settings.pdf_pictorial_min_chars,
|
||||
)
|
||||
# The stats (log-only — the sync ``detail`` API shape is untouched).
|
||||
if descriptions:
|
||||
summary.pdfs_pictorial += 1
|
||||
summary.pdf_pictorial_pages += len(descriptions)
|
||||
# The assembly: the extracted text's page blocks, the described
|
||||
# candidates carrying their illustration block (A4 — the page's
|
||||
# text STAYS; the description appends after it).
|
||||
parts: list[str] = []
|
||||
for page, text in page_texts.items():
|
||||
block = text
|
||||
if page in descriptions:
|
||||
block += f"\n\n[Page {page} illustration: {descriptions[page]}]"
|
||||
parts.append(f"{PAGE_SEPARATOR.format(page=page)}\n{block}")
|
||||
return "\n\n".join(parts)
|
||||
|
||||
|
||||
async def _index_pdf_embedded_images(
|
||||
session: Session,
|
||||
*,
|
||||
|
||||
Reference in New Issue
Block a user