phase: 140_pictorial_pdf_pages
Build and Push Containers / build-and-push-app (push) Successful in 2m26s
Build and Push Containers / build-and-push-db (push) Successful in 12s

Everything is verified green. Final report:

## Phase 140 final verification pass — report

- Verified all 4 tasks already complete (code in working tree, task files in `complete/`): `pictorial_pages` gate + 2 settings (`BOR_PDF_PICTORIAL`, `BOR_PDF_PICTORIAL_MIN_CHARS` + validator), `PICTORIAL_PAGE_MODE`/`describe_pdf_page` seam + `render_pdf_pages` page selector + mock branch, `_pictorial_pdf_pass` in `_index_pdf_file` with log-only counters, dedicated E2E. No code changes needed.
- Test results (exact commands):
  - `uv run pytest --cov=app --cov-report=term-missing` → **3320 passed, 11 skipped, 0 failed; TOTAL 99%** (>90% gate)
  - `uv run pytest tests/e2e/test_pictorial_pdf.py -v --no-cov` → **3 passed in isolation** (Playwright, mock-only)
  - Pre-existing suites unedited: `test_pdf_support.py` 6 passed; `test_garbage_pdf.py` 2 passed + 1 skipped (owner fixture); `test_pdf_image_context.py` 4 passed
  - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
- Completion criteria:
  1. IKEA tar import repro — pinned by 2 integration tests; **skipped loudly here** (`test_sources/ikea_instructions.tar` absent on this machine — verified repo-wide; same house convention as completed phase-133 HNB tests; runs real on the owner's machine). Equivalent logic fully covered by unit pins on synthetic IKEA-class (vector-drawn, zero-raster) PDFs.
  2. Chat grounded on described page — **green** (E2E test b: phrase-only-in-illustration question, PDF in `done.sources`, answer echoes the block)
  3. Clean-text no-op (zero vision calls) + garbage precedence pin — **green** (`test_zero_candidates_is_a_noop` unit; `test_garbage_pdf_with_pictorial_on_takes_the_fallback_not_the_pass` integration, verified alone)
  4. `BOR_PDF_PICTORIAL=0` byte-identical — **green** (unit pin on synthetic PDF; IKEA variant runs on owner's machine)
  5. Full gate + E2E isolation + pre-existing suites — **green** (above)
  6. ruff + pyright — **clean**
  7. Commit/phase-move — delegated to harness per run rules (left in working tree: 16 modified + 8 new files, uncommitted)
- Deviations: none beyond the expected owner-fixture skips (11, all pre-existing fixture-missing skips, including 2 IKEA).
- Next pending phase: **none** — `todo/` holds only 140; 139 is already complete.
This commit is contained in:
2026-09-30 19:16:35 -04:00
parent a86544a083
commit 9633eb7653
45 changed files with 3895 additions and 36 deletions
+40
View File
@@ -396,6 +396,34 @@ class Settings(BaseSettings):
#: upload): the served copy must outlive the source file.
image_dir: str = "~/bor-sources/images"
# --- Pictorial PDF pages (phase 140: sparse pages rendered + vision-described) ---
#: Master switch for the pictorial PDF pass (phase 140, LOCKED A3,
#: ``BOR_PDF_PICTORIAL``; ``0``/``false`` = off). While on, a PDF
#: whose text layer passes the phase-133 usability gate ALSO gets the
#: sparse-page pass: every page whose stripped text is under
#: :attr:`pdf_pictorial_min_chars` chars (``app.rag.pdf.pictorial_pages``
#: — the deterministic gate, LOCKED A1) is rendered to a PNG and
#: vision-described, and the description appends to that page's
#: indexed content block (``[Page N illustration: …]``, LOCKED A4) —
#: the embeddings then carry what the thin text layer couldn't (the
#: IKEA assembly-instruction class: 20 of 24 pages are 4–73 chars of
#: markers around vector pictograms). NO page cap: every candidate
#: page is described (PLAN A21 — the LLM calls are free; a slower
#: sync is acceptable, a lower-quality KB is not). While off, a
#: PDF's indexed content is byte-identical to pre-phase-140 (the
#: ``BOR_IMAGES`` naming pattern — the kill switch).
pdf_pictorial: bool = True
#: Sparseness bar for the pictorial pass (phase 140, LOCKED A1,
#: ``BOR_PDF_PICTORIAL_MIN_CHARS``): a page whose STRIPPED text is
#: STRICTLY shorter than this many characters is a pictorial
#: candidate — a page with exactly this many stripped chars is NOT
#: sparse (the boundary is pinned in the unit tests). Default 100 —
#: the IKEA fixture's sparse pages run 4–73 chars and its text-heavy
#: pages 3,091+, so 100 sits between the classes. ``0`` = every page
#: is a candidate (PLAN A21); a negative value is a typo — the
#: validator fails startup loudly (the ``agent_max_rounds`` pattern).
pdf_pictorial_min_chars: int = 100
# --- Chat image questions (phase 123: attach an image to a question) ---
#: Where a user's question image bytes are stored (phase 123,
#: ``BOR_CHAT_IMAGE_DIR`` — the ``image_dir`` convention: a sibling of
@@ -572,6 +600,18 @@ class Settings(BaseSettings):
raise ValueError("read_max_chars must be >= 0 (chars)")
return v
@field_validator("pdf_pictorial_min_chars")
@classmethod
def _pdf_pictorial_min_chars_non_negative(cls, v: int) -> int:
"""``0`` is the every-page-is-a-candidate mode (PLAN A21 — the
LLM calls are free) — a negative threshold is a typo (the
``agent_max_rounds`` pattern, phase 140)."""
if v < 0:
raise ValueError(
"pdf_pictorial_min_chars must be >= 0 (0 = every page is a candidate)"
)
return v
@field_validator("embed_question_max_chars")
@classmethod
def _embed_question_max_chars_positive(cls, v: int) -> int: