phase: 140_pictorial_pdf_pages
Everything is verified green. Final report: ## Phase 140 final verification pass — report - Verified all 4 tasks already complete (code in working tree, task files in `complete/`): `pictorial_pages` gate + 2 settings (`BOR_PDF_PICTORIAL`, `BOR_PDF_PICTORIAL_MIN_CHARS` + validator), `PICTORIAL_PAGE_MODE`/`describe_pdf_page` seam + `render_pdf_pages` page selector + mock branch, `_pictorial_pdf_pass` in `_index_pdf_file` with log-only counters, dedicated E2E. No code changes needed. - Test results (exact commands): - `uv run pytest --cov=app --cov-report=term-missing` → **3320 passed, 11 skipped, 0 failed; TOTAL 99%** (>90% gate) - `uv run pytest tests/e2e/test_pictorial_pdf.py -v --no-cov` → **3 passed in isolation** (Playwright, mock-only) - Pre-existing suites unedited: `test_pdf_support.py` 6 passed; `test_garbage_pdf.py` 2 passed + 1 skipped (owner fixture); `test_pdf_image_context.py` 4 passed - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors - Completion criteria: 1. IKEA tar import repro — pinned by 2 integration tests; **skipped loudly here** (`test_sources/ikea_instructions.tar` absent on this machine — verified repo-wide; same house convention as completed phase-133 HNB tests; runs real on the owner's machine). Equivalent logic fully covered by unit pins on synthetic IKEA-class (vector-drawn, zero-raster) PDFs. 2. Chat grounded on described page — **green** (E2E test b: phrase-only-in-illustration question, PDF in `done.sources`, answer echoes the block) 3. Clean-text no-op (zero vision calls) + garbage precedence pin — **green** (`test_zero_candidates_is_a_noop` unit; `test_garbage_pdf_with_pictorial_on_takes_the_fallback_not_the_pass` integration, verified alone) 4. `BOR_PDF_PICTORIAL=0` byte-identical — **green** (unit pin on synthetic PDF; IKEA variant runs on owner's machine) 5. Full gate + E2E isolation + pre-existing suites — **green** (above) 6. ruff + pyright — **clean** 7. Commit/phase-move — delegated to harness per run rules (left in working tree: 16 modified + 8 new files, uncommitted) - Deviations: none beyond the expected owner-fixture skips (11, all pre-existing fixture-missing skips, including 2 IKEA). - Next pending phase: **none** — `todo/` holds only 140; 139 is already complete.
This commit is contained in:
@@ -396,6 +396,34 @@ class Settings(BaseSettings):
|
||||
#: upload): the served copy must outlive the source file.
|
||||
image_dir: str = "~/bor-sources/images"
|
||||
|
||||
# --- Pictorial PDF pages (phase 140: sparse pages rendered + vision-described) ---
|
||||
#: Master switch for the pictorial PDF pass (phase 140, LOCKED A3,
|
||||
#: ``BOR_PDF_PICTORIAL``; ``0``/``false`` = off). While on, a PDF
|
||||
#: whose text layer passes the phase-133 usability gate ALSO gets the
|
||||
#: sparse-page pass: every page whose stripped text is under
|
||||
#: :attr:`pdf_pictorial_min_chars` chars (``app.rag.pdf.pictorial_pages``
|
||||
#: — the deterministic gate, LOCKED A1) is rendered to a PNG and
|
||||
#: vision-described, and the description appends to that page's
|
||||
#: indexed content block (``[Page N illustration: …]``, LOCKED A4) —
|
||||
#: the embeddings then carry what the thin text layer couldn't (the
|
||||
#: IKEA assembly-instruction class: 20 of 24 pages are 4–73 chars of
|
||||
#: markers around vector pictograms). NO page cap: every candidate
|
||||
#: page is described (PLAN A21 — the LLM calls are free; a slower
|
||||
#: sync is acceptable, a lower-quality KB is not). While off, a
|
||||
#: PDF's indexed content is byte-identical to pre-phase-140 (the
|
||||
#: ``BOR_IMAGES`` naming pattern — the kill switch).
|
||||
pdf_pictorial: bool = True
|
||||
#: Sparseness bar for the pictorial pass (phase 140, LOCKED A1,
|
||||
#: ``BOR_PDF_PICTORIAL_MIN_CHARS``): a page whose STRIPPED text is
|
||||
#: STRICTLY shorter than this many characters is a pictorial
|
||||
#: candidate — a page with exactly this many stripped chars is NOT
|
||||
#: sparse (the boundary is pinned in the unit tests). Default 100 —
|
||||
#: the IKEA fixture's sparse pages run 4–73 chars and its text-heavy
|
||||
#: pages 3,091+, so 100 sits between the classes. ``0`` = every page
|
||||
#: is a candidate (PLAN A21); a negative value is a typo — the
|
||||
#: validator fails startup loudly (the ``agent_max_rounds`` pattern).
|
||||
pdf_pictorial_min_chars: int = 100
|
||||
|
||||
# --- Chat image questions (phase 123: attach an image to a question) ---
|
||||
#: Where a user's question image bytes are stored (phase 123,
|
||||
#: ``BOR_CHAT_IMAGE_DIR`` — the ``image_dir`` convention: a sibling of
|
||||
@@ -572,6 +600,18 @@ class Settings(BaseSettings):
|
||||
raise ValueError("read_max_chars must be >= 0 (chars)")
|
||||
return v
|
||||
|
||||
@field_validator("pdf_pictorial_min_chars")
|
||||
@classmethod
|
||||
def _pdf_pictorial_min_chars_non_negative(cls, v: int) -> int:
|
||||
"""``0`` is the every-page-is-a-candidate mode (PLAN A21 — the
|
||||
LLM calls are free) — a negative threshold is a typo (the
|
||||
``agent_max_rounds`` pattern, phase 140)."""
|
||||
if v < 0:
|
||||
raise ValueError(
|
||||
"pdf_pictorial_min_chars must be >= 0 (0 = every page is a candidate)"
|
||||
)
|
||||
return v
|
||||
|
||||
@field_validator("embed_question_max_chars")
|
||||
@classmethod
|
||||
def _embed_question_max_chars_positive(cls, v: int) -> int:
|
||||
|
||||
Reference in New Issue
Block a user