phase: 151_pdf_image_extraction
The 149/150 CSS changes touch the sources-page sync UI visible behind the modal, so the regenerated screenshot is the accurate record of the current UI (house convention — the suite rewrites it each run; it was re-committed in phases 144/145/148 the same way). I'll keep it in the working tree. All verification is complete. Final state: **Phase 151 — final verification pass: complete, project 100% green.** - **Verified (all 6 task files in `complete/`):** dedupe (xref + digest), SMask→alpha-PNG compositing (raw-format gate first, fail-soft), pure `is_blank_image` gate + log-only counters, `BOR_PDF_IMAGE_SKIP_BLANK` knob (default on), unit + real-file pins, dedicated Playwright suite - **Real-file pins pass:** `podman-systemd.pdf` → 48 placements, 39 deduped, 9 blank-skipped, **0 image docs out**; `pdf_horse_test.pdf` → masked JPEG composited to alpha PNG, kept; `players_handbook.pdf` → 586 `jpx` images untouched (raw mime, ext-gate skipped) - **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → **3511 passed, TOTAL 99%** (pdf.py 100%, importer.py 99%, config.py 100%); dedicated E2E `tests/e2e/test_pdf_image_extraction.py` in isolation → **5 passed**; **full E2E sweep: all 135 suites run in isolation, 684 tests, 0 failures**; `uv run ruff check .` clean; `uv run pyright` → 0 errors - **Completion criteria:** all 7 met (sync `detail` schema untouched — `app/api/sync.py`/`app/schemas.py` unmodified; counters in the shared `import_sources` → `summary.log()` path; commit left to the harness per instructions) - **Docs:** README.md and AGENTS.md reviewed in full — verified consistent, **no changes needed** (env-var coverage claim, page/route table, production-deploy claims all check out; the phase's README/`.env.example` additions match the code). Note: PLAN.md §7.1's 46rem chat column is superseded by phase 100 (AGENTS.md rule 5 states this correctly; PLAN.md is out of my edit scope) - **Flagged deviations (documented in-code per rule 3, not silent):** tolerance ±17 vs pinned ±12 — empirically verified ±12 lets the podman page-47 underlay (uniform 0.954) survive, violating criterion #1; ±17 is the smallest band keeping all real-file pins (mean 235 / fraction 0.98 unchanged); gate measures the base stream, not the premultiplied composite (which darkens transparent regions and would miss the underlays) - **Next pending phase:** `152_sync_progress_image_pass`
This commit is contained in:
@@ -442,6 +442,23 @@ class Settings(BaseSettings):
|
||||
#: validator fails startup loudly (the ``agent_max_rounds`` pattern).
|
||||
pdf_pictorial_min_chars: int = 100
|
||||
|
||||
# --- PDF embedded images (phase 151: the blank/decorative gate) ---
|
||||
#: Skip blank/decorative PDF-embedded images (phase 151, task 04,
|
||||
#: ``BOR_PDF_IMAGE_SKIP_BLANK``; ``0``/``false`` = off — the kill
|
||||
#: switch, the ``pdf_pictorial``/``images`` naming pattern). While
|
||||
#: on (the DEFAULT) the parser's blank/decorative gate
|
||||
#: (``app.rag.pdf.is_blank_image`` — near-white AND nearly uniform,
|
||||
#: owner-locked A2) drops the near-white underlays (the table-
|
||||
#: shading "blank white" images, TODO L4) from the embedded-image
|
||||
#: pass: no row, no copy, no vision call. While off, EVERY
|
||||
#: extracted image is indexed again — the pre-phase-151 behavior
|
||||
#: (a KB of scanned/decorative pages where a near-white image
|
||||
#: still matters). Dedupe (task 01) and soft-mask compositing
|
||||
#: (task 02) are UNAFFECTED by this knob — they are correctness
|
||||
#: fixes. A plain bool like ``pdf_pictorial``/``images`` — no
|
||||
#: validator.
|
||||
pdf_image_skip_blank: bool = True
|
||||
|
||||
# --- Chat image questions (phase 123: attach an image to a question) ---
|
||||
#: Where a user's question image bytes are stored (phase 123,
|
||||
#: ``BOR_CHAT_IMAGE_DIR`` — the ``image_dir`` convention: a sibling of
|
||||
|
||||
Reference in New Issue
Block a user