phase: 163_pdf_strip_additive_selfheal

All gates verified. Final report:

**Phase 163 — final verification pass (all 5 tasks already complete; verification-only run)**

- Verified D1–D4 in code: `page_strip_mode` column + migration 0027, `DocContent` absent-never-null rule keyed off `page_count`; `_persist_pdf_pages` render → atomic tmp+`os.replace` writes → stale-tail-last + `strip_class` stamp; `_regate_pdf_strip` on both unchanged paths (mirror gate, heal, repair, fail-soft, one INFO line only when acting); viewer mode branch + Text/Pages toggle (Text default, `aria-pressed`, no innerHTML, lazy imgs).
- `uv run pytest --cov=app --cov-report=term-missing` → **3796 tests, exit 0, coverage 99%** (>90%)
- `uv run pytest tests/e2e/test_pdf_strip_additive_selfheal.py -v --no-cov` → **5 passed** in isolation
- `test_pdf_pages_in_sources + test_pictorial_pdf + test_garbage_pdf` → 10 passed; `test_document_viewer(_context) + test_image_documents` → 20 passed
- `uv run ruff check .` clean; `uv run pyright` → 0 errors; alembic **0027 downgrade→upgrade CLI round-trip clean** (+ integration pins)
- Completion criteria: additive view w/ toggle ✅ · primary DOM byte-identity ✅ · write-first order/mid-write-failure/atomic pins ✅ · heal + repair E2E ✅ · zero vision/LLM on fast path (mock-captured) ✅ · non-qualifying payload byte-identity ✅ · docs pass ✅
- **Documentation**: README.md + AGENTS.md reviewed in full — the phase's updates are accurate; all config-table defaults cross-checked against `app/config.py` (no drift), every page route, script, and E2E command referenced exists, every `.env.example` var documented — **verified consistent, no changes needed**.
- Notable: live `Letumdei` repair is owner-runtime post-deploy per D5 (next production Sync self-heals; no live changes made); the modified `.agents/screenshots/doc_modal_scrollbar_1280.png` is an E2E-regenerated artifact left for the pipeline commit.
- **Next pending phase:** `164_chat_error_honesty`

README.md

@@ -239,9 +239,11 @@ idempotent and delta-based: each file's RAW BYTES are hashed (sha256,
 streamed) and checked against the last index's stamp *before* any
 extraction or vision work — an unchanged file costs one hash + one row
 lookup (no re-extract, no vision, no re-embed), while a changed file
-re-indexes fully. A refresh after a normal editing session takes
-seconds — pictorial PDFs included (their expensive first-index vision
-burns only once per file version):
+re-indexes fully. The one exception is a PDF row that still claims a
+page strip: it is re-gated from its current bytes (no vision, no LLM —
+see "The strip repairs itself" below). A refresh after a normal editing
+session takes seconds — pictorial PDFs included (their expensive
+first-index vision burns only once per file version):
 
 ```bash
 # After editing/adding/removing notes:
@@ -844,11 +846,42 @@ uploaded archives):
   shows the PAGES: a vertical strip of lazy page images with
   "Page N" captions, in place of the text content and the Summary
   panel (the per-page descriptions stay in the DB for Q&A — the
-  embeddings and summary are untouched, just not displayed). A lost
+  embeddings and summary are untouched, just not displayed). Since
+  phase 163 this strip-only shape is the PRIMARY class only — the
+  pictorial class keeps its text behind a Text/Pages toggle, per the
+  next bullet. A lost
   copy shows a "Page N unavailable" note for that page only. Text-
   heavy PDFs (usable layer, zero pictorial candidates — and the whole
   sparse class while `BOR_PDF_PICTORIAL=0`) keep the text view,
   byte-identical to pre-phase.
+- **Which kind of strip it is (phase 163).** The two qualifying classes
+  are not the same problem: the screenshot class has no usable text to
+  show, while a book with one sparse page (an illustration map page in a
+  36-page text) does. The document row therefore records the strip's
+  CLASS beside the page count (`documents.page_strip_mode` —
+  `primary` = the screenshots ARE the content, so the strip is the view;
+  `additive` = a usable text layer, so the text + Summary view stays and
+  the pages sit behind a Text/Pages toggle), and the document-content
+  payload carries it as `page_strip_mode`. The key is ABSENT — never
+  `null` — when there is no strip, which is also how a row written before
+  this phase serializes; the viewer treats an absent class as `primary`,
+  so every existing document renders exactly as before until the next
+  sync stamps it (the sync's self-heal pass, phase 163).
+- **The strip repairs itself (phase 163).** An unchanged PDF is not
+  assumed to still have its page images on disk. On every sync, a row
+  that claims a strip is re-gated deterministically from the current
+  file bytes and the current settings — the same gate the changed path
+  uses (`BOR_PDF_PICTORIAL` and `BOR_PDF_PICTORIAL_MIN_CHARS`
+  included), with **zero vision or LLM calls** (rendering is CPU-only):
+  the class is re-stamped, any `<doc-id>-p{N}.png` missing from
+  `BOR_PDF_PAGE_DIR` is re-rendered (write-before-delete, atomically per
+  file), and a strip that no longer qualifies — the kill switch turned
+  off, the sparse-page bar lowered past it — is cleared along with its
+  copies. So a wiped pages directory, or a copy set lost to an
+  interrupted sync, repairs itself on the next sync instead of leaving
+  "Page N unavailable" forever. It logs one `PDF strip re-gate …` line
+  only when it actually acts, and a failure there never fails the sync.
+  PDFs without a strip pay nothing extra (no extra read at all).
 
 ---
 
@@ -975,7 +1008,7 @@ to `.env` and adjust.
 | `BOR_PDF_PICTORIAL` | `1` | Master switch for the pictorial PDF pass (phase 140, on by default): a usable-text PDF's sparse pages (stripped text under `BOR_PDF_PICTORIAL_MIN_CHARS`) are rendered + vision-described into the indexed content (`[Page N illustration: …]`); `0` ⇒ content byte-identical to pre-phase-140 (no render, no vision call) |
 | `BOR_PDF_PICTORIAL_MIN_CHARS` | `100` | Sparseness bar for the pictorial pass (phase 140): a page whose STRIPPED text is STRICTLY shorter than this many chars is a candidate (a page with exactly this many is not); `0` = every page (A21); a negative value fails startup loudly |
 | `BOR_PDF_IMAGE_SKIP_BLANK` | `1` | Skip blank/decorative images embedded in PDFs (phase 151, on by default): near-white, nearly-uniform images (the table-shading "blank white" underlays) are dropped from the embedded-image pass — no row, no copy, no vision call; `0` ⇒ every extracted image is indexed again (pre-phase behavior; dedupe + soft-mask compositing unaffected) |
-| `BOR_PDF_PAGE_DIR` | `~/bor-sources/pages` | Persistent home for the rendered PDF page PNGs (phase 148): a page-qualifying PDF (the phase-133 screenshot class or the phase-140 pictorial class) gets every page rendered at index time as `<doc-id>-p{N}.png` here, so the document viewer shows the actual pages (the descriptions stay embedded for Q&A, just not displayed); text-heavy PDFs keep the text view |
+| `BOR_PDF_PAGE_DIR` | `~/bor-sources/pages` | Persistent home for the rendered PDF page PNGs (phase 148): a page-qualifying PDF (the phase-133 screenshot class or the phase-140 pictorial class) gets every page rendered at index time as `<doc-id>-p{N}.png` here, so the document viewer shows the actual pages (the descriptions stay embedded for Q&A, just not displayed); text-heavy PDFs keep the text view. Clearing the directory is self-healing — the next sync re-gates any strip that still claims pages and re-renders what is missing, with zero vision calls (phase 163) |
 | `BOR_GIT_SOURCES` | — (empty) | CSV of git repo URLs — **fallback while the admin Git sources page's list is empty** |
 | `BOR_SOURCES_DIR` | `~/bor-sources` | Where git repos are cloned/pulled |
 | `BOR_UPLOAD_DIR` | `~/bor-sources/uploads` | Where uploaded source archives are unpacked |