phase: 163_pdf_strip_additive_selfheal
Build and Push Containers / build-and-push-app (push) Successful in 2m3s
Build and Push Containers / build-and-push-db (push) Successful in 21s

All gates verified. Final report:

**Phase 163 — final verification pass (all 5 tasks already complete; verification-only run)**

- Verified D1–D4 in code: `page_strip_mode` column + migration 0027, `DocContent` absent-never-null rule keyed off `page_count`; `_persist_pdf_pages` render → atomic tmp+`os.replace` writes → stale-tail-last + `strip_class` stamp; `_regate_pdf_strip` on both unchanged paths (mirror gate, heal, repair, fail-soft, one INFO line only when acting); viewer mode branch + Text/Pages toggle (Text default, `aria-pressed`, no innerHTML, lazy imgs).
- `uv run pytest --cov=app --cov-report=term-missing` → **3796 tests, exit 0, coverage 99%** (>90%)
- `uv run pytest tests/e2e/test_pdf_strip_additive_selfheal.py -v --no-cov` → **5 passed** in isolation
- `test_pdf_pages_in_sources + test_pictorial_pdf + test_garbage_pdf` → 10 passed; `test_document_viewer(_context) + test_image_documents` → 20 passed
- `uv run ruff check .` clean; `uv run pyright` → 0 errors; alembic **0027 downgrade→upgrade CLI round-trip clean** (+ integration pins)
- Completion criteria: additive view w/ toggle ✅ · primary DOM byte-identity ✅ · write-first order/mid-write-failure/atomic pins ✅ · heal + repair E2E ✅ · zero vision/LLM on fast path (mock-captured) ✅ · non-qualifying payload byte-identity ✅ · docs pass ✅
- **Documentation**: README.md + AGENTS.md reviewed in full — the phase's updates are accurate; all config-table defaults cross-checked against `app/config.py` (no drift), every page route, script, and E2E command referenced exists, every `.env.example` var documented — **verified consistent, no changes needed**.
- Notable: live `Letumdei` repair is owner-runtime post-deploy per D5 (next production Sync self-heals; no live changes made); the modified `.agents/screenshots/doc_modal_scrollbar_1280.png` is an E2E-regenerated artifact left for the pipeline commit.
- **Next pending phase:** `164_chat_error_honesty`
This commit is contained in:
2026-10-07 13:57:30 -04:00
parent 74eb2e265c
commit 6ba46e735a
52 changed files with 4729 additions and 212 deletions
+64 -44
View File
@@ -24,11 +24,14 @@ format — the upload gate admits it) and arriving through the REAL admin
upload flow → the RAG page's Sync button:
* (a) the document appears on the Sources page and the document viewer
shows the PDF's ACTUAL pages (phase 148 — a described sparse page
makes the doc page-QUALIFYING: the page strip REPLACES the text
content AND the Summary panel, the owner's TODO L5 contract): two
``.doc-page`` figures with loaded lazy images + ``Page N`` captions,
NO ``.doc-raw``, NO ``.doc-summary``; the described content itself
opens the PDF in its ADDITIVE view (phase 163 — a described sparse
page makes the doc page-QUALIFYING, and a usable text layer makes
the strip ADDITIVE: the text stays, the pages sit behind a
Text/Pages toggle — LOCKED A1/A6): the Text view first (``.doc-raw``
+ ``.doc-summary`` visible, ``Text`` pressed), the strip one
``Pages`` click away — two ``.doc-page`` figures with loaded lazy
images + ``Page N`` captions, the text view hidden while it shows;
the described content itself
(page 1's prose WITHOUT an illustration block, page 2 WITH
``[Page 2 illustration: <the mock's canned PICTORIAL_PAGE_ANSWER>]``
— the task-02 ``PICTORIAL_PAGE_MODE`` branch, deterministic) stays in
@@ -83,14 +86,17 @@ Test → observable mapping (Playwright Mapping Rule):
Sources page reads 1 doc / 3 chunks (2 page-anchored content chunks +
the summary chunk — no image doc), the drill-down table lists the PDF
as a plain text row (LOCKED A4 — a text document, no thumbnail), and
the same-page document viewer (the row's path link) shows the
PHASE-148 PAGE STRIP under the METADATA title (the fixture is
page-qualifying — the pictorial pass described its sparse page):
exactly two ``.doc-page`` figures, each a LOADED lazy ``<img>``
(naturalWidth > 0) with the ``Page N of <title>`` alt + the
``Page N`` caption, NO ``.doc-raw`` / NO ``.doc-summary`` (the
page descriptions stay in the DB for Q&A, not displayed — the
owner's contract).
the same-page document viewer (the row's path link) opens the
PHASE-163 ADDITIVE VIEW under the METADATA title (the fixture is
page-qualifying with a usable text layer — the pictorial pass
described its sparse page): the TEXT view first — ``.doc-raw`` +
``.doc-summary`` visible with the ``[Page 2 illustration: …]``
block among them, the toggle's ``Text`` pressed, the strip mounted
but hidden; pressing ``Pages`` shows exactly two ``.doc-page``
figures, each a LOADED lazy ``<img>`` (naturalWidth > 0) with the
``Page N of <title>`` alt + the ``Page N`` caption while the text
view hides (mounted for the flip-back — the text is never
destroyed).
2. ``test_chat_question_grounds_and_echoes_the_illustration`` — the raw
SSE over the scripted seeded-read question (httpx — the ``done``
event is the wire lens the task pins; the UI flow's premise is the
@@ -578,8 +584,8 @@ def seeded(
)
# The non-markdown doc got its lite summary (the mock's
# SUMMARY_MODE digest) — the summary chunk exists (the
# viewer's Summary panel is suppressed by the phase-148
# strip — the summary itself stays in the DB).
# additive view shows the Summary panel; the summary
# itself stays in the DB regardless).
assert doc.summary
# Phase 148: the described sparse page makes this doc
# page-QUALIFYING — the strip's payload (page_urls) rides
@@ -602,18 +608,19 @@ def test_sources_page_and_viewer_show_the_described_page(
content chunks + the phase-30 summary chunk — no image doc; the
vector figure is invisible to the embedded-image pipeline). The
drill-down table lists the PDF as a plain text row (LOCKED A4), and
the same-page document viewer (the row's path link, phase 26) shows
the PHASE-148 PAGE STRIP under the METADATA title: the fixture is
page-qualifying (the pictorial pass described its sparse page), so
the strip REPLACES the text content AND the Summary panel — two
``.doc-page`` figures, each a loaded lazy ``<img>`` with the
``Page N of <title>`` alt + the ``Page N`` caption, NO
``.doc-raw``, NO ``.doc-summary``. The described content (page 1's
prose WITHOUT an illustration block, page 2 WITH
``[Page 2 illustration: <canned>]`` — LOCKED A4) stays in the DB
for Q&A: the seed asserted it byte-exact, the chat test proves it
reaches the model — it is simply not displayed (the owner's
contract)."""
the same-page document viewer (the row's path link, phase 26)
opens the PHASE-163 ADDITIVE VIEW (LOCKED A1/A6): the TEXT view
first — Summary panel + ``.doc-raw`` visible, the Text/Pages
toggle's ``Text`` pressed — with the ``[Page 2 illustration:
<canned>]`` block riding the SHOWN text (the text is never hidden
for the pictorial class). Pressing ``Pages`` reveals the strip —
two ``.doc-page`` figures, each a loaded lazy ``<img>`` with the
``Page N of <title>`` alt + the ``Page N`` caption — while the
text view is hidden (mounted, not removed: flipping back shows it
with zero refetch). The described content (page 1's prose WITHOUT
an illustration block, page 2 WITH the block — LOCKED A4) also
stays in the DB for Q&A: the seed asserted it byte-exact, the chat
test proves it reaches the model."""
login(page, app_url) # lands on /sources.html
expect(page.locator("#stat-docs")).to_have_text("1")
expect(page.locator("#stat-chunks")).to_have_text("3")
@@ -630,13 +637,31 @@ def test_sources_page_and_viewer_show_the_described_page(
expect(modal).to_be_visible(timeout=30_000)
expect(page.locator("#doc-modal-title")).to_have_text(PDF_TITLE)
# Phase 148: the page-qualifying fixture shows its ACTUAL pages —
# a .doc-pages strip with exactly two .doc-page figures (the
# 2-page fixture), each a LOADED lazy <img> (naturalWidth > 0 —
# the route served a real PNG) with the "Page N of <title>" alt +
# the "Page N" caption. The text content AND the Summary panel are
# SUPPRESSED (the owner's contract — the descriptions stay in the
# DB for Q&A, not displayed).
# Phase 163 (task 04, LOCKED A1/A6): the page-qualifying fixture
# with a usable text layer is an ADDITIVE doc — it opens on its
# TEXT view: Summary panel + content visible, Text pressed, the
# strip mounted but hidden (its lazy imgs never fetched).
toggle = modal.locator(".doc-view-toggle")
expect(toggle).to_have_count(1)
expect(toggle.get_by_role("button", name="Text")).to_have_attribute(
"aria-pressed", "true"
)
expect(toggle.get_by_role("button", name="Pages")).to_have_attribute(
"aria-pressed", "false"
)
expect(modal.locator("pre.doc-raw")).to_be_visible()
expect(modal.locator(".doc-summary")).to_be_visible()
expect(modal.locator(".doc-pages")).to_be_hidden()
# The canned page description RIDES the shown text (phase-140
# blocks are part of the content — the additive view never hides
# the text). The node stays in the DOM, so a visibility-scoped
# inner_text() check would miss it — use the DOM text.
assert PICTORIAL_PAGE_ANSWER in modal.evaluate("el => el.textContent"), (
"the additive text view shows the indexed content including "
"the described sparse page"
)
# …and the strip is one Pages click away (phase 148's ask):
toggle.get_by_role("button", name="Pages").click()
strip = modal.locator(".doc-pages")
expect(strip).to_have_count(1)
figures = strip.locator(".doc-page")
@@ -657,17 +682,12 @@ def test_sources_page_and_viewer_show_the_described_page(
expect(figures.nth(i).locator("figcaption.doc-page-num")).to_have_text(
f"Page {n}"
)
# The suppression contract: no text content block, no Summary
# panel, no image block (the PDF is not an image document).
expect(modal.locator("pre.doc-raw")).to_have_count(0)
expect(modal.locator(".doc-summary")).to_have_count(0)
# While the strip shows, the text view is HIDDEN — mounted for
# the flip back (LOCKED A1), never removed. No image block (the
# PDF is not an image document).
expect(modal.locator("pre.doc-raw")).not_to_be_visible()
expect(modal.locator(".doc-summary")).not_to_be_visible()
expect(modal.locator(".doc-image-img")).to_have_count(0)
# The canned description is in the DB (the seed asserted it)
# but nowhere in the rendered modal.
assert PICTORIAL_PAGE_ANSWER not in modal.inner_text(), (
"the described page's text is in the DB for Q&A — it must not "
"be shown to the user (the owner's contract)"
)
# ---------------------------------------------------------------------------