phase: 163_pdf_strip_additive_selfheal
All gates verified. Final report: **Phase 163 — final verification pass (all 5 tasks already complete; verification-only run)** - Verified D1–D4 in code: `page_strip_mode` column + migration 0027, `DocContent` absent-never-null rule keyed off `page_count`; `_persist_pdf_pages` render → atomic tmp+`os.replace` writes → stale-tail-last + `strip_class` stamp; `_regate_pdf_strip` on both unchanged paths (mirror gate, heal, repair, fail-soft, one INFO line only when acting); viewer mode branch + Text/Pages toggle (Text default, `aria-pressed`, no innerHTML, lazy imgs). - `uv run pytest --cov=app --cov-report=term-missing` → **3796 tests, exit 0, coverage 99%** (>90%) - `uv run pytest tests/e2e/test_pdf_strip_additive_selfheal.py -v --no-cov` → **5 passed** in isolation - `test_pdf_pages_in_sources + test_pictorial_pdf + test_garbage_pdf` → 10 passed; `test_document_viewer(_context) + test_image_documents` → 20 passed - `uv run ruff check .` clean; `uv run pyright` → 0 errors; alembic **0027 downgrade→upgrade CLI round-trip clean** (+ integration pins) - Completion criteria: additive view w/ toggle ✅ · primary DOM byte-identity ✅ · write-first order/mid-write-failure/atomic pins ✅ · heal + repair E2E ✅ · zero vision/LLM on fast path (mock-captured) ✅ · non-qualifying payload byte-identity ✅ · docs pass ✅ - **Documentation**: README.md + AGENTS.md reviewed in full — the phase's updates are accurate; all config-table defaults cross-checked against `app/config.py` (no drift), every page route, script, and E2E command referenced exists, every `.env.example` var documented — **verified consistent, no changes needed**. - Notable: live `Letumdei` repair is owner-runtime post-deploy per D5 (next production Sync self-heals; no live changes made); the modified `.agents/screenshots/doc_modal_scrollbar_1280.png` is an E2E-regenerated artifact left for the pipeline commit. - **Next pending phase:** `164_chat_error_honesty`
This commit is contained in:
@@ -24,11 +24,14 @@ format — the upload gate admits it) and arriving through the REAL admin
|
||||
upload flow → the RAG page's Sync button:
|
||||
|
||||
* (a) the document appears on the Sources page and the document viewer
|
||||
shows the PDF's ACTUAL pages (phase 148 — a described sparse page
|
||||
makes the doc page-QUALIFYING: the page strip REPLACES the text
|
||||
content AND the Summary panel, the owner's TODO L5 contract): two
|
||||
``.doc-page`` figures with loaded lazy images + ``Page N`` captions,
|
||||
NO ``.doc-raw``, NO ``.doc-summary``; the described content itself
|
||||
opens the PDF in its ADDITIVE view (phase 163 — a described sparse
|
||||
page makes the doc page-QUALIFYING, and a usable text layer makes
|
||||
the strip ADDITIVE: the text stays, the pages sit behind a
|
||||
Text/Pages toggle — LOCKED A1/A6): the Text view first (``.doc-raw``
|
||||
+ ``.doc-summary`` visible, ``Text`` pressed), the strip one
|
||||
``Pages`` click away — two ``.doc-page`` figures with loaded lazy
|
||||
images + ``Page N`` captions, the text view hidden while it shows;
|
||||
the described content itself
|
||||
(page 1's prose WITHOUT an illustration block, page 2 WITH
|
||||
``[Page 2 illustration: <the mock's canned PICTORIAL_PAGE_ANSWER>]``
|
||||
— the task-02 ``PICTORIAL_PAGE_MODE`` branch, deterministic) stays in
|
||||
@@ -83,14 +86,17 @@ Test → observable mapping (Playwright Mapping Rule):
|
||||
Sources page reads 1 doc / 3 chunks (2 page-anchored content chunks +
|
||||
the summary chunk — no image doc), the drill-down table lists the PDF
|
||||
as a plain text row (LOCKED A4 — a text document, no thumbnail), and
|
||||
the same-page document viewer (the row's path link) shows the
|
||||
PHASE-148 PAGE STRIP under the METADATA title (the fixture is
|
||||
page-qualifying — the pictorial pass described its sparse page):
|
||||
exactly two ``.doc-page`` figures, each a LOADED lazy ``<img>``
|
||||
(naturalWidth > 0) with the ``Page N of <title>`` alt + the
|
||||
``Page N`` caption, NO ``.doc-raw`` / NO ``.doc-summary`` (the
|
||||
page descriptions stay in the DB for Q&A, not displayed — the
|
||||
owner's contract).
|
||||
the same-page document viewer (the row's path link) opens the
|
||||
PHASE-163 ADDITIVE VIEW under the METADATA title (the fixture is
|
||||
page-qualifying with a usable text layer — the pictorial pass
|
||||
described its sparse page): the TEXT view first — ``.doc-raw`` +
|
||||
``.doc-summary`` visible with the ``[Page 2 illustration: …]``
|
||||
block among them, the toggle's ``Text`` pressed, the strip mounted
|
||||
but hidden; pressing ``Pages`` shows exactly two ``.doc-page``
|
||||
figures, each a LOADED lazy ``<img>`` (naturalWidth > 0) with the
|
||||
``Page N of <title>`` alt + the ``Page N`` caption while the text
|
||||
view hides (mounted for the flip-back — the text is never
|
||||
destroyed).
|
||||
2. ``test_chat_question_grounds_and_echoes_the_illustration`` — the raw
|
||||
SSE over the scripted seeded-read question (httpx — the ``done``
|
||||
event is the wire lens the task pins; the UI flow's premise is the
|
||||
@@ -578,8 +584,8 @@ def seeded(
|
||||
)
|
||||
# The non-markdown doc got its lite summary (the mock's
|
||||
# SUMMARY_MODE digest) — the summary chunk exists (the
|
||||
# viewer's Summary panel is suppressed by the phase-148
|
||||
# strip — the summary itself stays in the DB).
|
||||
# additive view shows the Summary panel; the summary
|
||||
# itself stays in the DB regardless).
|
||||
assert doc.summary
|
||||
# Phase 148: the described sparse page makes this doc
|
||||
# page-QUALIFYING — the strip's payload (page_urls) rides
|
||||
@@ -602,18 +608,19 @@ def test_sources_page_and_viewer_show_the_described_page(
|
||||
content chunks + the phase-30 summary chunk — no image doc; the
|
||||
vector figure is invisible to the embedded-image pipeline). The
|
||||
drill-down table lists the PDF as a plain text row (LOCKED A4), and
|
||||
the same-page document viewer (the row's path link, phase 26) shows
|
||||
the PHASE-148 PAGE STRIP under the METADATA title: the fixture is
|
||||
page-qualifying (the pictorial pass described its sparse page), so
|
||||
the strip REPLACES the text content AND the Summary panel — two
|
||||
``.doc-page`` figures, each a loaded lazy ``<img>`` with the
|
||||
``Page N of <title>`` alt + the ``Page N`` caption, NO
|
||||
``.doc-raw``, NO ``.doc-summary``. The described content (page 1's
|
||||
prose WITHOUT an illustration block, page 2 WITH
|
||||
``[Page 2 illustration: <canned>]`` — LOCKED A4) stays in the DB
|
||||
for Q&A: the seed asserted it byte-exact, the chat test proves it
|
||||
reaches the model — it is simply not displayed (the owner's
|
||||
contract)."""
|
||||
the same-page document viewer (the row's path link, phase 26)
|
||||
opens the PHASE-163 ADDITIVE VIEW (LOCKED A1/A6): the TEXT view
|
||||
first — Summary panel + ``.doc-raw`` visible, the Text/Pages
|
||||
toggle's ``Text`` pressed — with the ``[Page 2 illustration:
|
||||
<canned>]`` block riding the SHOWN text (the text is never hidden
|
||||
for the pictorial class). Pressing ``Pages`` reveals the strip —
|
||||
two ``.doc-page`` figures, each a loaded lazy ``<img>`` with the
|
||||
``Page N of <title>`` alt + the ``Page N`` caption — while the
|
||||
text view is hidden (mounted, not removed: flipping back shows it
|
||||
with zero refetch). The described content (page 1's prose WITHOUT
|
||||
an illustration block, page 2 WITH the block — LOCKED A4) also
|
||||
stays in the DB for Q&A: the seed asserted it byte-exact, the chat
|
||||
test proves it reaches the model."""
|
||||
login(page, app_url) # lands on /sources.html
|
||||
expect(page.locator("#stat-docs")).to_have_text("1")
|
||||
expect(page.locator("#stat-chunks")).to_have_text("3")
|
||||
@@ -630,13 +637,31 @@ def test_sources_page_and_viewer_show_the_described_page(
|
||||
expect(modal).to_be_visible(timeout=30_000)
|
||||
expect(page.locator("#doc-modal-title")).to_have_text(PDF_TITLE)
|
||||
|
||||
# Phase 148: the page-qualifying fixture shows its ACTUAL pages —
|
||||
# a .doc-pages strip with exactly two .doc-page figures (the
|
||||
# 2-page fixture), each a LOADED lazy <img> (naturalWidth > 0 —
|
||||
# the route served a real PNG) with the "Page N of <title>" alt +
|
||||
# the "Page N" caption. The text content AND the Summary panel are
|
||||
# SUPPRESSED (the owner's contract — the descriptions stay in the
|
||||
# DB for Q&A, not displayed).
|
||||
# Phase 163 (task 04, LOCKED A1/A6): the page-qualifying fixture
|
||||
# with a usable text layer is an ADDITIVE doc — it opens on its
|
||||
# TEXT view: Summary panel + content visible, Text pressed, the
|
||||
# strip mounted but hidden (its lazy imgs never fetched).
|
||||
toggle = modal.locator(".doc-view-toggle")
|
||||
expect(toggle).to_have_count(1)
|
||||
expect(toggle.get_by_role("button", name="Text")).to_have_attribute(
|
||||
"aria-pressed", "true"
|
||||
)
|
||||
expect(toggle.get_by_role("button", name="Pages")).to_have_attribute(
|
||||
"aria-pressed", "false"
|
||||
)
|
||||
expect(modal.locator("pre.doc-raw")).to_be_visible()
|
||||
expect(modal.locator(".doc-summary")).to_be_visible()
|
||||
expect(modal.locator(".doc-pages")).to_be_hidden()
|
||||
# The canned page description RIDES the shown text (phase-140
|
||||
# blocks are part of the content — the additive view never hides
|
||||
# the text). The node stays in the DOM, so a visibility-scoped
|
||||
# inner_text() check would miss it — use the DOM text.
|
||||
assert PICTORIAL_PAGE_ANSWER in modal.evaluate("el => el.textContent"), (
|
||||
"the additive text view shows the indexed content including "
|
||||
"the described sparse page"
|
||||
)
|
||||
# …and the strip is one Pages click away (phase 148's ask):
|
||||
toggle.get_by_role("button", name="Pages").click()
|
||||
strip = modal.locator(".doc-pages")
|
||||
expect(strip).to_have_count(1)
|
||||
figures = strip.locator(".doc-page")
|
||||
@@ -657,17 +682,12 @@ def test_sources_page_and_viewer_show_the_described_page(
|
||||
expect(figures.nth(i).locator("figcaption.doc-page-num")).to_have_text(
|
||||
f"Page {n}"
|
||||
)
|
||||
# The suppression contract: no text content block, no Summary
|
||||
# panel, no image block (the PDF is not an image document).
|
||||
expect(modal.locator("pre.doc-raw")).to_have_count(0)
|
||||
expect(modal.locator(".doc-summary")).to_have_count(0)
|
||||
# While the strip shows, the text view is HIDDEN — mounted for
|
||||
# the flip back (LOCKED A1), never removed. No image block (the
|
||||
# PDF is not an image document).
|
||||
expect(modal.locator("pre.doc-raw")).not_to_be_visible()
|
||||
expect(modal.locator(".doc-summary")).not_to_be_visible()
|
||||
expect(modal.locator(".doc-image-img")).to_have_count(0)
|
||||
# The canned description is in the DB (the seed asserted it)
|
||||
# but nowhere in the rendered modal.
|
||||
assert PICTORIAL_PAGE_ANSWER not in modal.inner_text(), (
|
||||
"the described page's text is in the DB for Q&A — it must not "
|
||||
"be shown to the user (the owner's contract)"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user