phase: 163_pdf_strip_additive_selfheal
All gates verified. Final report: **Phase 163 — final verification pass (all 5 tasks already complete; verification-only run)** - Verified D1–D4 in code: `page_strip_mode` column + migration 0027, `DocContent` absent-never-null rule keyed off `page_count`; `_persist_pdf_pages` render → atomic tmp+`os.replace` writes → stale-tail-last + `strip_class` stamp; `_regate_pdf_strip` on both unchanged paths (mirror gate, heal, repair, fail-soft, one INFO line only when acting); viewer mode branch + Text/Pages toggle (Text default, `aria-pressed`, no innerHTML, lazy imgs). - `uv run pytest --cov=app --cov-report=term-missing` → **3796 tests, exit 0, coverage 99%** (>90%) - `uv run pytest tests/e2e/test_pdf_strip_additive_selfheal.py -v --no-cov` → **5 passed** in isolation - `test_pdf_pages_in_sources + test_pictorial_pdf + test_garbage_pdf` → 10 passed; `test_document_viewer(_context) + test_image_documents` → 20 passed - `uv run ruff check .` clean; `uv run pyright` → 0 errors; alembic **0027 downgrade→upgrade CLI round-trip clean** (+ integration pins) - Completion criteria: additive view w/ toggle ✅ · primary DOM byte-identity ✅ · write-first order/mid-write-failure/atomic pins ✅ · heal + repair E2E ✅ · zero vision/LLM on fast path (mock-captured) ✅ · non-qualifying payload byte-identity ✅ · docs pass ✅ - **Documentation**: README.md + AGENTS.md reviewed in full — the phase's updates are accurate; all config-table defaults cross-checked against `app/config.py` (no drift), every page route, script, and E2E command referenced exists, every `.env.example` var documented — **verified consistent, no changes needed**. - Notable: live `Letumdei` repair is owner-runtime post-deploy per D5 (next production Sync self-heals; no live changes made); the modified `.agents/screenshots/doc_modal_scrollbar_1280.png` is an E2E-regenerated artifact left for the pipeline commit. - **Next pending phase:** `164_chat_error_honesty`
This commit is contained in:
@@ -30,6 +30,17 @@ page-qualifying PDF row (``page_count`` set) adds ``page_urls`` = the
|
||||
exact 1-based ``/api/documents/<id>/pages/<i>`` list over
|
||||
``1..page_count``.
|
||||
|
||||
Phase 163 (task 01) — the content payload's STRIP CLASS signal
|
||||
(``page_strip_mode``: ``"primary"`` = the fallback class, the strip IS
|
||||
the view; ``"additive"`` = the pictorial class, text + a Text/Pages
|
||||
toggle): the model-level omission rule (ABSENT when ``None``, never
|
||||
``null``), and the endpoint (stubbed session) — the key rides
|
||||
``page_urls`` for a qualifying row, a PRE-PHASE row (``page_count`` set,
|
||||
mode NULL — the exact live ``Letumdei`` state) serializes it ABSENT while
|
||||
keeping ``page_urls``, and the whole strip block keys off
|
||||
``page_count``, so a row with a stray mode and no strip carries neither
|
||||
key (every non-qualifying payload byte-identical to pre-phase).
|
||||
|
||||
Phase 145 (task 03) — the content payload's git origin fields + the
|
||||
``renderDocument`` "Original" anchor (modal AND /document.html, one
|
||||
site):
|
||||
@@ -316,10 +327,11 @@ def _text_doc_content(
|
||||
origin_url: str | None = None,
|
||||
origin_branch: str | None = None,
|
||||
page_urls: list[str] | None = None,
|
||||
page_strip_mode: str | None = None,
|
||||
) -> DocContent:
|
||||
"""One text doc's content payload (the pre-phase shape) with the
|
||||
phase-145 origin fields + the phase-148 ``page_urls`` optionally
|
||||
set."""
|
||||
phase-145 origin fields + the phase-148 ``page_urls`` + the
|
||||
phase-163 ``page_strip_mode`` optionally set."""
|
||||
return DocContent(
|
||||
source="S",
|
||||
path="a.md",
|
||||
@@ -332,6 +344,7 @@ def _text_doc_content(
|
||||
origin_url=origin_url,
|
||||
origin_branch=origin_branch,
|
||||
page_urls=page_urls,
|
||||
page_strip_mode=page_strip_mode,
|
||||
)
|
||||
|
||||
|
||||
@@ -367,9 +380,29 @@ def test_doc_content_page_urls_absent_on_the_wire_when_unset() -> None:
|
||||
assert dumped["page_urls"] == urls # the exact 1-based route list
|
||||
|
||||
|
||||
def _page_pdf_doc() -> Document:
|
||||
def test_doc_content_page_strip_mode_absent_on_the_wire_when_unset() -> None:
|
||||
"""Phase 163 (task 01, LOCKED A2) — the model-level omission rule
|
||||
(the ``image_url`` / ``page_urls`` precedent): no strip class
|
||||
(``page_strip_mode`` ``None`` — every non-qualifying doc AND every
|
||||
pre-phase row) serializes the key ABSENT — never ``null`` — so those
|
||||
payloads are byte-identical to pre-phase. A stamped row carries the
|
||||
class verbatim, beside ``page_urls``."""
|
||||
dumped = _text_doc_content().model_dump()
|
||||
assert "page_strip_mode" not in dumped # absent — never null
|
||||
for mode in ("primary", "additive"):
|
||||
dumped = _text_doc_content(
|
||||
page_urls=["/api/documents/d/pages/1"], page_strip_mode=mode
|
||||
).model_dump()
|
||||
assert dumped["page_strip_mode"] == mode # the class verbatim
|
||||
|
||||
|
||||
def _page_pdf_doc(
|
||||
page_count: int | None = 2, page_strip_mode: str | None = None
|
||||
) -> Document:
|
||||
"""A page-QUALIFYING PDF row (phase 148): ``page_count`` set — the
|
||||
endpoint derives ``page_urls`` from it (the exact 1-based list)."""
|
||||
endpoint derives ``page_urls`` from it (the exact 1-based list).
|
||||
``page_strip_mode`` (phase 163) is the strip CLASS; ``None`` is the
|
||||
PRE-PHASE row shape (the column NULL while the strip stands)."""
|
||||
doc = Document(
|
||||
id=uuid.uuid4(),
|
||||
source="Homelab",
|
||||
@@ -379,7 +412,8 @@ def _page_pdf_doc() -> Document:
|
||||
content="[Page 1 illustration: the parts tray]…",
|
||||
content_hash="e" * 64,
|
||||
is_image=False, # phase 122: the stub session never applies defaults
|
||||
page_count=2,
|
||||
page_count=page_count,
|
||||
page_strip_mode=page_strip_mode,
|
||||
)
|
||||
doc.indexed_at = datetime(2026, 8, 22, 1, 2, 3, tzinfo=UTC)
|
||||
doc.created_at = datetime(2026, 8, 20, 9, 0, 0, tzinfo=UTC)
|
||||
@@ -410,6 +444,79 @@ database): a page-qualifying PDF row (``page_count`` set) carries
|
||||
] # the exact 1-based list, count == page_count
|
||||
assert body["is_image"] is False
|
||||
assert "image_url" not in body # absent — never null (not an image doc)
|
||||
# Phase 163 (task 01): the pre-phase row (mode NULL) keeps the strip
|
||||
# and serializes the class ABSENT — the viewer treats that as
|
||||
# "primary", i.e. today's DOM.
|
||||
assert "page_strip_mode" not in body
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", ["primary", "additive"])
|
||||
def test_content_page_strip_mode_carried_for_a_stamped_strip(mode: str) -> None:
|
||||
"""Phase 163 (task 01, LOCKED A2/A3) — the endpoint (stubbed
|
||||
session, no
|
||||
database): a page-qualifying PDF whose row carries the class
|
||||
(``"primary"`` = the fallback class, ``"additive"`` = the pictorial
|
||||
class) carries ``page_strip_mode`` VERBATIM next to ``page_urls`` —
|
||||
the signal the viewer branches on so an additive strip never hides
|
||||
the text (task 04). Nothing else in the payload moves."""
|
||||
doc = _page_pdf_doc(page_strip_mode=mode)
|
||||
with _client_with_row((doc, 4)) as client:
|
||||
r = client.get(
|
||||
"/api/documents/content",
|
||||
params={"source": "Homelab", "path": "ikea/besta.pdf"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["page_strip_mode"] == mode # the class verbatim
|
||||
assert body["page_urls"] == [
|
||||
f"/api/documents/{doc.id}/pages/1",
|
||||
f"/api/documents/{doc.id}/pages/2",
|
||||
] # the strip rides along, unchanged
|
||||
|
||||
|
||||
def test_content_strip_block_keys_off_page_count_never_mode_alone() -> None:
|
||||
"""Phase 163 (task 01) — the class never appears alone: the whole
|
||||
strip block keys off ``page_count``, so a row whose column somehow
|
||||
holds a mode while ``page_count`` is NULL serializes NEITHER
|
||||
``page_strip_mode`` nor ``page_urls`` — the payload is byte-identical
|
||||
to a plain non-qualifying document (there is no strip to show, and
|
||||
the viewer must never be told a class for pages that do not exist)."""
|
||||
doc = _page_pdf_doc(page_count=None, page_strip_mode="additive")
|
||||
with _client_with_row((doc, 1)) as client:
|
||||
r = client.get(
|
||||
"/api/documents/content",
|
||||
params={"source": "Homelab", "path": "ikea/besta.pdf"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert "page_strip_mode" not in body # absent — the strip block is off
|
||||
assert "page_urls" not in body # absent — never null
|
||||
|
||||
|
||||
def test_content_non_qualifying_payload_key_set_is_byte_identical() -> None:
|
||||
"""Phase 163 (task 01) — the byte-identity pin: a non-qualifying PDF
|
||||
row (``page_count`` NULL, mode NULL) carries EXACTLY the pre-phase
|
||||
key set — neither strip key joins it (``is_image`` always present,
|
||||
``image_url`` / the origin keys / both strip keys absent)."""
|
||||
doc = _page_pdf_doc(page_count=None)
|
||||
with _client_with_row((doc, 1)) as client:
|
||||
r = client.get(
|
||||
"/api/documents/content",
|
||||
params={"source": "Homelab", "path": "ikea/besta.pdf"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert set(r.json()) == {
|
||||
"source",
|
||||
"path",
|
||||
"title",
|
||||
"format",
|
||||
"summary",
|
||||
"created_at",
|
||||
"content",
|
||||
"indexed_at",
|
||||
"chunks",
|
||||
"is_image",
|
||||
}
|
||||
|
||||
|
||||
def test_content_origin_url_carried_for_a_matching_git_row() -> None:
|
||||
|
||||
Reference in New Issue
Block a user