phase: 163_pdf_strip_additive_selfheal
Build and Push Containers / build-and-push-app (push) Successful in 2m3s
Build and Push Containers / build-and-push-db (push) Successful in 21s

All gates verified. Final report:

**Phase 163 — final verification pass (all 5 tasks already complete; verification-only run)**

- Verified D1–D4 in code: `page_strip_mode` column + migration 0027, `DocContent` absent-never-null rule keyed off `page_count`; `_persist_pdf_pages` render → atomic tmp+`os.replace` writes → stale-tail-last + `strip_class` stamp; `_regate_pdf_strip` on both unchanged paths (mirror gate, heal, repair, fail-soft, one INFO line only when acting); viewer mode branch + Text/Pages toggle (Text default, `aria-pressed`, no innerHTML, lazy imgs).
- `uv run pytest --cov=app --cov-report=term-missing` → **3796 tests, exit 0, coverage 99%** (>90%)
- `uv run pytest tests/e2e/test_pdf_strip_additive_selfheal.py -v --no-cov` → **5 passed** in isolation
- `test_pdf_pages_in_sources + test_pictorial_pdf + test_garbage_pdf` → 10 passed; `test_document_viewer(_context) + test_image_documents` → 20 passed
- `uv run ruff check .` clean; `uv run pyright` → 0 errors; alembic **0027 downgrade→upgrade CLI round-trip clean** (+ integration pins)
- Completion criteria: additive view w/ toggle ✅ · primary DOM byte-identity ✅ · write-first order/mid-write-failure/atomic pins ✅ · heal + repair E2E ✅ · zero vision/LLM on fast path (mock-captured) ✅ · non-qualifying payload byte-identity ✅ · docs pass ✅
- **Documentation**: README.md + AGENTS.md reviewed in full — the phase's updates are accurate; all config-table defaults cross-checked against `app/config.py` (no drift), every page route, script, and E2E command referenced exists, every `.env.example` var documented — **verified consistent, no changes needed**.
- Notable: live `Letumdei` repair is owner-runtime post-deploy per D5 (next production Sync self-heals; no live changes made); the modified `.agents/screenshots/doc_modal_scrollbar_1280.png` is an E2E-regenerated artifact left for the pipeline commit.
- **Next pending phase:** `164_chat_error_honesty`
This commit is contained in:
2026-10-07 13:57:30 -04:00
parent 74eb2e265c
commit 6ba46e735a
52 changed files with 4729 additions and 212 deletions
+112 -5
View File
@@ -30,6 +30,17 @@ page-qualifying PDF row (``page_count`` set) adds ``page_urls`` = the
exact 1-based ``/api/documents/<id>/pages/<i>`` list over
``1..page_count``.
Phase 163 (task 01) — the content payload's STRIP CLASS signal
(``page_strip_mode``: ``"primary"`` = the fallback class, the strip IS
the view; ``"additive"`` = the pictorial class, text + a Text/Pages
toggle): the model-level omission rule (ABSENT when ``None``, never
``null``), and the endpoint (stubbed session) — the key rides
``page_urls`` for a qualifying row, a PRE-PHASE row (``page_count`` set,
mode NULL — the exact live ``Letumdei`` state) serializes it ABSENT while
keeping ``page_urls``, and the whole strip block keys off
``page_count``, so a row with a stray mode and no strip carries neither
key (every non-qualifying payload byte-identical to pre-phase).
Phase 145 (task 03) — the content payload's git origin fields + the
``renderDocument`` "Original" anchor (modal AND /document.html, one
site):
@@ -316,10 +327,11 @@ def _text_doc_content(
origin_url: str | None = None,
origin_branch: str | None = None,
page_urls: list[str] | None = None,
page_strip_mode: str | None = None,
) -> DocContent:
"""One text doc's content payload (the pre-phase shape) with the
phase-145 origin fields + the phase-148 ``page_urls`` optionally
set."""
phase-145 origin fields + the phase-148 ``page_urls`` + the
phase-163 ``page_strip_mode`` optionally set."""
return DocContent(
source="S",
path="a.md",
@@ -332,6 +344,7 @@ def _text_doc_content(
origin_url=origin_url,
origin_branch=origin_branch,
page_urls=page_urls,
page_strip_mode=page_strip_mode,
)
@@ -367,9 +380,29 @@ def test_doc_content_page_urls_absent_on_the_wire_when_unset() -> None:
assert dumped["page_urls"] == urls # the exact 1-based route list
def _page_pdf_doc() -> Document:
def test_doc_content_page_strip_mode_absent_on_the_wire_when_unset() -> None:
"""Phase 163 (task 01, LOCKED A2) — the model-level omission rule
(the ``image_url`` / ``page_urls`` precedent): no strip class
(``page_strip_mode`` ``None`` — every non-qualifying doc AND every
pre-phase row) serializes the key ABSENT — never ``null`` — so those
payloads are byte-identical to pre-phase. A stamped row carries the
class verbatim, beside ``page_urls``."""
dumped = _text_doc_content().model_dump()
assert "page_strip_mode" not in dumped # absent — never null
for mode in ("primary", "additive"):
dumped = _text_doc_content(
page_urls=["/api/documents/d/pages/1"], page_strip_mode=mode
).model_dump()
assert dumped["page_strip_mode"] == mode # the class verbatim
def _page_pdf_doc(
page_count: int | None = 2, page_strip_mode: str | None = None
) -> Document:
"""A page-QUALIFYING PDF row (phase 148): ``page_count`` set — the
endpoint derives ``page_urls`` from it (the exact 1-based list)."""
endpoint derives ``page_urls`` from it (the exact 1-based list).
``page_strip_mode`` (phase 163) is the strip CLASS; ``None`` is the
PRE-PHASE row shape (the column NULL while the strip stands)."""
doc = Document(
id=uuid.uuid4(),
source="Homelab",
@@ -379,7 +412,8 @@ def _page_pdf_doc() -> Document:
content="[Page 1 illustration: the parts tray]…",
content_hash="e" * 64,
is_image=False, # phase 122: the stub session never applies defaults
page_count=2,
page_count=page_count,
page_strip_mode=page_strip_mode,
)
doc.indexed_at = datetime(2026, 8, 22, 1, 2, 3, tzinfo=UTC)
doc.created_at = datetime(2026, 8, 20, 9, 0, 0, tzinfo=UTC)
@@ -410,6 +444,79 @@ database): a page-qualifying PDF row (``page_count`` set) carries
] # the exact 1-based list, count == page_count
assert body["is_image"] is False
assert "image_url" not in body # absent — never null (not an image doc)
# Phase 163 (task 01): the pre-phase row (mode NULL) keeps the strip
# and serializes the class ABSENT — the viewer treats that as
# "primary", i.e. today's DOM.
assert "page_strip_mode" not in body
@pytest.mark.parametrize("mode", ["primary", "additive"])
def test_content_page_strip_mode_carried_for_a_stamped_strip(mode: str) -> None:
"""Phase 163 (task 01, LOCKED A2/A3) — the endpoint (stubbed
session, no
database): a page-qualifying PDF whose row carries the class
(``"primary"`` = the fallback class, ``"additive"`` = the pictorial
class) carries ``page_strip_mode`` VERBATIM next to ``page_urls`` —
the signal the viewer branches on so an additive strip never hides
the text (task 04). Nothing else in the payload moves."""
doc = _page_pdf_doc(page_strip_mode=mode)
with _client_with_row((doc, 4)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "ikea/besta.pdf"},
)
assert r.status_code == 200
body = r.json()
assert body["page_strip_mode"] == mode # the class verbatim
assert body["page_urls"] == [
f"/api/documents/{doc.id}/pages/1",
f"/api/documents/{doc.id}/pages/2",
] # the strip rides along, unchanged
def test_content_strip_block_keys_off_page_count_never_mode_alone() -> None:
"""Phase 163 (task 01) — the class never appears alone: the whole
strip block keys off ``page_count``, so a row whose column somehow
holds a mode while ``page_count`` is NULL serializes NEITHER
``page_strip_mode`` nor ``page_urls`` — the payload is byte-identical
to a plain non-qualifying document (there is no strip to show, and
the viewer must never be told a class for pages that do not exist)."""
doc = _page_pdf_doc(page_count=None, page_strip_mode="additive")
with _client_with_row((doc, 1)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "ikea/besta.pdf"},
)
assert r.status_code == 200
body = r.json()
assert "page_strip_mode" not in body # absent — the strip block is off
assert "page_urls" not in body # absent — never null
def test_content_non_qualifying_payload_key_set_is_byte_identical() -> None:
"""Phase 163 (task 01) — the byte-identity pin: a non-qualifying PDF
row (``page_count`` NULL, mode NULL) carries EXACTLY the pre-phase
key set — neither strip key joins it (``is_image`` always present,
``image_url`` / the origin keys / both strip keys absent)."""
doc = _page_pdf_doc(page_count=None)
with _client_with_row((doc, 1)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "ikea/besta.pdf"},
)
assert r.status_code == 200
assert set(r.json()) == {
"source",
"path",
"title",
"format",
"summary",
"created_at",
"content",
"indexed_at",
"chunks",
"is_image",
}
def test_content_origin_url_carried_for_a_matching_git_row() -> None: