phase: 147_sync_changed_only
All defaults match. Verification complete — everything is green with no defects found. **Phase 147 final verification pass — report** - All 3 task files found in `complete/147_sync_changed_only/` → verification pass only; no code changes needed. - Verified: migration 0025 (`documents.file_hash` nullable String(64), no index) + model field; fast path at top of `_index_file` (streamed 1 MiB sha256 → row lookup → `_unchanged_doc`); stamping on all three branches' create/update + shared helper; pre-phase NULL-hash fallthrough; retriever/`content_hash` untouched. - **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3438 passed, `app/` coverage **99%** (>90% ✓); `uv run pytest tests/e2e/test_sync_changed_only.py -v --no-cov` → 3 passed in isolation ✓; `uv run ruff check . && uv run pyright` → all checks passed, 0 errors ✓; prior-phase `tests/e2e/test_sync_button.py` → 3 passed (no regression) ✓. - **Completion criteria:** all 7 pass — (1) column + round-trip ✓ (2) unchanged text/image/garbage-PDF/pictorial-PDF cost zero LLM/extraction/vision, `summary.unchanged` counted, rows byte-still ✓ (3) changed file re-indexes + re-stamps; bytes-changed/content-same stamps only ✓ (4) backfill/D1/D4 + log wordings pinned verbatim ✓ (5) `content_hash`/retriever byte-identical ✓ (6) all gates green ✓ (7) no prior-phase behavior change ✓. - **Documentation:** full review of README.md (1,031 lines) and AGENTS.md against the code: every `.env.example`/code `BOR_*` var documented, all config defaults match `app/config.py`, quick-start/QA commands and referenced test files exist, `file://` URL + troubleshooting claims accurate; the phase's README "Updating the documents" edit is accurate. **Verified consistent — no changes needed.** - **Notable:** no live-infrastructure changes (dev Postgres container used for the standard test gates only). Working tree left uncommitted per harness protocol. - **Next pending phase:** `148_pdf_pages_in_sources`.
This commit is contained in:
@@ -28,11 +28,12 @@ fixtures; the real tar is pinned in
|
||||
others are described), the unreadable-page placeholder skip (the
|
||||
zero-byte ``render_pdf_pages`` slot burns no vision call, the page
|
||||
keeps its text only).
|
||||
* the digest-stability rerun — a stable file + a stable vision reply
|
||||
re-syncs as ``unchanged`` (no re-embed, byte-stable content +
|
||||
digest); the vision calls are re-issued before the unchanged check
|
||||
can skip them — the documented no-cache cost (the phase-133
|
||||
convention, A21 endorses it).
|
||||
* the phase-147 rerun — an unchanged file re-syncs through the FAST
|
||||
PATH (the raw bytes are the change signal): ``unchanged``, no
|
||||
re-embed, byte-stable content + digest — and the vision calls are
|
||||
NOT re-issued (the pass never runs on an unchanged file; the
|
||||
pre-phase "vision re-issued before the unchanged check" no-cache
|
||||
cost is gone).
|
||||
|
||||
The E2E surface (the real UI + the mock LLM's ``PICTORIAL_PAGE_MODE``
|
||||
branch) is pinned in ``tests/e2e/test_pictorial_pdf.py`` (task 04).
|
||||
@@ -565,18 +566,22 @@ def test_unreadable_page_placeholder_skips_the_vision_call(
|
||||
_cleanup_source(db, "halfblind")
|
||||
|
||||
|
||||
def test_rerun_lands_unchanged_with_byte_stable_content(
|
||||
def test_rerun_lands_unchanged_fast_path_no_revision(
|
||||
db, tmp_path: Path
|
||||
) -> None:
|
||||
"""Digest stability (A4 — the phase-126/133 convention): the
|
||||
digest runs over the FINAL content, so a re-sync of an unchanged
|
||||
IKEA-class PDF (same pages + the stable fake's same replies)
|
||||
re-imports as UNCHANGED: no re-chunk, no re-embed, no re-summary,
|
||||
no KB change (not ``updated``, no changed path), and the content +
|
||||
digest are byte-stable. The vision calls ARE re-issued before the
|
||||
unchanged check can skip them — the documented no-cache cost (the
|
||||
phase-133 convention; a non-deterministic vision model re-describes
|
||||
the row each sync — a cost, not a bug; A21 endorses it)."""
|
||||
"""Phase 147 (task 02) — the owner's "really long sync times for
|
||||
PDFs that are mostly pictoral" complaint, fixed: a re-sync of an
|
||||
unchanged IKEA-class PDF (same raw bytes) takes the FAST PATH at
|
||||
the top of ``_index_file`` (the file hash matches) — the
|
||||
pictorial pass does NOT re-run: no re-render, the vision calls are
|
||||
NOT re-issued (``pictorial_calls`` stays at run 1's 6 — the
|
||||
pre-phase "vision re-issued before the unchanged check can skip
|
||||
them" no-cache cost is gone), ``pdfs_pictorial`` stays 0 on run 2.
|
||||
The row re-imports as UNCHANGED: no re-chunk, no re-embed, no
|
||||
re-summary, no KB change (not ``updated``, no changed path), and
|
||||
the content + digest are byte-stable. (A non-deterministic vision
|
||||
model no longer flips the row's digest either — the raw bytes
|
||||
decide.)"""
|
||||
root = tmp_path / "stable"
|
||||
root.mkdir()
|
||||
pdf = root / "ikea.pdf"
|
||||
@@ -586,17 +591,18 @@ def test_rerun_lands_unchanged_with_byte_stable_content(
|
||||
try:
|
||||
s1 = asyncio.run(import_sources([root], llm, session=db))
|
||||
assert (s1.added, s1.pdfs_pictorial, s1.pdf_pictorial_pages) == (1, 1, 6)
|
||||
assert llm.pictorial_calls == 6
|
||||
assert llm.pictorial_calls == 6 # one per candidate — first index
|
||||
embeds1 = len(llm.calls)
|
||||
doc1 = _pdf_doc(db, "stable", "ikea.pdf")
|
||||
content1, digest1 = doc1.content, doc1.content_hash
|
||||
|
||||
s2 = asyncio.run(import_sources([root], llm, session=db))
|
||||
# The pass re-ran (gate re-fired, vision re-issued — the
|
||||
# documented cost) — but the row itself re-imported unchanged.
|
||||
# The fast path fired (file hash match) — the pass never ran
|
||||
# on run 2: no render, no vision re-issued, no re-embed, no KB
|
||||
# change.
|
||||
assert (s2.added, s2.updated, s2.unchanged) == (0, 0, 1)
|
||||
assert s2.pdfs_pictorial == 1 # the pass re-fired
|
||||
assert llm.pictorial_calls == 12 # the vision calls were re-issued
|
||||
assert s2.pdfs_pictorial == 0 # the pass did not re-fire
|
||||
assert llm.pictorial_calls == 6 # NO re-issued vision call
|
||||
assert len(llm.calls) == embeds1, "unchanged: no re-embed"
|
||||
assert s2.changed_paths == [], "unchanged: no KB change recorded"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user