phase: 147_sync_changed_only
All defaults match. Verification complete — everything is green with no defects found. **Phase 147 final verification pass — report** - All 3 task files found in `complete/147_sync_changed_only/` → verification pass only; no code changes needed. - Verified: migration 0025 (`documents.file_hash` nullable String(64), no index) + model field; fast path at top of `_index_file` (streamed 1 MiB sha256 → row lookup → `_unchanged_doc`); stamping on all three branches' create/update + shared helper; pre-phase NULL-hash fallthrough; retriever/`content_hash` untouched. - **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3438 passed, `app/` coverage **99%** (>90% ✓); `uv run pytest tests/e2e/test_sync_changed_only.py -v --no-cov` → 3 passed in isolation ✓; `uv run ruff check . && uv run pyright` → all checks passed, 0 errors ✓; prior-phase `tests/e2e/test_sync_button.py` → 3 passed (no regression) ✓. - **Completion criteria:** all 7 pass — (1) column + round-trip ✓ (2) unchanged text/image/garbage-PDF/pictorial-PDF cost zero LLM/extraction/vision, `summary.unchanged` counted, rows byte-still ✓ (3) changed file re-indexes + re-stamps; bytes-changed/content-same stamps only ✓ (4) backfill/D1/D4 + log wordings pinned verbatim ✓ (5) `content_hash`/retriever byte-identical ✓ (6) all gates green ✓ (7) no prior-phase behavior change ✓. - **Documentation:** full review of README.md (1,031 lines) and AGENTS.md against the code: every `.env.example`/code `BOR_*` var documented, all config defaults match `app/config.py`, quick-start/QA commands and referenced test files exist, `file://` URL + troubleshooting claims accurate; the phase's README "Updating the documents" edit is accurate. **Verified consistent — no changes needed.** - **Notable:** no live-infrastructure changes (dev Postgres container used for the standard test gates only). Working tree left uncommitted per harness protocol. - **Next pending phase:** `148_pdf_pages_in_sources`.
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
"""documents.file_hash: the raw source-file-bytes digest (phase 147, task 01)
|
||||
|
||||
Revision ID: 0025
|
||||
Revises: 0024
|
||||
Create Date: 2026-09-30
|
||||
|
||||
Phase 147 (sync only what changed — task 01, storage only):
|
||||
|
||||
* ``documents.file_hash`` — VARCHAR(64) NULLABLE, NO server default:
|
||||
the sha256 of the RAW SOURCE FILE's bytes at index time — the sync's
|
||||
change signal, compared at the top of the importer's ``_index_file``
|
||||
BEFORE the image/text/PDF dispatch (an unchanged file then costs one
|
||||
streamed hash + one lookup — no extraction, no vision, no embed;
|
||||
task 02). NULL = "unknown" for every pre-phase row: a backfill is
|
||||
impossible in a migration (it cannot see the source files on disk),
|
||||
so the first post-upgrade sync runs the full pipeline once and
|
||||
STAMPS the hash — fast forever after.
|
||||
|
||||
DISTINCT from ``content_hash`` (0001): that column is the digest of
|
||||
the FINAL indexed content (a pictorial PDF's includes vision text),
|
||||
is indexed (a query/exposure surface — the retriever's
|
||||
dedupe/caching identity, ``app/rag/retriever.py``) and stays
|
||||
untouched. ``file_hash`` is internal bookkeeping — it is compared in
|
||||
Python AFTER the unique ``(source, path)`` lookup, so it carries no
|
||||
index of its own.
|
||||
|
||||
One additive, fully reversible migration (A13); no other schema
|
||||
change. The fast path itself (task 02) is importer code — this
|
||||
revision only carries the column.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "0025"
|
||||
down_revision = "0024"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# No index (the fast path compares the value in Python after the
|
||||
# unique (source, path) lookup — it is never a query surface) and
|
||||
# no server default (NULL is the meaningful pre-phase state).
|
||||
op.add_column("documents", sa.Column("file_hash", sa.String(length=64), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# The file_hash column is the only 0025 artefact — dropping it
|
||||
# leaves 0024's schema byte-identical (A13, fully reversible). The
|
||||
# hash is re-computable from the source files at any time, so no
|
||||
# data is lost by the drop.
|
||||
op.drop_column("documents", "file_hash")
|
||||
Reference in New Issue
Block a user