phase: 184_sync_bounded_session
All gates green; no code changes were needed — the prior failure was a missing final report, not a test failure. **Verified / implemented (phase 184 final pass)** - All 4 tasks present in `.agents/phases/complete/184_sync_bounded_session/`; confirmed their output in the worktree: `sync_session_reset_files=200` / `sync_prune_batch=500` on `Settings` (+ non-negative validators), both `SettingMeta` rows under `"Import & sources"`, `.env.example` + `README.md` entries. - `import_sources` expunges at **exactly two** sites (per-file loop top L972, source boundary L1012) — grep proves no third `expunge`/`expire_all` anywhere in `app/`; `app/db.py:30` untouched. - `_prune` uses `load_only(id, source, path, is_image, image_path, page_count).order_by(Document.id)` + per-batch commits, with the phase-122/126 guards and `changed_paths` intact. **Tests / lint / coverage (all 100% green)** - `uv run pytest --cov=app --cov-report=term-missing` → **4506 passed**, `app/` TOTAL **99%** (>90%). - `uv run pytest tests/integration/test_import_session_retention.py -v --no-cov` → **12 passed**. - `uv run pytest tests/e2e/test_sync_session_reset.py -v --no-cov` → **4 passed in isolation**. - Regression sweep of neighbouring sync suites (each isolated): `test_sync_changed_only` 3, `test_sync_summary_visibility` 3, `test_prune_cascade_cleanup` 3, `test_sync_git_freeze` 2 — all passed. - `uv run ruff check .` → All checks passed. `uv run pyright` → 0 errors, 0 warnings. **Completion criteria** 1. Settings + CATALOG + `.env.example` + README; `test_settings_catalog.py` → ✅ green 2. Retention test green with the post-fix bound and both measured numbers in the docstring (pre-fix 12-row monotone staircase / ≈265 MB extrapolated; post-fix ends at 0, add-path peak 14 vs 21→49) → ✅ 3. Expunge only at the two documented-safe points; light-row batched prune with guards intact → ✅ 4. Every existing importer/sync/removal test green with unchanged expectations → ✅ 5. Full suite green, coverage >90%, ruff + pyright clean → ✅ 6. Phase E2E green in isolation → ✅ 7. README + AGENTS accurate/consistent → ✅ 8. Atomic commit — left to the harness (I ran no git write commands). **Documentation** — Full README (1886 lines) + AGENTS.md (189 lines) review: **verified consistent — no changes needed.** Programmatic cross-checks: every `Settings` field appears in `.env.example` and in the README config reference; every `.env.example` var is documented in that reference; every `tests/…` and `app/…` path cited in README/AGENTS exists; all 10 README page URLs are registered in `app/main.py`; documented E2E port vars (`E2E_APP_PORT_SYNCRESET[_ZERO]`) exist in the test. The only reference-var without a `Settings` field is `BOR_THEME`, which the README explicitly documents as retired (correct as written). **Notable decisions/deviations** — None. No code, test, or doc edits were made this pass; changes remain entirely in the working tree for the harness to stage. `00_phase.md` still sits in `todo/` while tasks 01–04 are in `complete/` — harness-owned, so I left it untouched. **Next pending phase:** `185_sync_defer_content_fast_path`.
This commit is contained in:
@@ -133,6 +133,45 @@
|
||||
`doc.chunks = [...]` **re-embed** path is the deliberate, documented
|
||||
exception (collection replacement legitimately needs the loaded
|
||||
collection; changed files only).
|
||||
13. **Release the sync session at the proven-safe points only:** an import
|
||||
run holds one session and `expire_on_commit=False` (`app/db.py`) means a
|
||||
commit releases nothing, so `_unchanged_doc`'s `file_hash` stamp keeps
|
||||
every walked `Document` — content included — resident in the session's
|
||||
own strong `dirty` set (not collectable): memory grew with the KB. The
|
||||
cadence is therefore `session.expunge_all()` every
|
||||
`BOR_SYNC_SESSION_RESET_FILES` files at the TOP of the per-file loop and
|
||||
once per source before the two post-passes (phase 184; `0` = never, the
|
||||
pre-phase rollback). `expunge_all` DISCARDS unflushed changes, so never
|
||||
move it into `_index_file`, `_unchanged_doc`, `_store_summary` or a
|
||||
post-pass — their locals are live across statements — and never add a
|
||||
reset before a file's per-file commit. Pinned by
|
||||
`tests/integration/test_import_session_retention.py` (bounded with the
|
||||
cadence on, unbounded with it `0`, byte-identical summaries and DB) and
|
||||
through the UI by `tests/e2e/test_sync_session_reset.py` (two module apps,
|
||||
one per cadence: every document exactly once, the all-`unchanged` re-sync,
|
||||
and the `0`-cadence run byte-identical to the `3`-cadence one).
|
||||
14. **The prune reads six columns and commits in batches:** `_prune`
|
||||
selects its candidates through
|
||||
`load_only(id, source, path, is_image, image_path, page_count)`
|
||||
`order_by(Document.id)` — the exact set the phase-122 images-off guard,
|
||||
the phase-126 parent-PDF rule and the image/page copy cleanup read, and
|
||||
NOT `content` (252 MB of markdown per full KB walk) — and commits every
|
||||
`BOR_SYNC_PRUNE_BATCH` deletes (`0` = the pre-phase single commit;
|
||||
phase 184). Never read another `Document` attribute inside that loop (a
|
||||
deferred column costs one lazy SELECT per document and unbounds the
|
||||
walk — no `raiseload` paper-over, add the column instead), and never
|
||||
swap the per-document `session.delete(doc)` for a bulk `delete()` (the
|
||||
guards, `changed_paths` and the log line depend on the per-document
|
||||
shape). A non-zero batch deliberately trades the all-or-nothing prune
|
||||
for a prefix (LOCKED A2): `seen` is recomputed from disk, so the next
|
||||
sync converges. Pinned by
|
||||
`tests/integration/test_import_session_retention.py` (the captured
|
||||
`SELECT`'s column list, exactly one document-wide `SELECT` per prune
|
||||
(a second one is a per-row lazy load), the commit counts at `10` and
|
||||
`0`, both guards on light rows) and
|
||||
`tests/e2e/test_sync_session_reset.py` (the five-document prune at batch
|
||||
`2` through the Sources page: both folders cleared, the survivors' chunks
|
||||
intact, zero orphan chunk rows).
|
||||
|
||||
## Quick reference
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user