phase: 183_sync_passive_delete_cascade

All gates green. Final state verified.

**Implemented / verified**
- Task 03's work was **never actually done** (its report was a stub, `tests/e2e/test_prune_cascade_cleanup.py` did not exist) → wrote it: 3 tests — 9-file local source synced from the Sources page (`9 added`), 6 files deleted upstream, re-sync (`0 added · 3 unchanged · 6 pruned`), documents vanished from the tree + `GET /api/documents/content` 404 + total 3, **0 orphan chunk rows** via the test's own SQL, survivor chunk total unchanged; partial-prune chat turn (grounded, non-deflected, `query_log` names the survivor and no pruned doc); total source removal (`DELETE /api/git-sources/{id}`) wipes every doc+chunk of the source with 0 orphans, sibling source untouched. Mock-only (`E2E_REAL_LLM=1` skipped), own port 8199, truncate before/after.
- Tasks 01/02 re-verified: `passive_deletes=True` + mapper pin, probe flipped to `EXPECTED_CHUNK_SELECTS_PER_PRUNED_DOC = 0`, `_prune`/`delete_git_source` docstrings.
- **Drift guard proven non-tautological by my own one-off run**: scratch DB (`alembic upgrade head`) → FK `confdeltype='c'`, guard PASSES, CASCADE delete leaves 0 chunks; FK recreated `ON DELETE NO ACTION` → guard FAILS verbatim and a `passive_deletes` delete raises `ForeignKeyViolation` (never silent orphans). Scratch DB dropped; shared test DB untouched (`'c'`).

**Gates**
- `uv run pytest --cov=app --cov-report=term-missing` → **4488 passed**, `app/` TOTAL **99%** (>90%)
- `uv run pytest tests/integration/test_prune_cascade_statements.py -v` → 4 passed
- `uv run pytest tests/e2e/test_prune_cascade_cleanup.py -v --no-cov` → **3 passed in isolation**; `test_source_removal_cleanup.py` 6 passed, `test_force_push_resync.py` 2 passed (both isolated)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors

**Completion criteria**: passive_deletes+pin ✅ · probe post-fix ✅ · drift guard green + proven-failing variant ✅ · full suite/coverage/lint ✅ · E2E in isolation ✅ · docs consistent ✅ · commit = harness-only (per pipeline rule I ran no git write commands).

**Docs**: fixed 4 pre-existing README inconsistencies — per-module E2E ports (claim of "one pair of fixed ports" predates phase 79), debugpy `launch.json` snippet vs the shipped file (`name`/`remoteRoot`), and the page-naming wobble (navbar labels `/sources.html` **RAG**, `/git-sources.html` **Sources** — the Sync button's home is now named correctly in 4 places); added the phase-183 E2E pointer. AGENTS.md verified accurate (rule 12 matches the delete-site audit).

**Deviation**: none from the locked decisions; the E2E is a behaviour pin (byte-identical owner-visible behaviour), the statement-count probe remains the memory teeth, as phase 183 D1/A4 specify.

**Next pending phase**: `184_sync_bounded_session`.

README.md

@@ -272,7 +272,8 @@ just looked at costs a bodiless 304 (see *Caching & deploys*).
 The admin-managed source registry: add or remove git repositories, upload
 source archives (`.tar`, `.tar.gz`, `.tgz`, `.zip`), and register local
 directories — all in one table with a `kind` discriminator. No `.env`
-editing, no restart. The **Sync sources** button clones/pulls/walks every
+editing, no restart. The **Sync sources** button — which lives on the
+**RAG** page (`/sources.html`), not here — clones/pulls/walks every
 source and re-imports in one click.
 
 ### Tuning (`/tuning.html`)
@@ -446,10 +447,17 @@ uv run python -m scripts.import_docs --source ~/SomeOtherDocs
 With **git-based sources** each run first pulls the latest commits of your
 repos, so this same command is the whole update loop: *commit → re-run*.
 
-Then check the **Sources** page (`http://localhost:8000/sources.html`):
+Then check the **RAG** page — the knowledge-base catalog at
+`http://localhost:8000/sources.html` (the navbar labels it **RAG**; the
+separate **Sources** page, `/git-sources.html`, is the source registry):
 the *documents* / *chunks* counters and *last indexed* timestamp should
 reflect the new files.
 
+- `--prune` removes a pruned file **completely** — its `documents` row and
+  its `chunks` rows (the embeddings) go together, the latter through the
+  database's own `ON DELETE CASCADE`, so the cost of a mass prune is
+  bounded by documents, not chunks (see *A mass prune costs documents,
+  not chunks* under [Sync from the UI](#sync-from-the-ui)).
 - The import prints one line per file and ends with a greppable summary,
   so it is safe to run from cron or after every commit.
 - Indexed formats: **md, markdown, txt, yaml, yml, json, py**, the Podman
@@ -539,9 +547,9 @@ BOR_SOURCES_DIR=~/bor-sources   # default; each repo lands in <dir>/<repo-name>/
 Upload a `.tar`, `.tar.gz`, `.tgz`, or `.zip` archive to make it a source.
 The form on the Git sources page — and the `POST /api/git-sources/upload`
 route behind it — accepts archives and unpacks + registers them (no scan —
-the upload indexes nothing; the **Sync sources** button on the RAG page
-performs the scan, which leaves room to edit the new source's ignore list
-first).
+the upload indexes nothing; the **Sync sources** button on the **RAG**
+page (`/sources.html`) performs the scan, which leaves room to edit the
+new source's ignore list first).
 
 - Re-uploading the same filename replaces the source **in place** (no
   second folder, no duplicate row).
@@ -572,8 +580,8 @@ discriminator: `git` | `local`), **one admin page**, and **one Sync button**.
 
 ### Sync from the UI
 
-The **Sync sources** button on the **Sources** page (admin only) runs the
-whole git-source refresh in one click:
+The **Sync sources** button on the **RAG** page (`/sources.html`, admin
+only) runs the whole source refresh in one click:
 
 1. **clone/pull + walk** every configured source — git repos and local
    directories, including uploaded archives.
@@ -625,6 +633,24 @@ whole git-source refresh in one click:
   above all the 2 s `GET /api/sync/status` poll that watches the run.
   Before this a big first clone stalled the loop thread and the entire
   app looked dead until git returned.
+- **A mass prune costs documents, not chunks** (phase 183). A pruned file
+  leaves the index completely — its `documents` row AND its `chunks` rows
+  (every embedding) — but the chunk rows go **in the database**, through
+  `chunks_document_id_fkey`'s `ON DELETE CASCADE` (alembic 0001), because
+  `Document.chunks` is declared `passive_deletes=True`. So a prune of 3,000
+  documents issues batched `DELETE FROM documents` statements and **zero**
+  `SELECT`s of the 768-dim vectors it is about to remove: the memory a
+  prune costs is bounded by the *document* count, not by chunks ×
+  ~24.8 KiB of hydrated embedding. Before this, `session.delete(doc)`
+  loaded every child collection first, and the 2026-10-15 sync's
+  3,008-document prune materialised ~72k embeddings (~720 MB) on a 3,904
+  MB VM with no swap. The same holds for removing a source on the
+  **Git sources** page (the other mass-delete surface). Pinned by
+  `tests/integration/test_prune_cascade_statements.py` — the statement
+  count, the zero-orphan-chunk-rows contract, and a live-FK drift guard
+  that fails the suite if the cascade clause ever disappears; the owner
+  flow itself (delete upstream → re-sync → exactly those documents and
+  chunks gone) is pinned by `tests/e2e/test_prune_cascade_cleanup.py`.
 - **Cancel**: while a run is in flight, a **Cancel** button appears
   next to the sync button; clicking it posts `POST /api/sync/cancel`
   (admin only) and registers a cooperative cancel (`202`) — the run
@@ -1610,18 +1636,22 @@ DEBUGPY=1 uv run uvicorn app.main:app
 # → log line: debugpy: remote debugging ENABLED, listening on 0.0.0.0:5678
 ```
 
-Attach from VS Code (`.vscode/launch.json`):
+Attach from VS Code — the repo ships `.vscode/launch.json` with the
+matching configuration (`"Python Debugger: Remote Attach`), mapped for a
+**local** dev run:
 ```json
 {
-  "name": "Attach to Brain of Reese",
+  "name": "Python Debugger: Remote Attach",
   "type": "debugpy",
   "request": "attach",
   "connect": { "host": "localhost", "port": 5678 },
   "pathMappings": [
-    { "localRoot": "${workspaceFolder}", "remoteRoot": "/app" }
+    { "localRoot": "${workspaceFolder}", "remoteRoot": "." }
   ]
 }
 ```
+Attaching to a run **inside the container** instead needs the container's
+own root: `"remoteRoot": "/app"` (that is where the image puts the code).
 
 The port is non-blocking and attach-on-demand: the app keeps running
 normally until you attach. Override the port with `DEBUGPY_PORT`.
@@ -1654,12 +1684,18 @@ uv run pytest tests/e2e/test_chat_rag.py -v --no-cov
 # ...one file per phase in tests/e2e/ (the stories live in
 # .agents/user_stories/, see .agents/phases/todo/).
 #
-# Isolation is REQUIRED, not a style preference: every suite shares one
-# Postgres and one pair of fixed ports (the app on 8123, the mock aipi on
-# 8901 — overridable with E2E_APP_PORT / E2E_MOCK_PORT), and each file
-# truncates + reseeds the KB for its own fixture. Running the whole
-# `tests/e2e/` directory in a single pytest session interleaves those
-# fixtures and fails tests that pass file by file.
+# Isolation is REQUIRED, not a style preference: every suite shares ONE
+# Postgres, one mock aipi port (8901 — E2E_MOCK_PORT) and the KB it seeds,
+# and each file truncates + reseeds the KB for its own fixture. Running
+# the whole `tests/e2e/` directory in a single pytest session interleaves
+# those fixtures and fails tests that pass file by file.
+#
+# Ports: the shared session app (tests/e2e/conftest.py) binds 8123
+# (E2E_APP_PORT). A suite that needs its OWN app env boots a per-module
+# uvicorn on its own port (phase 79) — e.g. E2E_APP_PORT_PRUNECASCADE for
+# tests/e2e/test_prune_cascade_cleanup.py — because a second uvicorn on
+# the session port dies on bind and the suite would drive the wrong
+# server. So "isolation" is about the shared DATABASE, not about ports.
 
 # To sweep every suite, loop over the files (one pytest process each):
 for f in tests/e2e/test_*.py; do uv run pytest "$f" --no-cov -q || echo "FAIL $f"; done