phase: 90_upload_no_scan

All verification is complete and green. Final report:

**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**

- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors

**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)

**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.

**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
This commit is contained in:
2026-09-09 14:08:18 -04:00
parent 0c8a7b9974
commit 3095c4c577
27 changed files with 2478 additions and 1224 deletions
+60 -27
View File
@@ -25,13 +25,16 @@ the test process and the app subprocess resolve the same ``.env``
(``BOR_SOURCES_DIR`` / ``BOR_UPLOAD_DIR``) and the same-host ``pathlib``
assertions hit the very directories the DELETE handler cleans.
The suite triggers **no sync** (the git rows are ``example.com`` URLs
that are never cloned); the only real artifact is the API-driven
upload of one small archive with a unique name (``phase69-<8-hex>.tar.gz``,
one ``.md`` file) — its background scan runs the mock-LLM pipeline
(no network beyond the app itself). Seeded ``Document`` rows
(``SessionLocal``, the ``test_git_sources_admin.py`` pattern) give the
prune assertions a deterministic KB.
The suite triggers **one sync** (test 1 only — phase 90: the upload's
background run unpacks + registers only, so the uploaded row is
imported via ``POST /api/sync`` on the mock-LLM pipeline, no network
beyond the app itself); the git rows are ``example.com`` URLs that
are never cloned, and every other test stays sync-free. The only real
on-disk artifact is the API-driven upload of one small archive with a
unique name (``phase69-<8-hex>.tar.gz``, one ``.md`` file). Seeded
``Document`` rows (``SessionLocal``, the
``test_git_sources_admin.py`` pattern) give the prune assertions a
deterministic KB.
Per-module app env (the conftest pattern, module-scoped): the same env
shape as ``test_git_sources_admin.py`` with ``BOR_GIT_SOURCES`` forced
@@ -41,14 +44,15 @@ the table); **no** ``BOR_SOURCES_DIR`` / ``BOR_UPLOAD_DIR`` override
Contract under test:
* **upload → modal → total removal**: the uploaded folder exists on
disk and its document is in ``GET /api/docs``; the row's Remove →
the alertdialog opens (``#remove-confirm-source`` = the upload path,
focus on ``#remove-confirm-cancel``) → "Remove source" → the
"Removing…" in-flight state (both buttons disabled) → settled: the
row is gone, the document is pruned, **the folder is gone from
disk**, exactly one DELETE went out, and the announcer carries the
success line;
* **upload → sync → modal → total removal** (phase 90: the upload
unpacks + registers only — the sync performs the scan): the
uploaded folder exists on disk and, after the sync, its document is
in ``GET /api/docs``; the row's Remove → the alertdialog opens
(``#remove-confirm-source`` = the upload path, focus on
``#remove-confirm-cancel``) → "Remove source" → the "Removing…"
in-flight state (both buttons disabled) → settled: the row is gone,
the document is pruned, **the folder is gone from disk**, exactly
one DELETE went out, and the announcer carries the success line;
* **git checkout removal**: a seeded git row + a hand-made checkout
dir (marker file) + a seeded document → modal removal → the row is
gone, **the checkout dir is gone from disk** (marker included) and
@@ -312,7 +316,10 @@ def _build_targz(path: Path, files: dict[str, str]) -> Path:
def _upload_and_wait_success(page: Page, app_url: str, archive: Path) -> dict[str, Any]:
"""POST the archive through the logged-in page's request context
(the admin cookie rides along) and poll the phase-64 status
endpoint to ``success`` — returns the terminal status body."""
endpoint to ``success`` — returns the terminal status body.
Phase 90: the upload run is unpack + register only — no scan — so
the caller follows with :func:`_run_sync` (the new owner flow) to
import the registered row."""
r = page.request.post(
f"{app_url}/api/git-sources/upload",
multipart={
@@ -333,9 +340,29 @@ def _upload_and_wait_success(page: Page, app_url: str, archive: Path) -> dict[st
if body["state"] == "success":
return body
if body["state"] == "failed":
raise AssertionError(f"the upload scan failed: {body}")
raise AssertionError(f"the upload run failed: {body}")
time.sleep(0.2)
raise AssertionError(f"the upload scan never settled: {body}")
raise AssertionError(f"the upload run never settled: {body}")
def _run_sync(page: Page, app_url: str) -> dict[str, Any]:
"""``POST /api/sync`` → poll ``GET /api/sync/status`` to a terminal
state (the ``test_sync_button.py`` idiom) — phase 90: the scan the
upload deferred lands here (the sync imports the uploaded
``kind='local'`` row with prune). Returns the terminal body."""
r = page.request.post(f"{app_url}/api/sync")
assert r.status == 202, f"sync POST failed: {r.status} {r.text}"
deadline = time.monotonic() + 60.0
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = page.request.get(f"{app_url}/api/sync/status")
assert r.status == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
assert body["state"] == "success", f"sync failed: {body}"
return body
time.sleep(0.2)
raise AssertionError(f"sync never settled: {body}")
def _open_remove_modal(page: Page, value: str) -> None:
@@ -385,11 +412,13 @@ def test_uploaded_source_removal_cleans_index_and_disk(
page: Page, app_url: str, db_ready: None, upload_dir: Path, tmp_path: Path
) -> None:
"""A uniquely named archive uploaded through the API (202 → status
success): the folder exists on disk and its document is in the KB;
then the row's Remove → the alertdialog (the upload path named,
focus on Cancel) → "Remove source" → the "Removing…" in-flight
state → settled: row gone, document pruned, **the folder is gone
from disk**, one DELETE, the announcer's success line."""
success — phase 90: unpack + register only, no scan), then the
sync that performs the scan (the new owner flow): the folder
exists on disk and the document is in the KB; then the row's
Remove → the alertdialog (the upload path named, focus on Cancel)
→ "Remove source" → the "Removing…" in-flight state → settled:
row gone, document pruned, **the folder is gone from disk**, one
DELETE, the announcer's success line."""
page.set_default_timeout(30_000)
# Unique per run — never collides with a real (or a crashed-run's)
# upload, so the disk assertions are safe on the shared dir.
@@ -407,14 +436,18 @@ def test_uploaded_source_removal_cleans_index_and_disk(
_admin_git_sources_page(page, app_url)
# The API-driven upload (202) + the background scan (success).
# The API-driven upload (202) + the background unpack + register
# (success — the no-count payload, phase 90). Phase 90: the upload
# does NOT scan — the scan is the sync's job (the new owner flow:
# upload → [edit ignore list] → sync), so the uploaded row is
# imported by a sync before the preconditions below.
status = _upload_and_wait_success(page, app_url, archive)
assert status["detail"]["source"] == name
assert status["detail"]["added"] == 1
assert status["detail"] == {"message": "uploaded"}
_run_sync(page, app_url)
# Preconditions — the artifact is real: the folder on disk (the
# app's resolved upload dir — this process resolved the same one),
# the document in the KB, the row in the registry.
# the document in the KB (via the sync), the row in the registry.
folder = upload_dir / name
assert (folder / "note.md").is_file(), f"{folder}/note.md missing on disk"
assert _docs(page, app_url) == [(name, "note.md")]