phase: 90_upload_no_scan
All verification is complete and green. Final report:
**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**
- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors
**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)
**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.
**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
This commit is contained in:
@@ -25,13 +25,16 @@ the test process and the app subprocess resolve the same ``.env``
|
||||
(``BOR_SOURCES_DIR`` / ``BOR_UPLOAD_DIR``) and the same-host ``pathlib``
|
||||
assertions hit the very directories the DELETE handler cleans.
|
||||
|
||||
The suite triggers **no sync** (the git rows are ``example.com`` URLs
|
||||
that are never cloned); the only real artifact is the API-driven
|
||||
upload of one small archive with a unique name (``phase69-<8-hex>.tar.gz``,
|
||||
one ``.md`` file) — its background scan runs the mock-LLM pipeline
|
||||
(no network beyond the app itself). Seeded ``Document`` rows
|
||||
(``SessionLocal``, the ``test_git_sources_admin.py`` pattern) give the
|
||||
prune assertions a deterministic KB.
|
||||
The suite triggers **one sync** (test 1 only — phase 90: the upload's
|
||||
background run unpacks + registers only, so the uploaded row is
|
||||
imported via ``POST /api/sync`` on the mock-LLM pipeline, no network
|
||||
beyond the app itself); the git rows are ``example.com`` URLs that
|
||||
are never cloned, and every other test stays sync-free. The only real
|
||||
on-disk artifact is the API-driven upload of one small archive with a
|
||||
unique name (``phase69-<8-hex>.tar.gz``, one ``.md`` file). Seeded
|
||||
``Document`` rows (``SessionLocal``, the
|
||||
``test_git_sources_admin.py`` pattern) give the prune assertions a
|
||||
deterministic KB.
|
||||
|
||||
Per-module app env (the conftest pattern, module-scoped): the same env
|
||||
shape as ``test_git_sources_admin.py`` with ``BOR_GIT_SOURCES`` forced
|
||||
@@ -41,14 +44,15 @@ the table); **no** ``BOR_SOURCES_DIR`` / ``BOR_UPLOAD_DIR`` override
|
||||
|
||||
Contract under test:
|
||||
|
||||
* **upload → modal → total removal**: the uploaded folder exists on
|
||||
disk and its document is in ``GET /api/docs``; the row's Remove →
|
||||
the alertdialog opens (``#remove-confirm-source`` = the upload path,
|
||||
focus on ``#remove-confirm-cancel``) → "Remove source" → the
|
||||
"Removing…" in-flight state (both buttons disabled) → settled: the
|
||||
row is gone, the document is pruned, **the folder is gone from
|
||||
disk**, exactly one DELETE went out, and the announcer carries the
|
||||
success line;
|
||||
* **upload → sync → modal → total removal** (phase 90: the upload
|
||||
unpacks + registers only — the sync performs the scan): the
|
||||
uploaded folder exists on disk and, after the sync, its document is
|
||||
in ``GET /api/docs``; the row's Remove → the alertdialog opens
|
||||
(``#remove-confirm-source`` = the upload path, focus on
|
||||
``#remove-confirm-cancel``) → "Remove source" → the "Removing…"
|
||||
in-flight state (both buttons disabled) → settled: the row is gone,
|
||||
the document is pruned, **the folder is gone from disk**, exactly
|
||||
one DELETE went out, and the announcer carries the success line;
|
||||
* **git checkout removal**: a seeded git row + a hand-made checkout
|
||||
dir (marker file) + a seeded document → modal removal → the row is
|
||||
gone, **the checkout dir is gone from disk** (marker included) and
|
||||
@@ -312,7 +316,10 @@ def _build_targz(path: Path, files: dict[str, str]) -> Path:
|
||||
def _upload_and_wait_success(page: Page, app_url: str, archive: Path) -> dict[str, Any]:
|
||||
"""POST the archive through the logged-in page's request context
|
||||
(the admin cookie rides along) and poll the phase-64 status
|
||||
endpoint to ``success`` — returns the terminal status body."""
|
||||
endpoint to ``success`` — returns the terminal status body.
|
||||
Phase 90: the upload run is unpack + register only — no scan — so
|
||||
the caller follows with :func:`_run_sync` (the new owner flow) to
|
||||
import the registered row."""
|
||||
r = page.request.post(
|
||||
f"{app_url}/api/git-sources/upload",
|
||||
multipart={
|
||||
@@ -333,9 +340,29 @@ def _upload_and_wait_success(page: Page, app_url: str, archive: Path) -> dict[st
|
||||
if body["state"] == "success":
|
||||
return body
|
||||
if body["state"] == "failed":
|
||||
raise AssertionError(f"the upload scan failed: {body}")
|
||||
raise AssertionError(f"the upload run failed: {body}")
|
||||
time.sleep(0.2)
|
||||
raise AssertionError(f"the upload scan never settled: {body}")
|
||||
raise AssertionError(f"the upload run never settled: {body}")
|
||||
|
||||
|
||||
def _run_sync(page: Page, app_url: str) -> dict[str, Any]:
|
||||
"""``POST /api/sync`` → poll ``GET /api/sync/status`` to a terminal
|
||||
state (the ``test_sync_button.py`` idiom) — phase 90: the scan the
|
||||
upload deferred lands here (the sync imports the uploaded
|
||||
``kind='local'`` row with prune). Returns the terminal body."""
|
||||
r = page.request.post(f"{app_url}/api/sync")
|
||||
assert r.status == 202, f"sync POST failed: {r.status} {r.text}"
|
||||
deadline = time.monotonic() + 60.0
|
||||
body: dict[str, Any] = {}
|
||||
while time.monotonic() < deadline:
|
||||
r = page.request.get(f"{app_url}/api/sync/status")
|
||||
assert r.status == 200, r.text
|
||||
body = r.json()
|
||||
if body["state"] in ("success", "failed"):
|
||||
assert body["state"] == "success", f"sync failed: {body}"
|
||||
return body
|
||||
time.sleep(0.2)
|
||||
raise AssertionError(f"sync never settled: {body}")
|
||||
|
||||
|
||||
def _open_remove_modal(page: Page, value: str) -> None:
|
||||
@@ -385,11 +412,13 @@ def test_uploaded_source_removal_cleans_index_and_disk(
|
||||
page: Page, app_url: str, db_ready: None, upload_dir: Path, tmp_path: Path
|
||||
) -> None:
|
||||
"""A uniquely named archive uploaded through the API (202 → status
|
||||
success): the folder exists on disk and its document is in the KB;
|
||||
then the row's Remove → the alertdialog (the upload path named,
|
||||
focus on Cancel) → "Remove source" → the "Removing…" in-flight
|
||||
state → settled: row gone, document pruned, **the folder is gone
|
||||
from disk**, one DELETE, the announcer's success line."""
|
||||
success — phase 90: unpack + register only, no scan), then the
|
||||
sync that performs the scan (the new owner flow): the folder
|
||||
exists on disk and the document is in the KB; then the row's
|
||||
Remove → the alertdialog (the upload path named, focus on Cancel)
|
||||
→ "Remove source" → the "Removing…" in-flight state → settled:
|
||||
row gone, document pruned, **the folder is gone from disk**, one
|
||||
DELETE, the announcer's success line."""
|
||||
page.set_default_timeout(30_000)
|
||||
# Unique per run — never collides with a real (or a crashed-run's)
|
||||
# upload, so the disk assertions are safe on the shared dir.
|
||||
@@ -407,14 +436,18 @@ def test_uploaded_source_removal_cleans_index_and_disk(
|
||||
|
||||
_admin_git_sources_page(page, app_url)
|
||||
|
||||
# The API-driven upload (202) + the background scan (success).
|
||||
# The API-driven upload (202) + the background unpack + register
|
||||
# (success — the no-count payload, phase 90). Phase 90: the upload
|
||||
# does NOT scan — the scan is the sync's job (the new owner flow:
|
||||
# upload → [edit ignore list] → sync), so the uploaded row is
|
||||
# imported by a sync before the preconditions below.
|
||||
status = _upload_and_wait_success(page, app_url, archive)
|
||||
assert status["detail"]["source"] == name
|
||||
assert status["detail"]["added"] == 1
|
||||
assert status["detail"] == {"message": "uploaded"}
|
||||
_run_sync(page, app_url)
|
||||
|
||||
# Preconditions — the artifact is real: the folder on disk (the
|
||||
# app's resolved upload dir — this process resolved the same one),
|
||||
# the document in the KB, the row in the registry.
|
||||
# the document in the KB (via the sync), the row in the registry.
|
||||
folder = upload_dir / name
|
||||
assert (folder / "note.md").is_file(), f"{folder}/note.md missing on disk"
|
||||
assert _docs(page, app_url) == [(name, "note.md")]
|
||||
|
||||
Reference in New Issue
Block a user