Files
brain-of-reese/.agents/phases/complete/90_upload_no_scan/00_phase.md
T
ducoterra 3095c4c577 phase: 90_upload_no_scan
All verification is complete and green. Final report:

**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**

- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors

**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)

**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.

**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
2026-09-09 14:08:18 -04:00

6.3 KiB
Raw Blame History

Phase 90 — Upload stops scanning: the RAG page's Sync button owns the scan

Source: TODO.md L3 — "Uploading a source archive should not trigger a scan - that should be left to the sync button on the RAG page. Right now the sync starts right away which doesn't give the user time to edit the ignore list. Update the button text as well to read "Upload" rather than "Upload and scan"" Story: n/a (TODO-derived — owner roadmap confirmation 2026-09-09). Context: POST /api/git-sources/upload (app/api/git_sources.py::upload_archive) does three inline gates (1. name/format 422, 2. one-at-a-time 409, 3. 1 MiB-chunk stream into a dotfile temp with the upload_max_mb 413 cap), returns 202 + UploadAccepted, then runs _run_upload in the background: 4. unpack (traversal/symlink/corrupt/over-cap → failed), 5. atomic swap-in (same-name re-upload replaces in place), 6. row upsert by path (kind='local', added_at preserved, the row's saved ignore_paths captured — phase 89), 7. fail-fast check_models, 8. import_sources(…, prune=True, progress=<hook>, ignore_by_root=…) + change-gated regenerate_overview, 9. INFO log line, 10. success with the UploadOut count fields in detail. GET /api/git-sources/upload/status mirrors the phase-32 sync-status key set (state, started_at, finished_at, detail, error, current_file, files_done, files_total) and is what frontend/assets/git-sources.js polls (2 s poll, startUploadPolling; boot re-attach initUploadStatus). The Sources view (#view-git-sources, static form in frontend/index.html, #archive-upload-btn labeled "Upload & scan") and the RAG view (#view-rag) where the admin-only #sync-btn ("Sync sources") lives; app/api/sync.py::_run_sync already mirrors every git_sources row, including kind='local' (the stored directory, re-verified, missing dir → local source missing: <path>), honoring each row's ignore_paths (phase 89) — so a registered upload row is imported on the next sync with no sync change. The phase-89 per-row "Ignore paths" editor (#ignore-editor-dialog in git-sources.js) is the UI the deferral exists for: upload → row visible → edit ignore list → sync.

Objective

Uploading a source archive stores it, unpacks it, and registers the source row — and nothing else: no model check, no import, no overview regeneration. The scan is left to the "Sync sources" button on the RAG page, giving the owner time to edit the new source's ignore list first. The upload button reads "Upload" (was "Upload & scan") and all surrounding copy says the upload unpacks only and the sync scans.

Dependencies

  • 89_source_ignore_paths (complete) — the per-row ignore-list editor is the workflow this phase unlocks; the upload still captures the row's saved ignore_paths on re-upload.

Design (shared by all tasks — the executor reads this, not the chat)

  • What the background run keeps (A1, owner-locked 2026-09-09). Steps 1–6 unchanged: the three inline gates, the streaming receive, unpack (with all its sanitized failed states), the zero-entry failed, the atomic swap-in, the row upsert (existing row left as-is — added_at preserved, ignore_paths preserved — new row inserted with kind='local'). The scan = steps 7–8 (and their success detail) leave the upload entirely.
  • Status contract (A2, owner-locked 2026-09-09). Same key set as today. Terminal success now carries detail = {"message": "uploaded"} (no count fields), current_file = null, files_done = files_total = 0. failed states unchanged in shape and wording (unpack/zero-entry/swap/row). current_file/files_done/files_total stay null/0/0 for the whole background run (unpack has no file-level progress hook) — the UI's in-flight label loses its "(n/m)" file count for uploads (sync keeps its live file label; phase 64).
  • Sync unchanged (A4, owner-locked 2026-09-09). _run_sync already imports kind='local' rows with prune + ignore lists; task 03 proves the full loop in E2E. No change to app/api/sync.py in this phase.
  • Copy (A3, owner-locked 2026-09-09). Button label exactly Upload (static label, restoreUploadButton, and any relabel sites). The success moment keeps the phase-64 toast ("Successfully uploaded — ") but the settled result line points at the RAG page: "Uploaded — press Sync sources to import it." #git-sources-hint and the table caption are re-worded: uploads unpack and register only; the Sync button scans.

Tasks

  1. 01_upload_defers_scan.md — the background run stops after the row upsert; success status is a no-count "uploaded" payload; unit tests.
  2. 02_upload_ui_and_copy.md — "Upload" button, in-flight/result/toast copy, #git-sources-hint + caption; affected existing suites updated in place.
  3. 03_e2e_upload_then_sync.md — the story's dedicated Playwright E2E: upload → nothing indexed → edit ignore list → Sync sources on the RAG page → docs land, ignores honored.

Testing & Quality

  • Unit/integration: the upload pipeline's new terminal states (row registered, KB untouched, no model call); the status payload shape; the sync-imports-a-fresh-upload-row path already covered by the sync suites (no change) — plus every affected existing test updated, none deleted without replacement.
  • Coverage: >90% on new/modified code (uv run pytest --cov=app --cov-report=term-missing).
  • This phase's Playwright E2E suite: tests/e2e/test_upload_no_scan.py, run in isolation (uv run pytest tests/e2e/test_upload_no_scan.py -v --no-cov).
  • Affected existing E2E suites (test_archive_upload_sources.py, test_sync_upload_progress.py) updated in place and green.

Completion Criteria

  • Uploading an archive on the Sources page settles with the "ready for sync" result and indexes zero documents; the source row is present with its ignore editor.
  • The upload button reads "Upload"; no "Upload and scan" copy remains anywhere (rg "Upload &" frontend/ → nothing).
  • "Sync sources" on the RAG page imports the uploaded source (respecting its edited ignore list) — proven by the isolated E2E.
  • Full test suite green, app/ coverage >90%, uv run ruff check . && uv run pyright clean.
  • One atomic Conventional Commits commit for the phase (--no-gpg-sign), .agents/ phase files moved to complete/ by the pipeline.