Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 90_upload_no_scan

Browse Source 
All verification is complete and green. Final report:

**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**

- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors

**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)

**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.

**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
This commit is contained in:
ducoterra
2026-09-09 14:08:18 -04:00
parent 0c8a7b9974
commit 3095c4c577
27 changed files with 2478 additions and 1224 deletions
Download Patch File Download Diff File Expand all files Collapse all files
frontend/assets/git-sources.js
+122 -97
View File
@@ -20,7 +20,7 @@
* shell — scoped lookups keep the module honest and testable).
* The router mounts a view ONCE (mount-once, hide-forever), so
* the bindings and the upload-progress state machine survive
* every switch: the scan poller is a self-chaining setTimeout
* every switch: the upload poller is a self-chaining setTimeout
* started when an upload begins (never at boot), so progress
* continues while the user is on another view, and nothing
* refetches on re-show. The single toast node/timer stay module
@@ -58,37 +58,43 @@
* instruction survives. 409/422 details are fixed generic strings
* (credential safety — the URL is never echoed).
* • upload — #archive-upload-form submit (phase 49, reworked to the
* phase-64 202 contract in task 05 — the phase-49 synchronous
* 200 paragraph is superseded): POST
* phase-64 202 contract in task 05, unpack-only in phase 90 —
* the phase-49 synchronous 200 paragraph is superseded): POST
* /api/git-sources/upload with a FormData file (NO manual
* Content-Type — the browser sets the multipart boundary). The
* §7.4 never-stale lifecycle keeps its shape — the button
* disables + relabels "Uploading…" while the request is out —
* but the transfer is now short: the 202 arrives the moment the
* archive is safely on disk (A1). 202 → the page-local
* ("Upload") disables + relabels "Uploading…" while the request
* is out — but the transfer is short: the 202 arrives the moment
* the archive is safely on disk (A1). 202 → the page-local
* "Successfully uploaded — <file>" toast fires (showUploadToast,
* the phase-55 share-toast pattern; A2: safe to navigate away),
* the file input clears, and the button hands over to the scan —
* the processing state ("Processing…", disabled, title cleared)
* plus startUploadPolling(): a 2 s poll of
* GET /api/git-sources/upload/status renders the live
* "Processing… <file> (n/m)" label (A4 — bare during unpack; the
* full path rides the button title) and settles it: success →
* the sync-style count line (fmtUploadResult, the role=status
* result line) + the "Archive uploaded: …" announce +
* loadSources (the new/updated row lands with the Local badge;
* a re-upload refreshes the row — no duplicate; NO second toast
* — A2); failure → the sanitized server error in the role=alert
* banner + loadSources, the file selection KEPT for a one-click
* the file input clears, and the button hands over to the
* background run — the processing state (bare "Processing…",
* disabled, title cleared) plus startUploadPolling(): a 2 s poll
* of GET /api/git-sources/upload/status that renders the bare
* "Processing…" label for the WHOLE run (phase 90, A2 — the run
* is unpack + register only: no file, no "(n/m)" counts, no
* title) and settles it: success → the ready-for-sync line
* ("Uploaded <name> — press Sync sources to import it.",
* fmtUploadResult off the status's {"message": "uploaded"}
* detail — the role=status result line) + the "Archive uploaded
* — press Sync sources to import it." announce + loadSources
* (the new/updated row lands with the Local badge; a re-upload
* refreshes the row — no duplicate; NO second toast — A2);
* failure → the sanitized server error in the role=alert banner
* + loadSources, the file selection KEPT for a one-click
* re-upload. 409 (an upload is already in progress) raises NO
* error banner — it re-attaches to the in-flight run (processing
* state + poll, never stale). Other non-2xx (422 format/name,
* 413 cap, 5xx) keep the phase-49 error banner + the kept file
* selection. The submit finally restores the button ONLY when no
* poll is active (§7.4). Boot re-attach (initUploadStatus, admin
* branch): a running scan re-enters the processing state + poll
* (a reload mid-scan re-attaches — no second upload), a terminal
* run re-renders its result line / error banner.
* branch): a running run re-enters the processing state + poll
* (a reload mid-run re-attaches — no second upload), a terminal
* run re-renders its result line / error banner (the safe name
* was page-local — lastUploadName is null after a reload — so the
* re-rendered line is the nameless "Uploaded — press Sync sources
* to import it.").
* • remove — a row's Remove button opens the page-local
* confirmation modal (#remove-confirm-dialog, a real
* role="alertdialog" — the native confirm() retired, phase 69):
@@ -182,9 +188,12 @@
* page's hint box matches. The Sync button still mirrors the
* remaining sources (upstream file churn is pruned on that run).
* The phase-49 upload is the other in-place exception: it unpacks
* and scans the single source in place (the phase-64 background task
* — 202 + status endpoint), and its counts render as the result
* line.
* and registers the source in place (the phase-64 background task —
* 202 + status endpoint) and STOPS THERE — no model check, no
* import, no overview refresh (phase 90, A1): the scan is the RAG
* page's "Sync sources" button's job (it imports the uploaded
* kind=local row with prune + the row's ignore list), and the result
* line points at that button.
*
* The shared header module loads through this script's own relative
* import ("./header.js") — a hoisted import evaluated before this body
@@ -209,7 +218,8 @@ export async function mount(root) {
const addError = root.querySelector("#git-source-error");
/* Phase 49: the archive upload form (replaces the phase-38 local
directory form — same card, a file input instead of a path input).
The response counts render in the role=status result line. */
The no-count result line (phase 90) renders in the role=status
result line. */
const uploadFormEl = root.querySelector("#archive-upload-form");
const uploadFileInput = root.querySelector("#archive-upload-file");
const uploadBtn = root.querySelector("#archive-upload-btn");
@@ -821,53 +831,57 @@ export async function mount(root) {
idleLabel: "Add source",
});
/* ---------- upload (POST /api/git-sources/upload) — phase 64 (task 05) -------
/* ---------- upload (POST /api/git-sources/upload) — phase 64 (task 05), unpack-only (phase 90) -------
* The archive upload form follows the phase-64 202 contract (A1):
* the file input's selection is posted as FormData (the browser sets
* the multipart boundary — no manual Content-Type), and the 202
* answers the moment the archive is safely on disk — the "Uploading…"
* label covers only that short receive. Then the button HANDS OVER to
* the scan: 202 → the page-local "Successfully uploaded — <file>"
* toast (showUploadToast — A2, safe to navigate away), the file input
* clears, and the processing state ("Processing…", disabled, title
* cleared) + startUploadPolling() own it — a 2 s poll of
* GET /api/git-sources/upload/status renders the live "Processing…
* <file> (n/m)" label (A4 — bare during unpack; the full path rides
* the button title) and settles it: success → the sync-style count
* line (fmtUploadResult) in the role=status result line + the
* "Archive uploaded: …" announce + loadSources (NO second toast — it
* already fired at the 202, A2); failure → the sanitized server
* error in the role=alert banner + loadSources, the file selection
* KEPT for a one-click re-upload. 409 (an upload is already in
* progress) raises NO error banner — it re-attaches to the in-flight
* run (processing state + poll, never stale); the phase-49 "server
* the background run (phase 90: UNPACK + REGISTER only — no scan):
* 202 → the page-local "Successfully uploaded — <file>" toast
* (showUploadToast — A2, safe to navigate away), the file input
* clears, and the processing state (bare "Processing…", disabled,
* title cleared) + startUploadPolling() own it — a 2 s poll of
* GET /api/git-sources/upload/status that renders the bare
* "Processing…" label for the WHOLE run (phase 90, A2 — the unpack
* has no file-level progress: no file, no "(n/m)" counts, no title)
* and settles it: success → the ready-for-sync line ("Uploaded
* <name> — press Sync sources to import it.", fmtUploadResult off
* the status's {"message": "uploaded"} detail) in the role=status
* result line + the "Archive uploaded — press Sync sources to
* import it." announce + loadSources (NO second toast — it already
* fired at the 202, A2); failure → the sanitized server error in
* the role=alert banner + loadSources, the file selection KEPT for
* a one-click re-upload. 409 (an upload is already in progress)
* raises NO error banner — it re-attaches to the in-flight run
* (processing state + poll, never stale); the phase-49 "server
* detail inline for 409" branch is superseded. Other non-2xx (422
* format/name, 413 cap, 5xx) keep the phase-49 error banner + the
* kept file selection; a network failure keeps the fixed line. The
* submit finally restores the button ONLY when no poll is active
* (PLAN §7.4 — while startUploadPolling owns the button it stays
* disabled / "Processing…"). Boot re-attach (initUploadStatus, the
* admin branch): a running scan re-enters the processing state + poll
* admin branch): a running run re-enters the processing state + poll
* (no second upload, no error); a terminal run re-renders its result
* line (success) or error banner (failed); idle does nothing.
* (The phase-49 synchronous 200 paragraph is superseded by phase 64.) */
* (The phase-49 synchronous 200 paragraph is superseded by phase 64;
* the phase-64 scan counts are superseded by phase 90.) */
/* The success line's text — the sync-result shape (sources.js's
fmtSyncResult convention): "N added" always leads, then updated /
unchanged / pruned — zero parts omitted (unchanged is shown
when nothing was added or updated). Reads exactly the keys the
upload status's detail carries (task 03's UploadOut-shaped dict). */
function fmtUploadResult(detail) {
const d = detail || {};
const added = d.added || 0;
const updated = d.updated || 0;
const parts = [`${added} added`];
if (updated > 0) parts.push(`${updated} updated`);
if ((d.unchanged || 0) > 0 || (added === 0 && updated === 0)) {
parts.push(`${d.unchanged || 0} unchanged`);
/* The result line's text (phase 90, A2 — the no-count contract): the
status success detail is exactly {"message": "uploaded"} — the
sync-style counts the phase-64 line rendered are gone (the scan —
and its counts — belong to the Sync button, which renders them on
the RAG page). `name` is the accepted 202's safe source name
(lastUploadName) when the run started on this page; it is null
after a reload or on the 409 re-attach (the line still points at
the next step, only without the name). */
function fmtUploadResult(detail, name) {
if (detail && detail.message === "uploaded") {
return name
? `Uploaded ${name} — press Sync sources to import it.`
: "Uploaded — press Sync sources to import it.";
}
if ((d.pruned || 0) > 0) parts.push(`${d.pruned} pruned`);
return parts.join(" · ");
return "The upload finished.";
}
/* Upload-success toast (phase 64 task 05, A2 — owner-locked): the
@@ -903,14 +917,15 @@ export async function mount(root) {
}, UPLOAD_TOAST_MS);
}
/* The scan poll (phase 64 task 05): a 2 s cadence — the SYNC_POLL_MS
* house value. Single timer, one loop at a time (the guard makes a
* double-start a no-op, and the submit finally reads this same
* variable to know whether the poll OWNS the button). Each tick
* fetches GET /api/git-sources/upload/status: running → the live
* "Processing… <file> (n/m)" label (A4 — bare "Processing…" during
* the unpack phase, before any file is indexed; the full untruncated
* path rides the button title) + reschedule; success → stop + the
/* The background-run poll (phase 64 task 05, unpack-only in phase
* 90): a 2 s cadence — the SYNC_POLL_MS house value. Single timer,
* one loop at a time (the guard makes a double-start a no-op, and
* the submit finally reads this same variable to know whether the
* poll OWNS the button). Each tick fetches
* GET /api/git-sources/upload/status: running → the bare
* "Processing…" label for the whole run (phase 90, A2 — the unpack
* has no file-level progress: no file, no "(n/m)" counts, the title
* stays clear) + reschedule; success → stop + the ready-for-sync
* result line + the announcement + the row reload (NO toast — it
* fired at the 202, A2); failed → stop + the sanitized server error
* banner + the row reload (a post-swap failure keeps the row — the
@@ -920,6 +935,7 @@ export async function mount(root) {
* retries next tick. */
const UPLOAD_POLL_MS = 2000; // the SYNC_POLL_MS house value
let uploadPollTimer = null; // null = no poll active (the finally's guard)
let lastUploadName = null; // phase 90: the accepted 202's safe source name — the result line's <name> (null after a reload / on the 409 re-attach)
function stopUploadPolling() {
if (uploadPollTimer !== null) {
@@ -929,8 +945,9 @@ export async function mount(root) {
}
/* The button's processing entry (the 202 + the 409 re-attach): from
* here the poll OWNS it — disabled, "Processing…", title cleared (a
* live file lands on it at the first tick). */
* here the poll OWNS it — disabled, bare "Processing…", title
* cleared (a live file never lands on it — phase 90: the run is
* unpack + register only, so the label stays bare). */
function enterUploadProcessingState() {
uploadBtn.disabled = true;
uploadBtn.textContent = "Processing…";
@@ -941,7 +958,7 @@ export async function mount(root) {
* finally, which calls this ONLY when no poll is active — PLAN §7.4). */
function restoreUploadButton() {
uploadBtn.disabled = false; // never stale — success OR failure
uploadBtn.textContent = "Upload & scan";
uploadBtn.textContent = "Upload";
uploadBtn.removeAttribute("title");
}
@@ -957,28 +974,27 @@ export async function mount(root) {
uploadPollTimer = setTimeout(tick, UPLOAD_POLL_MS);
return;
}
// running: the live file label (A4 — bare "Processing…" during
// the unpack phase, before any file is indexed).
// running: the bare label for the whole background run (phase
// 90, A2 — the unpack has no file-level progress, so no file,
// no counts, and the title stays clear).
if (status.state === "running") {
uploadBtn.textContent =
"Processing…" +
(status.current_file ? ` ${status.current_file}` : "") +
(status.files_total > 0 ? ` (${status.files_done}/${status.files_total})` : "");
uploadBtn.title = status.current_file || ""; // full path on hover
uploadBtn.textContent = "Processing…";
uploadBtn.title = "";
uploadPollTimer = setTimeout(tick, UPLOAD_POLL_MS);
return;
}
stopUploadPolling();
if (status.state === "success") {
// The scan finished: the result line (the existing helper reads
// exactly these keys), the announcement, the row lands. NO toast
// here — it already fired at the 202 (A2).
// The run finished (unpack + register only — phase 90): the
// ready-for-sync line (fmtUploadResult reads the no-count
// detail), the announcement, the row lands. NO toast here — it
// already fired at the 202 (A2).
const detail = status.detail || {};
if (uploadResult) {
uploadResult.textContent = fmtUploadResult(detail);
uploadResult.textContent = fmtUploadResult(detail, lastUploadName);
uploadResult.hidden = false;
}
announce(`Archive uploaded: ${detail.source}.`);
announce("Archive uploaded — press Sync sources to import it.");
uploadFileInput.value = "";
restoreUploadButton();
loadSources(); // the row lands / refreshes
@@ -990,7 +1006,7 @@ export async function mount(root) {
// one-click re-upload, and the list reloads (a post-swap failure
// keeps the row — the list state may have changed).
if (uploadError) {
uploadError.textContent = status.error || "The upload scan failed.";
uploadError.textContent = status.error || "The upload failed.";
uploadError.hidden = false;
}
restoreUploadButton();
@@ -1005,12 +1021,15 @@ export async function mount(root) {
}
/* Boot re-attach (phase 64 task 05, the admin branch only): fetch the
* upload status ONCE — a running scan re-enters the processing state
* + the poll (a reload mid-scan re-attaches instead of dead-ending —
* upload status ONCE — a running run re-enters the processing state
* + the poll (a reload mid-run re-attaches instead of dead-ending —
* no second upload, no error); a finished run re-renders its result
* line ONLY (no announce, no toast — the toast fired at the 202, A2);
* a failed run re-renders its error banner; idle does nothing (and a
* blip is a no-op — the page boots honest either way). */
* line ONLY (no announce, no toast — the toast fired at the 202, A2;
* the name is unknown after a reload — lastUploadName is null — so
* the line is the nameless "Uploaded — press Sync sources to import
* it.", phase 90); a failed run re-renders its error banner; idle
* does nothing (and a blip is a no-op — the page boots honest
* either way). */
async function initUploadStatus() {
if (!uploadBtn) return;
let status;
@@ -1029,15 +1048,17 @@ export async function mount(root) {
}
if (status.state === "success") {
// The last run's result line only — no announce, no toast (A2).
// The safe name was page-local (lastUploadName is null after a
// reload) — the line still points at the next step (phase 90).
if (uploadResult) {
uploadResult.textContent = fmtUploadResult(status.detail);
uploadResult.textContent = fmtUploadResult(status.detail, lastUploadName);
uploadResult.hidden = false;
}
return;
}
if (status.state === "failed") {
if (uploadError) {
uploadError.textContent = status.error || "The upload scan failed.";
uploadError.textContent = status.error || "The upload failed.";
uploadError.hidden = false;
}
}
@@ -1070,11 +1091,13 @@ export async function mount(root) {
body: new FormData(uploadFormEl),
});
if (r.status === 202) {
// The archive is safely on disk (A1) — the "successfully
// uploaded" moment: the toast fires NOW (A2), the file input
// clears, and the scan's poll takes over the button. The 202
// body (UploadAccepted) carries the safe source name; a body
// parse failure degrades to the picked file's name.
// The archive is safely on disk (phase 64 A1) — the
// "successfully uploaded" moment: the toast fires NOW (A2),
// the file input clears, and the background run's poll takes
// over the button. The 202 body (UploadAccepted) carries the
// safe source name (the settled result line's <name>, phase
// 90 — recorded page-locally); a body parse failure degrades
// to the picked file's name.
let name = file.name;
try {
const data = await r.json();
@@ -1082,6 +1105,7 @@ export async function mount(root) {
} catch {
/* body parse failure — the picked file's name degrades fine */
}
lastUploadName = name;
showUploadToast(`Successfully uploaded — ${name}`);
uploadFileInput.value = ""; // 202: the archive is on the server
enterUploadProcessingState();
@@ -1111,7 +1135,7 @@ export async function mount(root) {
}
} finally {
// Never stale (PLAN §7.4) — but ONLY when no poll owns the
// button: while startUploadPolling tracks the scan (202 / 409)
// button: while startUploadPolling tracks the run (202 / 409)
// it stays disabled / "Processing…", so a finally restore here
// would race the poll. No poll → the button is ours to restore.
if (uploadPollTimer === null) restoreUploadButton();
@@ -1159,8 +1183,9 @@ export async function mount(root) {
fetch /api/git-sources (the Sources-page gate pattern). */
root.addEventListener("bor:view-refresh", () => loadSources());
await loadSources();
// Phase 64 (task 05): re-attach a running scan (a reload mid-scan
// resumes the Processing state) or re-render a terminal run's
// result line / error banner.
// Phase 64 (task 05): re-attach a running run (a reload mid-run
// resumes the bare Processing state) or re-render a terminal
// run's result line / error banner (phase 90: the unpack-only,
// ready-for-sync line).
await initUploadStatus();
}