feat(web): move the chat action cluster to the pinned bottom and align the button sets

- task 01: relocate the .chat-actions row (New chat + Share, comments byte-identical with a Phase 65 note) from the top of the column to the bottom of .chat-shell, directly above the composer - task 02 (owner-locked A1): wrap the row + #composer in ONE sticky .chat-bottom unit (position: sticky; bottom: env(safe-area-inset-bottom, 0), no z-index) — the pills stay at the bottom of the screen at every scroll position and settle into flow above the footer - task 03 (owner-locked A2): right-align the bottom row to the column's right edge (justify-content: flex-end), mirroring the right-aligned Save-as-doc corner; the five action pills share one 44px / 999px-pill geometry - task 04: dedicated Playwright suite tests/e2e/test_bottom_chat_actions.py (resting geometry, the A1 pin across the sticky range, A2 alignment + DOM order + mobile stack + 360px overflow bound + 44px touch targets, New chat / Share click-through) — green in isolation - task 05: regression matrix green in isolation (pinned_composer 4, save_share_ux 5, chat_persistence 4, share_chat 4, chat_history 5, smoke 3); full gate green — unit + integration pass, app/ coverage 99% (>90%), ruff + pyright clean
2026-09-02 01:08:18 -04:00
parent 4677d86f49
commit 8a1f99cb38
30 changed files with 1376 additions and 147 deletions
@@ -0,0 +1,49 @@
+# Phase 64 — Real-Time Progress for Sync + Upload, Background Upload
+
+**Source:** `TODO.md` L3 — "Need better indication of sync and upload progress. Both should show current file being processed in real time, not just 'syncing' or 'uploading'." (+ the navigate-away / toast half of the same item)
+**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-09-01)
+**Context:** `app/api/sync.py` (phase 32 — background sync + the 2 s `GET /api/sync/status` polling pattern, the template for everything here), `app/api/git_sources.py` (phase 49 — the upload route whose scan runs **synchronously in the request**: the browser `fetch` blocks until unpack + import finish, so navigating away mid-upload aborts it — the exact defect this phase removes), `app/rag/importer.py` (`import_sources` loops file-by-file with **no** progress hook — the single place both flows can be instrumented), `frontend/assets/sources.js` (the sync button's §7.4 never-stale lifecycle + load-time re-attach), `frontend/assets/git-sources.js` (the upload form's "Uploading…" lifecycle), `frontend/assets/styles.css` (`.toast` — the phase-55 share-success toast, reused verbatim for the new "successfully uploaded" toast). E2E conventions: `tests/e2e/test_archive_upload_sources.py` (archive fixture builder, mock LLM, `BOR_UPLOAD_DIR` scratch) and `tests/e2e/test_sync_button.py`.
+
+## Objective
+Both long-running KB jobs report **which file is being processed right now** (not just "Syncing…"/"Uploading…"), and the archive upload is fully **backgrounded**: `POST /api/git-sources/upload` answers 202 the moment the archive is on disk (a "successfully uploaded" toast fires — the user may navigate away), the unpack/scan continues server-side with a status endpoint, and the sync button on the RAG page (`/sources.html`) animates with the upload's current file while that scan runs.
+
+## Dependencies
+- `63_unambiguous_document_listing` (complete) — the immediately preceding phase (the todo queue was empty at authoring; this phase builds on no unfinished work).
+
+## Tasks
+1. `01_importer_progress_hook.md` — optional per-file progress callback on `import_sources` (source, rel path, done/total).
+2. `02_sync_status_current_file.md` — `SyncStatus` + `GET /api/sync/status` carry `current_file` (+ counts), wired through the hook.
+3. `03_upload_background_202.md` — `POST /upload` → 202 + background task + `GET /api/git-sources/upload/status` (phase-32 pattern, incl. `current_file`).
+4. `04_sync_button_live_file.md` — RAG-page sync button: live file label for sync runs AND for in-flight upload scans, catalog refresh + settle on upload completion, load-time re-attach.
+5. `05_upload_toast_progress.md` — Sources-page upload UI: "successfully uploaded" toast on 202, live "Processing… <file>" label via status polling, 409/load-time re-attach, failure banner.
+6. `06_e2e_sync_upload_progress.md` — the story Playwright suite + `test_archive_upload_sources.py` adaptation + regressions + commit.
+
+## Testing & Quality
+- Unit: the progress-hook contract in `tests/unit/test_importer.py`; status shapes in `tests/unit/test_sync_button.py`; the 202/background/upload-status contract in `tests/unit/test_archive_upload.py` (adapted from the synchronous expectations).
+- Frontend source pins (house pattern): a new `tests/unit/test_frontend_sync_upload.py` — label builders, the polling decision trees, toast-on-202, re-attach paths.
+- Coverage: **>90%** on `app/` (`validate.sh` gate).
+- E2E (mandatory, A16): `tests/e2e/test_sync_upload_progress.py`, run in isolation; `test_archive_upload_sources.py` updated to the 202 + toast + polling flow; regressions `test_sync_button.py`, `test_git_sources_admin.py`, `test_sync_model_down.py` in isolation.
+
+## Completion Criteria
+- [ ] `GET /api/sync/status` and `GET /api/git-sources/upload/status` both report `current_file` / `files_done` / `files_total` while their job runs (null/0 idle).
+- [ ] `POST /api/git-sources/upload` returns **202** with the safe source name once the archive is fully received; unpack + scan continue in a background task; one upload at a time (409 in flight).
+- [ ] The "Successfully uploaded — <file>" toast appears on the Sources page at 202 (before the scan finishes); navigating to `/sources.html` mid-scan shows the sync button animating with the upload's current file; on completion the button settles and the catalog shows the new documents.
+- [ ] The sync button's label shows the current file during sync runs ("Syncing… <file>") and during upload scans ("Importing <file>"); the upload area shows "Processing… <file>" during the scan.
+- [ ] `uv run pytest` green; `app/` coverage >90%.
+- [ ] `uv run pytest tests/e2e/test_sync_upload_progress.py -v --no-cov` green in isolation (DB up); `test_archive_upload_sources.py`, `test_sync_button.py`, `test_git_sources_admin.py`, `test_sync_model_down.py` green in isolation.
+- [ ] `uv run ruff check . && uv run pyright` clean.
+- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
+
+## Locked decisions
+- **Owner-locked (2026-09-01, roadmap confirmation):**
+  - **A1 — the upload goes fully background.** 202 means "archive safely on disk"; unpack → swap → row upsert → model check → import → overview run in a background task; `GET /api/git-sources/upload/status` is the single source of truth (phase-32 `SyncStatus` pattern). One upload at a time stays (409). This is the only way "navigate away before it finishes" works.
+  - **A2 — toast timing/scope.** The "successfully uploaded" toast fires on the Sources page **at 202** (file received), auto-dismisses (~5 s, phase-55 `.toast`); failures are announced by the existing `#archive-upload-error` banner, never a toast.
+  - **A3 — RAG-page sync button during an upload scan.** It enters its running animation showing the upload's current file ("Importing <file>"); on upload completion it settles to "Sync sources" and the catalog refreshes (`loadDocs()`). The upload's count line itself stays on the Sources (git-sources) page — the sync-result line never renders upload counts.
+  - **A4 — progress granularity.** Per importable file: labels show `source/relative/path` (truncated with ellipsis; the full path lives in the button `title` + the aria-live result line). The unpack phase (before any file is indexed) shows no file yet — just "Processing…"/"Syncing…".
+  - **A5 — post-202 failures.** Unpack / zero-entry / swap / model / import failures land in the upload status as `failed` with a sanitized error (the `_sanitize_error` credential mask). Pre-swap failures leave the KB, folders, and rows untouched; post-swap failures keep the folder + row so a re-upload or the next sync retries idempotently (the existing phase-49 step-7 semantics, now in the status state instead of an HTTP error).
+- **A9/A10/A16/A17 honoured** — the admin-only API surface stays stateless apart from the in-memory run state (phase 32 precedent); one story E2E suite; one atomic commit.
+
+## Commit
+```bash
+git add -A .agent/ app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(sources): real-time file progress for sync and upload — background upload with success toast"
+```
@@ -0,0 +1,40 @@
+# Task 06 — E2E suite, regressions, commit
+
+**Phase:** `64_sync_upload_progress` · **Source:** `TODO.md:3` — the whole item (the executable proof of every locked decision).
+**Story:** n/a (TODO-derived)
+
+## Objective
+The story's Playwright suite proves the full contract end-to-end (toast on receive → navigate away → sync button animating with the upload's current file → catalog refreshed; live file labels on both jobs; reload re-attach), `test_archive_upload_sources.py` is brought onto the 202 contract, the regression suites stay green in isolation, and the phase is committed.
+
+## Work
+1. `tests/e2e/test_sync_upload_progress.py` — new suite (house conventions: `tests/e2e/conftest.py` + `mock_llm.py`, the admin sign-in and fixture style of `tests/e2e/test_git_sources_admin.py`, the archive builder + `BOR_UPLOAD_DIR` scratch of `tests/e2e/test_archive_upload_sources.py`, the local-source sync fixture style of `tests/e2e/test_sync_button.py`):
+   - **Fixture note (timing):** the mock LLM indexes fast — the in-progress state is real but brief. Build the upload archive from **20+ small `.md` files** so the scan outlasts the 2 s poll, and assert the live-file label at TWO layers: the deterministic one is the status endpoint (`page.request.get("/api/git-sources/upload/status")` / `"/api/sync/status"` — `state == "running"` with non-null `current_file` observed at some tick); the UI one polls the button label for the `Importing ` / `Syncing…` / `Processing…` prefix plus a file path (generous timeout).
+   - `test_upload_toast_then_navigate_away` — on `/git-sources.html`: pick the multi-file archive, submit → the `Successfully uploaded — <archive>` toast (`.toast.is-visible`, `role="status"`) appears **while the scan is still running**; immediately navigate to `/sources.html` → the sync button is present, animating (icon `is-spinning`, `aria-busy`) with the `Importing` label; wait for the settle → button idle ("Sync sources"), no error UI, and the catalog table shows the uploaded documents (the phase-63 listing is untouched).
+   - `test_upload_progress_shows_current_file` — during the scan: the status endpoint reports a non-null `current_file` (`source/relative/path` shape) at some running tick, and the upload button label shows `Processing…` with a file path (UI layer) before the result line (`fmtUploadResult` counts) lands; the toast fired earlier in the run (not after the result).
+   - `test_sync_live_file_label` — a multi-file local source configured (the `test_sync_button.py` fixture style): on `/sources.html` click **Sync sources** → the label shows `Syncing…` with a file path while running (endpoint layer: `/api/sync/status` `current_file` non-null; UI layer: label poll), then the success settle with the counts result line (the pre-phase-64 sync UX is preserved, plus the file).
+   - `test_upload_reattach_after_reload` — start the upload, and **during the scan** reload `/git-sources.html` → the button is in the Processing state (disabled) with **no** error banner and no second upload (the status endpoint's single run is still the one from before the reload); it then settles with the result line and the list shows exactly one row for the archive (in-place identity preserved).
+2. `tests/e2e/test_archive_upload_sources.py` — adapt to the 202 contract (keep every scenario, move observation points):
+   - upload → the toast appears at 202, the button goes `Uploading…` → `Processing…` → restore; the result line + list now come from the status `success` (counts unchanged in shape).
+   - the inline 422/413 gates are unchanged (asserted exactly as today — name, format, cap, file kept).
+   - a bad-archive scenario (if present) now lands as the **error banner** via status `failed` instead of a 422 response; re-upload after it still works.
+   - the re-upload-in-place-replace scenario: the second run's status `detail` shows the prune/refresh counts; the list still has exactly ONE row for the archive name.
+   - anonymous: the gate/form stay hidden, `POST /api/git-sources/upload` 403, and `GET /api/git-sources/upload/status` 403 (new endpoint, same wall).
+3. Run in isolation (DB up, the AGENTS.md rule-9 command): `test_sync_upload_progress.py` (new), `test_archive_upload_sources.py` (updated), `test_sync_button.py`, `test_git_sources_admin.py`, `test_sync_model_down.py`.
+4. Full gate: `uv run pytest --cov=app --cov-report=term-missing` (suite green, `app/` >90%), `uv run ruff check . && uv run pyright`.
+5. Commit + hand-off:
+   ```bash
+   git add -A .agent/ app/ frontend/ tests/
+   git commit --no-gpg-sign -m "feat(sources): real-time file progress for sync and upload — background upload with success toast"
+   ```
+   then move `.agent/phases/todo/64_sync_upload_progress/` → `.agent/phases/complete/64_sync_upload_progress/` (the pipeline's `validate.sh` gate is the move's precondition).
+
+## Testing & Quality
+- E2E: the new suite + the five isolated runs above are this phase's A16 gate.
+- Coverage: **>90%** on `app/` (no new app code in this task — the gate guards against drift from tasks 01–03).
+
+## Completion Criteria
+- [ ] `uv run pytest tests/e2e/test_sync_upload_progress.py -v --no-cov` green in isolation (DB up).
+- [ ] `test_archive_upload_sources.py`, `test_sync_button.py`, `test_git_sources_admin.py`, `test_sync_model_down.py` green in isolation.
+- [ ] `uv run pytest --cov=app` green with `app/` >90%; `uv run ruff check . && uv run pyright` clean.
+- [ ] One `--no-gpg-sign` Conventional Commits commit; phase dir in `.agent/phases/complete/`.
+- [ ] `TODO.md` contains only `# TODO` (cleared by the phase pipeline's conversion step — items now live here).