31bd903a9361ab38be41fd2307880ed3dbf895af
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ducoterra
|
c275735db4 |
phase: 154_sync_status_freshness
All 4 task files exist in `complete/`, and only `00_phase.md` remains in `todo/`. This is the final verification pass. I'll read through the task files and check the current state of the code. |
||
|
ducoterra
|
12fa69b7ea |
phase: 152_sync_progress_image_pass
All criteria verified. The CLI path is hook-less (only `app/api/sync.py` passes the hook), and the byte-identity is pinned by `test_on_image_progress_none_run_is_byte_identical`. ## Final Report — Phase 152 final verification pass - **Verified (no new code needed):** `SyncStatus.images_done/images_total` + reset/clear rules in `app/api/sync.py`; `on_image_progress` threaded through the importer's three PDF vision spans with the continuous per-file counter; the `_image_hook` closure; `sources.js` `· image n/m` label part (box + title + aria). - **C1** status climbs 1→N + UI shows `· image n/m`: ✅ unit 3/15→15/15, integration 1/15→7/15, E2E 1/4→4/4 (title/aria ride the full string) - **C2** never frozen: ✅ E2E asserts ≥3 distinct positions, starting at 1, non-decreasing, ≤ total, reaching it - **C3** cleared at overview/summaries + all terminals; per-file reset via `_hook`; no stale position for image-less files: ✅ unit pins + E2E settle (image part gone in box/title/aria, endpoint 0/0) - **C4** `/api/sync/status` additive-only: ✅ full-dict pin passes (all pre-existing keys unchanged + the two new 0/0 keys) - **C5** hook-less caller byte-identical: ✅ CLI passes no hook; `test_on_image_progress_none_run_is_byte_identical` green - **C6** `uv run pytest`: **3526 passed, exit 0** (95s); dedicated E2E in isolation: **2 passed** (95s); `uv run ruff check . && uv run pyright`: **clean**; coverage: **99% on `app/` (>90% gate)** - Regression: all five neighbouring E2E suites green in isolation (progress_box 8, chunk_progress 2, label_mobile 4, pdf_support 6, image_documents 4) - **C7** atomic commit: left to the harness (no git index/history touched per rules) - **Docs:** README.md — sync-label paragraph already updated this phase, accurately; all `.env.example` vars documented; routes/commands verified — **no changes needed**. AGENTS.md — all references (PLAN, user_stories, test_smoke.py, .env.example, gpg config, .gitignore) verified — **no changes needed** - **Defects found in prior work:** none (one prior env-leak defect was already fixed inside this phase's task 04, conftest) - No live-infrastructure touched; no ntfy used. **Next pending phase:** none — `todo/` held only this phase |
||
|
ducoterra
|
edf6048a0d |
phase: 150_drop_sync_result
**Phase 150 final verification pass — complete, 100% green.** All 6 task files were already in `complete/`, so this was verification + doc pass. - Verified: `#sync-progress` is the sole sync text surface + sole `aria-live` region (`role="status"`, polite, no `aria-hidden`); `span#sync-result` gone from HTML/JS/CSS/tests (incl. runtime-built `RETIRED_ID` absence pins in the unit suites); `setSyncProgress` single writer with text-driven visibility; running-only box write in `setSyncLabel`; all 9 settle/cancel sites routed; `requestCancel` pre-fetch clear is a NO-OP. - Tests: `uv run pytest` → **3463 passed**; `uv run pytest --cov=app` → **99% on app/** (>90%); `uv run ruff check .` + `uv run pyright` → clean (0 errors); `tests/e2e/test_sync_progress_box.py --no-cov` → **8 passed in isolation**; all 18 migrated sync/image-PDF E2E files green in isolation (77 tests). - Completion criteria: 1) grep clean ✅ 2) sole live region, attrs correct ✅ 3) in-flight/settle/cancel/idle box states proven by E2E + unit pins ✅ 4) icon-only running button + restore ✅ 5) full gates green ✅ 6) no completed-phase behavior change (18 family suites + full suite green) ✅ 7) commit — left to harness per pass instructions (working tree untouched, no `git add`/`commit`). - Documentation: README.md fixed 2 pre-existing inconsistencies (stale `~/Homelab`+`~/Deployments` source model in the intro — retired in phase 61 — now describes the git/local/archive Sources model; "Requires AIPI_KEY" in retrieval-quality now states `BOR_LLM_API_KEY` in `.env` or `$AIPI_KEY`, matching the scripts' real convention). Everything else (URL table, env defaults vs `app/config.py`, sync/phase-150 copy, prod claims) verified accurate. AGENTS.md: verified consistent — no changes needed. - Notable: no code changes were required — the phase implementation was correct as left; no live infrastructure touched. - Next pending phase: `151_pdf_image_extraction` in `.agents/phases/todo/`. |
||
|
ducoterra
|
dd524246ec |
phase: 149_sync_progress_box
All verification complete. The `ink-soft`/`--surface` pair the box uses is already pinned AA by the existing theme E2E suites (8.2:1 dark, ~6.9:1 light). Everything is green — no fixes were needed.
**Phase 149 final verification report**
- Verified (no re-work needed): `#sync-progress` box wiring (single `setSyncLabel` writer with `running` flag; `is-running` class travels with the spin; aria-label owned by the writer), the panel/icon-only-button CSS, and the 5-test dedicated E2E; all 6 pre-existing sync E2Es' running pins migrated to the box, settled pins untouched.
- Dedicated E2E: `uv run pytest tests/e2e/test_sync_progress_box.py -v --no-cov` → **5 passed in 198.6 s** (isolation, slow-proxy mock; under the 600 s budget).
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **3463 passed, app/ coverage 99%** (>90% gate).
- Migrated sync E2Es, each in isolation: `test_sync_button` 3 passed, `test_sync_cancel` 3, `test_sync_chunk_progress` 2, `test_sync_label_mobile` 4, `test_sync_summary_visibility` 3, `test_sync_upload_progress` 4 — all green.
- Lint/types: `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Criteria: in-flight spinner-only button + full untruncated wrapped box (desktop & mobile 390×844, `scrollWidth ≤ clientWidth`) ✓; settle byte-identical ("Synced HH:MM" + `3 added`, box hidden, aria restored) ✓; `#sync-result` sole live region (box `aria-hidden`), mid-run reload re-attaches ✓; WCAG (AA contrast both themes via the pinned ink-soft/surface pair, focus-visible, reduced-motion kill intact at styles.css L2201) ✓; cancel/409/error/upload/status API unchanged (green suites above) ✓.
- Documentation: README.md reviewed in full (sync-states paragraph now matches the shipped UI; env-var table matches `app/config.py` and covers all `.env.example` vars; pages/commands verified) and AGENTS.md reviewed in full — **verified consistent, no changes needed**.
- Notable: no defects found in this or prior work; no live infrastructure touched.
- Next pending phase: none — `todo/` will be empty once the harness moves 149 (only 149 was pending).
|
||
|
ducoterra
|
fcac876d40 |
phase: 134_sync_cancel
**Phase 134 final verification pass — all green** (all 3 task files already in `complete/`; no implementation was missing) - Verified: 5-state `SyncStatus` + `cancelled` terminal (error None, final counts kept), `POST /api/sync/cancel` (202/409/403), cooperative `_check_cancelled` boundaries, shutdown `CancelledError` contract preserved, `#sync-cancel-btn` show/hide + neutral "Sync cancelled" settle, 8-branch poll tree, re-attach — code, unit, integration, and E2E layers all present - `uv run pytest --cov=app --cov-report=term-missing` → **3150 passed**, `app/` coverage **99%** (gate: >90%) - `uv run pytest tests/e2e/test_sync_cancel.py -v --no-cov` (isolated) → **3 passed** (47.6 s) - `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings **Completion criteria:** ① cancel endpoint 202/409/403 ✅ (unit + E2E) ② `cancelled` terminal reported by status, error null, counts kept ✅ (integration `test_cancel_mid_run_lands_in_cancelled_then_resyncs_clean`) ③ unrequested-cancel pin ✅ (`test_unrequested_cancel_is_never_swallowed_and_never_converts`) ④ UI lifecycle + clean re-sync (sentinel row, full 40-file import) ✅ (E2E) ⑤ suite/coverage/E2E gates ✅ ⑥ uncancelled runs byte-identical ✅ (all 3150 pre-existing tests green untouched) - Docs: README.md reviewed in full against code (pages, auth endpoints, every config default vs `app/config.py`, palette 9+8, fail-loud, cancel section) — consistent, no changes needed. AGENTS.md: fixed one pre-existing inaccuracy (rule 2 "move its file" → "move its directory" — phases are directories); everything else verified accurate - No live infrastructure touched; no deviations **Next pending phase:** `135_git_source_branch` |
||
|
ducoterra
|
2d0f608246 |
phase: 129_sync_chunk_progress
All green. Final report:
**Phase 129 final verification pass — 100% green** (previous work intact; this pass re-verified everything and fixed two pre-existing README inconsistencies)
- Verified on-spec: `LLMClient.embed` per-batch `on_progress` hook; importer `on_chunk_progress` plumbing (text/image/PDF branches; summary embed excluded); `SyncStatus.current_chunk/chunks_total` reset+clear contract; UI `.sync-label-chunk` pinned part with untruncated `title`/`#sync-result`
- `uv run pytest --cov=app -q`: exit 0, ~3041 passed, 0 failed, `app/` coverage **99%** (>90% gate)
- `uv run pytest tests/e2e/test_sync_chunk_progress.py -v --no-cov` (isolated, DB up): **2 passed** (endpoint chunk climb with true `chunk_document` total + UI `· chunk n/m` part)
- `uv run ruff check .`: all checks passed · `uv run pyright`: 0 errors, 0 warnings
- Completion criteria: all 5 ✅ (incl. no behavior change in completed phases — full suite green, CLI/chat no-op paths byte-identical)
- Docs: README.md fixed 2 pre-existing inconsistencies (upload section claimed "unpacks + scans immediately" — phase 90 made it unpack+register only, scan deferred to Sync; troubleshooting claimed "120s client timeout" — actual guard is the 300 s pre-token `TURN_TIMEOUT_MS`). Phase-129 label behavior already documented. AGENTS.md re-verified consistent — no changes
- Removed stray untracked `NOTES.md` (8-byte artifact); no live-infra changes; no commits — all changes left in working tree for the harness
- Next pending phase: `130_chat_answer_clusters`
|
||
|
ducoterra
|
1139f007cf |
phase: 128_sync_label_mobile
Everything verified and 100% green. The earlier rejection was a report-format issue, not a test failure — all work is in the working tree. Final report: **Phase 128 final pass — verified (all 3 tasks complete; no code defects found)** - Verified: `#sync-label` parts markup, `setSyncLabel` writer, part builders (A4 untruncated title/aria-live), flex-row CSS + mobile media query — all match design 1:1. - `uv run pytest --cov=app --cov-report=term`: 3024 passed; TOTAL coverage **99%** (>90% gate). - `uv run pytest tests/e2e/test_sync_label_mobile.py -v --no-cov`: 4 passed (119s, isolated). - `uv run ruff check .`: clean; `uv run pyright`: 0 errors. - Criteria: (1) mobile 390×844 label in-viewport, count visible in import+summaries — ✅ E2E; (2) desktop strings/cap/title unchanged — ✅ E2E+unit; (3) E2E isolated green — ✅; (4) full suite green — ✅. - Docs: README.md fixed 5 pre-existing gaps (PDF phase-126 section + `pdf` token, hidden-folders toggle ph.105, document dates ph.106, private git-source masked tokens ph.121, doc-draft Discard ph.115); AGENTS.md verified consistent — no changes. - Notable: double-`-q` (addopts `-q` + CLI `-q`) suppresses pytest's summary line — standard `-qq` behavior, not a bug; exit codes confirm green. - Next pending phase: `129_sync_chunk_progress`. |
||
|
ducoterra
|
f665a83b1a |
phase: 98_sync_summary_visibility
All verification complete. Final report: **Phase 98 — Sync summary visibility: final verification pass** (all 5 tasks already complete; implementation verified against the design, no defects found, no code changes needed) - **Implementation checked:** `SyncStatus` phase machine (4 new keys, terminal-keep counts), `on_progress` hook in `generate_folder_summaries`, `summary_pending` on `KbTreeSource`/`KbTreeFolder` + D3 rule in `build_kb_tree`, phase-aware sync labels + pending UI in `sources.js`, `.kb-summary-pending` CSS — all match decisions D1–D5. - **Unit + integration:** `uv run pytest` → 2184 tests, 0 failed/errors (exit 0) - **Coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **99%** on `app/` (criterion >90% ✓; `app/api/sync.py` and `app/rag/folder_summaries.py` at 100%) - **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings - **Phase E2E (isolation):** `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` → **3 passed** (phase machine, live label, pending markers + gap-fill self-heal) - **Regression suites (each isolated, `--no-cov`):** test_kb_tree ✓, test_ls_tree_drilldown 3 ✓, test_sync_button 3 ✓, test_sync_upload_progress 4 ✓, test_oneshot_llm_retry 2 ✓, test_local_directory_sources 3 ✓ - **Completion criteria:** all 7 verified green — status phase fields + terminal semantics; `Writing KB overview…`/`Summarizing folders… (n/m)` labels (title + aria-live); pending set == `missing_folder_summaries` (integration cross-check pinned at `test_docs_api.py:428`); CLI/`ls` byte-identity (no changes to those paths, pins green); suite/coverage/lint gates; dedicated + regression E2E. Commit left to the harness per protocol (no `git add`/`commit` run). - **Decisions/deviations:** none — no fixes were required this pass. - **Next pending phase:** `99_kb_tree_table_and_back_nav`. |
||
|
ducoterra
|
ad7585d474 |
phase: 97_kb_tree_catalog
All completion criteria verified — everything is green, no defects found. Final report: ## Phase 97 final verification pass — ALL GREEN **Verified (no code changes needed):** - `GET /api/docs/tree` (admin), `build_kb_tree` pure builder, `PATCH /api/folders/summary`, migration 0018 (`manually_edited`, head confirmed), generator skip/keep + `kept_manual` stat, RAG tree UI + edit affordance in `sources.js`/`index.html`/`styles.css` - `tests/e2e/test_kb_tree.py`: 8 passed — top level, drill source/folder, edit round-trip, clear, manual-desc-survives-sync, reload fallback, anonymous gate - Integration: tree shape/order/403/empty/indexed-only + PATCH update/create/root/clear/404/403/no-LLM + stat-walk equivalence (in `test_docs_api.py`); 3-field `folder_summaries=` import token preserved **Gates (exact commands):** - `uv run pytest --cov=app --cov-report=term-missing` → **2053 passed**, TOTAL coverage **99%** (>90% ✓) - `uv run ruff check . && uv run pyright` → **All checks passed / 0 errors** - `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` → **8 passed** in isolation - 30 story/RAG-view E2E suites run **one per process**: all passed, incl. `test_ls_tree_drilldown` (agent `ls` byte-identical ✓), `test_import_documents`, `test_edit_summaries`, `test_admin_auth`, `test_kb_overview` **Completion criteria:** tree view ✓ · edit round-trip + clear ✓ · manual persists/clear resets ✓ · `ls` unchanged ✓ · pytest/coverage/lint ✓ · E2E isolation ✓ · commit — left to harness per protocol (working tree untouched, `git add/commit` not run) **Deviations:** none. **Next pending phase:** none — `todo/` contains only 97 (96 already committed). |
||
|
ducoterra
|
3095c4c577 |
phase: 90_upload_no_scan
All verification is complete and green. Final report:
**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**
- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors
**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)
**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.
**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
|
||
|
ducoterra
|
f664aafeed |
fix(ui): give the mobile refresh controls labels and full width
At <=640px the RAG "Sync sources" pill and the History "Refresh" pill squeezed down to tiny icon-only buttons — hard to discover and tap on a phone. They are now full-width labeled pills: - the RAG page-head row wraps so the Sync pill drops below the "Knowledge base" title at full width; the History page-head already wrapped the pill below its title block - the Sync label's min(16rem, 40vw) cap lifts on mobile (min-width: 0 engages the ellipsis) so the live-file text truncates against the full width instead of the 40vw cap - the Refresh glyph joins its visible label (it stays hidden on desktop, where the label carries the pill) This matches the established mobile full-width pill language (New chat / Share / stale-ban Regenerate). The three unit tests that pinned the old icon-only CSS are updated to pin the new behavior. |
||
|
ducoterra
|
ffa919b8bf |
fix(chat): keep in-flight answers alive across in-app view switches
Root cause (owner repro, verified in a real browser 2026-09-06): the five navbar views (Chat, RAG, Sources, Tuning, History) were separate HTML documents, so a navbar click was a REAL cross-document navigation — the chat page unloaded, the in-flight SSE fetch was aborted, and the phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled") stopped the model. Observed: send question -> click RAG mid-stream -> click Chat -> the answer never finished: no `query_log` row, and on return a dangling question with no brain record (the pre-token pagehide partial persist skips because `acc` is empty). Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06, flagged per AGENTS.md rule 3, not silently deviated): "real navigation cancels the fetch" now means LEAVING THE APP — tab close, external/other-document navigation, the Stop button. In-app navbar switches are client-side view switches and no longer cancel. Fix — Option A (SPA shell), chosen over B (Service Worker owns the stream) and C (server-side turn registry + resume): - frontend/index.html is the shell: ONE `<main id="main">` holds the five `<section class="view">` blocks; hidden views carry BOTH `hidden` and `inert` (WCAG — no focus/keyboard traversal). The shared header, the single `doc-modal-*` skeleton, and the `#app-version` footer each exist exactly once; the per-view copies from the four folded pages are dropped. - New frontend/assets/router.js (vanilla module — no framework, no bundler, No-CDN rule intact): lazy-imports a view module on FIRST show only (mount-once, hide-forever — the chat view's in-flight SSE reader persists across switches; that persistence IS the fix); intercepts same-shell navbar links with preventDefault + history.pushState (never a document load); handles popstate; single writer of `.nav-link` active state (is-active + aria-current), document.title, and the per-view meta description (values carried over from the old pages' heads, brand-resolved at write time). - Each folded page's JS becomes `export async function mount(root)` — root-scoped queries; `initSharedHeader()` dropped (the header boots once in the shell via the chat module; the admin flag comes from the same cached `fetchIsAdmin()` promise — zero extra requests). - app/main.py: a small list-driven route factory serves the shell for /tuning.html, /sources.html, /git-sources.html, /history.html — registered AFTER the API routers and BEFORE the static catch-all (routes-first). The phase-33 caching middleware applies no-cache + `?v=` rewriting unchanged; app/core/caching.py needed NO change (the view paths did not change — pinned by the integration tests). - The four old view .html files are DELETED (one source of truth); deep links to the old URLs keep working (the router picks the view from the pathname); `/?chat=<id>` is unaffected; the Containerfile bundles router.js (inlining the lazy view modules) and drops the folded page files. - app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell keeps long saved answers in the chat, and the old cap (stricter than the 24_000-char total history budget) 422-rejected any second turn in such a chat (found by the phase-42 E2E suite on the shell). Boundaries: login.html, shared.html, doc-edit.html, document.html REMAIN separate documents (flow pages, not navbar tabs); a mid-stream navigation to doc-edit/document.html still cancels per phase 48 (follow-up candidate, out of scope). The SSE API is unchanged. Real departures still cancel the turn — phase 48 intact (pinned by tests/e2e/test_stop_generation.py, unchanged, and by the new suite's real-departure control). Tests: - Phase-20 suite REWRITTEN to the new semantics (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer cancels — the stream survives the switch and the FULL answer settles; the pagehide partial persist REMAINS for real departures (the partial's exact shape — first streamed chunk prefix, no done metadata — is still pinned there). - NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock LLM): the owner repro (send -> RAG mid-stream -> Chat: window sentinel survives = same document, FULL answer, exactly one brain turn in bor.chat.v1, exactly one settled query_log row, auto-saved row matches) + the same mid-stream switch against the other three views + the real-departure-still-cancels control + the no-switch baseline. - tests/unit/test_frontend_router.py: source-level pins of the router invariants (click interceptor targets ONLY same-shell view paths, pushState-only switches, mount-once guard, hidden+inert pair, single-writer active state/title); shell-route integration tests (each folded path serves the shell with no-cache + `?v=` body; a non-view path still 404s); the file-reading unit pins re-pointed at the shell (the four view files are gone — the shell is the source of truth). Verification (this commit): full suite green — 1565 unit+integration tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the phase's E2E suites green in isolation (house protocol, AGENTS.md rule 9). Owner repro verified in a real browser against the real LLM (dev server :8010, headful Chromium): "tell me about everquest" -> RAG mid-stream -> Chat — the answer completed with one brain bubble and no error banner, `query_log` gained exactly one settled row (deflected=True: the dev KB holds no EverQuest docs — the settle, not the topic, is the proof), zero "chat: turn cancelled" lines for that turn; the control (real navigation to /shared.html mid-stream) still cancelled (no settled row, the cancel line logged, the partial persisted on return). Screenshots: .agents/screenshots/76_manual_*. Phase 76 (76_spa_nav_shell) complete — moved to .agents/phases/complete/. |
||
|
ducoterra
|
4677d86f49 | feat(sources): real-time file progress for sync and upload — background upload with success toast |