diff --git a/.agent/phases/todo/52_pinned_composer/00_phase.md b/.agent/phases/todo/52_pinned_composer/00_phase.md new file mode 100644 index 0000000..55c389d --- /dev/null +++ b/.agent/phases/todo/52_pinned_composer/00_phase.md @@ -0,0 +1,41 @@ +# Phase 52 — Pinned Message Composer + +**Source:** `TODO.md` L3 — "The message input text box needs to be pinned to the bottom of the screen so it doesn't \"run away\" from the user as they try to click \"stop\"" +**Story:** n/a (TODO-derived — owner instruction 2026-08-30: convert without confirmation) +**Context:** The chat page (`frontend/index.html`) scrolls at the document level: `.chat-shell` (the centered 46rem column, PLAN §7) is a flex column — kb-banner, steering panel, New chat, Save/Share, `.messages`, and finally the `.composer` form (`#message-input` + `#send-btn`). The composer is NOT sticky — in a long conversation it sits below the fold, and since the page never auto-scrolls while a turn streams (phase 42), the Stop button (phase 48: `#send-btn` morphs into the enabled Stop control in flight) can be off-screen exactly when the user wants to click it. The sticky app header is the only sticky chrome (z-index 20, 2px hairline below); the document modal is the topmost layer (z-index 1000). House frontend testing: source pins (`tests/unit/test_frontend_feedback.py` style — `test_frontend_scroll.py` / `test_history_page.py` are the closest precedents) plus one isolated Playwright suite per story (A16). + +## Objective +The composer (input + Send/Stop button) is pinned to the bottom of the viewport at every scroll position — the Stop control is always reachable mid-turn without scrolling, and no new auto-scroll behaviour is introduced (the phase-42 contract stays intact). + +## Dependencies +- `48_stop_generation` (complete) — the Send↔Stop morph; Stop is clicked FROM the pinned composer (the original "run away" scenario). +- `42_no_reply_autoscroll` (complete) — the no-autoscroll-while-streaming contract the pin must not revise. +- `07_story_responsive_polish` (complete) — the 46rem column / responsive rules the pinned composer must sit within. + +## Tasks +1. `01_sticky_composer.md` — the `position: sticky; bottom` pin on `.composer` + safe-area inset + the frontend source pins. +2. `02_e2e_pinned_composer.md` — the story Playwright suite + regressions + commit. + +## Testing & Quality +- Unit: `tests/unit/test_pinned_composer.py` — source pins: `.composer` carries `position: sticky` with a `bottom` offset (safe-area inset) in `styles.css`; `app.js` gains NO new page-scroll call site (the phase-42 invariant — the one page scroll is still `scrollReveal`). +- Coverage: **>90%** on `app/` (validate.sh gate — this phase makes no `app/` changes; the gate must stay green). +- E2E (mandatory, A16): `tests/e2e/test_pinned_composer.py`, run in isolation. + +## Completion Criteria +- [ ] With an over-viewport conversation, scrolled to the top: the composer is fully visible (bounding box inside the viewport) at the bottom edge. +- [ ] In flight, scrolled up to read earlier content: the Stop button is visible and clickable; clicking it (no scrolling) stops the turn — partial kept + persisted with `stopped: true` (the phase-48 contract, unchanged), no error banner, no window scroll (phase 42). +- [ ] On an empty/short chat the composer renders in its normal flow position (the pin does not float it over the footer or shift the layout). +- [ ] `uv run pytest` green; coverage TOTAL >90%. +- [ ] `uv run pytest tests/e2e/test_pinned_composer.py -v --no-cov` green in isolation (DB up). +- [ ] Regression E2E suites green in isolation: `test_stop_generation.py`, `test_no_reply_autoscroll.py`, `test_chat_persistence.py`, `test_mobile_hamburger_nav.py`. +- [ ] `uv run ruff check . && uv run pyright` clean. +- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`. + +## Locked decisions +- **Recorded assumptions (TODO conversion, 2026-08-30 — owner asked for no confirmation):** (1) the pin is CSS-only — `position: sticky; bottom: env(safe-area-inset-bottom)` on the existing `.composer` inside the existing `.chat-shell` column; no `index.html` DOM change, no JS; (2) the composer keeps its current solid `--surface` background + border + shadow (no glass/transparency), so scrolled messages never show through it; (3) NO z-index change — the composer already paints above `.messages` by DOM order, never overlaps the sticky header, and stays under the z-1000 document modal; (4) the phase-42 never-auto-scroll contract is strictly upheld — the pin adds zero scroll call sites. +- **A16/A17 honoured** — one story E2E suite, one atomic commit. + +## Commit +```bash +git add -A .agent/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): pin the composer to the viewport bottom — Stop is always reachable while reading" +``` diff --git a/.agent/phases/todo/52_pinned_composer/01_sticky_composer.md b/.agent/phases/todo/52_pinned_composer/01_sticky_composer.md new file mode 100644 index 0000000..8dfcec5 --- /dev/null +++ b/.agent/phases/todo/52_pinned_composer/01_sticky_composer.md @@ -0,0 +1,23 @@ +# Task 01 — Sticky Bottom Composer + +**Phase:** `52_pinned_composer` · **Source:** `TODO.md:3` — "The message input text box needs to be pinned to the bottom of the screen so it doesn't \"run away\" from the user as they try to click \"stop\"" +**Story:** n/a (TODO-derived) + +## Objective +The composer stays pinned to the bottom of the viewport at every scroll position — a CSS-only change inside the existing chat column. + +## Work +1. `frontend/assets/styles.css` — in the `/* ---------- Composer ---------- */` block (`.composer`, ~L1133): add `position: sticky;` and `bottom: env(safe-area-inset-bottom);` to `.composer`. The page scrolls at the document level and `.chat-shell` is the composer's containing column, so the box sticks to the viewport's bottom edge (offset by the mobile safe-area inset) while `.messages` scrolls behind it; at the document bottom it settles back into its normal flow position above the footer. Keep the existing solid `background: var(--surface)`, border, radius and `box-shadow: var(--shadow)` — messages must never show through the pinned box. +2. `frontend/index.html` — verify NO change needed: the composer is already the LAST child of `.chat-shell` (the sticky context), and the `#message-input` / `#send-btn` / `#send-status` markup is untouched. +3. Do NOT touch `frontend/assets/app.js` — the pin must not add any scroll call site (phase-42 invariant; the one page scroll in the file stays `scrollReveal`). +4. `tests/unit/test_pinned_composer.py` (new, house pin style — see `tests/unit/test_frontend_scroll.py`): assert `styles.css` declares `position: sticky` AND a `bottom:` offset on `.composer` (the sticky-bottom pair, matched inside the `.composer` rule); assert `app.js` is unchanged in its scroll surface (the phase-42 single-`scrollReveal` pin still holds — reuse the same assertion approach `test_no_reply_autoscroll.py`'s companion pins use). +- ASSUMPTION: `bottom: env(safe-area-inset-bottom)` (not `bottom: 0` + extra padding) — the standard notch-aware inset; on desktop `env()` resolves to 0, so the box sits flush with the viewport bottom. +- ASSUMPTION: no `z-index` added — DOM order already stacks the composer above `.messages`; the sticky header (z 20) and doc modal (z 1000) are unaffected. + +## Testing & Quality +- Unit: `tests/unit/test_pinned_composer.py` (the pins above) — `uv run pytest tests/unit/test_pinned_composer.py -v` green. +- Coverage: **>90%** — no `app/` change; the gate stays green. + +## Completion Criteria +- [ ] `.composer` in `styles.css` carries `position: sticky` + the `bottom` safe-area offset; the `frontend/` diff contains no JS change. +- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. diff --git a/.agent/phases/todo/52_pinned_composer/02_e2e_pinned_composer.md b/.agent/phases/todo/52_pinned_composer/02_e2e_pinned_composer.md new file mode 100644 index 0000000..a7f9242 --- /dev/null +++ b/.agent/phases/todo/52_pinned_composer/02_e2e_pinned_composer.md @@ -0,0 +1,25 @@ +# Task 02 — E2E: Pinned Composer + Regressions + Commit + +**Phase:** `52_pinned_composer` · **Source:** `TODO.md:3` — "The message input text box needs to be pinned to the bottom of the screen so it doesn't \"run away\" from the user as they try to click \"stop\"" (this task verifies it in the browser) +**Story:** n/a (TODO-derived) + +## Objective +One isolated Playwright story suite proving the composer never runs away from the user — including the original scenario: clicking **Stop** while reading a streaming answer from a scrolled-up position. + +## Work +1. `tests/e2e/test_pinned_composer.py` (new) — the story suite (isolated run; `mock_llm` deterministic; DB up per the e2e prerequisite): + - **Pinned while reading:** build an over-viewport conversation — ask ~8 short questions through the UI (each turn adds user + brain bubbles with meta rows; at the house 1280×720 viewport this exceeds the fold; see the ASSUMPTION below for the fallback if it proves insufficient). `page.evaluate("window.scrollTo(0, 0)")` (a test scroll — the app never scrolls itself, phase 42). Assert `page.locator("#composer").bounding_box()` is fully inside the viewport (`y >= 0`, `y + height <= viewport height`) with its bottom edge at the viewport bottom (± a few px for the safe-area inset). + - **The run-away scenario — Stop from scrolled-up, in flight:** submit one question; let the turn enter streaming (the mock LLM streams deltas; wait for the Send label to read "Stop" per the phase-48 contract); scroll the window to the top (the user reads earlier content — phase 42 leaves them there; record `window.scrollY`); assert the Stop button (`#send-btn`, `.is-stop`) is visible WITHOUT scrolling; click it; assert: the turn settled (no in-flight state), the partial answer is on screen with the `.stopped-note` rendered, no error banner, `window.scrollY` UNCHANGED by the click (the pin adds no scroll), and a fresh page load restores the `stopped` record (the phase-48 persistence contract through the normal `bor.chat.v1` path). + - **Natural bottom:** a fresh empty chat — `#composer`'s bounding box bottom is at or above the viewport bottom and the `.app-footer` is present in normal flow (the pin must not float the composer over the footer on a short page). +2. Regressions, each in isolation (`uv run pytest tests/e2e/ -v --no-cov`): `test_stop_generation.py`, `test_no_reply_autoscroll.py`, `test_chat_persistence.py`, `test_mobile_hamburger_nav.py` (the mobile nav sits in the sticky header — the pin must not break the header/dropdown stacking at ≤640px). +3. One `--no-gpg-sign` commit staging `.agent/ frontend/ tests/` (message per the phase overview); move `.agent/phases/todo/52_pinned_composer/` to `.agent/phases/complete/`. +- ASSUMPTION: over-viewport overflow is produced by ~8 UI questions against the mock LLM (short deterministic answers, but each turn adds two bubbles + meta rows). If the suite shows that is not enough to exceed 720px, fall back to a saved long conversation via the phase-50 path (Save a multi-turn chat, reload with `/?chat=`); no new fixture or API surface. + +## Testing & Quality +- E2E (mandatory, A16): `tests/e2e/test_pinned_composer.py` green in isolation. +- The four regression suites green in isolation (no assertion edits outside the scope the phase-48 revised contract already owns — if `test_stop_generation.py` needs a revision it must be the pinned-composer contract, nothing else). + +## Completion Criteria +- [ ] `uv run pytest tests/e2e/test_pinned_composer.py -v --no-cov` green in isolation (DB up). +- [ ] `test_stop_generation.py`, `test_no_reply_autoscroll.py`, `test_chat_persistence.py`, `test_mobile_hamburger_nav.py` green in isolation. +- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`. diff --git a/.agent/phases/todo/53_stale_saved_chats/00_phase.md b/.agent/phases/todo/53_stale_saved_chats/00_phase.md new file mode 100644 index 0000000..9c2e3cb --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/00_phase.md @@ -0,0 +1,56 @@ +# Phase 53 — Invalidate Saved Chats on Sources Sync + +**Source:** `TODO.md` L4 — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" +**Story:** n/a (TODO-derived — owner instruction 2026-08-30: convert without confirmation) +**Context:** Phase 50 stores explicitly saved conversations in `saved_chats` (JSONB `bor.chat.v1` records; admin-only CRUD under `/api/chats` in `app/api/chats.py`; the History page is a full-width table per AGENTS.md rule 5; `/?chat=` re-opens a row pixel-identical through the phase-14 restore path). Phase 51 added the public snapshot read (`/shared/` → `SharedChatOut`, no admin dependency). Sources are synced through two canonical paths: the admin **Sync** button (`POST /api/sync` → `app/api/sync.py::_run_sync`: `check_models` → resolve effective sources → clone/pull or local-dir re-verify → `import_sources(prune=True)` → change-gated `regenerate_overview`) and the CLI/quadlet `scripts/import_docs.py` (same `import_sources`; `--limit` debug runs and unchanged re-imports are change-gated on `added + updated`). Neither path records *when the KB last changed*, so a saved answer can silently predate the current index. `retryLastTurn(wrap)` in `frontend/assets/app.js` (phase 49) is the existing redo-in-place mechanism for the LAST brain bubble (re-asks the preceding user question, `runTurn(text, { reask: true })`, no user append, no scroll) — the Regenerate action reuses it. Single-row table precedent: `kb_overview` (id = 1, `app/models.py::KbOverview`). + +## Objective +Every sync that actually changes the knowledge base bumps a sources version; saved chats stamp that version at save time; a chat saved against an older version is surfaced as **stale** (History table badge + a banner when opened) with a **Regenerate** action that re-asks the last question against the new index and re-saves the row — a stale answer can no longer masquerade as current. + +## Dependencies +- `50_chat_history` (complete) — the `saved_chats` row, `/api/chats` CRUD, the `?chat=` boot load, the linked-row Save upsert (`saveCurrentChat` in `app.js`). +- `51_share_chat` (complete) — the public `/shared/` snapshot read (stays a frozen snapshot; untouched by this phase). +- `49_retry_answer` (complete) — the `retryLastTurn` redo-in-place the Regenerate action drives. +- `52_pinned_composer` (todo, preceding) — the chat-page layout is stable while the banner lands; ordering keeps the chat UI quiet (no shared-file conflict beyond `styles.css`/`app.js` regions). + +## Tasks +1. `01_sources_version_and_migration.md` — the single-row `sources_meta` table + `saved_chats.sources_version` (migration 0010) + the current/bump helpers + the migration test. +2. `02_sync_version_bump.md` — the version bump on both sync paths (Sync button + CLI), change-gated. +3. `03_chats_api_staleness.md` — stamp-on-save + the `stale` flag on the list/detail responses. +4. `04_history_stale_badge.md` — the Stale column on the History table. +5. `05_stale_banner_and_regenerate.md` — the chat-page banner + Regenerate (`retryLastTurn` reuse) + the auto re-save. +6. `06_e2e_stale_saved_chats.md` — the story Playwright suite + regressions + commit. + +## Testing & Quality +- Unit: `tests/unit/test_sources_meta.py` (helpers: absent row → 0, first bump → 1, second bump → 2, idempotent reads); the sync bump gates — extend the existing sync coverage (`tests/integration/test_sync_api.py` + the `tests/fakes.py` override patterns) and the CLI coverage (`tests/integration/test_import_docs_overview.py` pins the change-gating pattern; the bump asserts sit alongside). +- Integration: `tests/integration/test_migration_0010.py` (house pattern, from `test_migration_0009.py`); extend `tests/integration/test_chats_api.py` (stamp + `stale` flag + share-unshare version immunity). +- Frontend source pins (house style): `history.js` stale-cell branch, `app.js` banner reveal / Regenerate wiring / post-regenerate persist / no-brain-bubble guard (the `test_history_page.py` / `test_save_chat_ui.py` pin patterns). +- Coverage: **>90%** on `app/` (validate.sh gate). +- E2E (mandatory, A16): `tests/e2e/test_stale_saved_chats.py`, run in isolation. + +## Completion Criteria +- [ ] A KB-changing sync (button or CLI) bumps `sources_meta.version` exactly once; an unchanged re-run, a `--limit` debug run, and a FAILED sync never bump. +- [ ] A Save/Re-Save stamps the row's `sources_version`; `GET /api/chats` + `GET /api/chats/` expose `stale` (true iff the row's version is behind the current one); share/unshare never touch the version. +- [ ] The History table shows the Stale marker exactly on rows saved before the last KB-changing sync (full-width table geometry unchanged, AGENTS.md rule 5). +- [ ] Opening a stale `/?chat=` shows the banner; Regenerate re-streams the last answer in place against the new index and re-saves the row — the banner clears, `GET /api/chats/` reports `stale: false`, the History marker is gone. +- [ ] A `/shared/` page is unchanged — the public snapshot carries no staleness surface. +- [ ] `uv run pytest` green; coverage TOTAL >90%. +- [ ] `uv run pytest tests/e2e/test_stale_saved_chats.py -v --no-cov` green in isolation (DB up). +- [ ] Regression E2E suites green in isolation: `test_chat_history.py`, `test_share_chat.py`, `test_sync_button.py`, `test_retry_answer.py`, `test_chat_persistence.py`. +- [ ] `uv run ruff check . && uv run pyright` clean. +- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`. + +## Locked decisions +- **Recorded assumptions (TODO conversion, 2026-08-30 — owner asked for no confirmation):** + 1. The invalidation marker is a monotonically increasing `sources_meta.version` (single row, the `kb_overview` id=1 precedent), bumped ONLY when a sync changed the KB — the gate is `added + updated + pruned > 0` (pruned counts: a deleted doc can invalidate an answer that cited it — deliberately broader than the overview gate's `added + updated > 0`). + 2. `stale = row.sources_version < current`, computed server-side in the chats API; the client never computes staleness. Pre-existing rows stamp 0 (the pre-counter KB) and go stale on the first bump. + 3. Regenerate = the phase-49 redo-in-place of the LAST brain bubble (re-ask the last user question with the full conversation context) followed by an auto re-save of the linked row — the owner does not press Save again; a stale chat with no brain answer shows the banner text without a Regenerate button. + 4. The public shared snapshot (`/shared/`) is deliberately untouched — a frozen snapshot by design; an owner who regenerates can re-share afterwards. + 5. A failed sync (git/embed/import error) aborts BEFORE the bump — a failed sync never invalidates chats; the bump commits even if the best-effort overview regeneration then fails (the index really did change). +- **A10 honoured** — `/api/chat` stays stateless; staleness is a property of the explicitly saved row, not of the chat endpoint. +- **A16/A17 honoured** — one story E2E suite, one atomic commit. + +## Commit +```bash +git add -A .agent/ app/ alembic/versions/ scripts/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(chat): invalidate saved chats on sources sync — versioned stamps, stale marker, Regenerate against the new index" +``` diff --git a/.agent/phases/todo/53_stale_saved_chats/01_sources_version_and_migration.md b/.agent/phases/todo/53_stale_saved_chats/01_sources_version_and_migration.md new file mode 100644 index 0000000..f129771 --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/01_sources_version_and_migration.md @@ -0,0 +1,27 @@ +# Task 01 — Sources Version Table + `saved_chats.sources_version` + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" +**Story:** n/a (TODO-derived) + +## Objective +A durable record of "which generation of the KB was this chat saved against": a single-row `sources_meta` version counter plus a `sources_version` stamp on every `saved_chats` row. + +## Work +1. `app/models.py` — add `SourcesMeta` (single row, the `KbOverview` id=1 precedent): `id` Integer PK `server_default="1"`, `version` Integer NOT NULL `server_default="0"`, `updated_at` timestamptz (`server_default=func.now()`, `onupdate=func.now()`). Add `SavedChat.sources_version: Mapped[int]` — Integer, NOT NULL, `server_default="0"`, with the model docstring noting: existing rows stamp 0 = "the pre-counter KB" and become stale on the first bump. +2. `alembic/versions/0010_sources_version.py` — ONE migration for the feature: `op.create_table("sources_meta", …)` + the seed row (id 1, version 0) + `op.add_column("saved_chats", sa.Column("sources_version", sa.Integer(), nullable=False, server_default="0"))`. Downgrade reverses in the safe order (drop column, drop table). `down_revision` = the 0009 revision (verify against `alembic/versions/0009_saved_chat_share_token.py`). +3. `app/rag/sources_meta.py` (new module) — `current_sources_version(db: Session) -> int` (PK read of the seed row; returns 0 when the row is absent — defensive, never raises) and `bump_sources_version(db: Session) -> int` (upsert the row, `version += 1`, `db.flush()` — the CALLER commits, because the two sync paths each own their session; returns the new version). +4. `tests/integration/test_migration_0010.py` — the house migration pattern (model from `tests/integration/test_migration_0009.py`): upgrade head → the seed row exists (version 0), `saved_chats.sources_version` is NOT NULL with default 0, an inserted row round-trips; downgrade → column + table gone. +5. `tests/unit/test_sources_meta.py` — the helpers on a real/fixture session (house DB-test pattern): absent row → `current` 0; first bump → 1; second bump → 2; `current` after a bump reflects it; two bumps in two sessions don't race to the same value (flush-order check is out of scope — one writer at a time is the deployment reality). + +- ASSUMPTION: both schema changes ship in ONE migration (0010) — one feature, one atomic schema change; the seed row is inserted in the migration (not lazily on first bump), so `current_sources_version` is a plain PK read. +- ASSUMPTION: the helper module lives at `app/rag/sources_meta.py` (rag domain — both sync paths already import from `app.rag.*`); no new setting, no new dependency. + +## Testing & Quality +- Integration: `tests/integration/test_migration_0010.py` green. +- Unit: `tests/unit/test_sources_meta.py` green. +- Coverage: **>90%** on `app/` (validate.sh gate). + +## Completion Criteria +- [ ] `uv run alembic upgrade head` applies 0010 cleanly on a fresh DB (and on a DB already at 0009); the seed row is present with version 0. +- [ ] `uv run pytest tests/integration/test_migration_0010.py tests/unit/test_sources_meta.py -v` green. +- [ ] `uv run pytest` green (no existing test broken by the new NOT NULL column); `uv run ruff check . && uv run pyright` clean. diff --git a/.agent/phases/todo/53_stale_saved_chats/02_sync_version_bump.md b/.agent/phases/todo/53_stale_saved_chats/02_sync_version_bump.md new file mode 100644 index 0000000..66221d9 --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/02_sync_version_bump.md @@ -0,0 +1,26 @@ +# Task 02 — Bump the Version on Every KB-Changing Sync + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" (this task wires the bump into the two canonical sync paths) +**Story:** n/a (TODO-derived) + +## Objective +Every sync path that changed the knowledge base bumps `sources_meta.version` exactly once — the in-app Sync button and the CLI — so the staleness marker tracks the index, not the click. + +## Work +1. `app/api/sync.py` — in `_run_sync`, after `summary = await import_sources(sources, llm, prune=True)` succeeds and the overview decision is made: when `summary.added + summary.updated + summary.pruned > 0`, open a short-lived `SessionLocal()` (the run's status is in-memory; the `effective_sources` call above sets the exact pattern — open, use, close in `finally`) and run `bump_sources_version(db)` + commit. Add `"sources_version": ` to `_status.detail` (the never-stale status object gains the new generation number). Any exception before this point (model check, source resolution, clone/pull, import failure) aborts the run in the `failed` state — a FAILED sync never bumps, so it never invalidates chats. + - Note the deliberate gate difference: the overview regenerates on `added + updated > 0`, but the version bump also covers `pruned > 0` — a pruned document can invalidate a saved answer that cited it. The two gates stay separate on purpose. +2. `scripts/import_docs.py` — in `_run` (the `asyncio.run` coroutine), the same bump with the same gate: after the `--limit`/unchanged early-returns, when `summary.added + summary.updated + summary.pruned > 0`, bump via a short `SessionLocal()` (the file already opens sessions for `_overview_row_exists`). `--limit` debug runs NEVER bump (an incomplete walk is debug-only — mirrors the existing `--limit` overview skip); an unchanged re-run (`added + updated + pruned == 0`) NEVER bumps. Add `sources_version=` to the summary `print` line (the cron/quadlet log surface). +3. Tests: + - `tests/integration/test_sync_api.py` — extend (fakes pattern from `tests/fakes.py`): a sync whose import changed the KB bumps the version exactly once AND the `/api/sync/status` detail carries `sources_version`; an unchanged re-sync (all `unchanged`) does NOT bump; a failing sync (e.g. the phase-41 model-down path) does NOT bump. + - `tests/integration/test_import_docs_overview.py` (or the existing `import_docs` gate test if the file that pins `--limit`/unchanged gating lives elsewhere — match the file that already asserts the overview gate) — extend with the bump asserts: changed run → bump + `sources_version` in the print line; `--limit` run → no bump, `skipped`; unchanged run → no bump, `skipped`. +- ASSUMPTION: the bump commits independently of the best-effort overview regeneration — a failed `lite` overview never rolls back the version (the index did change), and the bump happens in its own short session so it survives an overview exception. +- ASSUMPTION: `scripts/import_docs.py --source ` runs DO bump when they change the KB — a manual import changes the index exactly like a git sync (the TODO says "if the docs are synced"; the CLI is the other canonical sync). + +## Testing & Quality +- Integration: the extended sync + import_docs suites green (`uv run pytest tests/integration/test_sync_api.py tests/integration/test_import_docs_overview.py -v`). +- Coverage: **>90%** on `app/` (the sync-path bump is in `app/api/sync.py`; the CLI bump is in `scripts/`, covered by its own tests). + +## Completion Criteria +- [ ] `uv run pytest` green; coverage TOTAL >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] A changed sync (both paths) bumps exactly once; unchanged / `--limit` / failed syncs never bump. +- [ ] No behavior change in completed phases (the sync pipeline's steps 1–5 and exit codes are untouched apart from the new bump + detail field / print token). diff --git a/.agent/phases/todo/53_stale_saved_chats/03_chats_api_staleness.md b/.agent/phases/todo/53_stale_saved_chats/03_chats_api_staleness.md new file mode 100644 index 0000000..f57c1d6 --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/03_chats_api_staleness.md @@ -0,0 +1,23 @@ +# Task 03 — Stamp-on-Save + `stale` on the Chats API + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" (this task exposes staleness on the admin chats API) +**Story:** n/a (TODO-derived) + +## Objective +A saved chat knows the KB generation it was saved against, and the API tells the client whether it is stale — no client-side staleness math. + +## Work +1. `app/api/chats.py` — `create_chat`: stamp `row.sources_version = current_sources_version(db)` on the PENDING row (ships in the same INSERT, the `share_token` precedent). `update_chat` (the Re-Save upsert): re-stamp `sources_version` to the current value — a Re-Save is the owner affirming this content against the current KB. Share/unshare are UNCHANGED (raw SQL touching only `share_token` — the version, like `updated_at`, is untouched). +2. `app/schemas.py` — add `stale: bool` to `SavedChatRow` (list shape) and `SavedChatOut` (detail shape). `SharedChatOut` is UNCHANGED — the public snapshot is frozen by design (phase 51; the staleness surface is admin-only). +3. `app/api/chats.py` — `list_chats` + `get_chat`: call `current_sources_version(db)` ONCE per request and compute `stale = row.sources_version < current` inside `_to_row` / `_to_out` (pass the current version in — the module functions stay pure). +4. `tests/integration/test_chats_api.py` — extend: create stamps the current version (assert via the helper read); bump (`bump_sources_version` in the test) → `GET /api/chats` rows and `GET /api/chats/` both report `stale: true`; a Re-Save (`PUT`) re-stamps → `stale: false` again; share/unshare leave `sources_version` untouched; the 403/anonymous contract is unchanged. +- ASSUMPTION: Re-Save ALWAYS re-stamps to the current version (not "only when stale") — uniform semantics, one fewer code path; a re-Save is also the manual escape hatch for a false-positive stale row. + +## Testing & Quality +- Integration: `tests/integration/test_chats_api.py` green (extended). +- Coverage: **>90%** on `app/` (validate.sh gate). + +## Completion Criteria +- [ ] `POST` / `PUT /api/chats` stamp `sources_version`; list + detail carry `stale`; anonymous 403s unchanged; `/api/shared/` body unchanged. +- [ ] `uv run pytest` green; coverage TOTAL >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] No behavior change in completed phases (existing `test_chats_api.py` / `test_share_chat.py` assertions still hold). diff --git a/.agent/phases/todo/53_stale_saved_chats/04_history_stale_badge.md b/.agent/phases/todo/53_stale_saved_chats/04_history_stale_badge.md new file mode 100644 index 0000000..a5aae8d --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/04_history_stale_badge.md @@ -0,0 +1,23 @@ +# Task 04 — Stale Column on the History Table + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" (this task surfaces staleness on the History page) +**Story:** n/a (TODO-derived) + +## Objective +The History page shows at a glance which saved chats predate the last KB-changing sync. + +## Work +1. `frontend/history.html` — add a `Stale` `` to the full-width table, between the `Updated` and `Actions` columns (house rule 5: a column in the full-width table — no skinny side list). +2. `frontend/assets/history.js` — render the cell from `row.stale` (the phase-50 row shape, extended in task 03): a rose `Stale` pill when true, an em-dash when false; the pill carries `title="Sources have changed since this chat was saved — open the chat to Regenerate"` and the `` an `aria-label` so the marker is conveyed without the visual (WCAG 2.1 AA). Non-stale rows render exactly as before. +3. `frontend/assets/styles.css` — the `.stale-pill` treatment: rose family (the Stop-treatment tokens, so "stale" reads in the same visual language as the in-flight control), theme-token based (both themes), contrast ≥4.5:1 on `--surface`. +4. Source pins (`tests/unit/test_history_page.py` extended, house pin style): the `history.js` stale-cell branch (pill class + em-dash fallback + the `aria-label`), the `Stale` `` in `history.html`, and the `.stale-pill` rule in `styles.css`. +- ASSUMPTION: the marker is a READ-ONLY badge — opening the row shows the chat-page banner (task 05) which carries the Regenerate action; no bulk "regenerate all" in v1, no per-row action button (the table's Actions column keeps its Open/Share/Delete trio). + +## Testing & Quality +- Unit: the extended `tests/unit/test_history_page.py` pins green. +- Coverage: **>90%** — no `app/` change; the gate stays green. + +## Completion Criteria +- [ ] The History table renders the Stale column; stale rows show the pill, fresh rows the em-dash; the table stays full-width (AGENTS.md rule 5). +- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. +- [ ] No behavior change in completed phases (the Open/Share/Delete flows and the cache-busting contract are untouched). diff --git a/.agent/phases/todo/53_stale_saved_chats/05_stale_banner_and_regenerate.md b/.agent/phases/todo/53_stale_saved_chats/05_stale_banner_and_regenerate.md new file mode 100644 index 0000000..2360578 --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/05_stale_banner_and_regenerate.md @@ -0,0 +1,30 @@ +# Task 05 — Stale Banner + Regenerate on the Chat Page + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" (this task delivers the regeneration on the chat page) +**Story:** n/a (TODO-derived) + +## Objective +Opening a saved chat that predates the current KB shows a banner; **Regenerate** re-asks the last question against the new index, in place, and re-saves the row so the stored answer is current again. + +## Work +1. `frontend/index.html` — add a `#stale-banner` section directly after `#kb-banner` (the chat-shell top-of-column position, `role="status"`, `hidden` by default): the text "The sources have been updated since this chat was saved." + a `#stale-regenerate` button labeled "Regenerate" (the brand-pill treatment, matched to the Save/Share pair; icon = the redo glyph, reused from the phase-49 Retry asset in `app.js`). +2. `frontend/assets/app.js`: + - `retryLastTurn(wrap)` (phase 49, ~L1381): make it RETURN the `runTurn(text, { reask: true })` promise (today it is `void runTurn(...)`) — the existing Retry click handler ignores the return value, so phase-49 behavior is byte-identical; the Regenerate path needs the turn's completion to know when to persist. + - The `?chat=` boot-load block (the phase-50 code around `fetch(\`/api/chats/${chatId}\`)` → `currentChatId = chatId` → `saveConversation()` → `history.replaceState`): when the fetched payload has `stale: true`, reveal `#stale-banner` (remove `hidden`). + - New handler for `#stale-regenerate`: call `retryLastTurn(wrap)` on the LAST brain bubble's rendered wrap (the phase-49 targeting — reuse the `markLastRetryable` / last-`.msg.brain`-wrap resolution; the guard `wrap !== lastBrainWrap` already protects against a stale click), AWAIT the returned turn promise; on completion WITHOUT an error banner, persist the linked row through the existing upsert path (linked → `PUT /api/chats/` — the server re-stamps `sources_version`, see task 03), then hide the banner and announce the outcome in the existing `#send-status` live region (PLAN §7.4 never-stale). A 404 PUT (row deleted from History meanwhile) follows the `saveCurrentChat` stale-link rule: unlink + recreate — the owner is never left with an unsaved conversation. + - Guard: a stale conversation with NO brain record (user-only / deflection-only) — the banner shows WITHOUT the Regenerate button (text only); `retryLastTurn` is never called in that state. +3. `frontend/assets/styles.css` — `#stale-banner` styling (the kb-banner family, distinct redo icon) + the Regenerate button; theme tokens only, contrast ≥4.5:1 in both themes; banner stacks correctly above `#kb-banner` when both are visible (e.g. empty-KB banner + stale chat — kb-banner wins the top slot, stale-banner directly below). +4. Source pins (house style, extend `tests/unit/test_save_chat_ui.py`): the banner reveal branch on `payload.stale`, the Regenerate → `retryLastTurn` wiring, the `retryLastTurn` return-promise change, the post-regenerate linked PUT, and the no-brain-record guard. +- ASSUMPTION: Regenerate = the phase-49 redo-in-place of the LAST brain bubble ONLY — the full conversation context is kept, earlier answers are not re-run (re-asking every question is out of scope for v1; the TODO's "a new answer" is satisfied by the answer the user was reading going stale). +- ASSUMPTION: the auto re-save after a successful regenerate is the linked `PUT` (the row's `updated_at` bumps — it IS a content edit, so the History "latest activity" order follows it); a regenerate that errors mid-stream (the error banner shows) leaves the row untouched — stale stays true; a regenerate STOPPED mid-stream (phase 48) persists the stopped partial via the same PUT (the owner engaged with the new index). +- ASSUMPTION: localStorage-only (unsaved) conversations are never stale — staleness is a property of the saved row; the banner appears only on the `?chat=` boot path, never on the phase-14 local restore. + +## Testing & Quality +- Unit: the extended `tests/unit/test_save_chat_ui.py` pins green. +- Coverage: **>90%** — no `app/` change in this task; the gate stays green. + +## Completion Criteria +- [ ] Opening a stale `?chat=` shows the banner; Regenerate streams the fresh answer in place and clears the banner after the linked row is re-saved. +- [ ] The phase-49 Retry button behaves exactly as before (pin-verified: the return-promise change is behavior-neutral for the existing click handler). +- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. +- [ ] No behavior change in completed phases (phase-14 local restore, phase-48 stop, phase-50 save/load flows). diff --git a/.agent/phases/todo/53_stale_saved_chats/06_e2e_stale_saved_chats.md b/.agent/phases/todo/53_stale_saved_chats/06_e2e_stale_saved_chats.md new file mode 100644 index 0000000..51121b1 --- /dev/null +++ b/.agent/phases/todo/53_stale_saved_chats/06_e2e_stale_saved_chats.md @@ -0,0 +1,28 @@ +# Task 06 — E2E: Stale Saved Chats + Regressions + Commit + +**Phase:** `53_stale_saved_chats` · **Source:** `TODO.md:4` — "Make sure the saved chats are invalidated if the docs are synced, that way it generates a new answer with new data" (this task verifies the full loop in the browser) +**Story:** n/a (TODO-derived) + +## Objective +One isolated Playwright story suite for the whole invalidation loop: save → KB-changing sync (version bump) → stale surfaced → Regenerate → current again. + +## Work +1. `tests/e2e/test_stale_saved_chats.py` (new) — the story suite (isolated run; `mock_llm` deterministic; real DB per the e2e prerequisite): + - As admin: ask a question (mock LLM answers deterministically), Save via the chat-page button → `GET /api/chats` (admin cookie) reports the row with `stale: false`. + - Produce the KB change the way a real sync does: the test process (which shares the app's environment) imports `bump_sources_version` from `app.rag.sources_meta` and bumps the seed row through a short `SessionLocal()` — deterministic, no dependency on configured git sources in the E2E environment (see the ASSUMPTION). + - `/history.html`: the row now carries the Stale pill; `GET /api/chats` carries `stale: true`. + - Open the row (`/?chat=`, the same URL the History table links): the `#stale-banner` is visible with the Regenerate button; click it → the fresh answer streams in place (the mock LLM's deterministic text replaces the old last brain bubble, phase-49 contract), the banner clears once the row is re-saved; `GET /api/chats/` reports `stale: false` and its last brain message is the fresh answer; the History pill is gone. + - Anonymous: share the (now fresh) chat, open `/shared/` without a session — the page renders the snapshot with NO staleness surface (phase 51 unchanged). +2. Regressions, each in isolation (`uv run pytest tests/e2e/ -v --no-cov`): `test_chat_history.py`, `test_share_chat.py`, `test_sync_button.py`, `test_retry_answer.py` (the `retryLastTurn` return-promise change), `test_chat_persistence.py`. +3. One `--no-gpg-sign` commit staging `.agent/ app/ alembic/versions/ scripts/ frontend/ tests/` (message per the phase overview); move `.agent/phases/todo/53_stale_saved_chats/` to `.agent/phases/complete/`. +- ASSUMPTION: the E2E bumps the version via `bump_sources_version` directly (a test-only DB step) — the Sync button's end-to-end clone/import path stays covered by `test_sync_button.py`, and the bump GATES are covered by this phase's integration tests (task 02); the E2E proves the user-visible invalidation loop, not git plumbing. + +## Testing & Quality +- E2E (mandatory, A16): `tests/e2e/test_stale_saved_chats.py` green in isolation. +- The five regression suites green in isolation; no assertion edits outside the new contract (the phase-49 retry pins, the phase-50/51 save/share pins stay intact). + +## Completion Criteria +- [ ] `uv run pytest tests/e2e/test_stale_saved_chats.py -v --no-cov` green in isolation (DB up). +- [ ] `test_chat_history.py`, `test_share_chat.py`, `test_sync_button.py`, `test_retry_answer.py`, `test_chat_persistence.py` green in isolation. +- [ ] `uv run pytest` + coverage TOTAL >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.