fix(chat): keep in-flight answers alive across in-app view switches

Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).

Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.

Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
  five `<section class="view">` blocks; hidden views carry BOTH
  `hidden` and `inert` (WCAG — no focus/keyboard traversal). The
  shared header, the single `doc-modal-*` skeleton, and the
  `#app-version` footer each exist exactly once; the per-view copies
  from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
  bundler, No-CDN rule intact): lazy-imports a view module on FIRST
  show only (mount-once, hide-forever — the chat view's in-flight SSE
  reader persists across switches; that persistence IS the fix);
  intercepts same-shell navbar links with preventDefault +
  history.pushState (never a document load); handles popstate; single
  writer of `.nav-link` active state (is-active + aria-current),
  document.title, and the per-view meta description (values carried
  over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
  root-scoped queries; `initSharedHeader()` dropped (the header boots
  once in the shell via the chat module; the admin flag comes from the
  same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
  /tuning.html, /sources.html, /git-sources.html, /history.html —
  registered AFTER the API routers and BEFORE the static catch-all
  (routes-first). The phase-33 caching middleware applies no-cache +
  `?v=` rewriting unchanged; app/core/caching.py needed NO change
  (the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
  deep links to the old URLs keep working (the router picks the view
  from the pathname); `/?chat=<id>` is unaffected; the Containerfile
  bundles router.js (inlining the lazy view modules) and drops the
  folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
  keeps long saved answers in the chat, and the old cap (stricter than
  the 24_000-char total history budget) 422-rejected any second turn
  in such a chat (found by the phase-42 E2E suite on the shell).

Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).

Tests:
- Phase-20 suite REWRITTEN to the new semantics
  (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
  cancels — the stream survives the switch and the FULL answer
  settles; the pagehide partial persist REMAINS for real departures
  (the partial's exact shape — first streamed chunk prefix, no done
  metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
  LLM): the owner repro (send -> RAG mid-stream -> Chat: window
  sentinel survives = same document, FULL answer, exactly one brain
  turn in bor.chat.v1, exactly one settled query_log row, auto-saved
  row matches) + the same mid-stream switch against the other three
  views + the real-departure-still-cancels control + the no-switch
  baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
  invariants (click interceptor targets ONLY same-shell view paths,
  pushState-only switches, mount-once guard, hidden+inert pair,
  single-writer active state/title); shell-route integration tests
  (each folded path serves the shell with no-cache + `?v=` body; a
  non-view path still 404s); the file-reading unit pins re-pointed at
  the shell (the four view files are gone — the shell is the source
  of truth).

Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.

Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.

.agents/phases/complete/76_spa_nav_shell/00_phase.md

@@ -0,0 +1,50 @@
+# Phase 76 — In-app view switches never halt a generating answer (the navbar views become one document)
+
+**Source:** Owner repro, verified in a real browser 2026-09-06: send a question → click **RAG** in the navbar mid-stream → click **Chat** → the answer never finishes (turn cancelled: no `query_log` row, and on return a dangling question with no brain record — the pre-token `pagehide` partial persist skips because `acc` is empty). Root cause: every navbar view is a separate HTML document (`RAG` = `<a href="/sources.html">`), so a navbar click is a REAL cross-document navigation — the chat page unloads, the in-flight `fetch` is aborted, and the phase-48 teardown (`app/api/chat.py` `finally`, `chat: turn cancelled`) stops the model.
+**Story:** n/a (TODO-derived — descendant of `.agents/user_stories/sources-midstream.md` (phase 20) and `TODO.md` L3 (phase 73))
+**Context:** `frontend/index.html` (chat page; the shared `<header class="app-header">` markup is copy-pasted into every page), `frontend/assets/app.js` (the chat logic — owns the in-flight SSE stream that must SURVIVE view switches; phase-20/48/73 machinery must stay intact and its source-level unit pins in `tests/unit/test_frontend_*.py` must keep passing), `frontend/assets/{sources,git-sources,tuning,history}.js` (page modules that boot at document load and call `initSharedHeader()`), `frontend/assets/header.js` (shared header: auth-gated links, active state, mobile hamburger), `app/main.py` (explicit routes + catch-all `StaticFiles` mount, API routes first), `app/core/caching.py` (`HTML_PAGES` list — the view PATHS do not change, so the no-cache + `?v=` rewrite contract applies to the shell routes untouched), `tests/e2e/test_sources_midstream_bug.py` (phase-20 suite that encodes the OLD "partial survives the navigation" semantics — rewritten in task 02), `tests/e2e/test_hidden_tab_stream.py` (phase 73 — must stay green unchanged), `tests/e2e/test_stop_generation.py` (phase 48 — must stay green unchanged).
+
+## Objective
+Collapse the five navbar views (Chat, RAG, Sources, Tuning, History) into views of ONE HTML shell so a navbar click is a client-side view switch (`history.pushState` + show/hide), never a document load. An in-flight answer keeps streaming through any navbar switch and completes when the user returns to Chat — the owner repro (send → RAG → Chat) must finish the FULL answer, settled server-side (one `query_log` row), with exactly one brain turn persisted. Real departures (tab close, leaving the app, the Stop button) still cancel the fetch and stop the model — the phase-48 contract, intact.
+
+## Owner decisions (chat, 2026-09-06 — recorded per AGENTS.md rule 3)
+- **Option A** (SPA-ify the navbar views) chosen over B (Service Worker owns the stream) and C (server-side turn registry + resume).
+- **LOCKED-DECISION REFINEMENT (phase 48):** "real navigation cancels the fetch" now means **leaving the app** (tab close, external/other-document navigation, Stop). In-app navbar switches no longer cancel. Owner-confirmed refinement — flagged, not silently deviated.
+- **Boundaries:** `login.html`, `shared.html`, `doc-edit.html`, `document.html` REMAIN separate documents (flow pages, not navbar tabs). A mid-stream navigation to doc-edit/document.html still cancels per phase 48 (follow-up candidate, out of scope). The SSE API is unchanged.
+- **No new technology:** vanilla JS modules, no framework, no bundler — consistent with the existing architecture and the No-CDN rule (AGENTS.md #6).
+
+## Design (shared by all tasks — the executor reads this, not the chat)
+- **Shell:** `frontend/index.html` becomes the shell. The shared header stays at the top of the shell (it is copy-pasted into every page today; in the shell it exists ONCE, so the header/nav/auth/steering ids — `#app-nav`, `#nav-*`, `#sign-in-link*`, `#sign-out-btn*`, `#steering-*`, `#nav-toggle` — deduplicate automatically). ONE `<main id="main" class="app-main" tabindex="-1">` container (skip-link target unchanged) holds the five `<section class="view" id="view-<name>" aria-label="…" tabindex="-1">` blocks; each folded page's own `<main class="app-main">` wrapper (all five pages carry the identical one — layout CSS is class-based, so nothing is lost) is DROPPED when its content moves. Only the active view is rendered: hidden views carry BOTH `hidden` AND `inert` (WCAG — hidden views must not receive focus or keyboard traversal; AGENTS.md #5). ONE shell footer (the chat page's `#app-version` footer): per-page footers from folded views are dropped (`app-version` otherwise duplicates — it exists in both `index.html` and `history.html`).
+- **Duplicate-id resolution (audited 2026-09-06, whole-file id scan of the five pages, comments stripped):** the only cross-page id collisions in VIEW markup are (a) the per-view `<main id="main">` — resolved by the single-main container above, and (b) the `doc-modal-*` family (9 ids: `doc-modal`, `-backdrop`, `-close`, `-content`, `-desc`, `-meta`, `-open`, `-panel`, `-title`), present in BOTH `index.html` and `sources.html` — the shell keeps EXACTLY ONE instance (the chat's, already in `index.html`), placed at body level (the modal is a `position: fixed` overlay; its DOM position is cosmetic). `document-modal.js` resolves all of them with document-level `querySelector` at MODULE IMPORT (its lines 44–52), so the single skeleton must be static markup present before any import — it is. `app.js` (chat chips) and `sources.js` (RAG rows) both call `openDocumentModal(...)` and work UNCHANGED against the one shared instance. The moved sources view markup DROPS its skeleton copy (task 02).
+- **Same-document proof (canonical E2E assertion for "no document load"):** set a `window` sentinel before the nav click (`window.__shell_boot = "phase76"`) and assert it is still readable after the switch + return — a real navigation wipes `window` globals. Do NOT use `performance.getEntriesByType("navigation").length` — a real load resets that counter to 1 in the fresh document, so it cannot distinguish pushState from a reload.
+- **View-scoped E2E absence pattern:** in the shell, hidden views' elements REMAIN IN THE DOM (hidden+inert). Any E2E absence assertion (`to_have_count(0)`) on a view-scoped id must be re-scoped to the visible view (e.g. `#view-rag #new-chat-btn`) or switched to a visibility assertion (`to_be_hidden`). Known instance: `tests/e2e/test_chat_persistence.py::test_persists_across_page_navigation` asserts `#new-chat-btn` count 0 on the Sources page (task 02 fixes it).
+- **Router (new `frontend/assets/router.js`, module, ~200 lines):** a `VIEW` map of `pathname → view name` (`"/"` → chat, plus one entry per folded view); on boot it reads `location.pathname`, lazy-`import()`s the view module on FIRST show only, calls `await module.mount(root)`, then shows it. **Mount-once, hide-forever:** a view's DOM (and JS state — for chat, the in-flight SSE reader) persists across switches; that persistence IS the fix. A delegated click handler on the navbar intercepts same-shell view links (`preventDefault` + `pushState` + switch — no document load); `popstate` switches views for back/forward. The router is the SINGLE WRITER of `.nav-link` active state (`is-active` + `aria-current="page"`), `document.title`, and the per-view `<meta name="description">` (values carried over from the old pages' `<head>`s).
+- **View modules:** each folded page's JS changes from top-level boot to an exported `async function mount(root)`; DOM queries scope to `root` (view element ids stay unique across the shell — verify, don't rename, to keep E2E selectors stable); the `initSharedHeader()` call is dropped (the header boots once in the shell via the chat module, as today).
+- **Boot order in the shell:** `brand.js` (classic) → `app.js` (module — the chat view; runs at shell boot exactly as today, so its boot/restore/streaming behavior is untouched) → `router.js` (module — lazy-imports non-chat views only).
+- **Server:** one small route factory in `app/main.py` serves the shell (`frontend/index.html`) for the folded view paths, registered BEFORE the static catch-all mount (API routes first, per existing comment). The phase-33 caching middleware already wraps the whole app and lists these paths in `HTML_PAGES`, so no-cache + `?v=` asset rewriting applies unchanged — verify, don't rewire. The old per-view `.html` files are DELETED in the same task that lands their shell route (one source of truth).
+- **Deep links:** every old view URL (`/sources.html`, …) keeps working as a direct load — the server serves the shell, the router picks the view from the pathname. `/?chat=<id>` (the saved-chat deep link read by `app.js` at boot) is unaffected — it already lives on the shell's own path.
+- **Auth flag in view modules:** the folded view modules today call `await initSharedHeader()` and (in `sources.js`, `git-sources.js`) USE ITS RETURN VALUE as the admin flag. In the shell the header boots once via the chat module — so each view module DROPS `initSharedHeader()` and reads the admin flag from `fetchIsAdmin()` instead (the SAME cached `/api/whoami` promise `header.js` exports — zero extra requests; `sources.js`/`history.js`/`tuning.js` already import it, `git-sources.js` must add it to its import).
+
+## Dependencies
+— (none)
+
+## Tasks
+1. `01_shell_router_tuning.md` — shell + router + shell-route factory; fold Tuning as the first non-chat view (proves the pattern end-to-end, gate included).
+2. `02_fold_rag_sources_views.md` — fold RAG (sources) + Sources (git-sources); REWRITE the phase-20 `test_sources_midstream_bug.py` to the new "the stream survives" semantics.
+3. `03_fold_history_view.md` — fold History; verify the `/?chat=<id>` deep link and history suites.
+4. `04_header_shell_wiring.md` — header is shell-owned (single-writer active state, auth gate, hamburger, per-view title/meta); the surviving documents keep their header copies; update the header/nav consistency suites.
+5. `05_stream_survival_e2e.md` — new story suite `test_nav_switch_keeps_stream.py`: the owner repro pinned on the mock LLM (mid-stream switch → FULL answer, one brain turn, settled turn) + the real-departure-still-cancels control.
+6. `06_regression_sweep_commit.md` — full suite + coverage floor + ruff/pyright; manual verification of the owner repro in a real browser; atomic commit; phase → complete.
+
+## Testing & Quality
+- Unit: `tests/unit/test_frontend_router.py` — source-level pins of the router invariants (house pattern: `tests/unit/test_frontend_hidden_tab.py` reads JS source without a browser): the click interceptor targets ONLY same-shell view paths; switches use `pushState` (no document load); mount-once guard; hidden views get `hidden` + `inert`; the router writes active state/title.
+- Integration: the shell-route tests — each folded path serves the shell (200, `text/html`, body is the index page carrying `?v=`-tagged asset refs, `Cache-Control: no-cache`), and a non-view path (e.g. `/nonexistent.html`) still 404s. `app/core/caching.py` must need NO change (the paths are unchanged) — pin that in the test's assertion set.
+- Coverage: **>90%** on `app/` (server delta is the small route factory + tests).
+- E2E: new story suite `tests/e2e/test_nav_switch_keeps_stream.py` run in isolation (`uv run pytest tests/e2e/test_nav_switch_keeps_stream.py -v --no-cov`) against the deterministic mock LLM (the ~8 s long stream gives the guaranteed mid-stream window — house pattern from `tests/e2e/test_hidden_tab_stream.py`). The phase-20 suite is REWRITTEN in task 02 (its premise — navbar click = navigation — is the behavior this phase removes); phase 73 (`test_hidden_tab_stream.py`) and phase 48 (`test_stop_generation.py`) must stay green UNCHANGED.
+
+## Completion Criteria
+- [ ] Owner repro passes in a REAL browser (real LLM): send → RAG mid-stream → Chat → the FULL answer completes, one brain turn, one `query_log` row (settled, no `turn cancelled`).
+- [ ] All five navbar views render in the shell; direct loads of `/`, `/sources.html`, `/git-sources.html`, `/tuning.html`, `/history.html` deep-link to the right view; the four old view `.html` files are deleted; the cache-busting E2E (`test_cache_busting.py`, `test_asset_cache_revalidation.py`) is green with the shell routes.
+- [ ] `uv run pytest tests/e2e/test_nav_switch_keeps_stream.py -v --no-cov` green in isolation, including the real-departure control (external navigation / Stop still cancels per phase 48).
+- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
+- [ ] One atomic `--no-gpg-sign` Conventional-Commits commit (e.g. `fix(chat): keep in-flight answers alive across in-app view switches`) whose body records the owner repro + the phase-48 refinement; phase dir moved to `.agents/phases/complete/`.