brain-of-reese/.agents/phases/complete/76_spa_nav_shell/00_phase.md at f665a83b1ad452a3a04884598f5892f8361ef7d4

Files

T

ducoterra ffa919b8bf fix(chat): keep in-flight answers alive across in-app view switches

Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).

Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.

Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
  five `<section class="view">` blocks; hidden views carry BOTH
  `hidden` and `inert` (WCAG — no focus/keyboard traversal). The
  shared header, the single `doc-modal-*` skeleton, and the
  `#app-version` footer each exist exactly once; the per-view copies
  from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
  bundler, No-CDN rule intact): lazy-imports a view module on FIRST
  show only (mount-once, hide-forever — the chat view's in-flight SSE
  reader persists across switches; that persistence IS the fix);
  intercepts same-shell navbar links with preventDefault +
  history.pushState (never a document load); handles popstate; single
  writer of `.nav-link` active state (is-active + aria-current),
  document.title, and the per-view meta description (values carried
  over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
  root-scoped queries; `initSharedHeader()` dropped (the header boots
  once in the shell via the chat module; the admin flag comes from the
  same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
  /tuning.html, /sources.html, /git-sources.html, /history.html —
  registered AFTER the API routers and BEFORE the static catch-all
  (routes-first). The phase-33 caching middleware applies no-cache +
  `?v=` rewriting unchanged; app/core/caching.py needed NO change
  (the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
  deep links to the old URLs keep working (the router picks the view
  from the pathname); `/?chat=<id>` is unaffected; the Containerfile
  bundles router.js (inlining the lazy view modules) and drops the
  folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
  keeps long saved answers in the chat, and the old cap (stricter than
  the 24_000-char total history budget) 422-rejected any second turn
  in such a chat (found by the phase-42 E2E suite on the shell).

Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).

Tests:
- Phase-20 suite REWRITTEN to the new semantics
  (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
  cancels — the stream survives the switch and the FULL answer
  settles; the pagehide partial persist REMAINS for real departures
  (the partial's exact shape — first streamed chunk prefix, no done
  metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
  LLM): the owner repro (send -> RAG mid-stream -> Chat: window
  sentinel survives = same document, FULL answer, exactly one brain
  turn in bor.chat.v1, exactly one settled query_log row, auto-saved
  row matches) + the same mid-stream switch against the other three
  views + the real-departure-still-cancels control + the no-switch
  baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
  invariants (click interceptor targets ONLY same-shell view paths,
  pushState-only switches, mount-once guard, hidden+inert pair,
  single-writer active state/title); shell-route integration tests
  (each folded path serves the shell with no-cache + `?v=` body; a
  non-view path still 404s); the file-reading unit pins re-pointed at
  the shell (the four view files are gone — the shell is the source
  of truth).

Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.

Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.

2026-09-06 06:31:31 -04:00

13 KiB

Raw Blame History

Source: Owner repro, verified in a real browser 2026-09-06: send a question → click RAG in the navbar mid-stream → click Chat → the answer never finishes (turn cancelled: no query_log row, and on return a dangling question with no brain record — the pre-token pagehide partial persist skips because acc is empty). Root cause: every navbar view is a separate HTML document (RAG = <a href="/sources.html">), so a navbar click is a REAL cross-document navigation — the chat page unloads, the in-flight fetch is aborted, and the phase-48 teardown (app/api/chat.py finally, chat: turn cancelled) stops the model. Story: n/a (TODO-derived — descendant of .agents/user_stories/sources-midstream.md (phase 20) and TODO.md L3 (phase 73)) Context: frontend/index.html (chat page; the shared <header class="app-header"> markup is copy-pasted into every page), frontend/assets/app.js (the chat logic — owns the in-flight SSE stream that must SURVIVE view switches; phase-20/48/73 machinery must stay intact and its source-level unit pins in tests/unit/test_frontend_*.py must keep passing), frontend/assets/{sources,git-sources,tuning,history}.js (page modules that boot at document load and call initSharedHeader()), frontend/assets/header.js (shared header: auth-gated links, active state, mobile hamburger), app/main.py (explicit routes + catch-all StaticFiles mount, API routes first), app/core/caching.py (HTML_PAGES list — the view PATHS do not change, so the no-cache + ?v= rewrite contract applies to the shell routes untouched), tests/e2e/test_sources_midstream_bug.py (phase-20 suite that encodes the OLD "partial survives the navigation" semantics — rewritten in task 02), tests/e2e/test_hidden_tab_stream.py (phase 73 — must stay green unchanged), tests/e2e/test_stop_generation.py (phase 48 — must stay green unchanged).

Objective

Collapse the five navbar views (Chat, RAG, Sources, Tuning, History) into views of ONE HTML shell so a navbar click is a client-side view switch (history.pushState + show/hide), never a document load. An in-flight answer keeps streaming through any navbar switch and completes when the user returns to Chat — the owner repro (send → RAG → Chat) must finish the FULL answer, settled server-side (one query_log row), with exactly one brain turn persisted. Real departures (tab close, leaving the app, the Stop button) still cancel the fetch and stop the model — the phase-48 contract, intact.

Owner decisions (chat, 2026-09-06 — recorded per AGENTS.md rule 3)

Option A (SPA-ify the navbar views) chosen over B (Service Worker owns the stream) and C (server-side turn registry + resume).
LOCKED-DECISION REFINEMENT (phase 48): "real navigation cancels the fetch" now means leaving the app (tab close, external/other-document navigation, Stop). In-app navbar switches no longer cancel. Owner-confirmed refinement — flagged, not silently deviated.
Boundaries: login.html, shared.html, doc-edit.html, document.html REMAIN separate documents (flow pages, not navbar tabs). A mid-stream navigation to doc-edit/document.html still cancels per phase 48 (follow-up candidate, out of scope). The SSE API is unchanged.
No new technology: vanilla JS modules, no framework, no bundler — consistent with the existing architecture and the No-CDN rule (AGENTS.md #6).

Design (shared by all tasks — the executor reads this, not the chat)

Shell: frontend/index.html becomes the shell. The shared header stays at the top of the shell (it is copy-pasted into every page today; in the shell it exists ONCE, so the header/nav/auth/steering ids — #app-nav, #nav-*, #sign-in-link*, #sign-out-btn*, #steering-*, #nav-toggle — deduplicate automatically). ONE <main id="main" class="app-main" tabindex="-1"> container (skip-link target unchanged) holds the five <section class="view" id="view-<name>" aria-label="…" tabindex="-1"> blocks; each folded page's own <main class="app-main"> wrapper (all five pages carry the identical one — layout CSS is class-based, so nothing is lost) is DROPPED when its content moves. Only the active view is rendered: hidden views carry BOTH hidden AND inert (WCAG — hidden views must not receive focus or keyboard traversal; AGENTS.md #5). ONE shell footer (the chat page's #app-version footer): per-page footers from folded views are dropped (app-version otherwise duplicates — it exists in both index.html and history.html).
Duplicate-id resolution (audited 2026-09-06, whole-file id scan of the five pages, comments stripped): the only cross-page id collisions in VIEW markup are (a) the per-view <main id="main"> — resolved by the single-main container above, and (b) the doc-modal-* family (9 ids: doc-modal, -backdrop, -close, -content, -desc, -meta, -open, -panel, -title), present in BOTH index.html and sources.html — the shell keeps EXACTLY ONE instance (the chat's, already in index.html), placed at body level (the modal is a position: fixed overlay; its DOM position is cosmetic). document-modal.js resolves all of them with document-level querySelector at MODULE IMPORT (its lines 44–52), so the single skeleton must be static markup present before any import — it is. app.js (chat chips) and sources.js (RAG rows) both call openDocumentModal(...) and work UNCHANGED against the one shared instance. The moved sources view markup DROPS its skeleton copy (task 02).
Same-document proof (canonical E2E assertion for "no document load"): set a window sentinel before the nav click (window.__shell_boot = "phase76") and assert it is still readable after the switch + return — a real navigation wipes window globals. Do NOT use performance.getEntriesByType("navigation").length — a real load resets that counter to 1 in the fresh document, so it cannot distinguish pushState from a reload.
View-scoped E2E absence pattern: in the shell, hidden views' elements REMAIN IN THE DOM (hidden+inert). Any E2E absence assertion (to_have_count(0)) on a view-scoped id must be re-scoped to the visible view (e.g. #view-rag #new-chat-btn) or switched to a visibility assertion (to_be_hidden). Known instance: tests/e2e/test_chat_persistence.py::test_persists_across_page_navigation asserts #new-chat-btn count 0 on the Sources page (task 02 fixes it).
Router (new frontend/assets/router.js, module, ~200 lines): a VIEW map of pathname → view name ("/" → chat, plus one entry per folded view); on boot it reads location.pathname, lazy-import()s the view module on FIRST show only, calls await module.mount(root), then shows it. Mount-once, hide-forever: a view's DOM (and JS state — for chat, the in-flight SSE reader) persists across switches; that persistence IS the fix. A delegated click handler on the navbar intercepts same-shell view links (preventDefault + pushState + switch — no document load); popstate switches views for back/forward. The router is the SINGLE WRITER of .nav-link active state (is-active + aria-current="page"), document.title, and the per-view <meta name="description"> (values carried over from the old pages' <head>s).
View modules: each folded page's JS changes from top-level boot to an exported async function mount(root); DOM queries scope to root (view element ids stay unique across the shell — verify, don't rename, to keep E2E selectors stable); the initSharedHeader() call is dropped (the header boots once in the shell via the chat module, as today).
Boot order in the shell: brand.js (classic) → app.js (module — the chat view; runs at shell boot exactly as today, so its boot/restore/streaming behavior is untouched) → router.js (module — lazy-imports non-chat views only).
Server: one small route factory in app/main.py serves the shell (frontend/index.html) for the folded view paths, registered BEFORE the static catch-all mount (API routes first, per existing comment). The phase-33 caching middleware already wraps the whole app and lists these paths in HTML_PAGES, so no-cache + ?v= asset rewriting applies unchanged — verify, don't rewire. The old per-view .html files are DELETED in the same task that lands their shell route (one source of truth).
Deep links: every old view URL (/sources.html, …) keeps working as a direct load — the server serves the shell, the router picks the view from the pathname. /?chat=<id> (the saved-chat deep link read by app.js at boot) is unaffected — it already lives on the shell's own path.
Auth flag in view modules: the folded view modules today call await initSharedHeader() and (in sources.js, git-sources.js) USE ITS RETURN VALUE as the admin flag. In the shell the header boots once via the chat module — so each view module DROPS initSharedHeader() and reads the admin flag from fetchIsAdmin() instead (the SAME cached /api/whoami promise header.js exports — zero extra requests; sources.js/history.js/tuning.js already import it, git-sources.js must add it to its import).

Dependencies

— (none)

Tasks

01_shell_router_tuning.md — shell + router + shell-route factory; fold Tuning as the first non-chat view (proves the pattern end-to-end, gate included).
02_fold_rag_sources_views.md — fold RAG (sources) + Sources (git-sources); REWRITE the phase-20 test_sources_midstream_bug.py to the new "the stream survives" semantics.
03_fold_history_view.md — fold History; verify the /?chat=<id> deep link and history suites.
04_header_shell_wiring.md — header is shell-owned (single-writer active state, auth gate, hamburger, per-view title/meta); the surviving documents keep their header copies; update the header/nav consistency suites.
05_stream_survival_e2e.md — new story suite test_nav_switch_keeps_stream.py: the owner repro pinned on the mock LLM (mid-stream switch → FULL answer, one brain turn, settled turn) + the real-departure-still-cancels control.
06_regression_sweep_commit.md — full suite + coverage floor + ruff/pyright; manual verification of the owner repro in a real browser; atomic commit; phase → complete.

Testing & Quality

Unit: tests/unit/test_frontend_router.py — source-level pins of the router invariants (house pattern: tests/unit/test_frontend_hidden_tab.py reads JS source without a browser): the click interceptor targets ONLY same-shell view paths; switches use pushState (no document load); mount-once guard; hidden views get hidden + inert; the router writes active state/title.
Integration: the shell-route tests — each folded path serves the shell (200, text/html, body is the index page carrying ?v=-tagged asset refs, Cache-Control: no-cache), and a non-view path (e.g. /nonexistent.html) still 404s. app/core/caching.py must need NO change (the paths are unchanged) — pin that in the test's assertion set.
Coverage: >90% on app/ (server delta is the small route factory + tests).
E2E: new story suite tests/e2e/test_nav_switch_keeps_stream.py run in isolation (uv run pytest tests/e2e/test_nav_switch_keeps_stream.py -v --no-cov) against the deterministic mock LLM (the ~8 s long stream gives the guaranteed mid-stream window — house pattern from tests/e2e/test_hidden_tab_stream.py). The phase-20 suite is REWRITTEN in task 02 (its premise — navbar click = navigation — is the behavior this phase removes); phase 73 (test_hidden_tab_stream.py) and phase 48 (test_stop_generation.py) must stay green UNCHANGED.

Completion Criteria

Owner repro passes in a REAL browser (real LLM): send → RAG mid-stream → Chat → the FULL answer completes, one brain turn, one query_log row (settled, no turn cancelled).
All five navbar views render in the shell; direct loads of /, /sources.html, /git-sources.html, /tuning.html, /history.html deep-link to the right view; the four old view .html files are deleted; the cache-busting E2E (test_cache_busting.py, test_asset_cache_revalidation.py) is green with the shell routes.
uv run pytest tests/e2e/test_nav_switch_keeps_stream.py -v --no-cov green in isolation, including the real-departure control (external navigation / Stop still cancels per phase 48).
uv run pytest green; uv run pytest --cov=app --cov-report=term-missing >90%; uv run ruff check . && uv run pyright clean.
One atomic --no-gpg-sign Conventional-Commits commit (e.g. fix(chat): keep in-flight answers alive across in-app view switches) whose body records the owner repro + the phase-48 refinement; phase dir moved to .agents/phases/complete/.

13 KiB Raw Blame History Unescape Escape

Phase 76 — In-app view switches never halt a generating answer (the navbar views become one document)