Files
brain-of-reese/tests/integration/test_containerfile_assets.py
T
ducoterra ffa919b8bf fix(chat): keep in-flight answers alive across in-app view switches
Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).

Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.

Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
  five `<section class="view">` blocks; hidden views carry BOTH
  `hidden` and `inert` (WCAG — no focus/keyboard traversal). The
  shared header, the single `doc-modal-*` skeleton, and the
  `#app-version` footer each exist exactly once; the per-view copies
  from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
  bundler, No-CDN rule intact): lazy-imports a view module on FIRST
  show only (mount-once, hide-forever — the chat view's in-flight SSE
  reader persists across switches; that persistence IS the fix);
  intercepts same-shell navbar links with preventDefault +
  history.pushState (never a document load); handles popstate; single
  writer of `.nav-link` active state (is-active + aria-current),
  document.title, and the per-view meta description (values carried
  over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
  root-scoped queries; `initSharedHeader()` dropped (the header boots
  once in the shell via the chat module; the admin flag comes from the
  same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
  /tuning.html, /sources.html, /git-sources.html, /history.html —
  registered AFTER the API routers and BEFORE the static catch-all
  (routes-first). The phase-33 caching middleware applies no-cache +
  `?v=` rewriting unchanged; app/core/caching.py needed NO change
  (the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
  deep links to the old URLs keep working (the router picks the view
  from the pathname); `/?chat=<id>` is unaffected; the Containerfile
  bundles router.js (inlining the lazy view modules) and drops the
  folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
  keeps long saved answers in the chat, and the old cap (stricter than
  the 24_000-char total history budget) 422-rejected any second turn
  in such a chat (found by the phase-42 E2E suite on the shell).

Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).

Tests:
- Phase-20 suite REWRITTEN to the new semantics
  (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
  cancels — the stream survives the switch and the FULL answer
  settles; the pagehide partial persist REMAINS for real departures
  (the partial's exact shape — first streamed chunk prefix, no done
  metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
  LLM): the owner repro (send -> RAG mid-stream -> Chat: window
  sentinel survives = same document, FULL answer, exactly one brain
  turn in bor.chat.v1, exactly one settled query_log row, auto-saved
  row matches) + the same mid-stream switch against the other three
  views + the real-departure-still-cancels control + the no-switch
  baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
  invariants (click interceptor targets ONLY same-shell view paths,
  pushState-only switches, mount-once guard, hidden+inert pair,
  single-writer active state/title); shell-route integration tests
  (each folded path serves the shell with no-cache + `?v=` body; a
  non-view path still 404s); the file-reading unit pins re-pointed at
  the shell (the four view files are gone — the shell is the source
  of truth).

Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.

Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.
2026-09-06 06:31:31 -04:00

316 lines
14 KiB
Python

"""Integration: Containerfile stage-1 (frontend) asset coverage pin (phase 23).
HERMETIC — no podman, no network, no database: this suite parses the
``Containerfile`` and ``frontend/`` as plain text and pins the image-build
coverage that the TODO L6 bug ("Fix Containerfile build not working")
demonstrated can silently rot in two independent ways:
* the stage-1 ``cp`` line only copies the pages that existed when it was
written (``document.html`` / ``login.html`` + their scripts +
``markdown.js`` were all missing from the image), and
* absolute module imports (``import … from "/assets/header.js"``) break
the esbuild bundle, so a "fixed" stage 1 can still ship a broken page.
The pins (each is one test, per the phase-23 task file):
1. every ``frontend/*.html`` page is copied into stage 1's ``/out``
— exactly (a new page without a ``cp`` entry fails; a ``cp`` of a
deleted page also fails);
2. every local ``assets/`` / ``/assets/`` ``src=``/``href=`` reference in
the pages is produced by a stage-1 line (``esbuild … --outfile`` or
``cp``) — the missing-``markdown.js``-style gap cannot reappear;
3. the set of ``type="module"`` page scripts the HTML references equals
the set of inputs esbuild ``--bundle``s in stage 1;
4. ``header.js`` is imported relatively by every page script and loaded
by NO direct ``<script>`` tag (single-evaluation design pin,
owner-confirmed 2026-08-24);
5. ``markdown.js`` is a classic script: stage-1 minify line WITHOUT
``--bundle``, and no top-level ``import``/``export`` in the source
(the source-level assumption that makes that build line safe);
6. the frontend stage pins a concrete ``esbuild@X.Y.Z`` (no floating
version — the exact pinned 0.25.5 is what the diagnosis reproduced
against).
"""
from __future__ import annotations
import re
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
FRONTEND = REPO_ROOT / "frontend"
ASSETS = FRONTEND / "assets"
CONTAINERFILE = REPO_ROOT / "Containerfile"
# A local asset reference: src="assets/…" or src/href="/assets/…"
# (data: URIs, "#main" anchors, and same-origin page links never match).
_ASSET_REF_RE = re.compile(r"""(?:src|href)\s*=\s*["']((?:/)?assets/[^"']+)["']""")
_SCRIPT_TAG_RE = re.compile(r"<script\b[^>]*>", re.IGNORECASE)
_MODULE_ATTR_RE = re.compile(r"""type\s*=\s*["']module["']""", re.IGNORECASE)
_SRC_ATTR_RE = re.compile(r"""src\s*=\s*["']([^"']+)["']""")
_TOP_LEVEL_MODULE_RE = re.compile(r"^\s*(?:import\b|export\b)", re.MULTILINE)
def _stage1_lines() -> list[str]:
"""The physical lines of the ``frontend`` build stage (FROM … up to,
but not including, the next FROM)."""
assert CONTAINERFILE.is_file(), f"missing Containerfile: {CONTAINERFILE}"
lines = CONTAINERFILE.read_text(encoding="utf-8").splitlines()
start = next(
i
for i, ln in enumerate(lines)
if re.match(r"^FROM\s+\S+\s+AS\s+frontend\b", ln, re.IGNORECASE)
)
end = len(lines)
for j in range(start + 1, len(lines)):
if re.match(r"^FROM\s", lines[j], re.IGNORECASE):
end = j
break
return lines[start:end]
def _is_esbuild_invocation(ln: str) -> bool:
"""True if the line actually RUNS esbuild (not just mentions it in a
comment or in the npm install line)."""
stripped = ln.lstrip()
return not stripped.startswith("#") and bool(re.search(r"(?:^|&&)\s*esbuild\b", ln))
def _esbuild_outputs(stage1: list[str]) -> dict[str, bool]:
"""basename of every asset a stage-1 esbuild line writes → was it
``--bundle``d? (Only lines with an ``--outfile`` count.)"""
outputs: dict[str, bool] = {}
for ln in stage1:
if not _is_esbuild_invocation(ln):
continue
m = re.search(r"--outfile=(\S+)", ln)
assert m, f"stage-1 esbuild line without --outfile: {ln.strip()}"
assert m.group(1).startswith("/out/assets/"), (
f"stage-1 asset must be written under /out/assets (the served "
f"/assets/ URL space), got {m.group(1)}"
)
outputs[m.group(1).rsplit("/", 1)[-1]] = "--bundle" in ln
return outputs
def _esbuild_bundle_inputs(stage1: list[str]) -> set[str]:
"""Basenames of the inputs to every stage-1 ``esbuild … --bundle``."""
inputs: set[str] = set()
for ln in stage1:
if _is_esbuild_invocation(ln) and "--bundle" in ln:
m = re.search(r"esbuild\s+(\S+)", ln)
assert m, f"stage-1 esbuild --bundle line without an input: {ln.strip()}"
inputs.add(m.group(1).rsplit("/", 1)[-1])
return inputs
def _cp_produced_basenames(stage1: list[str]) -> set[str]:
"""Basenames written into the image by stage-1 ``cp`` lines (the last
argument is the destination; a ``/``-suffixed one is a directory)."""
produced: set[str] = set()
for ln in stage1:
m = re.search(r"\bcp\s+(.+?)(?:\s*\\)?\s*$", ln)
if not m:
continue
args = m.group(1).split()
assert len(args) >= 2, f"malformed cp line in stage 1: {ln.strip()}"
dest = args[-1]
if dest.endswith("/"):
produced.update(a.rsplit("/", 1)[-1] for a in args[:-1])
else: # single-file rename: the image sees the destination name
produced.add(dest.rsplit("/", 1)[-1])
return produced
def _local_asset_basenames() -> set[str]:
"""Every local assets/ reference (basenames) across all frontend pages."""
refs: set[str] = set()
for html in sorted(FRONTEND.glob("*.html")):
text = html.read_text(encoding="utf-8")
refs.update(m.group(1).rsplit("/", 1)[-1] for m in _ASSET_REF_RE.finditer(text))
return refs
def _module_script_basenames() -> set[str]:
"""Basenames of the ``<script type="module" src=…>`` page scripts."""
names: set[str] = set()
for html in sorted(FRONTEND.glob("*.html")):
for tag in _SCRIPT_TAG_RE.findall(html.read_text(encoding="utf-8")):
if _MODULE_ATTR_RE.search(tag):
m = _SRC_ATTR_RE.search(tag)
assert m, f"{html.name}: module <script> without src: {tag}"
names.add(m.group(1).rsplit("/", 1)[-1])
return names
# ---------- 1. page coverage ----------
def test_every_html_page_is_copied_into_stage1() -> None:
"""Every ``frontend/*.html`` page is copied into stage 1's ``/out`` —
and the ``cp`` set is EXACTLY the pages on disk: a new page without a
matching cp entry (image 404s) and a cp of a deleted page (stale build
step) both fail here."""
stage1 = _stage1_lines()
on_disk = {p.name for p in FRONTEND.glob("*.html")}
assert on_disk, "frontend/ contains no .html pages — test is blind"
copied: set[str] = set()
for ln in stage1:
if ".html" not in ln:
continue
m = re.search(r"\bcp\s+(.+?)(?:\s*\\)?\s*$", ln)
if not m:
continue
args = m.group(1).split()
dest = args[-1]
assert dest.rstrip("/") == "/out", f"pages must be copied into /out, got {dest}"
copied.update(a.rsplit("/", 1)[-1] for a in args[:-1] if a.endswith(".html"))
missing = on_disk - copied
extra = copied - on_disk
assert not missing, (
f"pages missing from stage 1's cp line (404 in the image): {sorted(missing)}"
)
assert not extra, (
f"stage 1 copies pages that no longer exist in frontend/: {sorted(extra)}"
)
# ---------- 2. asset-reference coverage ----------
def test_every_local_asset_reference_is_produced() -> None:
"""Every local assets/… src/href in the pages is produced by a stage-1
line — esbuild ``--outfile=/out/assets/<name>`` or a ``cp`` of it.
This is what catches a missing-markdown.js-style gap: a page that
references an asset stage 1 never builds is a 404 in the image."""
stage1 = _stage1_lines()
produced = set(_esbuild_outputs(stage1)) | _cp_produced_basenames(stage1)
refs = _local_asset_basenames()
assert refs, "no local assets/ references found in the pages — test is blind"
missing = refs - produced
assert not missing, (
f"local assets referenced by the pages but not produced by stage 1 "
f"(404 in the image): {sorted(missing)}"
)
# ---------- 3. page-module ⇄ bundle-input parity ----------
def test_page_module_scripts_are_bundled() -> None:
"""The set of ``type="module"`` page scripts the HTML references
(basenames) EQUALS the set of inputs esbuild ``--bundle``s in stage 1
(today: app.js, sources.js, document.js, login.js). A new page script
without a bundle entry would 404 in the image; a bundle input whose
page no longer references it is dead build weight."""
stage1 = _stage1_lines()
html_modules = _module_script_basenames()
bundled = _esbuild_bundle_inputs(stage1)
assert html_modules, "no module page scripts found in the pages — test is blind"
assert html_modules == bundled, (
f"page module scripts {sorted(html_modules)} != esbuild bundle inputs "
f"{sorted(bundled)} — stage 1 must bundle exactly the pages' modules"
)
# ---------- 4. single-evaluation design pin ----------
def test_header_module_is_imported_not_directly_loaded() -> None:
"""Owner-confirmed design (2026-08-24, A4-2): NO page loads header.js
with a direct ``<script>`` tag — in the image the bundled page script
already contains the header code, so a raw header.js tag would evaluate
the module TWICE (duplicate sign-out listener, double init). Every page
script imports it relatively instead (``from "./header.js"``) — a
hoisted import that guarantees evaluation order in dev AND in the
bundle, and the only form esbuild can resolve."""
htmls = sorted(FRONTEND.glob("*.html"))
assert htmls, "no frontend pages — test is blind"
for html in htmls:
text = html.read_text(encoding="utf-8")
for tag in _SCRIPT_TAG_RE.findall(text):
m = _SRC_ATTR_RE.search(tag)
assert m is None or m.group(1).rsplit("/", 1)[-1] != "header.js", (
f"{html.name}: direct header.js <script> tag — double-evaluation "
f"trap in the image: {tag}"
)
page_scripts = _module_script_basenames()
assert page_scripts, "no module page scripts found — test is blind"
for name in sorted(page_scripts):
js_path = ASSETS / name
assert js_path.is_file(), f"page script missing: {js_path}"
body = js_path.read_text(encoding="utf-8")
# Phase 76: the shell's router (router.js) is NOT a view module
# — the shell's shared header boots via the chat module (app.js,
# which imports header.js), and the view modules the router
# lazy-imports (tuning.js, …) import header.js themselves, so
# the single-evaluation contract holds without the router
# importing it (it owns the view switch, not the header).
if name == "router.js":
assert 'from "./header.js"' not in body, (
"router.js must not import the header module — the chat "
"module boots the shell's ONE header"
)
continue
assert re.search(r"""from\s+["']\./header\.js["']""", body), (
f"{name}: must import the shared header module relatively "
f'("from \\"./header.js\\"")'
)
assert '"/assets/header.js"' not in body, (
f"{name}: absolute header import breaks the esbuild stage-1 bundle"
)
# ---------- 5. markdown.js is a classic script ----------
def test_markdown_js_is_a_produced_classic_script() -> None:
"""markdown.js ships minified WITHOUT --bundle (it is a classic global
script — window.markdownRender — loaded by index.html and
document.html), and the source has no top-level import/export: that
source-level fact is exactly what makes the no-bundle build line safe.
Pinning both sides keeps the assumption honest."""
stage1 = _stage1_lines()
outputs = _esbuild_outputs(stage1)
assert "markdown.js" in outputs, (
"markdown.js has no stage-1 esbuild --outfile line — index.html and "
"document.html would 404 the renderer in the image"
)
assert outputs["markdown.js"] is False, (
"markdown.js must NOT be --bundled: it is a classic global script, and "
"bundling it would strip the window-level globals the pages rely on"
)
md_path = ASSETS / "markdown.js"
assert md_path.is_file(), f"missing {md_path}"
hits = _TOP_LEVEL_MODULE_RE.findall(md_path.read_text(encoding="utf-8"))
assert not hits, f"markdown.js has top-level module syntax: {hits!r}"
# ---------- 6. pinned esbuild ----------
def test_esbuild_stays_pinned() -> None:
"""The frontend stage installs esbuild at a CONCRETE X.Y.Z. The phase-23
diagnosis was reproduced against the exact pinned 0.25.5; a floating
version (latest, ^, ~, or no spec) would let an upstream release change
the bundle behavior with no diff to see."""
stage1 = _stage1_lines()
install_lines = [ln for ln in stage1 if re.search(r"\bnpm\s+(?:install|i|add)\b", ln)]
assert install_lines, "no npm install line in the frontend stage"
pinned = [
v for ln in install_lines for v in re.findall(r"\besbuild@(\d+\.\d+\.\d+)\b", ln)
]
assert pinned, (
"the frontend stage must pin esbuild@X.Y.Z — a floating esbuild is how "
"the bundle behavior rots silently"
)
for ln in install_lines:
for m in re.finditer(r"\besbuild(@\S*)?", ln):
spec = m.group(1) or ""
assert re.fullmatch(r"@\d+\.\d+\.\d+", spec), (
f"esbuild must be installed with a concrete pin, got "
f"{m.group(0)!r} in: {ln.strip()}"
)