fix(chat): keep generating while the tab is hidden

Root cause (task 01): none of C1-C3 - in Chromium 151 (real mode) a
merely-hidden tab neither stops the stream (frames arrive at full rate;
turn completes) nor fires pagehide on tab switch; C1's double-record
path was proven latent via a synthetic pagehide (trigger is
browser-dependent, e.g. Safari) and C2 (the 120s pre-token guard) was
confirmed to fire while hidden.

- C1: the pagehide partial-persist is correlated with the turn's settle
  (leavePartialIndex) - the done/stop settle REPLACES it in place
  (identity-guarded rememberBrainTurn in-place mode), so bor.chat.v1
  and the auto-saved saved_chats row keep exactly ONE brain turn per
  question; a real navigation never runs a settle, so the leave-save
  is unchanged.
- C2: the visibility re-arm gives the still-armed pre-token guard a
  fresh TURN_TIMEOUT_MS when the tab returns to visible - hidden time
  no longer counts toward the 120s guard.
- Phase-48 teardown contract untouched: Stop / tab close / real
  navigation still cancel the fetch and stop the model.
- Unit pins: tests/unit/test_frontend_hidden_tab.py (the app.js
  mechanisms without a browser).
- E2E pins: tests/e2e/test_hidden_tab_stream.py - synthetic pagehide
  mid-stream completes exactly once with one brain turn (localStorage
  + auto-saved row), reload restores one bubble, no-event baseline,
  and the fake-clock pre-token guard re-arm (discriminating: fails
  with the re-arm disabled).
This commit is contained in:
2026-09-05 14:34:33 -04:00
parent 45c3fa2863
commit a16130c71d
28 changed files with 4127 additions and 10 deletions
@@ -0,0 +1,10 @@
**Task 01 complete — repro performed, root cause pinned.**
- Real-browser repro (Chromium 151 headful, 2 real tabs in 1 window, 90s re-paced mock stream, scratch DB — no owner data touched; all instrumentation throwaway, no repo changes):
- **Scenario A:** real tab switch mid-stream → 65s hidden: stream kept arriving at full wire rate, turn completed on return, no error, `bor.chat.v1` = 1 brain turn; server: `total_ms=90143`, **no** `turn cancelled` → C3 ruled out. Tab switch fired only `visibilitychange` — **no `pagehide`** in Chromium.
- **Scenario C:** synthetic `pagehide` mid-stream → **2 brain turns** (336-char partial + 5379-char full) for one question; reload renders both → C1 corruption branch proven latent (trigger is browser-dependent, e.g. Safari).
- **Scenario B:** 130s pre-token hold + hidden tab → 120s `TURN_TIMEOUT_MS` guard fired **while hidden** ("That's taking a long time — the answer may be stuck."), server `turn cancelled total_ms=119999` (client-initiated) → C2 confirmed.
- Verdict: **none of C1–C3 fired on a simple tab switch in the test browser** (answer continues + completes; no stall, no cancel, no pagehide); C1 latent path + C2 conditional path pinned with evidence (`.agents/reports/73_hidden_tab_stream/task01_findings.md` + JSONs/screenshots/server log).
- Gates: `uv run ruff check .` clean; `uv run pyright` 0 errors; `uv run pytest --cov=app` → **1511 passed, app/ coverage 99%** (>90%); E2E `test_smoke.py` 3 passed.
- Notable: first attempt's "hidden" state was an X11 focus flake (XWayland); re-run used stable-focus assertions + contamination detection (exactly 1 hidden/1 visible event). No timer throttling observed in this environment — noted, not an app defect.
- Next pending task: `73_hidden_tab_stream/02_fix_hidden_tab.md`.
@@ -0,0 +1,77 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 19%]
........................................................................ [ 23%]
........................................................................ [ 28%]
........................................................................ [ 33%]
........................................................................ [ 38%]
........................................................................ [ 42%]
........................................................................ [ 47%]
........................................................................ [ 52%]
........................................................................ [ 57%]
........................................................................ [ 61%]
........................................................................ [ 66%]
........................................................................ [ 71%]
........................................................................ [ 76%]
........................................................................ [ 81%]
........................................................................ [ 85%]
........................................................................ [ 90%]
........................................................................ [ 95%]
....................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2946 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
@@ -0,0 +1,16 @@
All green. Task 02 is complete.
**Implemented (frontend-only, `frontend/assets/app.js`):**
- **C1 hardening (unconditional):** module-scope `leavePartialIndex` (reset in `runTurn`); the `pagehide` handler records the pushed partial's index after `rememberBrainTurn`; `rememberBrainTurn` gained an identity-guarded in-place mode (`replaceIndex >= 0 && conversation[replaceIndex]?.who === "brain"` → replace, else append) with `saveConversation()`+`persistConversation()` running once on the written record; the `done` and stop-path settles pass `leavePartialIndex` so the final record *replaces* the partial (plus the zero-frame fallback, passed for uniform correlation though unreachable with a partial). Real-navigation leave-save unchanged (no settle runs on unload).
- **C2 fix (task 01 confirmed C2):** `turnTimeoutCb` stores the guard callback; a `visibilitychange` listener re-arms the guard with a fresh `TURN_TIMEOUT_MS` when the tab returns to visible with the guard still armed — hidden time no longer counts toward the 120s pre-token guard.
- Stop/teardown contract (phase 48) untouched. New unit pins: `tests/unit/test_frontend_hidden_tab.py` (8 tests).
- Fixed one pre-existing-pin break I introduced then resolved by keeping the done-call on one line (`test_chat_persistence.py` format pin).
**Results (all green):**
- `uv run pytest --cov=app --cov-report=term-missing` → 1519 passed, app/ coverage **99%** (>90% floor)
- `uv run ruff check . && uv run pyright` → clean (0 errors)
- Regression E2E in isolation: `test_stop_generation.py` 3✓, `test_chat_persistence.py` 4✓, `test_chat_history.py` 5✓, `test_retry_answer.py` 4✓, `test_chat_rag.py` 3✓, `test_sources_midstream_bug.py` 4✓, `test_loading_feedback.py` 5✓
**Decisions:** all three runTurn settle write-sites pass the correlation index (task named two; the fallback is defensive, provably unreachable with a partial). Commit is left to task 03 per phase plan (root-cause line goes in that commit body).
**Next pending task:** `.agents/phases/todo/73_hidden_tab_stream/03_e2e_hidden_tab_stream.md`
@@ -0,0 +1,78 @@
........................................................................ [ 4%]
........................................................................ [ 9%]
........................................................................ [ 14%]
........................................................................ [ 18%]
........................................................................ [ 23%]
........................................................................ [ 28%]
........................................................................ [ 33%]
........................................................................ [ 37%]
........................................................................ [ 42%]
........................................................................ [ 47%]
........................................................................ [ 52%]
........................................................................ [ 56%]
........................................................................ [ 61%]
........................................................................ [ 66%]
........................................................................ [ 71%]
........................................................................ [ 75%]
........................................................................ [ 80%]
........................................................................ [ 85%]
........................................................................ [ 90%]
........................................................................ [ 94%]
........................................................................ [ 99%]
....... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
-----------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 22 0 100%
app/api/chat.py 176 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 7 0 100%
app/api/doc_drafts.py 93 0 100%
app/api/docs.py 50 0 100%
app/api/git_sources.py 212 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 8 0 100%
app/api/sync.py 101 0 100%
app/config.py 126 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 20 0 100%
app/core/caching.py 108 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/logging.py 13 0 100%
app/db.py 21 0 100%
app/main.py 52 0 100%
app/models.py 86 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 222 0 100%
app/rag/archive_upload.py 128 0 100%
app/rag/chunker.py 206 4 98%
app/rag/git_sources.py 14 0 100%
app/rag/importer.py 180 3 98%
app/rag/llm.py 216 0 100%
app/rag/overview.py 71 0 100%
app/rag/prompts.py 65 0 100%
app/rag/retriever.py 150 3 98%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 24 0 100%
app/schemas.py 205 0 100%
-----------------------------------------------
TOTAL 2946 12 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
validation OK
Binary file not shown.

After

Width:  |  Height:  |  Size: 291 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 320 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 305 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 309 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 357 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 345 KiB

File diff suppressed because one or more lines are too long
Binary file not shown.

After

Width:  |  Height:  |  Size: 369 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 354 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 345 KiB

@@ -0,0 +1,127 @@
{
"scenario": "B \u2014 C2 probe: 120s guard in a hidden pre-token window (130s silence)",
"wall_seconds": {
"hide": 6.5,
"return": 138.6
},
"probes": [
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 16.7
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 26.7
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 36.7
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 46.7
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 56.7
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 66.8
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 76.8
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 86.8
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 96.8
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 106.8
},
{
"vis": "hidden",
"bubble": 16,
"btn": "Stop",
"status": "Brain of Reese is thinking",
"banner": "",
"dt": 116.8
},
{
"vis": "hidden",
"bubble": 53,
"btn": "Send",
"status": "",
"banner": "That's taking a long time \u2014 the answer may be stuck. Try again \u2014 if this persists, check the LLM is reachable.",
"dt": 126.8
},
{
"vis": "hidden",
"bubble": 53,
"btn": "Send",
"status": "",
"banner": "That's taking a long time \u2014 the answer may be stuck. Try again \u2014 if this persists, check the LLM is reachable.",
"dt": 136.8
}
],
"final_state": {
"vis": "visible",
"bubble": 53,
"btn": "Send",
"status": "",
"banner": "That's taking a long time \u2014 the answer may be stuck. Try again \u2014 if this persists, check the LLM is reachable."
},
"localStorage_messages": [
{
"who": "user",
"len": 53,
"head": "please write a long answer about your favorite topics"
}
]
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 379 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 364 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 385 KiB

@@ -0,0 +1,30 @@
{
"scenario": "C \u2014 synthetic pagehide mid-stream (C1 double-record path)",
"mid_bubble_chars": 327,
"brain_turns_after_completion": [
{
"len": 336,
"head": "1. Step 1: configure node-1 with the homelab defaults and verify that step 1 of the long walkthrough",
"tail": "figure node-3 with the homelab defaults and verify that step"
},
{
"len": 5379,
"head": "1. Step 1: configure node-1 with the homelab defaults and verify that step 1 of the long walkthrough",
"tail": "ng walkthrough is complete before moving on.\nLONG-ANSWER-END"
}
],
"restored_messages": [
{
"who": "user",
"len": 53
},
{
"who": "brain",
"len": 327
},
{
"who": "brain",
"len": 5228
}
]
}
@@ -0,0 +1,150 @@
# Phase 73 / Task 01 — Repro + root-cause findings
**Bug (TODO.md L3):** "Clicking on another tab while an answer is generating
stops that answer from being generated. Responses should continue to generate
unless you outright close the tab."
**Environment (the "test browser"):** real (headful) Chromium
`151.0.0.0` (Playwright chromium-1234 build, X11/XWayland on the owner's
KDE Plasma Wayland desktop, display :0), launched with ONLY
`--disable-popup-blocking` added (needed for the scripted second tab) —
**no** `--disable-*-background*` flags, so real hidden-tab behavior is
under test. The app ran against the deterministic mock LLM
(`tests/e2e/mock_llm.py`) behind a re-pacing proxy (0.2 s per 12-char
SSE frame → a ~90 s stream, a guaranteed mid-stream hidden window), on a
scratch Postgres DB (`bor_repro73`) so no owner data was touched.
Method: two REAL tabs in ONE window (second tab opened via
`window.open`, tab switched via CDP `Target.activateTarget` /
`Page.bringToFront` — verified real by the `document.visibilityState`
transitions). In-page instrumentation (temporary, removed — no repo
changes): `pagehide`/`pageshow`/`visibilitychange`/`freeze`/`resume`/
`beforeunload`/window focus/blur events with timestamps; a 400 ms
recorder (visibility, `hasFocus`, live-bubble text length = frame
arrivals, status/banner/button text); CDP `Network.*` lifecycle of
`POST /api/chat`; `performance.getEntriesByType('resource')`;
localStorage `bor.chat.v1`; server (uvicorn) log.
## Scenario A — mid-stream real tab switch, 65 s hidden (main repro)
`task01_A_evidence.json`, screenshots `task01_A_0…5_*.png`,
server log `server_A.log`.
1. Long answer started; at ~7 s (bubble ≈ 460 chars, `streaming`) the
driver switched to the other tab. Tab A went `hidden` — exactly ONE
`visibilitychange {state:"hidden"}` event, at the switch.
2. **While hidden (65 s) the answer kept generating at full wire rate**:
the live bubble grew linearly 456 → 4141 chars (≈ 58 chars/s = the
stream's 12 chars / 0.2 s frame pace) — SSE frames kept arriving the
entire time; no stall, no drop, no error copy, no banner.
(The 400 ms recorder cadence never degraded — no background timer
throttling was observed in this XWayland/KWin environment; noted as an
environment property, not an app defect.)
3. Returned to the tab at ~72 s: `visibilitychange {state:"visible"}`;
the turn settled ~18 s later — button back to "Send", no banner.
4. `bor.chat.v1`: exactly ONE brain turn (the full 5 379-char answer,
ending `LONG-ANSWER-END`) for the one question.
5. **`pagehide` did NOT fire** on the tab switch (nor `freeze`/`resume`/
`beforeunload`/`pageshow`). In real-mode Chromium 151, a merely-hidden
tab fires only `visibilitychange`.
6. Network: `POST /api/chat` → `requestWillBeSent` →
`responseReceived (200)` → `loadingFinished`. **No `loadingFailed`**
(verified on a clean re-probe: the earlier duplicate/failed entries
were a bug in the repro's own CDP event router, not the browser).
7. Server: the turn's settle line shows `total_ms=90143/90145` — the
server streamed the FULL 90 s to the hidden tab; there is **NO
`chat: turn cancelled`** line for this turn (the phase-48 line that
would prove a real consumer departure).
**⇒ In the test browser, a tab switch does NOT stop the answer (C3 and
C4 ruled out; C1's trigger did not occur; the guard was already cleared
by the first frame, so C2 was not on the path).**
## Scenario C — C1 double-record path (synthetic `pagehide` mid-stream)
`task01_C_evidence.json`, screenshots `task01_C_1…3_*.png`.
Chromium does not fire `pagehide` on a tab switch, but browsers that DO
(e.g. Safari, which bfcaches/freeze-then-`pagehide`s hidden tabs) would
run the app's `pagehide` handler (`frontend/assets/app.js` ~L2162)
mid-turn. Dispatching a synthetic `PageTransitionEvent('pagehide',
{persisted:true})` at ~7 s into the stream (bubble ≈ 327 chars) — the
EXACT code path a real `pagehide` executes — then letting the turn
complete:
- `bor.chat.v1` ended with **TWO brain turns for ONE question**:
the 336-char partial (the pagehide persist) AND the full 5 379-char
answer (the `done` persist).
- After a reload, the restore renders BOTH: user → brain (327) → brain
(5 228) — the duplicated / "truncated-looking" conversation the bug
report describes.
**⇒ C1's corruption branch is real and deterministic whenever the
browser fires `pagehide` on a merely-hidden tab; it never double-persists
in Chromium because Chromium never fires that event for a tab switch.
This is the branch task 02 hardens unconditionally.**
## Scenario B — C2 probe: the 120 s guard in a hidden pre-token window
`task01_B_evidence.json`, screenshots `task01_B_1…3_*.png`.
The proxy was re-run holding the FIRST frame for 130 s (pre-token
silence > `TURN_TIMEOUT_MS`). The tab was switched away at ~7 s and
stayed hidden past the guard's deadline:
- Probes every 10 s while hidden: no banner through t≈117 s; at
t≈127 s the button had returned to "Send" and the banner read
**"That's taking a long time — the answer may be stuck."** — the guard
fired at its 120 s deadline **during the hidden window** and errored
the turn.
- The first frame would have arrived at t≈130 s — too late: the guard
had already aborted the fetch. On return the error banner is still up;
the user perceives "switching tabs killed the answer", but the trigger
is the pre-token silence, not the switch.
- `bor.chat.v1` holds only the user message (pre-token abort persists
nothing brain-side — phase-20 convention).
- Server: `chat: turn cancelled … total_ms=119999` — the cancellation is
the CLIENT's guard abort (client-initiated), not a browser/proxy drop
of the hidden tab.
**⇒ C2 is confirmed as a real, conditional stop path: any turn whose
first frame takes > 120 s errors out if the tab is hidden at the 120 s
mark (the guard is a plain `setTimeout` with no visibility awareness;
where a browser DOES throttle background timers, its fire time drifts
relative to token arrival, as the C2 branch describes).**
## Decision-tree verdict
- **C1 — `pagehide` on tab switch:** did NOT fire in the test browser
(Chromium 151 real mode fires only `visibilitychange` for a hidden
tab). Its double-record corruption path was nonetheless demonstrated
deterministically (synthetic `pagehide`) — latent in browsers that do
fire `pagehide` on hide (e.g. Safari); the trigger is
browser-dependent, so task 02's fix applies unconditionally.
- **C2 — 120 s guard while hidden:** CONFIRMED to fire (scenario B) in
the pre-token-only window, erroring the turn with the "stuck" copy.
- **C3 — real connection cancel:** ruled out (no `turn cancelled` for
the hidden-window turn; `loadingFinished`, no `loadingFailed`; full
90 s server-side stream completed).
- **C4 — none of the above:** in the test browser a merely-hidden tab
did NOT stop the answer — it kept generating at full rate and
completed on return. The owner's symptom did not reproduce on
Chromium 151 desktop; the reproducible stop paths are C2 (slow
pre-token + hidden) and C1's corruption (browsers that fire
`pagehide` on hide, e.g. Safari — where a hidden tab is also the
likelier place for the stream to actually pause).
**One-line root cause (for the phase-73 commit message body):**
`none of C1–C3 — in Chromium 151 (real mode) a merely-hidden tab neither
stops the stream (frames arrive at full rate; turn completes) nor fires
pagehide on tab switch; C1's double-record path was proven latent via a
synthetic pagehide (trigger is browser-dependent, e.g. Safari) and C2
(the 120s pre-token guard) was confirmed to fire while hidden.`
## Cleanup
- No permanent code changes: `git status` shows only this (untracked)
evidence directory; all instrumentation lived in throwaway scripts
under `/tmp/repro73/` and in the browser profiles.
- Scratch DB `bor_repro73` dropped; repro stack (mock 8901 / proxy 8900
/ app 8123) terminated.