2ac3fc89c2f7cfa3cf230b6122ace2a60346984a
57
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ducoterra
|
ffa919b8bf |
fix(chat): keep in-flight answers alive across in-app view switches
Root cause (owner repro, verified in a real browser 2026-09-06): the five navbar views (Chat, RAG, Sources, Tuning, History) were separate HTML documents, so a navbar click was a REAL cross-document navigation — the chat page unloaded, the in-flight SSE fetch was aborted, and the phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled") stopped the model. Observed: send question -> click RAG mid-stream -> click Chat -> the answer never finished: no `query_log` row, and on return a dangling question with no brain record (the pre-token pagehide partial persist skips because `acc` is empty). Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06, flagged per AGENTS.md rule 3, not silently deviated): "real navigation cancels the fetch" now means LEAVING THE APP — tab close, external/other-document navigation, the Stop button. In-app navbar switches are client-side view switches and no longer cancel. Fix — Option A (SPA shell), chosen over B (Service Worker owns the stream) and C (server-side turn registry + resume): - frontend/index.html is the shell: ONE `<main id="main">` holds the five `<section class="view">` blocks; hidden views carry BOTH `hidden` and `inert` (WCAG — no focus/keyboard traversal). The shared header, the single `doc-modal-*` skeleton, and the `#app-version` footer each exist exactly once; the per-view copies from the four folded pages are dropped. - New frontend/assets/router.js (vanilla module — no framework, no bundler, No-CDN rule intact): lazy-imports a view module on FIRST show only (mount-once, hide-forever — the chat view's in-flight SSE reader persists across switches; that persistence IS the fix); intercepts same-shell navbar links with preventDefault + history.pushState (never a document load); handles popstate; single writer of `.nav-link` active state (is-active + aria-current), document.title, and the per-view meta description (values carried over from the old pages' heads, brand-resolved at write time). - Each folded page's JS becomes `export async function mount(root)` — root-scoped queries; `initSharedHeader()` dropped (the header boots once in the shell via the chat module; the admin flag comes from the same cached `fetchIsAdmin()` promise — zero extra requests). - app/main.py: a small list-driven route factory serves the shell for /tuning.html, /sources.html, /git-sources.html, /history.html — registered AFTER the API routers and BEFORE the static catch-all (routes-first). The phase-33 caching middleware applies no-cache + `?v=` rewriting unchanged; app/core/caching.py needed NO change (the view paths did not change — pinned by the integration tests). - The four old view .html files are DELETED (one source of truth); deep links to the old URLs keep working (the router picks the view from the pathname); `/?chat=<id>` is unaffected; the Containerfile bundles router.js (inlining the lazy view modules) and drops the folded page files. - app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell keeps long saved answers in the chat, and the old cap (stricter than the 24_000-char total history budget) 422-rejected any second turn in such a chat (found by the phase-42 E2E suite on the shell). Boundaries: login.html, shared.html, doc-edit.html, document.html REMAIN separate documents (flow pages, not navbar tabs); a mid-stream navigation to doc-edit/document.html still cancels per phase 48 (follow-up candidate, out of scope). The SSE API is unchanged. Real departures still cancel the turn — phase 48 intact (pinned by tests/e2e/test_stop_generation.py, unchanged, and by the new suite's real-departure control). Tests: - Phase-20 suite REWRITTEN to the new semantics (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer cancels — the stream survives the switch and the FULL answer settles; the pagehide partial persist REMAINS for real departures (the partial's exact shape — first streamed chunk prefix, no done metadata — is still pinned there). - NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock LLM): the owner repro (send -> RAG mid-stream -> Chat: window sentinel survives = same document, FULL answer, exactly one brain turn in bor.chat.v1, exactly one settled query_log row, auto-saved row matches) + the same mid-stream switch against the other three views + the real-departure-still-cancels control + the no-switch baseline. - tests/unit/test_frontend_router.py: source-level pins of the router invariants (click interceptor targets ONLY same-shell view paths, pushState-only switches, mount-once guard, hidden+inert pair, single-writer active state/title); shell-route integration tests (each folded path serves the shell with no-cache + `?v=` body; a non-view path still 404s); the file-reading unit pins re-pointed at the shell (the four view files are gone — the shell is the source of truth). Verification (this commit): full suite green — 1565 unit+integration tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the phase's E2E suites green in isolation (house protocol, AGENTS.md rule 9). Owner repro verified in a real browser against the real LLM (dev server :8010, headful Chromium): "tell me about everquest" -> RAG mid-stream -> Chat — the answer completed with one brain bubble and no error banner, `query_log` gained exactly one settled row (deflected=True: the dev KB holds no EverQuest docs — the settle, not the topic, is the proof), zero "chat: turn cancelled" lines for that turn; the control (real navigation to /shared.html mid-stream) still cancelled (no settled row, the cancel line logged, the partial persisted on return). Screenshots: .agents/screenshots/76_manual_*. Phase 76 (76_spa_nav_shell) complete — moved to .agents/phases/complete/. |
||
|
ducoterra
|
7e567bddf3 | prepping for SPA conversion | ||
|
ducoterra
|
0e4651c779 |
feat(docs): save the whole chat session as a doc
Phase 75 (TODO.md L4): "Save as doc" now drafts a document from the
ENTIRE chat session — every question and answer up to the click, in
order — instead of only the clicked bubble's answer; the existing
doc-edit screen's free-form body editing is how the user edits out
anything they don't want to keep from previous replies (no new UI
surface).
Task 01 (frontend):
- app.js buildSessionTranscript(): walks the bor.chat.v1 conversation
record in order — a numbered section per user turn ("## N.
<question, raw>" + blank line + the raw answer text; more answers
join under the same heading), sections blank-line separated, all
trailing whitespace collapsed to one final newline. Only the raw
persisted text travels (m.who + m.text — no thinking blocks, no
source chips, no tune metadata); a brain record before the first
user record is skipped; a heading-only section marks a user turn
whose answer never landed (A6, owner-confirmed 2026-09-08).
- saveAsDoc(btn): the draft body is buildSessionTranscript(); the
dead single-bubble markdown parameter is dropped (the button's
appendSaveAsDocButton signature is unchanged — one button per
bubble). Title/path/double-click guard/hand-off are unchanged
(defaultDocTitle: the last question, whitespace-collapsed,
<=120 chars; docs/<slug>.md).
- Unit: the app.js source pins move to the transcript shape (whole
session, no thinking, no dead parameter).
Task 02 (E2E):
- tests/e2e/test_save_doc_session.py (bare-repo fixture, the
phase-59 convention — git as source of truth): three DISTINCT
on-topic turns in one session (turn 1 carries the phase-17
"think out loud" trigger so its record has a thinking block the
transcript must exclude) -> save on the LAST bubble -> the
prefilled body is ## 1./## 2./## 3. in order, byte-exact against
the deterministic mock, thinking-free -> edit the whole
section-2 block out of the body -> push -> git show
bor-docs:<path> equals the EDITED body byte-for-byte (section 2's
question and answer provably absent; sections 1 and 3 byte-exact;
the UI's sha prefix is git rev-parse bor-docs). Second test:
the button on the FIRST bubble still drafts the whole session
(A6 — the transcript is the session at click time, title stays
the last question); canceling leaves the branch tip untouched.
- tests/e2e/test_response_to_docs.py: the phase-59 single-turn body
expectation moves to the transcript shape ("## 1. <question>" +
the answer's markdown) — the rest of the suite unchanged.
Also lands the phase-74 file moves (00_phase.md /
03_mock_marker_e2e.md -> complete/) and the phase reports — the
house convention of committing .agents/ with the phase.
|
||
|
ducoterra
|
055c0b5d85 |
feat(rag): pass chat history with prior thinking to the LLM
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH the conversation so far — every prior user/brain turn and the prior thinking blocks on brain turns (preserve-thinking) — while POST /api/chat stays stateless (A10): the client provides the history in the request body and the server stores nothing new. Server (task 01): - ChatRequest.history: optional list[HistoryTurn] (who: user|brain, text, optional thinking) — absent/empty keeps the request byte-identical to pre-phase-74 (the two-message [system, user] request; the kill-switch semantics are pinned in the integration suite). - app.rag.prompts.history_to_messages: pure mapper — walks the turns newest-first against the settings budgets (history_max_turns=40 / history_max_chars=24000, BOR_HISTORY_MAX_TURNS / BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut mid-answer); the kept window is returned oldest-first; brain turns carry their thinking as reasoning_content (A4) only when non-empty. - Both branches feed it: the deflected path splices it between the system prompt and the current user message (the phase-71 recovery still rebuilds from messages[1:]), the grounded agent receives run_agent(..., history=hist); llm.py's message params widen to list[dict[str, Any]] (string-only messages stay byte-identical on the wire — the SDK passes message dicts through verbatim). - The per-turn log line (PLAN §9) gains history_msgs=N after kb_chars=N. - Pins: tests/unit/test_history.py (mapper: mapping, reasoning gating, both budgets, drop-whole, ordering, empty default), tests/unit/test_config.py (the two settings + env overrides), tests/unit/test_agent.py (the history splice + the default), tests/integration/test_chat_api.py (deflected AND grounded forward the history incl. reasoning_content, no-history byte-identity, 422 pins, the log field). Client (task 02): - runTurn — the single funnel for fresh send / phase-49 retry / phase-53 stale-regen — sends history = the conversation record minus the current question, with thinking only on brain records that streamed one (undefined drops the key from the JSON, the record's convention); the question is never duplicated into the history. Wire proof (task 03): - The mock's echo my history marker (HISTORY_TRIGGER) answers with the deterministic history echo — history: N prior messages; last answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both turn branches whatever the gate says; the module docstring records the user/assistant-only history invariant that keeps every existing (tool-result-classified) marker flow unaffected. - tests/e2e/test_llm_history.py (isolated): a grounded follow-up and a deflected follow-up both receive history: 2 prior messages + thinking: yes + the byte-exact tail of turn 1's answer (derived from the persisted bor.chat.v1 record — the same array the client maps into the body); a cold start receives history: 0 prior messages / last answer tail: none / thinking: no. - Regressions green in isolation: chat_rag, chat_history (phase 50), agent_document_tools, harness_aligned_tools, stop_generation, retry_answer, response_to_docs. |
||
|
ducoterra
|
a16130c71d |
fix(chat): keep generating while the tab is hidden
Root cause (task 01): none of C1-C3 - in Chromium 151 (real mode) a merely-hidden tab neither stops the stream (frames arrive at full rate; turn completes) nor fires pagehide on tab switch; C1's double-record path was proven latent via a synthetic pagehide (trigger is browser-dependent, e.g. Safari) and C2 (the 120s pre-token guard) was confirmed to fire while hidden. - C1: the pagehide partial-persist is correlated with the turn's settle (leavePartialIndex) - the done/stop settle REPLACES it in place (identity-guarded rememberBrainTurn in-place mode), so bor.chat.v1 and the auto-saved saved_chats row keep exactly ONE brain turn per question; a real navigation never runs a settle, so the leave-save is unchanged. - C2: the visibility re-arm gives the still-armed pre-token guard a fresh TURN_TIMEOUT_MS when the tab returns to visible - hidden time no longer counts toward the 120s guard. - Phase-48 teardown contract untouched: Stop / tab close / real navigation still cancel the fetch and stop the model. - Unit pins: tests/unit/test_frontend_hidden_tab.py (the app.js mechanisms without a browser). - E2E pins: tests/e2e/test_hidden_tab_stream.py - synthetic pagehide mid-stream completes exactly once with one brain turn (localStorage + auto-saved row), reload restores one bubble, no-event baseline, and the fake-clock pre-token guard re-arm (discriminating: fails with the re-arm disabled). |
||
|
ducoterra
|
45c3fa2863 |
chore(agent): phase roadmap from TODO.md (phases 73-75), clear the file
Convert the two unchecked TODO items into executable phases (Protocol B, appended after the 72 completed phases): - 73_hidden_tab_stream (TODO L3): a merely-hidden browser tab must never stop a generating answer; repro/root-cause decision tree + the pagehide partial-correlation fix + the hidden-tab E2E pin. - 74_llm_chat_history (TODO L4, history): client-provided history in POST /api/chat (stateless, A10) mapped through both the deflected and grounded agent paths, prior thinking blocks preserved via reasoning_content, capped oldest-first; mock echo marker + E2E. - 75_save_doc_full_session (TODO L4, save-as-doc): the Save-as-doc draft body becomes the full session transcript; edit-out happens in the existing doc-edit body; multi-turn git-verified E2E. Owner-confirmed assumptions A1-A7 are recorded as ASSUMPTION lines in the task files. TODO.md is cleared (items now live in .agents/phases/todo/). |
||
|
ducoterra
|
dbf2af26c6 |
refactor(agents): migrate .agent/ planning tree to .agents/
Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched. |