brain-of-reese/.agents/phases/complete/158_thinking_timeline/00_phase.md

Phase 158 — Chronological thinking / tool-call timeline

Source: owner report (live interactive-browser session, 2026-10-04, shared page https://brain.reeseapps.com/shared/95dd7a2a-8cda-478b-b694-53936070b9fb — a real turbo turn with 6 tool calls): "the thinking block happens first but LLM sent a normal response, not a thinking response, and then went back to thinking. This is awkward. Then the tool calling happens, and the thinking block reads strangely because the tool calls are shown below the thinking block. … show the thinking / tool call / responses in order, rather than in single big blocks each." Story: n/a (owner bug report — live site, 2026-10-04) Context: app/rag/agent.py::run_agent (yields thinking / content / ToolCallPiece / ToolResultPiece in TRUE chronological order per round — thinking → tool call → thinking → … → answer; the SSE mapping in app/api/chat.py preserves that order, so NO backend stream change is needed); frontend/assets/app.js — ensureThinkingBlock (~L1053, ONE details.thinking per message, idempotent; phase 109 D15 re-opens it on every later thinking frame), closeThinkingBlock (~L1075), closeToolCalls (~L1086), appendToolLine (~L1136, ONE details.tool-calls-disclosure per message — phase 117 D3 open-live/fold-at-rest), armToolLineClock (~L1479, the phase-87 clock queries .tool-calls .tool-call:last-child), the turn handler in runTurn (thinking ~L3148 / tool ~L3193 / delta ~L3300 / done ~L3313 branches; thinkingAcc ~L3071, toolAcc ~L3080, segRuns — NEW — module scope beside them; the done save point ~L3355, the tool_result branch ~L3389, the stream-drop finalizeFailedTurn call ~L3417), rememberBrainTurn (L2589, meta spread — the pagehide partial save reuses it), finalizeFailedTurn (L2983); renderStoredMessage (~L2089: the m.thinking block + the m.tools loop, both rendered collapsed); frontend/assets/shared.js — addThinkingBlock (L126) + addToolLines (L175, the byte-parity copy, rendered closed); app/schemas.py — ChatMessage (L1103, extra="forbid", phase-83 value caps, the wrap serializer that omits absent image keys), ToolCall (L1017), HistoryTurn (L58 — the history wire, UNCHANGED); tests/e2e/mock_llm.py (_tool_flow L1820, _tool_call_stream L3813, _sse_stream L3727, compose_thinking L3589, the trigger-dispatch order in chat_completions L3872 — the more-specific-phrase-wins convention); the E2E suites whose pins this phase updates (task 05): test_turn_progress_loader.py (the phase-109 D15 re-open pin), plus the unit source pins in test_frontend_feedback.py, test_frontend_turn_loader.py, test_frontend_tool_states.py, test_chat_persistence.py, test_shared_page.py, test_tool_call_compact.py, test_thinking_scroll.py, test_big_read_progress.py, test_read_range_tool_lines.py, test_read_truncation_frontend.py, test_search_tool_scope.py, test_frontend_failed_turn.py (each pins ensureThinkingBlock / appendToolLine / the clock in different depths — inspect each; ONLY the pins the redesign breaks change, the rest stay green untouched).

Bug basis (code-traced, 2026-10-04)

The server streams the turn in true chronological order (run_agent yields per round: thinking → tool call → thinking → … → answer), but the client LUMPS: app.js funnels every thinking frame of the turn into ONE details.thinking block (ensureThinkingBlock is idempotent — the phase-109 D15 re-open re-opens that same block for each later agent round) and every tool frame into ONE details.tool-calls-disclosure. DOM order is therefore always [ALL thinking] → [ALL tool calls] → [answer] regardless of the actual order — the owner's "thought everything, then called all the tools, then answered" reading. The "thinking → answer → back to thinking" awkwardness is the D15 re-open live: the model emits a short mid-round reply alongside a tool call (React style), the first delta closes the one thinking block and the answer starts, then the next round's thinking frame re-opens the SAME block above the tool lines — the answer begins, then thinking resumes on top of it. The persisted bor.chat.v1 record only stores thinking (one concatenated string) + tools (one array) — the per-run order is never captured, so the shared page (shared.js) renders the same lumped layout, collapsed.

Objective

Render each agent turn as a chronological timeline — each CONTIGUOUS run of thinking frames in its own details.thinking block, each CONTIGUOUS run of tool calls in its own details.tool-calls-disclosure (Tool call(s) (N) per run), segments in arrival order — live, on restore, and on the shared page; persist the run order in a new optional segments array on the brain record (additive; records without it — all pre-phase — restore byte-identically through the untouched legacy path).

Owner decisions (chat, 2026-10-04 — recorded per AGENTS.md rule 3)

• D1 — Per-run timeline. Each contiguous thinking run = its own Thinking block; each contiguous tool-call run = its own disclosure. Consecutive frames of the same kind extend the current run; the first frame of a new kind closes the previous run and opens the new one.
• D2 — Option B mid-round placement (owner-chosen, shown as a live DOM demo in the owner's browser 2026-10-04). ALL segments render ABOVE the answer bubble, in arrival order — including runs that start AFTER the first delta (the mid-round reply case). The bubble stays ONE element; its content accumulates in arrival order (the mid-round text and the final answer are one streamed text). Rejected: Option A (post-answer segments below the bubble — the bubble grows above its own scratchpad).
• D3 — Plain Thinking label on every block (owner-chosen). The phase-17 pinned summary literal stays byte-identical across runs (zero new literals; position distinguishes the runs). The disclosure summary stays the phase-117 Tool call(s) (N) — now per run.
• D4 — Additive persistence. The brain record gains an OPTIONAL segments array; thinking (full concatenated text — feeds the history wire + the per-turn log) and tools (all call records) are KEPT (old-client graceful degradation; the phase-113 lesson applied up front). segments is omitted when absent (the undefined-drops-the-key house pattern) — a no-thinking, no-tools record stays byte-identical to pre-phase.
• D5 — Open live, fold at rest (extends phase 117 D3 + the phase-17 open-while-thinking contract): the active run's block is OPEN while its frames stream; a new run's first frame closes the previous run; the first delta, done, and the stop/abort settle fold the open runs; the restore path and the shared page render everything COLLAPSED.
• D6 — Server: a validated segments field on ChatMessage. extra="forbid" means the new key would 422 every saved-chat write without a schema field (the phase-113 lesson: a missing key silently UNSAVED every grounded brain record). The field is a discriminated union (kind = "thinking" | "tool") with phase-83-style value caps, omitted-when-None in the serializer (the phase-50 round-trip contract). No migration (the record is opaque JSONB), no SSE contract change, no history-wire change.

Design (shared by all tasks — the executor reads this, not the chat)

The segments entry shape (D4)

segments: [
  { kind: "thinking", text: "<that run's full text>" },
  { kind: "tool", name: "…", argument?: "…", offset?: int, limit?: int,
    path?: "…", truncated?: true, chars_shown?: int, chars_total?: int },
  …
]

• One thinking entry per RUN (the run's accumulated text — the same text that streams live into that run's block). One tool entry per TOOL CALL (the exact toolAcc record shape — the phase-37 {name, argument} plus the phase-139 offset/limit and phase-146 path additive fields, and the phase-95 truncated/chars_shown/chars_total stamp when present).
• Built at save time from the run tracker: an EMPTY thinking run stores NOTHING (a zero-char run — e.g. a BOR_STREAM_THINKING-suppressed turn — is absent); a turn with no thinking and no tools stores no segments key at all.
• thinking (the full concatenation, for the history wire + the log) and tools (every call record, for old clients) are UNCHANGED in the record.

app/schemas.py (task 01)

• ChatThinkingSegment: kind: Literal["thinking"], text: str (min_length=1, max_length=32_000 — HistoryTurn's cap, phase 83).
• ChatToolSegment: kind: Literal["tool"] + the EXACT ToolCall field set (name/argument/offset/limit/path/truncated/chars_shown/chars_total with their existing caps) — one model, no duplication of the field definitions where practical (the docstring cross-references ToolCall).
• ChatSegment = Annotated[Union[ChatThinkingSegment, ChatToolSegment], Field(discriminator="kind")].
• ChatMessage.segments: list[ChatSegment] | None = Field(default=None, max_length=50) (the phase-83 cap style — far past any realistic turn: the round cap is 10 → ≤ 20 entries; 50 matches the tools cap).
• The EXISTING wrap serializer (_drop_image_when_absent) gains the rule: segments is None → pop "segments" (ABSENT, never null — the phase-50 contract; a record WITH segments keeps them). HistoryTurn and the history_to_messages mapper are UNTOUCHED (segments never travel the wire — the concatenated thinking travels exactly as today).

frontend/assets/app.js — the live turn (task 02)

• Turn state: segRuns = [] (module scope, reset in runTurn beside thinkingAcc/toolAcc — an array of run objects { kind, text, records }), currentRun = null (turn-local).
• Insertion anchor (D2): every run element is inserted before body.querySelector(".bubble") (get-or-create the bubble first, as today). There is no other anchor — segments ALWAYS sit above the bubble, in arrival order. ensureThinkingBlock's old .tool-calls-disclosure ?? .tool-calls ?? .bubble anchor is REPLACED by this.
• thinking frame (per frame): settleToolLine() (unchanged); thinkingAcc += ev.text + sawThinking + clearTurnTimeout() + if (!wrap) wrap = addMessage("brain","") + removeTyping() (all unchanged). Then the run logic: if currentRun?.kind === "thinking" → extend it (re-render renderMarkdown(run.text) into THAT block's .thinking-text — the existing pre-update pin-state / bottom-follow logic moves verbatim into this branch, now per-block); else → close currentRun (if any, details.open = false), create details.thinking (open = true, <summary>Thinking</summary> + .thinking-text, inserted before the bubble), push {kind:"thinking",text:"", records:[]} to segRuns, currentRun = it, extend.
• tool frame (per frame): the existing guards stay byte-identical (the phase-139 frameOffset/frameLimit, the phase-146 framePath well-formedness; toolAcc.push(toolRecord); clearTurnTimeout(); the toolStatus label logic; the phase-70 legacy-name mapping). Then the run logic: if currentRun?.kind === "tool" → extend that disclosure; else → close currentRun (if any), create details.tool-calls-disclosure (open = true, summary.tool-calls-summary + the div.tool-calls list role=list aria-label="Tool calls", inserted before the bubble), push {kind:"tool",text:"",records:[]} to segRuns, currentRun = it; extend = build the line EXACTLY as today (appendToolLine — now with the container passed in: appendToolLine(container, name, argument, offset, limit, scope), the line bytes + the four pinned line.textContent label literals + the phase-139 range suffix + the phase-146 scope span UNCHANGED, the summary count updated per append — Tool call(s) (N) per run), then armToolLineClock(wrap, lineEl) (the NEW signature — task 02's clock fix).
• armToolLineClock(wrap, lineEl) (the phase-87 multi-disclosure fix): store the lineEl reference (module scope, reset with the wrap); the interval appends the .tool-elapsed suffix to lineEl instead of querying .tool-calls .tool-call:last-child (that query breaks with multiple disclosures — it would find the last line of the FIRST list). The visible behavior is identical (the suffix on the newest line; settleToolLine still removes every .tool-elapsed in the wrap + resets the reference).
• delta handler: settleToolLine() + acc += ev.text + the thinking→streaming transition UNCHANGED; replace closeThinkingBlock(wrap) + closeToolCalls(wrap) with closeCurrentRun() (folds currentRun if open; idempotent; no-op without a run).
• The other settle-pair sites — finalizeFailedTurn's if (wrap) block (L2990–L2991), the STOP path (L3477–L3478), and every remaining closeThinkingBlock(wrap); closeToolCalls(wrap); pair (grep — the redesign removes both functions): closeAllRuns(wrap) (closes EVERY open details.thinking + details.tool-calls-disclosure in the wrap — defensive; by construction at most the last run is open, but a late double-close must be a no-op). The rest of done is unchanged.
• Save payloads (the FIVE sites): buildSegments(segRuns) (a helper — maps runs: thinking run → {kind:"thinking",text} when non-empty; tool run → one {kind:"tool",…record} per record, in order) — the done save point (~L3355); the finalizeFailedTurn parameter + its if (wrap) saved record (the two call sites — stream-drop ~L3431, network-error ~L3519 — pass buildSegments(segRuns)); the STOP path's save meta (L3487–L3491, stopped: true); the pagehide partial (L3642–L3650 — segRuns is module scope, readable there). All include segments non-empty only. The zero-frame fallback save needs nothing (zero frames → no runs → no key). The thinking/tools keys are UNCHANGED at every site.
• The UI state machine, #send-status copy, the 300 s pre-token guard, the stream-drop guard, and the thinking-without-answer fallback are ALL UNCHANGED (§7.4 never-stale intact). BOR_STREAM_THINKING=0 needs no client change (no thinking frames arrive → no thinking runs).

frontend/assets/app.js — the restore path (task 03)

• renderStoredMessage: after the bubble exists, IF Array.isArray(m.segments) && m.segments.length → restoreTimeline(wrap, m.segments) and SKIP the legacy m.thinking block + the m.tools loop (segments take precedence — they carry the same data in order); ELSE the legacy path is BYTE-IDENTICAL (a pre-phase record — or a BOR_STREAM_THINKING=0 turn saved with the old client — restores exactly as today).
• restoreTimeline: iterate m.segments in order; group CONSECUTIVE kind:"tool" entries into ONE details.tool-calls-disclosure (created CLOSED — D5; summary Tool call(s) (N); each line through the SAME appendToolLine container-signature path with the EXISTING tOffset/tLimit/tPath guards + the existing appendTruncatedNote call for t.truncated records — the current loop's guards move verbatim inside the grouping); a kind:"thinking" entry → its OWN details.thinking (created CLOSED; .thinking-text = renderMarkdown(entry.text) — the restore convention stays escape-first); unknown kind values are SKIPPED (never render from storage — textContent only, the house rule). All elements inserted before the bubble.

frontend/assets/shared.js — the shared page (task 03)

• addSegments(wrap, segments) — the shared-page parity copy of restoreTimeline (pure render, always COLLAPSED — D5): the same grouping, the same line bytes (the four label literals + the phase-95 truncation note
  • the phase-139/146 suffixes, already parity-pinned in addToolLines), the same summary count; the addThinkingBlock/addToolLines legacy path is BYTE-IDENTICAL when the record has no segments. The record-dispatch site (~L456): if (Array.isArray(m.segments) && m.segments.length) addSegments(wrap, m.segments); else { the existing addThinkingBlock + addToolLines }.

tests/e2e/mock_llm.py (task 04)

• THINK_TOOLS_TRIGGER = "think out loud and use your tools" — checked in the streaming branch BEFORE the _tool_flow dispatch (the more-specific-phrase-wins convention — the phrase contains both think out loud and use your tools, so it must not fall through to the plain answer path; no other existing trigger phrase is a substring of it, and no existing E2E question carries it — verify both before wiring).
• _thinking_tools_flow(body) — stateless, shaped like _tool_flow (gated on the tools parameter + the <tools> section, like the other marker flows):
  • request 1 (tools offered, no tool results yet): thinking = "Step 1: list the documents to find the reference server config." → _tool_call_stream("ls", {}, "call_0", thinking=thinking);
  • request 2 (a tool-role top-level source listing — the agent's ls tree format, no file lines yet): "Step 2: drill into the first source." → the _drill_target scoped ls (id call_1), with thinking;
  • request 3 (a tool-role folder listing with file lines — parse the FIRST file line's labeled source: / path: fields, the phase-63/70 format): "Step 3: read the file line's document." → read on the JOINED combined source/path (id call_2), with thinking;
  • request 4 (a tool-role read result — content starting with the agent's "Document <source/path>:" header): "Step 4: compose the answer from the read document." → the phase-37 answer convention: Read <source/path>. <first 80 chars of the read document's content> (content stream, with thinking ahead of it).
• _tool_call_stream(name, arguments, call_id, thinking="") — the optional thinking param: when non-empty, yield reasoning_content chunks FIRST (the 12-char split — the _sse_stream convention), then the existing tool_calls frames + finish_reason: "tool_calls" (unchanged). Default "" → every pre-phase 3-arg caller streams BYTE-IDENTICAL.
• Docstring: the new trigger entry (the phase-158 flow) — the mock stays mock-only by design (E2E_REAL_LLM note unchanged).

tests/e2e/test_thinking_timeline.py (task 04 — dedicated suite, run in isolation)

The conftest app_server/page/seeded_kb fixtures + the phase-37 admin login + the trigger question (the KB must be seeded so request 4's read result exists — mirror test_agent_document_tools.py). Six tests:

1. test_timeline_order_after_completed_turn — after done: the brain wrap's .msg-body children before the bubble are EXACTLY [details.thinking, details.tool-calls-disclosure, details.thinking, details.tool-calls-disclosure, details.thinking, details.tool-calls-disclosure, details.thinking] (7 elements); the four .thinking-texts carry DISTINCT Step 1…Step 4 markers (distinctness = the runs are separate blocks, not one lump); each disclosure summary is Tool call (1); the answer bubble carries the Read <path> echo.
2. test_runs_fold_at_rest — after done: NO .thinking / .tool-calls-disclosure has the open attribute; the summaries are visible (D5).
3. test_live_runs_open_while_streaming — during the turn (an expect.poll predicate over the mock's 0.1 s pacing window): at least one .thinking or .tool-calls-disclosure is open while the answer bubble is still empty (the D5 live-open contract — the live feedback the owner keeps).
4. test_restored_timeline_and_record — RELOAD (same context): the restored message renders the same 7-element timeline, all closed; the localStorage bor.chat.v1 record for that brain turn carries a segments array of EXACTLY 7 entries in the pattern thinking, tool, thinking, tool, thinking, tool, thinking (4 thinking runs
   • 3 tool entries).
5. test_shared_page_timeline — the phase-51 save + share flow, a FRESH context on /shared/<token>: the same 7-element timeline renders, all collapsed (the shared.js parity — addSegments).
6. test_no_thinking_turn_single_disclosure — a plain use your tools question (NO thinking trigger): the turn renders EXACTLY ONE .tool-calls-disclosure (3 lines, Tool calls (3)) and ZERO .thinking (the run-grouping contract: no thinking frames → one tool run — the phase-117 behavior for no-thinking turns is preserved).

Tasks

1. 01_schema_segments.md — the discriminated ChatSegment models + the ChatMessage.segments field + the serializer omission + the integration contract (accept / caps / byte-identical round-trip / shared read).
2. 02_live_timeline.md — the per-run live rendering (run tracker, D2 anchor, the clock re-target, the settle sites, the save payloads) + the JS source unit pins.
3. 03_restore_and_shared.md — the restoreTimeline dispatch in renderStoredMessage + the shared.js addSegments parity + the unit pins.
4. 04_mock_and_e2e.md — the mock THINK_TOOLS_TRIGGER flow + the dedicated test_thinking_timeline.py story suite, run in isolation.
5. 05_regression_update_and_commit.md — the phase-109 suite pin update + the remaining unit-pin updates, the full gate, the documentation pass, the atomic commit, the phase move.

Testing & Quality

• Unit (new): tests/unit/test_thinking_timeline.py — the source-level pins: the per-run creation (a NEW details.thinking when the run kind changes, insert-before-bubble anchor — D2), the plain Thinking literal (D3), the per-run disclosure + the Tool call(s) (N) per-run count, the armToolLineClock reference (the :last-child query is GONE), the segRuns → buildSegments mapping (empty-run skip, the tool-record shape), the delta closeCurrentRun + the settle-pair sites → closeAllRuns, the FIVE save-payload sites (done / finalizeFailedTurn + its 2 call sites / stop path / pagehide), the restore dispatch (segments vs legacy — the legacy path byte-identical), the shared.js addSegments parity, and innerHTML ABSENT from the restore/shared builders (the house rule).
• Integration: tests/integration/test_chats_api.py (extended) — a record WITH segments (both kinds, every additive tool field) is accepted and round-trips through create/get; segments > 50 → 422; a thinking-segment text > 32 000 → 422; an unknown segment kind → 422 (extra="forbid" intact — a stray <b>-ish key still 422s); a record WITHOUT segments round-trips with the key ABSENT (never null — the phase-50 byte-identical contract, pinned); the shared read (/api/chats/<id>/shared or the public read path) carries segments (the public shape).
• Existing unit suites MUST stay green: every pin NOT broken by the redesign (the tool-line label literals, the phase-117 deboxed CSS, the phase-87 .tool-elapsed behavior, the phase-139/146 suffix pins, the emoji guard, the test_tool_call_compact.py pins — a no-thinking turn is one run, so the phase-117 E2E contract is UNCHANGED).
• E2E: tests/e2e/test_thinking_timeline.py (new — the six tests above), green in isolation (uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov), mock-only.
• Regression E2E (task 05, in isolation): test_thinking_display.py (a no-tools thinking turn is ONE run — the suite's contract is unchanged), test_tool_call_compact.py (unchanged — no-thinking single-run), test_turn_progress_loader.py (UPDATED in task 05 — the D15 pin → the per-run contract), test_big_read_progress.py (the clock's visible behavior is identical — must stay green UNCHANGED), test_agent_document_tools.py, test_share_chat.py, test_chat_persistence.py, test_smoke.py.
• Coverage: uv run pytest --cov=app --cov-report=term-missing >90% on app/ (task 01's schema code is unit/integration-covered; the rest is held by the suite).
• Lint/types: uv run ruff check . && uv run pyright clean.
• The phase is "Complete" only when the entire project test suite passes 100% (every test in the project, not just this phase's).

Documentation & Consistency

• The final pass reviews README.md and AGENTS.md in full against the project as it stands after this phase — fixing every inconsistency found, including pre-existing ones not caused by the phase.
• Tasks update README.md/AGENTS.md immediately when their own changes are user-visible (the timeline is a visible UI change — name it where the chat thinking/tool-call display is described, if anywhere).

Completion Criteria

• A completed tool+thinking turn renders Thinking → Tool call (1) → Thinking → Tool call (1) → … → answer — the E2E's 7-element order pin (task 04 test 1); the runs fold at rest (test 2) and stay open while streaming (test 3).
• A reload restores the same timeline + the record's segments array (test 4); a fresh context on /shared/<token> renders the same timeline (test 5).
• A no-thinking tool turn is ONE disclosure, 3 lines (test 6) — the phase-117 contract is preserved; test_tool_call_compact.py + test_thinking_display.py + test_big_read_progress.py green in isolation UNCHANGED.
• A pre-phase saved record (no segments) restores byte-identically (the integration pin + the E2E regression suites).
• uv run pytest green (100% of the project suite); uv run pytest --cov=app --cov-report=term-missing > 90%; tests/e2e/test_thinking_timeline.py green in isolation; uv run ruff check . && uv run pyright clean.
• git diff --stat limited to app/schemas.py, frontend/assets/app.js, frontend/assets/shared.js, tests/, and the phase files.
• README.md and AGENTS.md are accurate and consistent with the whole project (verified in the final pass).
• ONE owner-directed exception to "no behavior change in completed phases": the phase-109 D15 re-open contract is REPLACED by the per-run blocks (owner direction 2026-10-04 — recorded above); test_turn_progress_loader.py' pin is updated in task 05 to the per-run contract (two thinking blocks, each carrying its own round, open in its live window, folded at rest).
• One atomic --no-gpg-sign commit (below); the phase dir moved to .agents/phases/complete/.

Commit

git add -A .agents/ app/schemas.py frontend/ tests/ README.md && git commit --no-gpg-sign -m "feat(ui): render thinking and tool runs as a chronological timeline (per-run blocks, segments persistence)"

Locked decisions

• The SSE contract is UNCHANGED (A15 intact). The frames already arrive in true chronological order (run_agent's yield order — verified in code); this phase is client-grouping + record persistence only. app/rag/ is untouched.
• D1–D6 (owner, 2026-10-04) — the per-run timeline, Option B placement, the plain Thinking label, the additive segments record, open-live/ fold-at-rest, and the validated ChatMessage.segments field (the extra="forbid" boundary preserved with phase-83-style caps).
• A10 history wire UNCHANGED — segments never travel to the LLM (the concatenated thinking travels as reasoning_content exactly as today; HistoryTurn + the mapper are untouched).
• A13 UNCHANGED — no migration (the record is opaque JSONB; the schema field is validation at the boundary only).
• A16 honoured — one new story E2E suite + the task-05 regression updates; mock-only by design. A17 honoured — one atomic --no-gpg-sign commit.
• The phase-87 clock re-target (the :last-child query → the tracked reference) is a bug fix in service of D1; its visible behavior (the suffix on the newest line) is byte-identical — test_big_read_progress.py stays green unchanged. No other anchor changed.