brain-of-reese/.agent/phases/complete/17_thinking_display/00_phase.md

Phase 17 — Model "Thinking" in the Chat UI

Story: .agent/user_stories/thinking-display.md (created by task 04) Context: PLAN §3/§4 (SSE chat transport, A15), §6 (locked prompt), §7.4/§7.5 (feedback contract + component inventory), §9 (per-turn log line); app/rag/llm.py (streaming client), app/api/chat.py (SSE mapping), frontend/assets/app.js (turn handler + phase-14 persistence), tests/e2e/mock_llm.py (deterministic mock).

Objective

The web interface supports and shows the model's "thinking": the aipi turbo model streams its reasoning in delta.reasoning_content chunks before the answer (verified live 2026-08-23 — the app currently discards those tokens, so the user stares at "Thinking…" with no visibility). This phase plumbs reasoning through the SSE contract as a new thinking event type and renders it in a collapsible "Thinking" block above the answer bubble — streaming open, auto-collapsing when the answer starts, user-toggleable afterwards, and persisted with the message (phase 14). Models/turns that emit no reasoning render exactly as before.

Dependencies

• All of 01–16 (complete). Specifically: 03_story_chat_rag (the SSE turn pipeline this extends), 06_story_loading_feedback (the idle → thinking → streaming → done | error → idle state machine + 120s guard the thinking display plugs into), 14_chat_persistence (localStorage record shape gains an optional thinking field), 08_story_dark_tech_theme (Phase-08 design tokens for the block), 16_admin_auth (chat stays public — no gating change).

Verified facts (probe, 2026-08-23, live aipi endpoint)

• POST /v1/chat/completions with stream=True against turbo emits choices[0].delta.reasoning_content chunks before the first delta.content chunk (deepseek/litellm wire convention). No request-side flag is needed — the model thinks on its own; how much it thinks is the model's call.
• Reasoning counts against max_tokens: at max_tokens=300 a probe produced 300 reasoning tokens and no answer content. With the locked default BOR_MAX_OUTPUT_TOKENS=32768 there is ample headroom, but an answer can in principle be empty — the UI must handle thinking-without-answer gracefully (existing empty-answer fallback).
• openai SDK 2.54 (this repo's version) preserves unknown delta fields: ChatCompletionChunk.model_validate(...) keeps reasoning_content in model_extra, reachable via getattr(delta, "reasoning_content", None). The design therefore needs no raw-HTTP parsing.

Design

• SSE contract (PLAN §4 extension, owner permission 2026-08-23 = this request): new event type {"type":"thinking","text":"…"}. Frames arrive before delta frames in practice (the model reasons first); the client must tolerate a late/interleaved thinking event defensively (append to the block, never reopen it once the answer started). The done event shape is unchanged (deflected, sources, suggestions) — thinking text never needs to travel again on done.
• Backend:
  • app/rag/llm.py — new StreamPiece (frozen dataclass: kind: "content" | "thinking", text: str); chat_stream yields StreamPiece instead of str. Per chunk: delta.reasoning_content (verified aipi field) → thinking piece; fallback delta.reasoning (future-proofing, same getattr pattern); delta.content → content piece. A chunk carrying both yields thinking before content. LLMError wrapping and generation params (model, temperature=0.4, max_tokens, stream=True) unchanged.
  • app/schemas.py — ChatThinkingEvent (type="thinking", text), sibling of ChatErrorEvent/ChatDoneEvent.
  • app/config.py — stream_thinking: bool = True (BOR_STREAM_THINKING; 0/false disables) — operator kill-switch. When off, thinking pieces are still counted for the log line but never emitted. (Reasoning is otherwise always on: the model emits it, and the whole point of this phase is to show it.)
  • app/api/chat.py — maps pieces to thinking/delta events; accumulates thinking_chars per turn; per-turn log line (PLAN §9) gains thinking_chars=N inserted immediately before total_ms=N. No schema change (A13 untouched), no new packages (A12 untouched).
• Frontend (frontend/assets/app.js, styles.css):
  • Block DOM (dynamic — index.html unchanged): inside .msg-body, before .bubble:

    <details class="thinking" open>
      <summary>Thinking</summary>
      <div class="thinking-text"></div>
    </details>
    

    renderStoredMessage renders the same block collapsed for restored messages carrying thinking.
  • Turn handler (handleSend): new turn-local state thinkingAcc, sawThinking, sawDone.
    • First thinking event: clearTurnTimeout() (the stream is alive — the 120s pre-token guard also clears on the first delta, as today); if no wrap exists yet, create it (addMessage("brain", "")); ensureThinkingBlock(wrap) (idempotent; creates the open <details> and returns it); the typing indicator is removed (the live block replaces it as the visible "thinking" feedback — the UI state stays thinking: button still disabled, label "Thinking…", #send-status still "Brain of Reese is thinking" — no state machine change); render .thinking-text.innerHTML = renderMarkdown(thinkingAcc) (escape- first renderer — XSS-safe; the model's scratchpad may contain markdown-ish formatting); while the block is open, pin .thinking-text scrolled to the bottom on each update.
    • First delta: if the UI state is still thinking, transition to streaming (replaces today's if (!wrap)-only transition so the label/status flip even when thinking created the wrap first); closeThinkingBlock(wrap) (idempotent — never reopens a block once the answer started); existing delta logic unchanged.
    • done: sawDone = true; existing logic (deflected styling, source chips, tune button) unchanged; the thinking block is closed if open; thinking-without-answer: when acc is empty but sawThinking, the bubble receives the existing empty-answer fallback string and that is what gets persisted (what the user saw is what is stored); persistence record gains thinking: thinkingAcc (only when non-empty — bor.chat.v1 shape: brain messages may carry an optional thinking field; no version bump: old records without it restore exactly as before).
    • Stream-drop guard (new, required now that streams run longer): after readSSE completes, if !sawDone && !aborted and at least one thinking/delta frame arrived → setUiState(error, "The stream ended before my answer finished — try again?") (previously a severed stream settled silently into idle with a half bubble). Zero frames + no wrap keeps today's "came back empty" fallback.
  • No live region on the thinking text (it is a scratchpad — announcing every chunk would be hostile to screen readers); the existing #send-status region + native <details> open/closed announcements cover accessibility.
  • Styling (Phase-08 tokens, WCAG AA): details.thinking — background: var(--surface), border: 1px solid var(--line), border-left: 3px solid var(--brand-soft), border-radius: var(--radius-sm), margin-bottom: 0.5rem, overflow: hidden. summary — flex row, padding: 0.5rem 0.75rem, min-height: 44px (mobile touch target), color: var(--brand-ink) (8.7:1 on surface), font-size: 0.9rem, cursor: pointer, list-style: none (+ ::-webkit-details-marker {display:none}), CSS chevron ::before ("▸", rotates 90° when open, 0.15s transform disabled under prefers-reduced-motion), :focus-visible 3px var(--brand) outline offset 2px. .thinking-text — padding: 0 0.75rem 0.75rem, color: var(--ink-soft) (6.9:1 on surface), font-size: 0.875rem, line-height: 1.55, max-height: 320px, overflow-y: auto; its inner paragraphs get reduced margins. No background animation — prefers-reduced-motion respected by construction (only the chevron transition, gated).
• E2E mock (tests/e2e/mock_llm.py): deterministic thinking via a new user-message trigger THINKING_TRIGGER = "think out loud" (same convention as the existing "write a long answer" / "pretend to think slowly" triggers). When the trigger is present the mock streams ~800 chars of deterministic reasoning_content chunks (a fixed "Step 1… Step 4…" scratchpad) before the normal content chunks; the non-streaming path includes reasoning_content in the message. Without the trigger the mock is byte-identical to today — every existing story suite is unaffected. (The suite is mock-only by design: E2E_REAL_LLM=1 against the real turbo — which thinks on every turn — would break the "no thinking block" regression test. Note that in the file header.)
• Non-goals: no thinking-budget/effort request parameters (the model decides; BOR_MAX_OUTPUT_TOKENS already bounds total output); no DB schema change (thinking is not stored server-side — query_log keeps its shape; only the log line counts chars); no in-UI toggle (the kill-switch is BOR_STREAM_THINKING); no done-event change; no changes to steering/suggestions/viewer.

Tasks

1. 01_backend_thinking_stream.md — StreamPiece in the LLM client, thinking SSE event end-to-end (schema, settings, chat mapping, log line), backend tests.
2. 02_frontend_thinking_block.md — the collapsible thinking block in app.js/styles.css (streaming, auto-collapse, persistence, stream-drop guard), frontend unit pins.
3. 03_e2e_mock_and_story_suite.md — mock thinking trigger + the dedicated test_thinking_display.py Playwright suite (5 scenarios), run in isolation.
4. 04_story_docs_plan_commit.md — user story file, README section, .env.example, PLAN §2/§4/§7/§9/§12 revisions (owner permission recorded), the single atomic commit, phase move to complete/.

Locked decisions

• A15 extended with owner permission (2026-08-23): the SSE contract gains the thinking event type; delta + done shapes unchanged. Recorded as a PLAN §4 revision (owner permission noted), not a silent deviation.
• A5 untouched — no new models/params sent to aipi; we only read a field the model already emits. A11 untouched — vanilla JS/CSS, no CDN (native <details>/<summary>). A12/A13 untouched — no new services, no migration. A16 untouched — one new story E2E suite + unit/integration extensions. A10 untouched — chat stays public; thinking is visible to everyone (homelab scope). No other anchor changed.

Testing & Quality

• Unit: tests/unit/test_llm_client.py (StreamPiece mapping: content, reasoning_content, reasoning fallback, both-in-one-chunk ordering, empty-skip, failure wrapping), tests/unit/test_sse_events.py (thinking frame shape), tests/unit/test_config.py (default + env parse of stream_thinking), tests/unit/test_frontend_feedback.py + tests/unit/test_chat_persistence.py (source-level pins: thinking event handling, block markers, auto-collapse, sawDone guard, persisted thinking field, .thinking CSS rules).
• Integration: tests/integration/test_chat_api.py — FakeRagLLM gains optional thinking; thinking frames precede all delta frames and reassemble; BOR_STREAM_THINKING=0 suppresses thinking frames while deltas are unchanged; existing suites stay green.
• Coverage: uv run pytest --cov=app --cov-report=term-missing >90% on app/.
• E2E: tests/e2e/test_thinking_display.py — five scenarios (see task 03), mock-only, green in isolation (uv run pytest tests/e2e/test_thinking_display.py -v --no-cov).
• Lint/types: uv run ruff check . && uv run pyright clean.

Completion Criteria

• uv run pytest green (unit + integration); uv run pytest --cov=app --cov-report=term-missing > 90%.
• uv run pytest tests/e2e/test_thinking_display.py -v --no-cov green in isolation (prereq: podman compose up -d db).
• Regression suites green in isolation: test_chat_rag.py, test_loading_feedback.py, test_chat_persistence.py, test_honest_deflection.py.
• uv run ruff check . && uv run pyright clean.
• Manual live check (dev server, real aipi): a chat turn shows the thinking block streaming, collapsing when the answer starts, toggleable afterwards; with BOR_STREAM_THINKING=0 no block.
• UI Structure Check (AGENTS.md rule 5): block uses Phase-08 tokens, all text contrast ≥4.5:1, summary is a real focusable control with ≥44px target, prefers-reduced-motion respected, no CDN tags, chat column still 46rem.
• .agent/user_stories/thinking-display.md exists; PLAN §2/§4/§7.4/ §7.5/§9/§12 carry the revision notes (owner permission 2026-08-23).
• One --no-gpg-sign commit (below); .agent/phases/todo/17_thinking_display/ moved to .agent/phases/complete/.

Commit

git add -A .agent/ app/ frontend/ tests/ README.md .env.example && git commit --no-gpg-sign -m "feat(chat): stream model thinking over SSE and show it in a collapsible block"