refactor(agents): migrate .agent/ planning tree to .agents/
Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched.
This commit is contained in:
@@ -0,0 +1,232 @@
|
||||
# Phase 17 — Model "Thinking" in the Chat UI
|
||||
|
||||
**Story:** `.agents/user_stories/thinking-display.md` (created by task 04)
|
||||
**Context:** PLAN §3/§4 (SSE chat transport, A15), §6 (locked prompt),
|
||||
§7.4/§7.5 (feedback contract + component inventory), §9 (per-turn log line);
|
||||
`app/rag/llm.py` (streaming client), `app/api/chat.py` (SSE mapping),
|
||||
`frontend/assets/app.js` (turn handler + phase-14 persistence),
|
||||
`tests/e2e/mock_llm.py` (deterministic mock).
|
||||
|
||||
## Objective
|
||||
The web interface **supports and shows the model's "thinking"**: the aipi
|
||||
`turbo` model streams its reasoning in `delta.reasoning_content` chunks
|
||||
before the answer (verified live 2026-08-23 — the app currently discards
|
||||
those tokens, so the user stares at "Thinking…" with no visibility). This
|
||||
phase plumbs reasoning through the SSE contract as a new `thinking` event
|
||||
type and renders it in a **collapsible "Thinking" block** above the answer
|
||||
bubble — streaming open, auto-collapsing when the answer starts,
|
||||
user-toggleable afterwards, and persisted with the message (phase 14).
|
||||
Models/turns that emit no reasoning render exactly as before.
|
||||
|
||||
## Dependencies
|
||||
- All of `01`–`16` (complete). Specifically: `03_story_chat_rag` (the SSE
|
||||
turn pipeline this extends), `06_story_loading_feedback` (the
|
||||
`idle → thinking → streaming → done | error → idle` state machine +
|
||||
120s guard the thinking display plugs into), `14_chat_persistence`
|
||||
(localStorage record shape gains an optional `thinking` field),
|
||||
`08_story_dark_tech_theme` (Phase-08 design tokens for the block),
|
||||
`16_admin_auth` (chat stays public — no gating change).
|
||||
|
||||
## Verified facts (probe, 2026-08-23, live aipi endpoint)
|
||||
- `POST /v1/chat/completions` with `stream=True` against `turbo` emits
|
||||
`choices[0].delta.reasoning_content` chunks **before** the first
|
||||
`delta.content` chunk (deepseek/litellm wire convention). No request-side
|
||||
flag is needed — the model thinks on its own; how much it thinks is the
|
||||
model's call.
|
||||
- Reasoning counts against `max_tokens`: at `max_tokens=300` a probe
|
||||
produced 300 reasoning tokens and **no answer content**. With the locked
|
||||
default `BOR_MAX_OUTPUT_TOKENS=32768` there is ample headroom, but an
|
||||
answer can in principle be empty — the UI must handle
|
||||
thinking-without-answer gracefully (existing empty-answer fallback).
|
||||
- `openai` SDK 2.54 (this repo's version) preserves unknown delta fields:
|
||||
`ChatCompletionChunk.model_validate(...)` keeps `reasoning_content` in
|
||||
`model_extra`, reachable via `getattr(delta, "reasoning_content", None)`.
|
||||
The design therefore needs no raw-HTTP parsing.
|
||||
|
||||
## Design
|
||||
- **SSE contract (PLAN §4 extension, owner permission 2026-08-23 = this
|
||||
request):** new event type `{"type":"thinking","text":"…"}`. Frames
|
||||
arrive before `delta` frames in practice (the model reasons first); the
|
||||
client must tolerate a late/interleaved `thinking` event defensively
|
||||
(append to the block, never reopen it once the answer started). The
|
||||
`done` event shape is **unchanged** (`deflected`, `sources`,
|
||||
`suggestions`) — thinking text never needs to travel again on `done`.
|
||||
- **Backend:**
|
||||
- `app/rag/llm.py` — new `StreamPiece` (frozen dataclass:
|
||||
`kind: "content" | "thinking"`, `text: str`); `chat_stream` yields
|
||||
`StreamPiece` instead of `str`. Per chunk: `delta.reasoning_content`
|
||||
(verified aipi field) → thinking piece; fallback `delta.reasoning`
|
||||
(future-proofing, same getattr pattern); `delta.content` → content
|
||||
piece. A chunk carrying both yields thinking **before** content.
|
||||
`LLMError` wrapping and generation params (model, `temperature=0.4`,
|
||||
`max_tokens`, `stream=True`) unchanged.
|
||||
- `app/schemas.py` — `ChatThinkingEvent` (`type="thinking"`, `text`),
|
||||
sibling of `ChatErrorEvent`/`ChatDoneEvent`.
|
||||
- `app/config.py` — `stream_thinking: bool = True`
|
||||
(`BOR_STREAM_THINKING`; `0`/`false` disables) — operator kill-switch.
|
||||
When off, thinking pieces are still **counted** for the log line but
|
||||
never emitted. (Reasoning is otherwise always on: the model emits it,
|
||||
and the whole point of this phase is to show it.)
|
||||
- `app/api/chat.py` — maps pieces to `thinking`/`delta` events;
|
||||
accumulates `thinking_chars` per turn; per-turn log line (PLAN §9)
|
||||
gains `thinking_chars=N` inserted immediately before `total_ms=N`.
|
||||
No schema change (A13 untouched), no new packages (A12 untouched).
|
||||
- **Frontend (`frontend/assets/app.js`, `styles.css`):**
|
||||
- **Block DOM** (dynamic — `index.html` unchanged): inside `.msg-body`,
|
||||
**before** `.bubble`:
|
||||
```html
|
||||
<details class="thinking" open>
|
||||
<summary>Thinking</summary>
|
||||
<div class="thinking-text"></div>
|
||||
</details>
|
||||
```
|
||||
`renderStoredMessage` renders the same block **collapsed** for
|
||||
restored messages carrying `thinking`.
|
||||
- **Turn handler** (`handleSend`): new turn-local state
|
||||
`thinkingAcc`, `sawThinking`, `sawDone`.
|
||||
- First `thinking` event: `clearTurnTimeout()` (the stream is alive —
|
||||
the 120s pre-token guard also clears on the first `delta`, as
|
||||
today); if no wrap exists yet, create it (`addMessage("brain", "")`);
|
||||
`ensureThinkingBlock(wrap)` (idempotent; creates the open
|
||||
`<details>` and returns it); the **typing indicator is removed**
|
||||
(the live block replaces it as the visible "thinking" feedback —
|
||||
the UI state stays `thinking`: button still disabled, label
|
||||
"Thinking…", `#send-status` still "Brain of Reese is thinking" —
|
||||
no state machine change); render
|
||||
`.thinking-text.innerHTML = renderMarkdown(thinkingAcc)` (escape-
|
||||
first renderer — XSS-safe; the model's scratchpad may contain
|
||||
markdown-ish formatting); while the block is open, pin
|
||||
`.thinking-text` scrolled to the bottom on each update.
|
||||
- First `delta`: if the UI state is still `thinking`, transition to
|
||||
`streaming` (replaces today's `if (!wrap)`-only transition so the
|
||||
label/status flip even when thinking created the wrap first);
|
||||
`closeThinkingBlock(wrap)` (idempotent — never reopens a block once
|
||||
the answer started); existing delta logic unchanged.
|
||||
- `done`: `sawDone = true`; existing logic (deflected styling, source
|
||||
chips, tune button) unchanged; the thinking block is closed if open;
|
||||
**thinking-without-answer:** when `acc` is empty but `sawThinking`,
|
||||
the bubble receives the existing empty-answer fallback string and
|
||||
that is what gets persisted (what the user saw is what is stored);
|
||||
persistence record gains `thinking: thinkingAcc` (only when
|
||||
non-empty — `bor.chat.v1` shape: brain messages may carry an
|
||||
optional `thinking` field; **no version bump**: old records without
|
||||
it restore exactly as before).
|
||||
- **Stream-drop guard (new, required now that streams run longer):**
|
||||
after `readSSE` completes, if `!sawDone && !aborted` and at least
|
||||
one `thinking`/`delta` frame arrived → `setUiState(error, "The
|
||||
stream ended before my answer finished — try again?")` (previously a
|
||||
severed stream settled silently into idle with a half bubble). Zero
|
||||
frames + no wrap keeps today's "came back empty" fallback.
|
||||
- **No live region on the thinking text** (it is a scratchpad —
|
||||
announcing every chunk would be hostile to screen readers); the
|
||||
existing `#send-status` region + native `<details>` open/closed
|
||||
announcements cover accessibility.
|
||||
- **Styling (Phase-08 tokens, WCAG AA):** `details.thinking` —
|
||||
`background: var(--surface)`, `border: 1px solid var(--line)`,
|
||||
`border-left: 3px solid var(--brand-soft)`,
|
||||
`border-radius: var(--radius-sm)`, `margin-bottom: 0.5rem`,
|
||||
`overflow: hidden`. `summary` — flex row, `padding: 0.5rem 0.75rem`,
|
||||
`min-height: 44px` (mobile touch target), `color: var(--brand-ink)`
|
||||
(8.7:1 on surface), `font-size: 0.9rem`, `cursor: pointer`,
|
||||
`list-style: none` (+ `::-webkit-details-marker {display:none}`),
|
||||
CSS chevron `::before` (`"▸"`, rotates 90° when open, 0.15s
|
||||
transform disabled under `prefers-reduced-motion`), `:focus-visible`
|
||||
3px `var(--brand)` outline offset 2px. `.thinking-text` —
|
||||
`padding: 0 0.75rem 0.75rem`, `color: var(--ink-soft)` (6.9:1 on
|
||||
surface), `font-size: 0.875rem`, `line-height: 1.55`,
|
||||
`max-height: 320px`, `overflow-y: auto`; its inner paragraphs get
|
||||
reduced margins. No background animation — `prefers-reduced-motion`
|
||||
respected by construction (only the chevron transition, gated).
|
||||
- **E2E mock (`tests/e2e/mock_llm.py`):** deterministic thinking via a new
|
||||
user-message trigger `THINKING_TRIGGER = "think out loud"` (same
|
||||
convention as the existing `"write a long answer"` /
|
||||
`"pretend to think slowly"` triggers). When the trigger is present the
|
||||
mock streams ~800 chars of deterministic `reasoning_content` chunks
|
||||
(a fixed "Step 1… Step 4…" scratchpad) **before** the normal
|
||||
`content` chunks; the non-streaming path includes `reasoning_content`
|
||||
in the message. Without the trigger the mock is byte-identical to
|
||||
today — every existing story suite is unaffected. (The suite is
|
||||
mock-only by design: `E2E_REAL_LLM=1` against the real `turbo` — which
|
||||
thinks on every turn — would break the "no thinking block" regression
|
||||
test. Note that in the file header.)
|
||||
- **Non-goals:** no thinking-budget/effort request parameters (the model
|
||||
decides; `BOR_MAX_OUTPUT_TOKENS` already bounds total output); no DB
|
||||
schema change (thinking is not stored server-side — `query_log` keeps
|
||||
its shape; only the log line counts chars); no in-UI toggle (the
|
||||
kill-switch is `BOR_STREAM_THINKING`); no `done`-event change; no
|
||||
changes to steering/suggestions/viewer.
|
||||
|
||||
## Tasks
|
||||
1. `01_backend_thinking_stream.md` — `StreamPiece` in the LLM client,
|
||||
`thinking` SSE event end-to-end (schema, settings, chat mapping,
|
||||
log line), backend tests.
|
||||
2. `02_frontend_thinking_block.md` — the collapsible thinking block in
|
||||
`app.js`/`styles.css` (streaming, auto-collapse, persistence,
|
||||
stream-drop guard), frontend unit pins.
|
||||
3. `03_e2e_mock_and_story_suite.md` — mock thinking trigger + the
|
||||
dedicated `test_thinking_display.py` Playwright suite (5 scenarios),
|
||||
run in isolation.
|
||||
4. `04_story_docs_plan_commit.md` — user story file, README section,
|
||||
`.env.example`, PLAN §2/§4/§7/§9/§12 revisions (owner permission
|
||||
recorded), the single atomic commit, phase move to complete/.
|
||||
|
||||
## Locked decisions
|
||||
- **A15 extended with owner permission (2026-08-23):** the SSE contract
|
||||
gains the `thinking` event type; `delta` + `done` shapes unchanged.
|
||||
Recorded as a PLAN §4 revision (owner permission noted), not a silent
|
||||
deviation.
|
||||
- **A5 untouched** — no new models/params sent to aipi; we only *read* a
|
||||
field the model already emits. **A11 untouched** — vanilla JS/CSS, no
|
||||
CDN (native `<details>/<summary>`). **A12/A13 untouched** — no new
|
||||
services, no migration. **A16 untouched** — one new story E2E suite +
|
||||
unit/integration extensions. **A10 untouched** — chat stays public;
|
||||
thinking is visible to everyone (homelab scope). No other anchor
|
||||
changed.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_llm_client.py` (StreamPiece mapping: content,
|
||||
`reasoning_content`, `reasoning` fallback, both-in-one-chunk ordering,
|
||||
empty-skip, failure wrapping), `tests/unit/test_sse_events.py`
|
||||
(thinking frame shape), `tests/unit/test_config.py` (default + env
|
||||
parse of `stream_thinking`), `tests/unit/test_frontend_feedback.py` +
|
||||
`tests/unit/test_chat_persistence.py` (source-level pins: thinking event
|
||||
handling, block markers, auto-collapse, `sawDone` guard, persisted
|
||||
`thinking` field, `.thinking` CSS rules).
|
||||
- **Integration:** `tests/integration/test_chat_api.py` — `FakeRagLLM`
|
||||
gains optional thinking; thinking frames precede all delta frames and
|
||||
reassemble; `BOR_STREAM_THINKING=0` suppresses thinking frames while
|
||||
deltas are unchanged; existing suites stay green.
|
||||
- **Coverage:** `uv run pytest --cov=app --cov-report=term-missing`
|
||||
**>90%** on `app/`.
|
||||
- **E2E:** `tests/e2e/test_thinking_display.py` — five scenarios (see
|
||||
task 03), **mock-only**, green **in isolation**
|
||||
(`uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest` green (unit + integration);
|
||||
`uv run pytest --cov=app --cov-report=term-missing` > 90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
green in isolation (prereq: `podman compose up -d db`).
|
||||
- [ ] Regression suites green in isolation: `test_chat_rag.py`,
|
||||
`test_loading_feedback.py`, `test_chat_persistence.py`,
|
||||
`test_honest_deflection.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual live check (dev server, real aipi): a chat turn shows the
|
||||
thinking block streaming, collapsing when the answer starts,
|
||||
toggleable afterwards; with `BOR_STREAM_THINKING=0` no block.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): block uses Phase-08 tokens,
|
||||
all text contrast ≥4.5:1, summary is a real focusable control with
|
||||
≥44px target, `prefers-reduced-motion` respected, no CDN tags,
|
||||
chat column still 46rem.
|
||||
- [ ] `.agents/user_stories/thinking-display.md` exists; PLAN §2/§4/§7.4/
|
||||
§7.5/§9/§12 carry the revision notes (owner permission
|
||||
2026-08-23).
|
||||
- [ ] One `--no-gpg-sign` commit (below);
|
||||
`.agents/phases/todo/17_thinking_display/` moved to
|
||||
`.agents/phases/complete/`.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add -A .agents/ app/ frontend/ tests/ README.md .env.example && git commit --no-gpg-sign -m "feat(chat): stream model thinking over SSE and show it in a collapsible block"
|
||||
```
|
||||
@@ -0,0 +1,120 @@
|
||||
# Task 01 — Backend: stream thinking pieces end-to-end
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agents/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
The chat pipeline carries the model's reasoning: `LLMClient.chat_stream`
|
||||
yields typed `StreamPiece`s (content vs thinking), and `POST /api/chat`
|
||||
emits a new `{"type":"thinking","text":…}` SSE event ahead of the
|
||||
`delta` events — counted in the per-turn log line, suppressible via
|
||||
`BOR_STREAM_THINKING=0`.
|
||||
|
||||
## Work
|
||||
1. `app/rag/llm.py`
|
||||
- Add a module-level frozen dataclass `StreamPiece` with fields
|
||||
`kind: Literal["content", "thinking"]` and `text: str` (export it in
|
||||
the module — `from app.rag.llm import StreamPiece` must work).
|
||||
- Change `chat_stream` to return `AsyncIterator[StreamPiece]` (was
|
||||
`AsyncIterator[str]`). Per streamed chunk with at least one choice:
|
||||
read `delta = chunk.choices[0].delta`, then
|
||||
- `reasoning = getattr(delta, "reasoning_content", None)` (the field
|
||||
aipi's `turbo` emits — verified live 2026-08-23; the openai SDK
|
||||
keeps unknown delta fields in `model_extra`, so `getattr` is the
|
||||
right accessor); if falsy fall back to
|
||||
`getattr(delta, "reasoning", None)` (future-proofing).
|
||||
If truthy, `yield StreamPiece("thinking", reasoning)`.
|
||||
- If `delta.content` is truthy, `yield StreamPiece("content", delta.content)`.
|
||||
A chunk carrying both fields yields the thinking piece **first**.
|
||||
Chunks with no choices are skipped, as today. The
|
||||
`LLMError` wrap (`except Exception → LLMError`) and the generation
|
||||
params (`model`, `temperature=0.4`, `max_tokens=
|
||||
settings.max_output_tokens`, `stream=True`) are unchanged.
|
||||
- Update the module + method docstrings: name the verified wire
|
||||
convention (`delta.reasoning_content` before `delta.content`) and
|
||||
that reasoning counts against `max_tokens` (an answer can in
|
||||
principle be empty — the UI handles that).
|
||||
2. `app/schemas.py` — add `ChatThinkingEvent(BaseModel)`:
|
||||
`type: str = "thinking"`, `text: str`, with a docstring referencing
|
||||
the phase-17 PLAN §4 extension (sibling of `ChatErrorEvent`).
|
||||
3. `app/config.py` — in the LLM section add
|
||||
`stream_thinking: bool = True` with a comment: operator kill-switch
|
||||
for the `thinking` SSE events (phase 17); when off, pieces are still
|
||||
counted for the log line but never emitted. Env: `BOR_STREAM_THINKING`
|
||||
(`0`/`false` → False — pydantic-settings parses bools).
|
||||
4. `app/api/chat.py`
|
||||
- Import `ChatThinkingEvent` and `StreamPiece` (for typing).
|
||||
- In the step-3 streaming loop, replace
|
||||
`for piece in llm.chat_stream(messages): yield sse_event({"type": "delta", "text": piece})`
|
||||
with a loop over `StreamPiece`s:
|
||||
- `kind == "thinking"`: accumulate `thinking_chars += len(piece.text)`;
|
||||
emit `sse_event(ChatThinkingEvent(text=piece.text).model_dump())`
|
||||
**only when** `settings.stream_thinking` is true (`settings` is
|
||||
already in scope from step 2).
|
||||
- `kind == "content"`: emit the `delta` event exactly as today.
|
||||
- Per-turn log line (PLAN §9): insert `thinking_chars=%d`
|
||||
immediately before `total_ms=%d` (add `thinking_chars` to the
|
||||
`logger.info` args). Keep every existing field and order.
|
||||
- Update the module docstring: the turn now streams `thinking` events
|
||||
(phase 17, PLAN §4 extension) ahead of `delta` events, with
|
||||
`BOR_STREAM_THINKING=0` suppressing them.
|
||||
5. `.env.example` — add `BOR_STREAM_THINKING=1` with a short comment
|
||||
(stream the model's thinking as `thinking` SSE events; set `0` to
|
||||
suppress).
|
||||
|
||||
## Testing & Quality
|
||||
- `tests/unit/test_llm_client.py`
|
||||
- Extend the `_chunk` helper: `_chunk(content=…, reasoning=None)` —
|
||||
build the delta `SimpleNamespace` with `reasoning_content` present
|
||||
only when `reasoning is not None` (mirror the real wire: the field
|
||||
exists only when the model sends it).
|
||||
- Adapt the existing chat-stream tests to the new yield type: `_collect`
|
||||
returns pieces; assertions compare
|
||||
`(p.kind, p.text)` pairs (or map to text where the old intent was
|
||||
"deltas in order").
|
||||
- New tests:
|
||||
- `test_chat_stream_maps_reasoning_content_to_thinking_pieces`
|
||||
- `test_chat_stream_falls_back_to_reasoning_field`
|
||||
- `test_chat_stream_thinking_yields_before_content_in_chunk` (one
|
||||
chunk with both fields → thinking piece first)
|
||||
- `test_chat_stream_interleaved_thinking_and_content_order_preserved`
|
||||
(thinking chunks → content chunks → order of the piece sequence
|
||||
matches the chunk order)
|
||||
- Existing `test_chat_stream_skips_empty_deltas_and_choiceless_chunks`
|
||||
and `test_chat_stream_wraps_failures_as_llm_error` keep passing
|
||||
(adapted to pieces where needed).
|
||||
- `tests/unit/test_sse_events.py` — `test_thinking_frame_serializes_exactly`:
|
||||
`sse_event(ChatThinkingEvent(text="…").model_dump())` round-trips to
|
||||
`{"type": "thinking", "text": "…"}`.
|
||||
- `tests/unit/test_config.py` — `stream_thinking` defaults to `True`;
|
||||
`Settings(_env_file=None, stream_thinking=False)` / env `0` parse.
|
||||
- `tests/integration/test_chat_api.py`
|
||||
- `FakeRagLLM` — add `thinking: str = ""`; its `chat_stream` yields
|
||||
`StreamPiece("thinking", …)` slices of `self.thinking` (12-char
|
||||
slices, same cadence as content) **before** the content pieces; with
|
||||
the default `thinking=""` it yields content-only pieces (today's
|
||||
behavior, new yield type).
|
||||
- New `test_chat_streams_thinking_before_deltas` (override `get_llm`
|
||||
with `FakeRagLLM(thinking="…")`, assert: ≥1 `thinking` frame, every
|
||||
`thinking` frame precedes every `delta` frame, thinking text
|
||||
reassembles to the input, `done` still last, sources unchanged).
|
||||
- New `test_chat_thinking_suppressed_when_disabled` — monkeypatch
|
||||
`chat_api.get_settings` to a `Settings(_env_file=None, …)` with
|
||||
`stream_thinking=False` **and the same `relevance_threshold` the
|
||||
module/conftest already use** (read it from the existing settings —
|
||||
don't hardcode a different gate), then: no `thinking` frames, deltas
|
||||
identical to the thinking-free case. Restore via `monkeypatch`.
|
||||
- Coverage: **>90%** on the touched `app/` modules
|
||||
(`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_llm_client.py tests/unit/test_sse_events.py tests/unit/test_config.py tests/integration/test_chat_api.py -v --no-cov`
|
||||
green (integration needs `podman compose up -d db`).
|
||||
- [ ] `uv run pytest` fully green; coverage > 90%;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] `curl -N -X POST localhost:8000/api/chat …` against the dev server
|
||||
+ real aipi shows `thinking` frames before `delta` frames (and the
|
||||
per-turn log line carries `thinking_chars=`); with
|
||||
`BOR_STREAM_THINKING=0` no `thinking` frames.
|
||||
- [ ] The frontend (phase-16 state, `app.js` untouched) still works:
|
||||
unknown `thinking` frames are ignored by its `readSSE` handler
|
||||
(it only branches on `delta`/`done`/`error`) — no chat regression.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Task 02 — Frontend: the collapsible Thinking block
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agents/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
`index.html` chat shows the model's thinking: a `<details class="thinking">`
|
||||
block above the answer bubble streams open while `thinking` events
|
||||
arrive, auto-collapses when the first answer token lands, stays
|
||||
user-toggleable, and survives reloads (phase-14 persistence gains an
|
||||
optional `thinking` field). Plus a stream-drop guard for severed
|
||||
streams.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/app.js` — turn handler (`handleSend`):
|
||||
- New turn-local vars next to `acc`/`aborted`: `let thinkingAcc = "";
|
||||
let sawThinking = false; let sawDone = false;`
|
||||
- **Two small helpers** (near `addTyping`/`removeTyping`):
|
||||
- `ensureThinkingBlock(wrap)` — returns the existing
|
||||
`.thinking` `details` in `wrap`, or creates one:
|
||||
`details.thinking` with `open = true`, containing
|
||||
`<summary>Thinking</summary>` + `<div class="thinking-text">`,
|
||||
inserted into `.msg-body` **before** the `.bubble`
|
||||
(`body.insertBefore(el, body.querySelector(".bubble"))`).
|
||||
- `closeThinkingBlock(wrap)` — sets `.open = false` on
|
||||
`wrap.querySelector(".thinking")` when present (no-op otherwise).
|
||||
- **`readSSE` onEvent** — new branch `ev.type === "thinking"`:
|
||||
- `thinkingAcc += ev.text || ""; sawThinking = true;`
|
||||
- `clearTurnTimeout()` (the stream is alive — same role the first
|
||||
`delta` already plays; keep the existing delta-side clear as is).
|
||||
- If `!wrap`: `wrap = addMessage("brain", "");` — **do not call
|
||||
`setUiState` here**: the UI state stays `thinking` (button still
|
||||
disabled/"Thinking…", `#send-status` still
|
||||
"Brain of Reese is thinking" — all still true); remove the typing
|
||||
indicator via `removeTyping()` since the live block replaces it as
|
||||
the visible feedback.
|
||||
- `const block = ensureThinkingBlock(wrap);` then
|
||||
`block.querySelector(".thinking-text").innerHTML =
|
||||
renderMarkdown(thinkingAcc);` (escape-first renderer — XSS-safe;
|
||||
the scratchpad may contain markdown-ish text). While
|
||||
`block.open`, pin the text to the bottom
|
||||
(`textEl.scrollTop = textEl.scrollHeight`) and
|
||||
`wrap.scrollIntoView({ behavior: SCROLL, block: "end" })`.
|
||||
- **`delta` branch** — replace the `if (!wrap) { setUiState(streaming);
|
||||
wrap = addMessage("brain",""); }` transition with:
|
||||
`if (uiState === UI_STATE.thinking) setUiState(UI_STATE.streaming);`
|
||||
then `if (!wrap) wrap = addMessage("brain", "");` then
|
||||
`closeThinkingBlock(wrap);` (idempotent — never reopens once the
|
||||
answer started; a late/interleaved `thinking` event appends to the
|
||||
closed block without reopening it). Rest of the delta logic
|
||||
unchanged.
|
||||
- **`done` branch** — set `sawDone = true;` and
|
||||
`closeThinkingBlock(wrap)`; existing logic (deflected class,
|
||||
maybe-try, sources, tune button) unchanged. Then:
|
||||
- **thinking-without-answer:** `const finalText = acc ||
|
||||
(sawThinking ? EMPTY_ANSWER_FALLBACK : "");` where
|
||||
`EMPTY_ANSWER_FALLBACK` is the existing fallback string
|
||||
("Hmm — that came back empty. Ask me again?" — hoist it to a
|
||||
const shared by the `done` branch and the post-stream `!wrap`
|
||||
fallback). When `finalText` was substituted, set the bubble's
|
||||
innerHTML to `renderMarkdown(finalText)`.
|
||||
- `rememberBrainTurn(finalText || acc, { thinking:
|
||||
thinkingAcc || undefined, deflected: !!ev.deflected, sources:
|
||||
ev.sources, suggestions: ev.suggestions })` — `undefined` drops
|
||||
the key from the JSON, so turns without thinking persist exactly
|
||||
as before.
|
||||
- **Stream-drop guard** — after the `await readSSE(…)` call, before
|
||||
the existing `if (!aborted && !wrap)` fallback:
|
||||
`if (!sawDone && !aborted && (acc || thinkingAcc)) {
|
||||
setUiState(UI_STATE.error, "The stream ended before my answer
|
||||
finished — try again?"); }` (ERROR_HINT is appended by the banner).
|
||||
The zero-frame case falls through to the existing "came back empty"
|
||||
fallback, unchanged.
|
||||
- **Persistence/restore:**
|
||||
- `renderStoredMessage` (brain branch): after `addMessage`, if
|
||||
`m.thinking` — `const block = ensureThinkingBlock(wrap);
|
||||
block.open = false;
|
||||
block.querySelector(".thinking-text").innerHTML =
|
||||
renderMarkdown(m.thinking);`
|
||||
- `loadStoredConversation` filter: unchanged (raw `text` still
|
||||
required; `thinking` is optional).
|
||||
- Header doc comment: extend the phase-14 persistence note (brain
|
||||
records may carry `thinking`) and the loading-feedback comment
|
||||
(the thinking block is the visible feedback while `thinking`
|
||||
events stream; the 120s guard clears on the first thinking *or*
|
||||
delta event).
|
||||
2. `frontend/assets/styles.css` — new rules (Phase-08 tokens; place with
|
||||
the other `.msg` styles):
|
||||
- `details.thinking` — `background: var(--surface)`,
|
||||
`border: 1px solid var(--line)`,
|
||||
`border-left: 3px solid var(--brand-soft)`,
|
||||
`border-radius: var(--radius-sm)`, `margin: 0 0 0.5rem`,
|
||||
`overflow: hidden`.
|
||||
- `details.thinking summary` — `display: flex`,
|
||||
`align-items: center`, `gap: 0.5rem`,
|
||||
`padding: 0.5rem 0.75rem`, `min-height: 44px`,
|
||||
`color: var(--brand-ink)` (8.7:1 on `--surface`),
|
||||
`font-size: 0.9rem`, `cursor: pointer`, `list-style: none`;
|
||||
`summary::-webkit-details-marker { display: none; }`;
|
||||
chevron `summary::before { content: "▸"; display: inline-block;
|
||||
transition: transform 0.15s ease; }` and
|
||||
`details.thinking[open] summary::before { transform:
|
||||
rotate(90deg); }`; `summary:focus-visible` — 3px `var(--brand)`
|
||||
outline, 2px offset.
|
||||
- `details.thinking .thinking-text` —
|
||||
`padding: 0 0.75rem 0.75rem`, `color: var(--ink-soft)` (6.9:1 on
|
||||
`--surface` — keep the ratio comment in-line with the file's
|
||||
convention), `font-size: 0.875rem`, `line-height: 1.55`,
|
||||
`max-height: 320px`, `overflow-y: auto`; reduce the margins of its
|
||||
direct `p`/`ul` (e.g. `margin: 0 0 0.5rem`).
|
||||
- In the existing `@media (prefers-reduced-motion: reduce)` block(s),
|
||||
disable the summary chevron `transition`.
|
||||
3. Frontend unit pins (source-level, matching the existing files' style):
|
||||
- `tests/unit/test_frontend_feedback.py` — new tests:
|
||||
`ev.type === "thinking"` is handled in `app.js`;
|
||||
`ensureThinkingBlock` + `closeThinkingBlock` markers exist;
|
||||
the auto-collapse marker (`closeThinkingBlock(wrap)` in the delta
|
||||
branch); the `sawDone` stream-drop guard marker; the
|
||||
`uiState === UI_STATE.thinking` → streaming transition.
|
||||
- `tests/unit/test_chat_persistence.py` — new tests: the stored brain
|
||||
record carries `thinking:` in `rememberBrainTurn`'s meta
|
||||
(assert the `thinking:` marker in the save call site) and
|
||||
`renderStoredMessage` restores `m.thinking`; `styles.css` contains
|
||||
the `.thinking` rules (`details.thinking`, `.thinking-text`).
|
||||
|
||||
## Testing & Quality
|
||||
- The browser behavior is E2E-covered by task 03's suite; here the
|
||||
source-level pins above catch silent regressions without a browser.
|
||||
- No backend code changes in this task.
|
||||
- Coverage: unaffected on `app/` (frontend) — keep
|
||||
`uv run pytest --cov=app` ≥ today's number.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_frontend_feedback.py tests/unit/test_chat_persistence.py -v --no-cov`
|
||||
green; `uv run pytest` fully green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] Manual (dev server + real aipi, or mock with a trigger once task
|
||||
03 lands): a turn with thinking shows the block open and streaming,
|
||||
collapsed after the answer; clicking the summary toggles it;
|
||||
reload restores the collapsed block with the same text; a plain
|
||||
turn (no thinking events) renders exactly as before (no block).
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5): summary is a real focusable
|
||||
control (≥44px target, `:focus-visible` outline), text contrast
|
||||
≥4.5:1 (8.7:1 / 6.9:1 as specified), no CDN tags, chat column
|
||||
still 46rem, `prefers-reduced-motion` respected.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Task 03 — E2E: mock thinking trigger + the story suite
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agents/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
Deterministic E2E coverage of the thinking display: the mock LLM gains a
|
||||
`"think out loud"` trigger that streams `reasoning_content` chunks before
|
||||
the answer, and the new dedicated suite
|
||||
`tests/e2e/test_thinking_display.py` (one story → one file, A16) covers
|
||||
streaming, auto-collapse, toggle, persistence-restore, deflection
|
||||
coexistence, and the no-thinking regression — green in isolation.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/mock_llm.py`
|
||||
- Module docstring: add the trigger to the documented list —
|
||||
*user message containing ``think out loud`` → the answer is
|
||||
preceded by ~800 chars of deterministic `reasoning_content`
|
||||
chunks (thinking-display story, phase 17).*
|
||||
- `THINKING_TRIGGER = "think out loud"` (checked case-insensitively
|
||||
on the user message — same convention as
|
||||
`LONG_ANSWER_TRIGGER`).
|
||||
- `compose_thinking(body) -> str` — deterministic scratchpad text of
|
||||
roughly 700–900 chars built from a fixed 4–5 line "Step 1… Step
|
||||
N:" template that quotes the first ~60 chars of the user question
|
||||
(unique per question, stable across runs). It must contain the
|
||||
line fragment `Step 2: Check my notes` (tests key off it).
|
||||
- `_sse_stream(answer, delay)` — extend to
|
||||
`_sse_stream(answer, delay, thinking="")`: when `thinking` is
|
||||
non-empty, first yield its 12-char slices as
|
||||
`choices[0].delta = {"reasoning_content": piece}` frames (same
|
||||
0.02s cadence, same `chunk_id`/envelope shape as content frames),
|
||||
then the content frames exactly as today. Without `thinking` the
|
||||
output is byte-identical to today.
|
||||
- `chat_completions` — streaming: pass
|
||||
`compose_thinking(body) if THINKING_TRIGGER in _user(body).lower()
|
||||
else ""` as `thinking`. Non-streaming: when the trigger is present,
|
||||
include `"reasoning_content": <same text>` in the message dict
|
||||
(harmless future-proofing; the app only uses streaming).
|
||||
- Confirm no trigger collision: existing E2E questions do not contain
|
||||
the substring `"think out loud"` (the loading-feedback trigger is
|
||||
`"pretend to think slowly"` — distinct).
|
||||
2. `tests/e2e/test_thinking_display.py` (new)
|
||||
- Header comment: **mock-only suite** — `E2E_REAL_LLM=1` is not
|
||||
supported here because the real `turbo` thinks on every turn and
|
||||
would break the no-thinking regression test.
|
||||
- Fixtures mirroring `tests/e2e/test_chat_persistence.py`:
|
||||
`seeded_kb` (truncate `chunks, documents, query_log`, import
|
||||
`tests/fixtures/docs` through the real `LLMClient` +
|
||||
`import_sources`, assert `summary.added == 8`, truncate in
|
||||
teardown), `db_ready`-style skip when Postgres is down (reuse the
|
||||
conftest `db_ready` fixture).
|
||||
- Helpers: `THINK_QUESTION = "think out loud — how is my kubernetes
|
||||
cluster set up?"` (on-topic → grounded answer + thinking);
|
||||
`PLAIN_QUESTION = "How is my Kubernetes cluster set up?"`;
|
||||
`THINK_DEFLECT_QUESTION = "think out loud — tell me about quantum
|
||||
wormhole cooling"` (off-topic → deflected + thinking).
|
||||
- `send_and_wait(page, question)` — type into `#message-input`,
|
||||
submit via `#composer`, then `expect` the last `.msg.brain` to
|
||||
settle (send button re-enabled) with a 30s timeout (the mock
|
||||
streams at 0.02s/chunk; thinking + answer ≈ a few seconds).
|
||||
- **The five scenarios** (also the story's Playwright Mapping Rule):
|
||||
1. `test_thinking_block_streams_open_then_collapses` — submit
|
||||
`THINK_QUESTION`. Assert `details.thinking` inside the last
|
||||
`.msg.brain` attaches within 10s (it appears at the first
|
||||
`thinking` event); immediately after attach, assert it is open
|
||||
(the mock's ~800-char thinking stream gives a multi-second
|
||||
open window — see determinism note) and `.thinking-text` is
|
||||
non-empty; once `.bubble` text is non-empty, assert the block is
|
||||
**closed**; at settle: `.thinking-text` contains `Step 2: Check
|
||||
my notes`, the bubble contains the mock's deterministic answer
|
||||
sentence, `.source-chip` count ≥ 1, send button re-enabled.
|
||||
2. `test_thinking_toggle_after_done` — after a settled
|
||||
`THINK_QUESTION` turn, the block is closed; click `summary` →
|
||||
`details[open]` and the full thinking text is visible; click
|
||||
again → closed. (Real keyboard-focusable control.)
|
||||
3. `test_thinking_restored_after_reload` — settle a
|
||||
`THINK_QUESTION` turn; capture the thinking text;
|
||||
`page.reload()`; the restored conversation contains the brain
|
||||
message with a **closed** `details.thinking` whose
|
||||
`.thinking-text` matches the captured text, and the answer
|
||||
bubble + source chips are intact (phase-14 restore path).
|
||||
4. `test_no_thinking_block_without_trigger` — submit
|
||||
`PLAIN_QUESTION`; at settle: `page.locator("details.thinking")`
|
||||
count is 0 (a model that doesn't think renders exactly as
|
||||
before — no layout regression).
|
||||
5. `test_thinking_with_deflection` — submit
|
||||
`THINK_DEFLECT_QUESTION`; at settle: the brain message has
|
||||
`.is-deflected`, a `.maybe-try` group with chips, and a closed
|
||||
`details.thinking` whose text contains `Step 2: Check my notes`
|
||||
(thinking and the honesty gate coexist).
|
||||
- Determinism note (comment in the file): the mock paces every SSE
|
||||
frame at 0.02s and the thinking text is ~700–900 chars (≈ 60–75
|
||||
frames ≈ 1.2–1.5s) before the first content frame, so
|
||||
"attach → assert open" runs well inside the open window on
|
||||
headless Chromium; all other assertions are made after the send
|
||||
button re-enables (fully settled state).
|
||||
3. Regression pass (run each **in isolation**, per A16):
|
||||
`uv run pytest tests/e2e/test_chat_rag.py -v --no-cov`,
|
||||
`… test_loading_feedback.py …`, `… test_chat_persistence.py …`,
|
||||
`… test_honest_deflection.py …` — all green (the mock is
|
||||
trigger-gated, so their behavior is unchanged; this pass proves it).
|
||||
|
||||
## Testing & Quality
|
||||
- The new suite is this story's Playwright gate (A16): it must pass in
|
||||
isolation: `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
(prereq: `podman compose up -d db`, Chromium installed).
|
||||
- No `app/` or `frontend/` code changes in this task — if a test exposes
|
||||
a real bug in tasks 01/02, fix it in the owning file (app/ vs
|
||||
frontend/) and re-run that task's tests; note the fix in the commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Without the trigger, mock behavior is byte-identical to before —
|
||||
the four regression suites from the Work section each green **one
|
||||
command at a time** (isolation is the A16 invariant).
|
||||
- [ ] `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
green in isolation (5/5).
|
||||
- [ ] `uv run pytest` (unit + integration) still green;
|
||||
`uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Task 04 — Story file, docs, PLAN revisions, the phase commit
|
||||
|
||||
**Phase:** `17_thinking_display` · **Story:** `.agents/user_stories/thinking-display.md`
|
||||
|
||||
## Objective
|
||||
Record the feature where it belongs: the user story file (AGENTS.md rule
|
||||
4 — one story per phase), the README section, `.env.example` parity, the
|
||||
PLAN.md revisions (owner permission noted per the A10-revision
|
||||
precedent of phase 16), and the single atomic `--no-gpg-sign` commit with
|
||||
the phase moved to `complete/`.
|
||||
|
||||
## Work
|
||||
1. `.agents/user_stories/thinking-display.md` (new — match the format of
|
||||
the sibling stories, e.g. `loading-feedback.md`):
|
||||
- Header: `**Phase:** 17_thinking_display · **E2E:**
|
||||
tests/e2e/test_thinking_display.py`.
|
||||
- Narrative: as a user, local reasoning models "think" before they
|
||||
answer (10–30s of silence today). I want to *see* Brain think —
|
||||
its reasoning streaming live, tucked away once the answer starts —
|
||||
so long turns feel transparent instead of frozen.
|
||||
- Acceptance criteria:
|
||||
1. Turns whose model stream carries reasoning show a "Thinking"
|
||||
block (collapsible, above the answer bubble) that streams open
|
||||
and auto-collapses on the first answer token; always
|
||||
user-toggleable afterwards.
|
||||
2. Turns without reasoning render exactly as before (no block, no
|
||||
layout shift).
|
||||
3. Thinking-without-answer (reasoning exhausted the token budget)
|
||||
shows the existing empty-answer fallback with the thinking
|
||||
block preserved.
|
||||
4. Thinking persists with the message (phase 14) and restores
|
||||
collapsed after reload; "New chat" clears it with everything
|
||||
else.
|
||||
5. Deflected turns show the amber bubble + "Maybe try" chips
|
||||
alongside the thinking block (honesty gate untouched).
|
||||
6. A stream that dies mid-thinking/mid-answer ends in the error
|
||||
state (retry hint) — never a silent half bubble.
|
||||
7. `BOR_STREAM_THINKING=0` suppresses `thinking` events server-side
|
||||
(log line still counts `thinking_chars`).
|
||||
- UI Visualization & Structure: the DOM contract
|
||||
(`details.thinking` > `summary` + `.thinking-text`, before
|
||||
`.bubble`), Phase-08 token values + computed contrasts (summary
|
||||
8.7:1, text 6.9:1), 44px summary target, native
|
||||
`<details>/<summary>` accessibility (no live region on the
|
||||
scratchpad — `#send-status` announces state), `max-height: 320px`
|
||||
scroll, reduced-motion note.
|
||||
- Playwright Mapping Rule: the five scenarios of
|
||||
`tests/e2e/test_thinking_display.py` verbatim from task 03.
|
||||
2. `README.md` — add a short "Thinking" section in the chat/features
|
||||
area (find the natural neighbor — e.g. after the description of the
|
||||
chat UI / loading feedback): what it is (the model's reasoning,
|
||||
streamed as `thinking` SSE events, shown in a collapsible block),
|
||||
that how much it thinks is the model's call, and the
|
||||
`BOR_STREAM_THINKING=0` kill-switch. No CDN rule, no other edits.
|
||||
3. `.env.example` — verify the task-01 line (`BOR_STREAM_THINKING=1`)
|
||||
is present with its comment; add nothing new.
|
||||
4. `.agents/PLAN.md` revisions — **record owner permission
|
||||
(2026-08-23) in each note**, exactly the style phase 16 used for the
|
||||
A10 revision:
|
||||
- Header revisions line: append
|
||||
`; thinking display (Phase 17)`.
|
||||
- **§4 SSE contract:** extend the example with
|
||||
`data: {"type":"thinking","text":"…"}` frames before the `delta`
|
||||
frames, and add the client rule: *render `thinking` text in a
|
||||
collapsible block above the answer; auto-collapse on the first
|
||||
`delta`; tolerate interleaved `thinking` events; the `done` shape
|
||||
is unchanged.*
|
||||
- **§7.4 table:** new row — **Thinking (model reasoning)**:
|
||||
collapsible `.thinking` block streams open (replaces the typing
|
||||
dots as the live indicator), auto-collapses on the first answer
|
||||
token, toggleable afterwards, persisted with the message (phase
|
||||
14); 120s guard clears on the first `thinking` *or* `delta` event.
|
||||
- **§7.5 component inventory:** add `.thinking`, `.thinking-text`
|
||||
(collapsible thinking block; plain `<summary>`, no id).
|
||||
- **§9 per-turn log line:**
|
||||
`question=… embed_ms=… top_score=… fts_hits=… tuning=N
|
||||
threshold=… deflected=… sources=… thinking_chars=… total_ms=…`
|
||||
- **§12 roadmap:** new row 17 — `17_thinking_display` /
|
||||
`thinking-display.md` / `tests/e2e/test_thinking_display.py`.
|
||||
- Do **not** touch the locked anchors themselves (A15's decision text
|
||||
stays; the §4 note carries the extension) and do not renumber
|
||||
anything.
|
||||
5. Final validation pass (all gates, AGENTS.md rules 5 + 9):
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` (> 90%),
|
||||
- `uv run pytest tests/e2e/test_thinking_display.py -v --no-cov`
|
||||
(in isolation), plus the four regression E2E suites (one command
|
||||
each, in isolation),
|
||||
- `uv run ruff check . && uv run pyright`,
|
||||
- confirm no CDN tags were introduced
|
||||
(`tests/integration/test_api.py::test_index_html_served_locally`
|
||||
covers index; the thinking block is dynamic JS, no template
|
||||
change).
|
||||
6. Commit + phase move (last step, only when all gates are green):
|
||||
```bash
|
||||
git add -A .agents/ app/ frontend/ tests/ README.md .env.example
|
||||
git commit --no-gpg-sign -m "feat(chat): stream model thinking over SSE and show it in a collapsible block"
|
||||
mv .agents/phases/todo/17_thinking_display .agents/phases/complete/
|
||||
```
|
||||
|
||||
## Testing & Quality
|
||||
- No new logic in this task — it is the record-keeping + validation pass
|
||||
of the phase; the gates above are the phase's final proof.
|
||||
- If validation fails, fix in the owning task's files, re-run that task's
|
||||
tests, and only then commit.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `.agents/user_stories/thinking-display.md` exists with all five
|
||||
sections (header, narrative, acceptance, UI visualization,
|
||||
Playwright Mapping Rule).
|
||||
- [ ] README "Thinking" section + `.env.example` line present;
|
||||
`.agents/PLAN.md` carries the §2-revision/§4/§7.4/§7.5/§9/§12 notes
|
||||
with the 2026-08-23 owner-permission wording; no anchor text
|
||||
altered.
|
||||
- [ ] All gates green (coverage > 90%, story E2E + 4 regressions in
|
||||
isolation, ruff + pyright clean).
|
||||
- [ ] Exactly one new commit, conventional, `--no-gpg-sign`;
|
||||
`.agents/phases/todo/17_thinking_display/` is now under
|
||||
`.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user