Files
brain-of-reese/tests/e2e/test_thinking_timeline.py
ducoterra 690fd98036
Build and Push Containers / build-and-push-app (push) Successful in 1m43s
Build and Push Containers / build-and-push-db (push) Successful in 15s
fix(rag): stop the per-run blocks from folding — the container is the one disclosure
The owner refinement (2026-10-06): the per-run thinking and tool-call
blocks inside the Reasoning bubble were individually collapsible
details elements — a fold inside the fold. A tool frame closed the
previous thinking block, so the visible thinking vanished mid-turn;
and the pinned-tail follow died on the append of every run after the
first (the pin was captured AFTER the new block's height growth
contaminated the 32px reading).

- app.js: the in-container runs are now plain divs —
  createThinkingRunEl (div.thinking + static .thinking-summary label +
  .thinking-text) and createToolRunEl (div.tool-calls-disclosure +
  static .tool-calls-summary label + the .tool-calls list); they never
  open or fold (the container is the ONE open/fold control, open
  through the turn, folded at settle — unchanged); the pin capture
  moves BEFORE ensureThinkingRun/ensureToolRun append (plus the
  first-frame edge: no container yet -> always pinned);
  closeCurrentRun just ends the run; closeAllRuns stays the defensive
  legacy guard; restoreTimeline/addSegments (shared.js) use the div
  builders; the legacy no-segments restore path keeps the old
  details-based blocks byte-identical
- styles.css: the in-container runs are distinct by label + rail
  (brand-ink Thinking label on a brand-soft rail; accent-ink
  Tool call(s) (N) label on a neutral rail); the interactive
  treatment (cursor + chevron) is scoped to the legacy <summary>; the
  inner windows stay disabled (the container's .reasoning-body is the
  ONE live scroll surface)
- unit + E2E: the D5-refinement contract is pinned — one
  details.reasoning disclosure, per-run blocks as plain divs, zero
  nested <details> inside the container, the container as the only
  open/fold control, the .reasoning-body window auto-follows the
  tail while streaming

unit+integration 3738 passed (coverage 99%), ruff + pyright clean, all
five affected E2E suites pass in isolation (reasoning_bubble 6,
thinking_timeline 6, thinking_display 5, thinking_scroll 8,
tool_call_compact 4), verified live: the follow holds through a long
thinking stream, the tool-frame transition, and the second thinking
run; restore renders the div structure without client-side errors

LLM-Generated: true
2026-10-06 14:44:41 -04:00

876 lines
36 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 158 E2E (Playwright, mock-only): the chronological thinking /
tool-call timeline (the owner's 2026-10-04 report — "the thinking block
happens first … and then went back to thinking. This is awkward. Then
the tool calling happens, and the thinking block reads strangely because
the tool calls are shown below the thinking block").
Source: owner report (live interactive-browser session, 2026-10-04,
shared page ``https://brain.reeseapps.com/shared/
95dd7a2a-8cda-478b-b694-53936070b9fb`` — a real ``turbo`` turn with 6
tool calls). Run in isolation (DB must be up:
``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate
is the deterministic marker flow in ``tests/e2e/mock_llm.py`` (user
message contains ``think out loud and use your tools``
(``THINK_TOOLS_TRIGGER``) **and** the system prompt carries the
``<tools>`` section of the HIGH prompt): the phase-37 single-read
shape (list → drill → read → the quoted answer) with a DISTINCT
``reasoning_content`` run streamed AHEAD of every tool call and ahead
of the final answer — the ``Step N`` markers make each run
individually assertable. The client renders each CONTIGUOUS run in its
own per-run block (D1) in arrival order (D2 — Option B,
owner-chosen) and persists the run order in the record's ``segments``
array (D4). Phase 161 (the documented supersession, owner TODO
direction 2026-10-05): the per-run blocks are NESTED inside ONE
``details.reasoning`` container (``.reasoning-body``) that sits
before the bubble — open during the turn, folded at done / on
restore / on the shared page. The D5 refinement (owner 2026-10-06):
the per-run blocks are plain divs — NOT collapsible (the container
is the ONE disclosure; a nested fold inside the fold used to hide
the live reasoning mid-turn), always visible while the turn is live.
The sequence pin is shape-unchanged, read from the container's
``.reasoning-body`` children (the container is the one
new parent — the pre-bubble ``.msg-body`` children are exactly
``["details.reasoning"]``).
KB fixture — mirrored from ``test_agent_document_tools.py`` (the
deterministic two-document pair): ``Deployments`` registered FIRST so
the mock's drill (first source of the top-level listing) — and
therefore the read — lands on the JSON file; the marker question
cosines ≈0.595 against the retrievable document (well past the E2E
0.30 threshold) plus FTS hits, so the turn is solidly grounded (HIGH
prompt → ``<tools>``).
Test → phase mapping (Playwright Mapping Rule):
1. ``test_timeline_order_after_completed_turn`` — D1/D2: after ``done``
the pre-bubble ``.msg-body`` children are EXACTLY the reasoning
container (the one new parent — phase 161), and its
``.reasoning-body`` children are EXACTLY the 7-element
arrival-order sequence (4 thinking blocks + 3 one-line
disclosures, alternating); the four ``.thinking-text``s carry the
DISTINCT ``Step N`` markers (distinctness = separate blocks, not
one lump); each disclosure summary is ``Tool call (1)``; the
answer bubble carries the ``Read <source/path>`` echo.
2. ``test_runs_fold_at_rest`` — D5 (phase 161 D2): after ``done``
the container has NO ``open`` attribute (the settle folded it) and
the inner runs are NOT collapsible (zero ``<details>`` inside the
container — the D5 refinement); the at-rest visible line is the
container's ``Reasoning`` summary (``7 steps``); opening it
reveals the always-visible run blocks.
3. ``test_live_runs_open_while_streaming`` — D5 (the live half):
DURING the turn (the mock's pacing window — the 0.1 s tool frames +
the thinking stream, the answer bubble still EMPTY) the container
is ``open`` with a run visible in its ``.reasoning-body`` (the
live feedback the owner keeps — the container is the ONE open
disclosure).
4. ``test_restored_timeline_and_record`` — the restore path + D4: a
same-context RELOAD re-renders the same 7-element sequence in ONE
collapsed reasoning container (``7 steps``, phase 161 D5/D6), and
the localStorage ``bor.chat.v1`` record (phase 14) carries a
``segments`` array of EXACTLY 7 entries in the
``thinking, tool, ×3`` pattern (the ``Step N`` text on the thinking
entries, the ``name``/``argument`` on the tool entries); the legacy
``thinking`` (full concatenation) + ``tools`` (3 records) keys are
STILL present (the old-client graceful degradation).
5. ``test_shared_page_timeline`` — the shared-page parity (phase 51
save + share flow): a FRESH context on ``/shared/<token>`` renders
the same 7-element sequence in ONE collapsed reasoning container
(``addSegments``), the ``Step N`` markers visible on container
click (the D5 refinement — the inner blocks are always visible
once the container opens), the answer bubble carries the
``Read <source/path>`` echo.
6. ``test_no_thinking_turn_single_disclosure`` — the run-grouping
contract for no-thinking turns (the phase-117 behavior preserved,
now wrapped — phase 161): a plain ``use your tools`` turn (NO
thinking trigger) renders ONE folded reasoning container (``1
steps``) wrapping EXACTLY ONE ``.tool-calls-disclosure`` inside its
``.reasoning-body`` (3 ``.tool-call`` lines, summary ``Tool calls
(3)``) and ZERO ``.thinking``.
"""
from __future__ import annotations
import hashlib
import re
import time
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Any
import httpx
from playwright.sync_api import Browser, BrowserContext, Page, expect
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, GitSource
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
# --------------------------------------------------------------------------
# Fixture documents (deterministic, token-controlled — mirrored from
# test_agent_document_tools.py so the flow is byte-stable)
# --------------------------------------------------------------------------
SEED_SOURCE = "Homelab"
SEED_PATH = "aws-route53.md"
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
READ_SOURCE = "Deployments"
READ_PATH = "example-record-file.json"
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
#: The retrievable document: references the JSON file "for the exact
#: JSON shape of reeselink.json" but never includes it. The repeated
#: record-file lines carry the marker question's key tokens (aws,
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
#: ≈0.595 cosine against the mock's embeddings (E2E threshold 0.30)
#: plus FTS hits, so the turn is solidly grounded.
ROUTE53_CONTENT = (
"# AWS Route 53 Notes\n\n"
"## Record file\n\n"
+ (
"The aws route53 hosted zone for reeselink keeps every record in "
"reseelink.json — the exact JSON shape of reeselink.json is "
"documented in example-record-file.json.\n"
)
* 10
+ "\n## Sync job\n\n"
"A cron job pushes reeselink.json to the aws route53 hosted zone "
"every fifteen minutes; the diff is applied through the route53 api.\n"
)
#: The referenced document: the exact JSON shape. Its FIRST line is
#: longer than 80 chars, so the mock's first-80-chars quote content
#: part is newline-free (the rendered-text assertions below match it
#: after the date line). Pinned by the assert below.
RECORD_FILE_CONTENT = (
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
'JSON shape of reeselink.json",\n'
' "hosted_zone_id": "Z0RESEELINK01",\n'
' "record_sets": [\n'
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
" ]\n"
"}\n"
)
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
#: The phase-158 trigger question (``THINK_TOOLS_TRIGGER`` + the KB
#: topic — the ≈0.595 cosine keeps the turn grounded over the mock-
#: calibrated E2E threshold).
TIMELINE_QUESTION = (
"Think out loud and use your tools: what is the exact JSON shape of "
"reseelink.json for my aws route53 hosted zone?"
)
#: The NO-thinking control — the plain ``use your tools`` marker
#: (``test_agent_document_tools.py`` / ``test_tool_call_compact.py``'s
#: question, mirrored so the 3-call flow is the phase-37/117 one).
PLAIN_TOOLS_QUESTION = (
"Use your tools: what is the exact JSON shape of reeselink.json "
"for my aws route53 hosted zone?"
)
#: The mock's deterministic single-read answer —
#: "Read <source/path>. <first 80 chars of the read document>" (the
#: phase-37 shape). Phase 106 (D5): the read result's ``date:`` SECOND
#: line rides into the first-80-chars quote — the quote is the date
#: line (the fixture's fixed ``created_at`` UTC date part, 17 chars) +
#: the first 63 content chars (80 − 17).
ANSWER_PREFIX = f"Read {READ_SP}."
ANSWER_QUOTE = "date: 2024-06-15" + RECORD_FILE_CONTENT[:63]
#: The mock's four DISTINCT reasoning runs (the ``Step N`` markers —
#: their distinctness proves the runs are separate blocks, not one
#: lump).
STEP_MARKERS = ("Step 1", "Step 2", "Step 3", "Step 4")
#: The 7-element sequence the timeline renders in the reasoning
#: container's ``.reasoning-body`` children, in arrival order (D1:
#: alternating thinking/tool runs — one block per run; phase 161: the
#: sequence is shape-unchanged — the container is the one new parent)
#: — tests 1/4 (chat page) and 5 (shared page) pin it.
TIMELINE_SEQ = [
"div.thinking",
"div.tool-calls-disclosure",
"div.thinking",
"div.tool-calls-disclosure",
"div.thinking",
"div.tool-calls-disclosure",
"div.thinking",
]
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
# --------------------------------------------------------------------------
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
# --------------------------------------------------------------------------
def _seed(db: Session) -> None:
"""The two-document pair (see the module docstring), plus the
phase-94 registry rows: both sources registered, ``Deployments``
FIRST — the mock's drill (first source of the top-level listing)
lands on the JSON file deterministically, independent of the
operator's ``BOR_GIT_SOURCES`` (a non-empty table ignores the env
fallback)."""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
full_path=f"/tmp/{SEED_PATH}",
title="AWS Route 53 Notes",
content=ROUTE53_CONTENT,
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
# (the mock's first-80-chars read quote carries the date line).
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
db.add(md)
db.flush()
# One chunk carrying the mock's own embedding → genuine token
# overlap between the marker question and this document.
db.add(
Chunk(
document_id=md.id,
position=0,
content=ROUTE53_CONTENT,
embedding=embed_text(ROUTE53_CONTENT),
)
)
# The referenced JSON: indexed, catalogued, readable — but NO
# chunks, so retrieval never puts it in context (the failure the
# tools fix).
db.add(
Document(
source=READ_SOURCE,
path=READ_PATH,
full_path=f"/tmp/{READ_PATH}",
title="Example Record File",
content=RECORD_FILE_CONTENT,
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
)
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""Truncate the KB (plus the prompt-shaping tables and the
phase-55 auto-save rows), then re-seed — the E2E isolation
pattern. ``steering_notes`` / ``kb_overview`` are truncated too,
so the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
``<tools>`` regardless of leftovers — byte-stable prompts,
byte-stable answers."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, "
"steering_notes, kb_overview, saved_chats, git_sources"
)
)
db.commit()
if seed is not None:
seed(db)
db.commit()
# --------------------------------------------------------------------------
# Page helpers
# --------------------------------------------------------------------------
#: The last ``.msg.brain`` wrap's ``.msg-body`` children BEFORE the
#: bubble, as ``tag.className`` strings. Phase 161 (D1): EXACTLY ONE
#: — the ``details.reasoning`` container (the one new anchor before
#: the bubble; the per-run blocks live inside it, not as direct
#: children; the ``.msg-meta`` row lands AFTER the bubble, so it
#: never enters the sequence).
PRE_BUBBLE_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return null;
const body = wrap.querySelector('.msg-body');
const out = [];
for (const child of body.children) {
if (child.classList.contains('bubble')) break;
out.push(child.tagName.toLowerCase() + '.' + child.className);
}
return out;
}
"""
#: The last ``.msg.brain`` wrap's reasoning container's
#: ``.reasoning-body`` children, as ``tag.className`` strings — the
#: timeline's order pin (phase 161: the 7-element arrival-order
#: sequence nested in the container — shape unchanged from phase 158).
BODY_SEQUENCE_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return null;
const body = wrap.querySelector('details.reasoning .reasoning-body');
if (!body) return null;
const out = [];
for (const child of body.children) {
out.push(child.tagName.toLowerCase() + '.' + child.className);
}
return out;
}
"""
#: D5's live predicate (the D5 refinement — the container is the ONE
#: open disclosure): the container is ``open`` with a per-run block
#: nested in its ``.reasoning-body`` while the answer bubble is STILL
#: EMPTY (the turn is in flight; the mock's pacing window — the 0.1 s
#: tool frames + the thinking stream — keeps the window comfortably
#: open for a 100 ms poll).
LIVE_OPEN_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return false;
const bubble = wrap.querySelector('.msg-body .bubble');
if (!bubble || bubble.textContent !== '') return false;
const cont = wrap.querySelector('details.reasoning');
if (!cont || !cont.hasAttribute('open')) return false;
return !!cont.querySelector(
'.reasoning-body div.thinking, ' +
'.reasoning-body div.tool-calls-disclosure');
}
"""
#: The phase-14 localStorage record's LAST brain message (the
#: ``bor.chat.v1`` — ``{ v: 1, chatId, messages: [...] }``; the
#: filter mirrors ``loadStoredRecord``'s well-formedness).
READ_RECORD_JS = """
() => {
const raw = localStorage.getItem('bor.chat.v1');
if (!raw) return null;
const data = JSON.parse(raw);
const brain = (data.messages || []).filter(
(m) => m && m.who === 'brain' && typeof m.text === 'string');
return brain.length ? brain[brain.length - 1] : null;
}
"""
def _submit(page: Page, question: str) -> None:
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (phase 48 — the in-flight button is the enabled Stop
control, so the label assertion carries the settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# --------------------------------------------------------------------------
# Phase-51 save + share flow (the test_share_chat.py pattern — the
# auto-saved row is shared from the chat page; the row is deleted in a
# finally)
# --------------------------------------------------------------------------
def _admin_cookies(page: Page) -> dict[str, str]:
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["chats"]
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
return next((c for c in rows if c["title"] == title), None)
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55 — auto-saves are SILENT,
so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
last = _find_row(_chats(app_url, cookies), title)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(headless Chromium requires the grant before ``writeText``
resolves — without it the inline-link fallback fires)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
"""Press the chat page's Share pill and pin the outcome on the
live region (the assertion branches on the API's availability —
the test_share_chat.py convention)."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
field = page.locator(".share-link-fallback")
expect(field).to_be_visible()
href = field.get_attribute("href")
assert href is not None
assert href.startswith(app_url)
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
# --------------------------------------------------------------------------
# 1. D1/D2: the completed turn renders the 7-element arrival-order
# timeline (the runs are separate blocks, not one lump)
# --------------------------------------------------------------------------
def test_timeline_order_after_completed_turn(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# Phase 161 (D1): the container is the ONE pre-bubble anchor —
# the per-run blocks are no longer direct ``.msg-body`` children.
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
"the pre-bubble children must be exactly the reasoning "
"container (phase 161 D1)"
)
# D1: the container's ``.reasoning-body`` sequence is EXACTLY the
# 7-element arrival-order timeline — thinking, tool, ×3 (one block
# per contiguous run; the plain ``use your tools`` flow's 3 calls
# land in 3 SEPARATE one-line disclosures, one per thinking run).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the in-container segment sequence is not the timeline: {seq}"
)
# D1/D3: the four thinking blocks carry the DISTINCT ``Step N``
# markers — distinctness proves the runs are separate blocks, not
# one lump (the pre-phase single-block layout would put all four
# markers in one ``.thinking-text``).
texts = page.locator("#messages .msg.brain .thinking-text")
expect(texts).to_have_count(4)
for i, marker in enumerate(STEP_MARKERS):
expect(texts.nth(i)).to_contain_text(marker)
# D3: each tool run's disclosure summary is per-run — one line →
# "Tool call (1)" (the phase-117 singular form).
summaries = page.locator("#messages .msg.brain .tool-calls-summary")
expect(summaries).to_have_count(3)
for i in range(3):
expect(summaries.nth(i)).to_have_text("Tool call (1)")
# The answer bubble carries the ``Read <source/path>`` echo (the
# read document reached the model and landed in the answer).
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(ANSWER_PREFIX)
expect(bubble).to_contain_text(ANSWER_QUOTE)
# --------------------------------------------------------------------------
# 2. D5: the runs fold at rest (done settled every open block)
# --------------------------------------------------------------------------
def test_runs_fold_at_rest(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# Phase 161 (D2): the container folds at done — the at-rest
# visible line is its ``Reasoning`` summary (the D6 settled status
# is the run count — 7 runs → "7 steps").
container = page.locator("#messages .msg.brain details.reasoning")
expect(container).to_have_count(1)
expect(container).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-summary").first
).to_be_visible()
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
# D5 refinement (owner 2026-10-06): the inner runs are NOT
# collapsible — ZERO <details> inside the container (the
# per-run blocks are plain divs, always visible while the
# container is open).
expect(
page.locator("#messages .msg.brain details.reasoning details")
).to_have_count(0)
# The inner runs stay in the DOM inside the fold — opening the
# container (the native toggle) reveals the 4 "Thinking" + 3
# "Tool call (1)" labels (the timeline stays readable at rest;
# the D5 refinement — no second toggle: the blocks are always
# visible once the container opens).
page.locator("#messages .msg.brain .reasoning-summary").click()
expect(container).to_have_attribute("open", "")
expect(
page.locator("#messages .msg.brain .thinking .thinking-summary").first
).to_be_visible()
expect(
page.locator("#messages .msg.brain .tool-calls-summary").first
).to_be_visible()
# --------------------------------------------------------------------------
# 3. D5: the active run's block stays OPEN while the turn streams
# --------------------------------------------------------------------------
def test_live_runs_open_while_streaming(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
page.fill("#message-input", TIMELINE_QUESTION)
page.click("#send-btn")
expect(page.locator(".msg.user .bubble").last).to_contain_text(TIMELINE_QUESTION)
# D5 (the live half): during the turn — the mock's pacing window
# (the 0.1 s tool frames + the thinking stream; the answer bubble
# is still EMPTY — the first delta has not landed) — the active
# run's block is ``open`` (the live feedback the owner keeps:
# thinking/tool lines stream into their OPEN blocks, folded only
# by the first delta / done / stop). The house deadline-poll
# pattern (the ``test_agent_document_tools.py`` ``_frames``
# convention): a 50 ms stride over the ~2 s turn comfortably
# samples the open windows.
seen_live_open = False
deadline = time.monotonic() + 30.0
while time.monotonic() < deadline:
if page.evaluate(LIVE_OPEN_JS):
seen_live_open = True
break
time.sleep(0.05)
assert seen_live_open, (
"no open thinking/tool run was observed while the answer "
"bubble was still empty (the D5 live-open window)"
)
# The turn settles (the poll caught the live window; the settle is
# the D5 fold — test 2's contract).
_wait_settled(page)
# --------------------------------------------------------------------------
# 4. The restore path (the same 7-element timeline, collapsed) + the
# record's ``segments`` array (D4 — the legacy keys ride along)
# --------------------------------------------------------------------------
def test_restored_timeline_and_record(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# RELOAD (same context) — the phase-14 record restores through the
# restore path (restoreTimeline — the segments take precedence).
page.reload()
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
# Phase 161 (D5/D6): the restore renders ONE collapsed container
# around the timeline (the D6 at-rest step count). The pre-bubble
# children are exactly the container (the live pin, D1).
expect(
page.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
page.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
"the restored pre-bubble children must be exactly the "
"container (phase 161 D1)"
)
# The restored message renders the SAME 7-element timeline
# (in-container), all COLLAPSED (D5 — the restore renders
# collapsed).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the restored in-container sequence is not the timeline: {seq}"
)
expect(
page.locator("#messages .msg.brain details.reasoning details")
).to_have_count(0)
# The restored answer (the restore is synchronous at boot).
expect(
page.locator(".msg.brain .bubble").last
).to_contain_text(ANSWER_PREFIX, timeout=30_000)
# D4: the localStorage record (``bor.chat.v1`` — phase 14) carries
# a ``segments`` array of EXACTLY 7 entries in the arrival-order
# pattern — 4 thinking runs (the ``Step N`` text) + 3 tool entries
# (the call records, in order).
rec = page.evaluate(READ_RECORD_JS)
assert rec is not None, "the phase-14 record must exist after the reload"
segments = rec.get("segments")
assert isinstance(segments, list), (
f"the record must carry a segments array, got {type(segments)}"
)
assert len(segments) == 7, f"expected EXACTLY 7 segments, got {len(segments)}"
assert [s.get("kind") for s in segments] == [
"thinking", "tool", "thinking", "tool", "thinking", "tool", "thinking"
], f"the kind sequence is not the arrival-order pattern: {segments}"
# The thinking entries carry the DISTINCT ``Step N`` runs.
for i, marker in enumerate(STEP_MARKERS):
text = segments[i * 2].get("text")
assert isinstance(text, str) and marker in text, (
f"thinking segment {i} must carry {marker!r}: {text!r}"
)
# The tool entries are the 3 call records, in call order.
assert segments[1].get("name") == "ls"
assert segments[1].get("argument") is None # the no-arg top-level ls
assert segments[3].get("name") == "ls"
assert segments[3].get("argument") == READ_SOURCE # the drill scope
assert segments[5].get("name") == "read"
assert segments[5].get("argument") == READ_SP # the combined identity
# D4 (the old-client graceful degradation): the legacy fallback
# fields are STILL present — the full ``thinking`` concatenation
# (feeds the history wire + the per-turn log) + every ``tools``
# record (the old-client render path).
thinking_full = rec.get("thinking")
assert isinstance(thinking_full, str) and thinking_full, (
"the full thinking concatenation must still be present"
)
for marker in STEP_MARKERS:
assert marker in thinking_full, (
f"the concatenated thinking must carry {marker!r}"
)
tools = rec.get("tools")
assert isinstance(tools, list) and len(tools) == 3, (
f"the tools record must still carry the 3 calls: {tools!r}"
)
# --------------------------------------------------------------------------
# 5. The shared page (phase-51 save + share flow): the same 7-element
# timeline, collapsed (the addSegments parity)
# --------------------------------------------------------------------------
def test_shared_page_timeline(
page: Page,
browser: Browser,
app_url: str,
mock_llm: int,
db_ready: None,
) -> None:
_reset_db(_seed)
page.set_default_timeout(30_000)
login(page, app_url, next="/")
expect(page).to_have_url(app_url + "/", timeout=30_000)
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# The phase-51 flow: the conversation is AUTO-SAVED by the time the
# answer settles (phase 55) — the Share click shares the linked row.
cookies = _admin_cookies(page)
_grant_clipboard(page, app_url)
_wait_saved_row(app_url, cookies, _auto_title(TIMELINE_QUESTION))
share = page.locator("#share-chat-btn")
expect(share).to_be_visible()
_click_share_and_assert_status(page, app_url)
row = _find_row(_chats(app_url, cookies), _auto_title(TIMELINE_QUESTION))
assert row is not None and row.get("share_url"), (
"the shared row must carry a share_url"
)
share_url: str = row["share_url"]
assert SHARE_URL_RE.fullmatch(share_url), f"bad share_url shape: {share_url}"
created: str = row["id"]
anon_ctx: BrowserContext | None = None
try:
# A FRESH context: a separate session with no cookies — the
# guest's only credential is the token in the URL.
anon_ctx = browser.new_context()
anon = anon_ctx.new_page()
anon.set_default_timeout(30_000)
anon.goto(app_url + share_url)
# The title = the auto-title (the shared chat's h1).
expect(anon.locator("#shared-title")).to_have_text(_auto_title(TIMELINE_QUESTION))
# Phase 161 (D5/D6): the shared page renders ONE collapsed
# container (the D6 at-rest step count) around the timeline.
expect(
anon.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
anon.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
anon.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
# The SAME 7-element timeline renders inside it (the
# ``addSegments`` parity), all COLLAPSED (D5 — the shared
# page is a finished conversation; a closed <details> has no
# ``open`` attribute).
seq = anon.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the shared in-container sequence is not the timeline: {seq}"
)
# D5 refinement (owner 2026-10-06): the inner runs are NOT
# collapsible — ZERO <details> inside the container (the
# per-run blocks are plain divs).
expect(
anon.locator("#messages .msg.brain details.reasoning details")
).to_have_count(0)
# The ``Step N`` markers are visible on the container's summary
# click (the D5 refinement — ONE toggle: the inner blocks are
# always visible once the container opens; no second toggle).
anon.locator("#messages .msg.brain .reasoning-summary").click()
first_text = anon.locator("#messages .msg.brain .thinking-text").first
expect(first_text).to_be_visible()
expect(first_text).to_contain_text(STEP_MARKERS[0])
# The answer bubble carries the ``Read <source/path>`` echo.
expect(
anon.locator(".msg.brain .bubble").first
).to_contain_text(ANSWER_PREFIX)
finally:
if anon_ctx is not None:
anon_ctx.close()
if created:
_delete_chat(app_url, cookies, created)
# --------------------------------------------------------------------------
# 6. The run-grouping contract for no-thinking turns (phase 117
# preserved): one disclosure, 3 lines, zero thinking blocks
# --------------------------------------------------------------------------
def test_no_thinking_turn_single_disclosure(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
# The plain ``use your tools`` marker (NO thinking trigger — the
# mock's tool frames carry no ``reasoning_content``): the 3 calls
# are ONE contiguous tool run.
_submit(page, PLAIN_TOOLS_QUESTION)
_wait_settled(page)
# Phase 161 (D1/D2): the tool frames create the container — it
# exists and folds at done; the D6 settled status is the single
# tool run ("1 steps").
expect(
page.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
page.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("1 steps")
# D1 (the grouping contract): no thinking frames → ONE tool run →
# EXACTLY ONE disclosure INSIDE THE CONTAINER (the phase-117
# behavior for no-thinking turns is preserved, now wrapped), with
# the 3 ``.tool-call`` lines and the plural summary.
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-calls-disclosure"
)
).to_have_count(1)
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-call"
)
).to_have_count(3)
expect(
page.locator("#messages .msg.brain .tool-calls-summary")
).to_have_text("Tool calls (3)")
# And ZERO thinking blocks inside (no thinking ran — no empty
# block).
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
"details.thinking"
)
).to_have_count(0)
# Folded at rest (the first delta / done settle) + the answer.
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-calls-disclosure[open]"
)
).to_have_count(0)
expect(page.locator(".msg.brain .bubble").last).to_contain_text(ANSWER_PREFIX)