Files
brain-of-reese/tests/e2e/test_thinking_timeline.py
T
ducoterra 051f62deb7
Build and Push Containers / build-and-push-app (push) Successful in 1m55s
Build and Push Containers / build-and-push-db (push) Successful in 14s
phase: 161_reasoning_bubble
All gates verified. Final check — AGENTS.md rules against the project (commands, gates, protocols) — the quick-reference commands all exist and the rules (PLAN §2/§7/§9 references, phase protocol, No CDN, Debugpy, git gates) accurately describe the project; the reasoning bubble is a pure UI change with no new commands or env vars, so no AGENTS.md change is needed.

**Verification complete — full results:**

- **Verified** (implemented by prior task passes, all in working tree): `createReasoningWrap`/`ensureReasoningWrap` live container (D1), `settleReasoning` fold at the 3 settle sites + not at delta (D2), 320px `.reasoning-body` window + pinned-tail follow (D3), zero persistence change (D4), collapsed restore + shared-page parity with D6 step counts, `styles.css` family treatment + reduced-motion still, mock `REASONING_MIDROUND_TRIGGER` flow, 6-test story suite, phase-158 pin updates
- **Full suite:** `uv run pytest` → **3738 passed, 0 failed** · `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99% (>90%)** · `uv run ruff check .` → all checks passed · `uv run pyright` → **0 errors, 0 warnings**
- **E2E in isolation** (DB up; port-8123 collision with an unrelated `undercooked` service → `E2E_APP_PORT=8125`/`E2E_MOCK_PORT=8902`, no live change made): `test_reasoning_bubble.py` **6/6** · `test_thinking_timeline.py` ✓ · `test_thinking_display.py` ✓ · `test_tool_call_compact.py` ✓ · `test_big_read_progress.py` ✓ · `test_turn_progress_loader.py` ✓ · `test_share_chat.py` ✓ · `test_chat_persistence.py` ✓ · `test_smoke.py` ✓ (all exit 0, zero failures)
- **Completion criteria:** all 9 checked against the code + test outcomes — container open/live/nested ✓, stays open past mid-round delta ✓, folds at done ✓, restore + shared collapsed with "3 steps" ✓, plain turn = no container ✓, regression suites green ✓, suite/coverage/lint green ✓, diff scope limited to `frontend/assets/{app.js,shared.js,styles.css}` + `tests/` + `README.md` + phase files (no `app/` — D4) ✓
- **Documentation:** README.md "Thinking" section names the Reasoning bubble accurately (320px window, open-until-settle incl. mid-round deltas, "N steps", byte-identical no-container, collapsed on restore/shared) — verified consistent, no changes needed; AGENTS.md — verified consistent, no changes needed; all `.env.example` vars documented
- **Decisions:** commit + phase-dir move left to the harness (per pass rules — no git index/history commands executed); `undercooked` dev server on port 8123 untouched, E2E used the env port overrides
- **Next pending phase:** none — `161_reasoning_bubble` is the last in `todo/`
2026-10-06 06:36:09 -04:00

881 lines
36 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 158 E2E (Playwright, mock-only): the chronological thinking /
tool-call timeline (the owner's 2026-10-04 report — "the thinking block
happens first … and then went back to thinking. This is awkward. Then
the tool calling happens, and the thinking block reads strangely because
the tool calls are shown below the thinking block").
Source: owner report (live interactive-browser session, 2026-10-04,
shared page ``https://brain.reeseapps.com/shared/
95dd7a2a-8cda-478b-b694-53936070b9fb`` — a real ``turbo`` turn with 6
tool calls). Run in isolation (DB must be up:
``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate
is the deterministic marker flow in ``tests/e2e/mock_llm.py`` (user
message contains ``think out loud and use your tools``
(``THINK_TOOLS_TRIGGER``) **and** the system prompt carries the
``<tools>`` section of the HIGH prompt): the phase-37 single-read
shape (list → drill → read → the quoted answer) with a DISTINCT
``reasoning_content`` run streamed AHEAD of every tool call and ahead
of the final answer — the ``Step N`` markers make each run
individually assertable. The client renders each CONTIGUOUS run in its
own ``details.thinking`` / ``details.tool-calls-disclosure`` (D1), all
above the answer bubble in arrival order (D2 — Option B,
owner-chosen), open while the run's frames stream and folded at rest
(D5), and persists the run order in the record's ``segments`` array
(D4). Phase 161 (the documented supersession, owner TODO direction
2026-10-05): the per-run blocks are NESTED inside ONE
``details.reasoning`` container (``.reasoning-body``) that sits
before the bubble — open during the turn, folded at done / on
restore / on the shared page; the per-run open/fold behavior itself
is unchanged. The sequence pin is shape-unchanged, read from the
container's ``.reasoning-body`` children (the container is the one
new parent — the pre-bubble ``.msg-body`` children are exactly
``["details.reasoning"]``).
KB fixture — mirrored from ``test_agent_document_tools.py`` (the
deterministic two-document pair): ``Deployments`` registered FIRST so
the mock's drill (first source of the top-level listing) — and
therefore the read — lands on the JSON file; the marker question
cosines ≈0.595 against the retrievable document (well past the E2E
0.30 threshold) plus FTS hits, so the turn is solidly grounded (HIGH
prompt → ``<tools>``).
Test → phase mapping (Playwright Mapping Rule):
1. ``test_timeline_order_after_completed_turn`` — D1/D2: after ``done``
the pre-bubble ``.msg-body`` children are EXACTLY the reasoning
container (the one new parent — phase 161), and its
``.reasoning-body`` children are EXACTLY the 7-element
arrival-order sequence (4 thinking blocks + 3 one-line
disclosures, alternating); the four ``.thinking-text``s carry the
DISTINCT ``Step N`` markers (distinctness = separate blocks, not
one lump); each disclosure summary is ``Tool call (1)``; the
answer bubble carries the ``Read <source/path>`` echo.
2. ``test_runs_fold_at_rest`` — D5 (phase 161 D2): after ``done``
the container AND every ``.thinking`` /
``.tool-calls-disclosure`` inside it has NO ``open`` attribute
(the settle folded the runs + the container); the at-rest visible
line is the container's ``Reasoning`` summary (``7 steps``);
opening it reveals the folded run summaries.
3. ``test_live_runs_open_while_streaming`` — D5 (the live half):
DURING the turn (the mock's pacing window — the 0.1 s tool frames +
the thinking stream, the answer bubble still EMPTY) the active run's
block is ``open`` (the live feedback the owner keeps).
4. ``test_restored_timeline_and_record`` — the restore path + D4: a
same-context RELOAD re-renders the same 7-element sequence in ONE
collapsed reasoning container (``7 steps``, phase 161 D5/D6), and
the localStorage ``bor.chat.v1`` record (phase 14) carries a
``segments`` array of EXACTLY 7 entries in the
``thinking, tool, ×3`` pattern (the ``Step N`` text on the thinking
entries, the ``name``/``argument`` on the tool entries); the legacy
``thinking`` (full concatenation) + ``tools`` (3 records) keys are
STILL present (the old-client graceful degradation).
5. ``test_shared_page_timeline`` — the shared-page parity (phase 51
save + share flow): a FRESH context on ``/shared/<token>`` renders
the same 7-element sequence in ONE collapsed reasoning container
(``addSegments``), the ``Step N`` markers visible on container +
summary click (expand ONE each), the answer bubble carries the
``Read <source/path>`` echo.
6. ``test_no_thinking_turn_single_disclosure`` — the run-grouping
contract for no-thinking turns (the phase-117 behavior preserved,
now wrapped — phase 161): a plain ``use your tools`` turn (NO
thinking trigger) renders ONE folded reasoning container (``1
steps``) wrapping EXACTLY ONE ``.tool-calls-disclosure`` inside its
``.reasoning-body`` (3 ``.tool-call`` lines, summary ``Tool calls
(3)``) and ZERO ``.thinking``.
"""
from __future__ import annotations
import hashlib
import re
import time
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Any
import httpx
from playwright.sync_api import Browser, BrowserContext, Page, expect
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, GitSource
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
# --------------------------------------------------------------------------
# Fixture documents (deterministic, token-controlled — mirrored from
# test_agent_document_tools.py so the flow is byte-stable)
# --------------------------------------------------------------------------
SEED_SOURCE = "Homelab"
SEED_PATH = "aws-route53.md"
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
READ_SOURCE = "Deployments"
READ_PATH = "example-record-file.json"
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
#: The retrievable document: references the JSON file "for the exact
#: JSON shape of reeselink.json" but never includes it. The repeated
#: record-file lines carry the marker question's key tokens (aws,
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
#: ≈0.595 cosine against the mock's embeddings (E2E threshold 0.30)
#: plus FTS hits, so the turn is solidly grounded.
ROUTE53_CONTENT = (
"# AWS Route 53 Notes\n\n"
"## Record file\n\n"
+ (
"The aws route53 hosted zone for reeselink keeps every record in "
"reseelink.json — the exact JSON shape of reeselink.json is "
"documented in example-record-file.json.\n"
)
* 10
+ "\n## Sync job\n\n"
"A cron job pushes reeselink.json to the aws route53 hosted zone "
"every fifteen minutes; the diff is applied through the route53 api.\n"
)
#: The referenced document: the exact JSON shape. Its FIRST line is
#: longer than 80 chars, so the mock's first-80-chars quote content
#: part is newline-free (the rendered-text assertions below match it
#: after the date line). Pinned by the assert below.
RECORD_FILE_CONTENT = (
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
'JSON shape of reeselink.json",\n'
' "hosted_zone_id": "Z0RESEELINK01",\n'
' "record_sets": [\n'
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
" ]\n"
"}\n"
)
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
#: The phase-158 trigger question (``THINK_TOOLS_TRIGGER`` + the KB
#: topic — the ≈0.595 cosine keeps the turn grounded over the mock-
#: calibrated E2E threshold).
TIMELINE_QUESTION = (
"Think out loud and use your tools: what is the exact JSON shape of "
"reseelink.json for my aws route53 hosted zone?"
)
#: The NO-thinking control — the plain ``use your tools`` marker
#: (``test_agent_document_tools.py`` / ``test_tool_call_compact.py``'s
#: question, mirrored so the 3-call flow is the phase-37/117 one).
PLAIN_TOOLS_QUESTION = (
"Use your tools: what is the exact JSON shape of reeselink.json "
"for my aws route53 hosted zone?"
)
#: The mock's deterministic single-read answer —
#: "Read <source/path>. <first 80 chars of the read document>" (the
#: phase-37 shape). Phase 106 (D5): the read result's ``date:`` SECOND
#: line rides into the first-80-chars quote — the quote is the date
#: line (the fixture's fixed ``created_at`` UTC date part, 17 chars) +
#: the first 63 content chars (80 − 17).
ANSWER_PREFIX = f"Read {READ_SP}."
ANSWER_QUOTE = "date: 2024-06-15" + RECORD_FILE_CONTENT[:63]
#: The mock's four DISTINCT reasoning runs (the ``Step N`` markers —
#: their distinctness proves the runs are separate blocks, not one
#: lump).
STEP_MARKERS = ("Step 1", "Step 2", "Step 3", "Step 4")
#: The 7-element sequence the timeline renders in the reasoning
#: container's ``.reasoning-body`` children, in arrival order (D1:
#: alternating thinking/tool runs — one block per run; phase 161: the
#: sequence is shape-unchanged — the container is the one new parent)
#: — tests 1/4 (chat page) and 5 (shared page) pin it.
TIMELINE_SEQ = [
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
]
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
# --------------------------------------------------------------------------
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
# --------------------------------------------------------------------------
def _seed(db: Session) -> None:
"""The two-document pair (see the module docstring), plus the
phase-94 registry rows: both sources registered, ``Deployments``
FIRST — the mock's drill (first source of the top-level listing)
lands on the JSON file deterministically, independent of the
operator's ``BOR_GIT_SOURCES`` (a non-empty table ignores the env
fallback)."""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
full_path=f"/tmp/{SEED_PATH}",
title="AWS Route 53 Notes",
content=ROUTE53_CONTENT,
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
# (the mock's first-80-chars read quote carries the date line).
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
db.add(md)
db.flush()
# One chunk carrying the mock's own embedding → genuine token
# overlap between the marker question and this document.
db.add(
Chunk(
document_id=md.id,
position=0,
content=ROUTE53_CONTENT,
embedding=embed_text(ROUTE53_CONTENT),
)
)
# The referenced JSON: indexed, catalogued, readable — but NO
# chunks, so retrieval never puts it in context (the failure the
# tools fix).
db.add(
Document(
source=READ_SOURCE,
path=READ_PATH,
full_path=f"/tmp/{READ_PATH}",
title="Example Record File",
content=RECORD_FILE_CONTENT,
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
)
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""Truncate the KB (plus the prompt-shaping tables and the
phase-55 auto-save rows), then re-seed — the E2E isolation
pattern. ``steering_notes`` / ``kb_overview`` are truncated too,
so the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
``<tools>`` regardless of leftovers — byte-stable prompts,
byte-stable answers."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, "
"steering_notes, kb_overview, saved_chats, git_sources"
)
)
db.commit()
if seed is not None:
seed(db)
db.commit()
# --------------------------------------------------------------------------
# Page helpers
# --------------------------------------------------------------------------
#: The last ``.msg.brain`` wrap's ``.msg-body`` children BEFORE the
#: bubble, as ``tag.className`` strings. Phase 161 (D1): EXACTLY ONE
#: — the ``details.reasoning`` container (the one new anchor before
#: the bubble; the per-run blocks live inside it, not as direct
#: children; the ``.msg-meta`` row lands AFTER the bubble, so it
#: never enters the sequence).
PRE_BUBBLE_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return null;
const body = wrap.querySelector('.msg-body');
const out = [];
for (const child of body.children) {
if (child.classList.contains('bubble')) break;
out.push(child.tagName.toLowerCase() + '.' + child.className);
}
return out;
}
"""
#: The last ``.msg.brain`` wrap's reasoning container's
#: ``.reasoning-body`` children, as ``tag.className`` strings — the
#: timeline's order pin (phase 161: the 7-element arrival-order
#: sequence nested in the container — shape unchanged from phase 158).
BODY_SEQUENCE_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return null;
const body = wrap.querySelector('details.reasoning .reasoning-body');
if (!body) return null;
const out = [];
for (const child of body.children) {
out.push(child.tagName.toLowerCase() + '.' + child.className);
}
return out;
}
"""
#: D5's live-open predicate — the active run's block is ``open`` while
#: the answer bubble is STILL EMPTY (the turn is in flight; the mock's
#: pacing window — the 0.1 s tool frames + the thinking stream — keeps
#: the window comfortably open for a 100 ms poll).
LIVE_OPEN_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return false;
const bubble = wrap.querySelector('.msg-body .bubble');
if (!bubble || bubble.textContent !== '') return false;
return !!wrap.querySelector(
'details.thinking[open], details.tool-calls-disclosure[open]');
}
"""
#: The phase-14 localStorage record's LAST brain message (the
#: ``bor.chat.v1`` — ``{ v: 1, chatId, messages: [...] }``; the
#: filter mirrors ``loadStoredRecord``'s well-formedness).
READ_RECORD_JS = """
() => {
const raw = localStorage.getItem('bor.chat.v1');
if (!raw) return null;
const data = JSON.parse(raw);
const brain = (data.messages || []).filter(
(m) => m && m.who === 'brain' && typeof m.text === 'string');
return brain.length ? brain[brain.length - 1] : null;
}
"""
def _submit(page: Page, question: str) -> None:
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (phase 48 — the in-flight button is the enabled Stop
control, so the label assertion carries the settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# --------------------------------------------------------------------------
# Phase-51 save + share flow (the test_share_chat.py pattern — the
# auto-saved row is shared from the chat page; the row is deleted in a
# finally)
# --------------------------------------------------------------------------
def _admin_cookies(page: Page) -> dict[str, str]:
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["chats"]
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
return next((c for c in rows if c["title"] == title), None)
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55 — auto-saves are SILENT,
so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
last = _find_row(_chats(app_url, cookies), title)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(headless Chromium requires the grant before ``writeText``
resolves — without it the inline-link fallback fires)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
"""Press the chat page's Share pill and pin the outcome on the
live region (the assertion branches on the API's availability —
the test_share_chat.py convention)."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
field = page.locator(".share-link-fallback")
expect(field).to_be_visible()
href = field.get_attribute("href")
assert href is not None
assert href.startswith(app_url)
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
# --------------------------------------------------------------------------
# 1. D1/D2: the completed turn renders the 7-element arrival-order
# timeline (the runs are separate blocks, not one lump)
# --------------------------------------------------------------------------
def test_timeline_order_after_completed_turn(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# Phase 161 (D1): the container is the ONE pre-bubble anchor —
# the per-run blocks are no longer direct ``.msg-body`` children.
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
"the pre-bubble children must be exactly the reasoning "
"container (phase 161 D1)"
)
# D1: the container's ``.reasoning-body`` sequence is EXACTLY the
# 7-element arrival-order timeline — thinking, tool, ×3 (one block
# per contiguous run; the plain ``use your tools`` flow's 3 calls
# land in 3 SEPARATE one-line disclosures, one per thinking run).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the in-container segment sequence is not the timeline: {seq}"
)
# D1/D3: the four thinking blocks carry the DISTINCT ``Step N``
# markers — distinctness proves the runs are separate blocks, not
# one lump (the pre-phase single-block layout would put all four
# markers in one ``.thinking-text``).
texts = page.locator("#messages .msg.brain .thinking-text")
expect(texts).to_have_count(4)
for i, marker in enumerate(STEP_MARKERS):
expect(texts.nth(i)).to_contain_text(marker)
# D3: each tool run's disclosure summary is per-run — one line →
# "Tool call (1)" (the phase-117 singular form).
summaries = page.locator("#messages .msg.brain .tool-calls-summary")
expect(summaries).to_have_count(3)
for i in range(3):
expect(summaries.nth(i)).to_have_text("Tool call (1)")
# The answer bubble carries the ``Read <source/path>`` echo (the
# read document reached the model and landed in the answer).
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(ANSWER_PREFIX)
expect(bubble).to_contain_text(ANSWER_QUOTE)
# --------------------------------------------------------------------------
# 2. D5: the runs fold at rest (done settled every open block)
# --------------------------------------------------------------------------
def test_runs_fold_at_rest(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# Phase 161 (D2): the container folds at done — the at-rest
# visible line is its ``Reasoning`` summary (the D6 settled status
# is the run count — 7 runs → "7 steps").
container = page.locator("#messages .msg.brain details.reasoning")
expect(container).to_have_count(1)
expect(container).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-summary").first
).to_be_visible()
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
# D5: at rest NOTHING inside is open — the ``done`` settle
# (closeAllRuns) folded every run (a closed <details> has no
# ``open`` attribute).
expect(
page.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
page.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The inner runs stay in the DOM inside the fold — opening the
# container (the native toggle) reveals the 4 "Thinking" + 3
# "Tool call (1)" lines (the timeline stays readable at rest).
page.locator("#messages .msg.brain .reasoning-summary").click()
expect(container).to_have_attribute("open", "")
expect(
page.locator("#messages .msg.brain details.thinking > summary").first
).to_be_visible()
expect(
page.locator("#messages .msg.brain .tool-calls-summary").first
).to_be_visible()
# --------------------------------------------------------------------------
# 3. D5: the active run's block stays OPEN while the turn streams
# --------------------------------------------------------------------------
def test_live_runs_open_while_streaming(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
page.fill("#message-input", TIMELINE_QUESTION)
page.click("#send-btn")
expect(page.locator(".msg.user .bubble").last).to_contain_text(TIMELINE_QUESTION)
# D5 (the live half): during the turn — the mock's pacing window
# (the 0.1 s tool frames + the thinking stream; the answer bubble
# is still EMPTY — the first delta has not landed) — the active
# run's block is ``open`` (the live feedback the owner keeps:
# thinking/tool lines stream into their OPEN blocks, folded only
# by the first delta / done / stop). The house deadline-poll
# pattern (the ``test_agent_document_tools.py`` ``_frames``
# convention): a 50 ms stride over the ~2 s turn comfortably
# samples the open windows.
seen_live_open = False
deadline = time.monotonic() + 30.0
while time.monotonic() < deadline:
if page.evaluate(LIVE_OPEN_JS):
seen_live_open = True
break
time.sleep(0.05)
assert seen_live_open, (
"no open thinking/tool run was observed while the answer "
"bubble was still empty (the D5 live-open window)"
)
# The turn settles (the poll caught the live window; the settle is
# the D5 fold — test 2's contract).
_wait_settled(page)
# --------------------------------------------------------------------------
# 4. The restore path (the same 7-element timeline, collapsed) + the
# record's ``segments`` array (D4 — the legacy keys ride along)
# --------------------------------------------------------------------------
def test_restored_timeline_and_record(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# RELOAD (same context) — the phase-14 record restores through the
# restore path (restoreTimeline — the segments take precedence).
page.reload()
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
# Phase 161 (D5/D6): the restore renders ONE collapsed container
# around the timeline (the D6 at-rest step count). The pre-bubble
# children are exactly the container (the live pin, D1).
expect(
page.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
page.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
"the restored pre-bubble children must be exactly the "
"container (phase 161 D1)"
)
# The restored message renders the SAME 7-element timeline
# (in-container), all COLLAPSED (D5 — the restore renders
# collapsed).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the restored in-container sequence is not the timeline: {seq}"
)
expect(
page.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
page.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The restored answer (the restore is synchronous at boot).
expect(
page.locator(".msg.brain .bubble").last
).to_contain_text(ANSWER_PREFIX, timeout=30_000)
# D4: the localStorage record (``bor.chat.v1`` — phase 14) carries
# a ``segments`` array of EXACTLY 7 entries in the arrival-order
# pattern — 4 thinking runs (the ``Step N`` text) + 3 tool entries
# (the call records, in order).
rec = page.evaluate(READ_RECORD_JS)
assert rec is not None, "the phase-14 record must exist after the reload"
segments = rec.get("segments")
assert isinstance(segments, list), (
f"the record must carry a segments array, got {type(segments)}"
)
assert len(segments) == 7, f"expected EXACTLY 7 segments, got {len(segments)}"
assert [s.get("kind") for s in segments] == [
"thinking", "tool", "thinking", "tool", "thinking", "tool", "thinking"
], f"the kind sequence is not the arrival-order pattern: {segments}"
# The thinking entries carry the DISTINCT ``Step N`` runs.
for i, marker in enumerate(STEP_MARKERS):
text = segments[i * 2].get("text")
assert isinstance(text, str) and marker in text, (
f"thinking segment {i} must carry {marker!r}: {text!r}"
)
# The tool entries are the 3 call records, in call order.
assert segments[1].get("name") == "ls"
assert segments[1].get("argument") is None # the no-arg top-level ls
assert segments[3].get("name") == "ls"
assert segments[3].get("argument") == READ_SOURCE # the drill scope
assert segments[5].get("name") == "read"
assert segments[5].get("argument") == READ_SP # the combined identity
# D4 (the old-client graceful degradation): the legacy fallback
# fields are STILL present — the full ``thinking`` concatenation
# (feeds the history wire + the per-turn log) + every ``tools``
# record (the old-client render path).
thinking_full = rec.get("thinking")
assert isinstance(thinking_full, str) and thinking_full, (
"the full thinking concatenation must still be present"
)
for marker in STEP_MARKERS:
assert marker in thinking_full, (
f"the concatenated thinking must carry {marker!r}"
)
tools = rec.get("tools")
assert isinstance(tools, list) and len(tools) == 3, (
f"the tools record must still carry the 3 calls: {tools!r}"
)
# --------------------------------------------------------------------------
# 5. The shared page (phase-51 save + share flow): the same 7-element
# timeline, collapsed (the addSegments parity)
# --------------------------------------------------------------------------
def test_shared_page_timeline(
page: Page,
browser: Browser,
app_url: str,
mock_llm: int,
db_ready: None,
) -> None:
_reset_db(_seed)
page.set_default_timeout(30_000)
login(page, app_url, next="/")
expect(page).to_have_url(app_url + "/", timeout=30_000)
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# The phase-51 flow: the conversation is AUTO-SAVED by the time the
# answer settles (phase 55) — the Share click shares the linked row.
cookies = _admin_cookies(page)
_grant_clipboard(page, app_url)
_wait_saved_row(app_url, cookies, _auto_title(TIMELINE_QUESTION))
share = page.locator("#share-chat-btn")
expect(share).to_be_visible()
_click_share_and_assert_status(page, app_url)
row = _find_row(_chats(app_url, cookies), _auto_title(TIMELINE_QUESTION))
assert row is not None and row.get("share_url"), (
"the shared row must carry a share_url"
)
share_url: str = row["share_url"]
assert SHARE_URL_RE.fullmatch(share_url), f"bad share_url shape: {share_url}"
created: str = row["id"]
anon_ctx: BrowserContext | None = None
try:
# A FRESH context: a separate session with no cookies — the
# guest's only credential is the token in the URL.
anon_ctx = browser.new_context()
anon = anon_ctx.new_page()
anon.set_default_timeout(30_000)
anon.goto(app_url + share_url)
# The title = the auto-title (the shared chat's h1).
expect(anon.locator("#shared-title")).to_have_text(_auto_title(TIMELINE_QUESTION))
# Phase 161 (D5/D6): the shared page renders ONE collapsed
# container (the D6 at-rest step count) around the timeline.
expect(
anon.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
anon.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
anon.locator("#messages .msg.brain .reasoning-status")
).to_have_text("7 steps")
# The SAME 7-element timeline renders inside it (the
# ``addSegments`` parity), all COLLAPSED (D5 — the shared
# page is a finished conversation; a closed <details> has no
# ``open`` attribute).
seq = anon.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the shared in-container sequence is not the timeline: {seq}"
)
expect(
anon.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
anon.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The ``Step N`` markers are visible on a summary click (expand
# ONE — the native <summary> is a real focusable toggle; phase
# 161: the container's own summary first — the inner runs'
# summaries sit inside its closed fold).
anon.locator("#messages .msg.brain .reasoning-summary").click()
anon.locator("#messages .msg.brain details.thinking > summary").first.click()
expect(
anon.locator("#messages .msg.brain details.thinking").first
).to_have_attribute("open")
first_text = anon.locator("#messages .msg.brain .thinking-text").first
expect(first_text).to_be_visible()
expect(first_text).to_contain_text(STEP_MARKERS[0])
# The answer bubble carries the ``Read <source/path>`` echo.
expect(
anon.locator(".msg.brain .bubble").first
).to_contain_text(ANSWER_PREFIX)
finally:
if anon_ctx is not None:
anon_ctx.close()
if created:
_delete_chat(app_url, cookies, created)
# --------------------------------------------------------------------------
# 6. The run-grouping contract for no-thinking turns (phase 117
# preserved): one disclosure, 3 lines, zero thinking blocks
# --------------------------------------------------------------------------
def test_no_thinking_turn_single_disclosure(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
# The plain ``use your tools`` marker (NO thinking trigger — the
# mock's tool frames carry no ``reasoning_content``): the 3 calls
# are ONE contiguous tool run.
_submit(page, PLAIN_TOOLS_QUESTION)
_wait_settled(page)
# Phase 161 (D1/D2): the tool frames create the container — it
# exists and folds at done; the D6 settled status is the single
# tool run ("1 steps").
expect(
page.locator("#messages .msg.brain details.reasoning")
).to_have_count(1)
expect(
page.locator("#messages .msg.brain details.reasoning")
).not_to_have_attribute("open")
expect(
page.locator("#messages .msg.brain .reasoning-status")
).to_have_text("1 steps")
# D1 (the grouping contract): no thinking frames → ONE tool run →
# EXACTLY ONE disclosure INSIDE THE CONTAINER (the phase-117
# behavior for no-thinking turns is preserved, now wrapped), with
# the 3 ``.tool-call`` lines and the plural summary.
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-calls-disclosure"
)
).to_have_count(1)
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-call"
)
).to_have_count(3)
expect(
page.locator("#messages .msg.brain .tool-calls-summary")
).to_have_text("Tool calls (3)")
# And ZERO thinking blocks inside (no thinking ran — no empty
# block).
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
"details.thinking"
)
).to_have_count(0)
# Folded at rest (the first delta / done settle) + the answer.
expect(
page.locator(
"#messages .msg.brain details.reasoning .reasoning-body "
".tool-calls-disclosure[open]"
)
).to_have_count(0)
expect(page.locator(".msg.brain .bubble").last).to_contain_text(ANSWER_PREFIX)