Files
brain-of-reese/tests/e2e/test_thinking_timeline.py
T
ducoterra 31bd903a93
Build and Push Containers / build-and-push-app (push) Successful in 8m38s
Build and Push Containers / build-and-push-db (push) Successful in 32s
phase: 158_thinking_timeline
All verification complete. Every gate is green and the docs check out. Final report:

## Phase 158 final verification pass — COMPLETE

**Verified (all tasks pre-completed; this pass re-ran the full gate):**
- Unit + integration: `uv run pytest --cov=app --cov-report=term-missing` → **3618 passed, 99% coverage on `app/`** (>90% gate)
- Phase E2E in isolation: `uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov` → **6/6 passed** (order pin, fold-at-rest, live-open, restore+`segments` array, shared page, no-thinking single disclosure)
- Regression E2E, each in isolation: `test_thinking_display` 5✓, `test_tool_call_compact` 4✓, `test_turn_progress_loader` 3✓ (per-run pin updated), `test_big_read_progress` 4✓, `test_agent_document_tools` 4✓, `test_share_chat` 4✓, `test_chat_persistence` 4✓, `test_smoke` 3✓
- Lint/types: `uv run ruff check . && uv run pyright` → **clean (0 errors)**

**Completion criteria:** all 9 met — timeline E2E pins ✓, restore/shared `segments` ✓, no-thinking single-disclosure + the three UNCHANGED regression suites green ✓, pre-phase byte-identical round-trip (integration pin: key absent, never null; >50 / >32 000 / empty text / unknown kind all 422) ✓, full suite + coverage + E2E + lint ✓, `git diff --stat` footprint limited to `app/schemas.py`, `frontend/assets/{app,shared}.js`, `tests/`, README, phase files (a pre-existing owner `TODO.md` edit is out of scope, not staged by the phase commit) ✓, D15→per-run supersession recorded in the updated pin ✓. (Commit + phase move are the harness's job; task dirs already in `complete/`.)

**Documentation:** README.md — Thinking section correctly describes the phase-158 timeline; config reference matches `.env.example` exactly; page table matches registered routes; no stale single-block wording. AGENTS.md — rules and quick-reference commands all accurate. **Verified consistent — no changes needed.**

**Decisions/deviations:** none. `test_tool_call_compact.py` carries a one-line docstring-only update (referenced the removed `closeToolCalls`); its test logic is byte-identical to the phase-117 contract and green.

**Next pending phase:** none — `todo/` contains only 158.
2026-10-05 02:52:55 -04:00

758 lines
31 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 158 E2E (Playwright, mock-only): the chronological thinking /
tool-call timeline (the owner's 2026-10-04 report — "the thinking block
happens first … and then went back to thinking. This is awkward. Then
the tool calling happens, and the thinking block reads strangely because
the tool calls are shown below the thinking block").
Source: owner report (live interactive-browser session, 2026-10-04,
shared page ``https://brain.reeseapps.com/shared/
95dd7a2a-8cda-478b-b694-53936070b9fb`` — a real ``turbo`` turn with 6
tool calls). Run in isolation (DB must be up:
``podman compose up -d db``):
uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate
is the deterministic marker flow in ``tests/e2e/mock_llm.py`` (user
message contains ``think out loud and use your tools``
(``THINK_TOOLS_TRIGGER``) **and** the system prompt carries the
``<tools>`` section of the HIGH prompt): the phase-37 single-read
shape (list → drill → read → the quoted answer) with a DISTINCT
``reasoning_content`` run streamed AHEAD of every tool call and ahead
of the final answer — the ``Step N`` markers make each run
individually assertable. The client renders each CONTIGUOUS run in its
own ``details.thinking`` / ``details.tool-calls-disclosure`` (D1), all
above the answer bubble in arrival order (D2 — Option B,
owner-chosen), open while the run's frames stream and folded at rest
(D5), and persists the run order in the record's ``segments`` array
(D4).
KB fixture — mirrored from ``test_agent_document_tools.py`` (the
deterministic two-document pair): ``Deployments`` registered FIRST so
the mock's drill (first source of the top-level listing) — and
therefore the read — lands on the JSON file; the marker question
cosines ≈0.595 against the retrievable document (well past the E2E
0.30 threshold) plus FTS hits, so the turn is solidly grounded (HIGH
prompt → ``<tools>``).
Test → phase mapping (Playwright Mapping Rule):
1. ``test_timeline_order_after_completed_turn`` — D1/D2: after ``done``
the brain wrap's ``.msg-body`` children before the bubble are
EXACTLY the 7-element arrival-order sequence (4 thinking blocks +
3 one-line disclosures, alternating); the four ``.thinking-text``s
carry the DISTINCT ``Step N`` markers (distinctness = separate
blocks, not one lump); each disclosure summary is ``Tool call
(1)``; the answer bubble carries the ``Read <source/path>`` echo.
2. ``test_runs_fold_at_rest`` — D5: after ``done`` NO ``.thinking`` /
``.tool-calls-disclosure`` has the ``open`` attribute (the settle
folded the runs); the summaries are still visible.
3. ``test_live_runs_open_while_streaming`` — D5 (the live half):
DURING the turn (the mock's pacing window — the 0.1 s tool frames +
the thinking stream, the answer bubble still EMPTY) the active run's
block is ``open`` (the live feedback the owner keeps).
4. ``test_restored_timeline_and_record`` — the restore path + D4: a
same-context RELOAD re-renders the same 7-element sequence (all
collapsed), and the localStorage ``bor.chat.v1`` record (phase 14)
carries a ``segments`` array of EXACTLY 7 entries in the
``thinking, tool, ×3`` pattern (the ``Step N`` text on the thinking
entries, the ``name``/``argument`` on the tool entries); the legacy
``thinking`` (full concatenation) + ``tools`` (3 records) keys are
STILL present (the old-client graceful degradation).
5. ``test_shared_page_timeline`` — the shared-page parity (phase 51
save + share flow): a FRESH context on ``/shared/<token>`` renders
the same 7-element sequence, all collapsed (``addSegments``), the
``Step N`` markers visible on summary click (expand ONE), the answer
bubble carries the ``Read <source/path>`` echo.
6. ``test_no_thinking_turn_single_disclosure`` — the run-grouping
contract for no-thinking turns (the phase-117 behavior preserved): a
plain ``use your tools`` turn (NO thinking trigger) renders EXACTLY
ONE ``.tool-calls-disclosure`` (3 ``.tool-call`` lines, summary
``Tool calls (3)``) and ZERO ``.thinking``.
"""
from __future__ import annotations
import hashlib
import re
import time
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Any
import httpx
from playwright.sync_api import Browser, BrowserContext, Page, expect
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, GitSource
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
# --------------------------------------------------------------------------
# Fixture documents (deterministic, token-controlled — mirrored from
# test_agent_document_tools.py so the flow is byte-stable)
# --------------------------------------------------------------------------
SEED_SOURCE = "Homelab"
SEED_PATH = "aws-route53.md"
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
READ_SOURCE = "Deployments"
READ_PATH = "example-record-file.json"
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
#: The retrievable document: references the JSON file "for the exact
#: JSON shape of reeselink.json" but never includes it. The repeated
#: record-file lines carry the marker question's key tokens (aws,
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
#: ≈0.595 cosine against the mock's embeddings (E2E threshold 0.30)
#: plus FTS hits, so the turn is solidly grounded.
ROUTE53_CONTENT = (
"# AWS Route 53 Notes\n\n"
"## Record file\n\n"
+ (
"The aws route53 hosted zone for reeselink keeps every record in "
"reseelink.json — the exact JSON shape of reeselink.json is "
"documented in example-record-file.json.\n"
)
* 10
+ "\n## Sync job\n\n"
"A cron job pushes reeselink.json to the aws route53 hosted zone "
"every fifteen minutes; the diff is applied through the route53 api.\n"
)
#: The referenced document: the exact JSON shape. Its FIRST line is
#: longer than 80 chars, so the mock's first-80-chars quote content
#: part is newline-free (the rendered-text assertions below match it
#: after the date line). Pinned by the assert below.
RECORD_FILE_CONTENT = (
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
'JSON shape of reeselink.json",\n'
' "hosted_zone_id": "Z0RESEELINK01",\n'
' "record_sets": [\n'
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
" ]\n"
"}\n"
)
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
#: The phase-158 trigger question (``THINK_TOOLS_TRIGGER`` + the KB
#: topic — the ≈0.595 cosine keeps the turn grounded over the mock-
#: calibrated E2E threshold).
TIMELINE_QUESTION = (
"Think out loud and use your tools: what is the exact JSON shape of "
"reseelink.json for my aws route53 hosted zone?"
)
#: The NO-thinking control — the plain ``use your tools`` marker
#: (``test_agent_document_tools.py`` / ``test_tool_call_compact.py``'s
#: question, mirrored so the 3-call flow is the phase-37/117 one).
PLAIN_TOOLS_QUESTION = (
"Use your tools: what is the exact JSON shape of reeselink.json "
"for my aws route53 hosted zone?"
)
#: The mock's deterministic single-read answer —
#: "Read <source/path>. <first 80 chars of the read document>" (the
#: phase-37 shape). Phase 106 (D5): the read result's ``date:`` SECOND
#: line rides into the first-80-chars quote — the quote is the date
#: line (the fixture's fixed ``created_at`` UTC date part, 17 chars) +
#: the first 63 content chars (80 − 17).
ANSWER_PREFIX = f"Read {READ_SP}."
ANSWER_QUOTE = "date: 2024-06-15" + RECORD_FILE_CONTENT[:63]
#: The mock's four DISTINCT reasoning runs (the ``Step N`` markers —
#: their distinctness proves the runs are separate blocks, not one
#: lump).
STEP_MARKERS = ("Step 1", "Step 2", "Step 3", "Step 4")
#: The 7-element pre-bubble segment sequence the timeline renders, in
#: arrival order (D1: alternating thinking/tool runs — one block per
#: run) — tests 1/4 (chat page) and 5 (shared page) pin it.
TIMELINE_SEQ = [
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
"details.tool-calls-disclosure",
"details.thinking",
]
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
# --------------------------------------------------------------------------
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
# --------------------------------------------------------------------------
def _seed(db: Session) -> None:
"""The two-document pair (see the module docstring), plus the
phase-94 registry rows: both sources registered, ``Deployments``
FIRST — the mock's drill (first source of the top-level listing)
lands on the JSON file deterministically, independent of the
operator's ``BOR_GIT_SOURCES`` (a non-empty table ignores the env
fallback)."""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
full_path=f"/tmp/{SEED_PATH}",
title="AWS Route 53 Notes",
content=ROUTE53_CONTENT,
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
# (the mock's first-80-chars read quote carries the date line).
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
db.add(md)
db.flush()
# One chunk carrying the mock's own embedding → genuine token
# overlap between the marker question and this document.
db.add(
Chunk(
document_id=md.id,
position=0,
content=ROUTE53_CONTENT,
embedding=embed_text(ROUTE53_CONTENT),
)
)
# The referenced JSON: indexed, catalogued, readable — but NO
# chunks, so retrieval never puts it in context (the failure the
# tools fix).
db.add(
Document(
source=READ_SOURCE,
path=READ_PATH,
full_path=f"/tmp/{READ_PATH}",
title="Example Record File",
content=RECORD_FILE_CONTENT,
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
)
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""Truncate the KB (plus the prompt-shaping tables and the
phase-55 auto-save rows), then re-seed — the E2E isolation
pattern. ``steering_notes`` / ``kb_overview`` are truncated too,
so the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
``<tools>`` regardless of leftovers — byte-stable prompts,
byte-stable answers."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, "
"steering_notes, kb_overview, saved_chats, git_sources"
)
)
db.commit()
if seed is not None:
seed(db)
db.commit()
# --------------------------------------------------------------------------
# Page helpers
# --------------------------------------------------------------------------
#: The last ``.msg.brain`` wrap's ``.msg-body`` children BEFORE the
#: bubble, as ``tag.className`` strings — the pre-bubble segment
#: sequence (the timeline's order pin; the ``.msg-meta`` row lands
#: AFTER the bubble, so it never enters the sequence).
BODY_SEQUENCE_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return null;
const body = wrap.querySelector('.msg-body');
const out = [];
for (const child of body.children) {
if (child.classList.contains('bubble')) break;
out.push(child.tagName.toLowerCase() + '.' + child.className);
}
return out;
}
"""
#: D5's live-open predicate — the active run's block is ``open`` while
#: the answer bubble is STILL EMPTY (the turn is in flight; the mock's
#: pacing window — the 0.1 s tool frames + the thinking stream — keeps
#: the window comfortably open for a 100 ms poll).
LIVE_OPEN_JS = """
() => {
const wrap = document.querySelector('#messages .msg.brain');
if (!wrap) return false;
const bubble = wrap.querySelector('.msg-body .bubble');
if (!bubble || bubble.textContent !== '') return false;
return !!wrap.querySelector(
'details.thinking[open], details.tool-calls-disclosure[open]');
}
"""
#: The phase-14 localStorage record's LAST brain message (the
#: ``bor.chat.v1`` — ``{ v: 1, chatId, messages: [...] }``; the
#: filter mirrors ``loadStoredRecord``'s well-formedness).
READ_RECORD_JS = """
() => {
const raw = localStorage.getItem('bor.chat.v1');
if (!raw) return null;
const data = JSON.parse(raw);
const brain = (data.messages || []).filter(
(m) => m && m.who === 'brain' && typeof m.text === 'string');
return brain.length ? brain[brain.length - 1] : null;
}
"""
def _submit(page: Page, question: str) -> None:
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (phase 48 — the in-flight button is the enabled Stop
control, so the label assertion carries the settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# --------------------------------------------------------------------------
# Phase-51 save + share flow (the test_share_chat.py pattern — the
# auto-saved row is shared from the chat page; the row is deleted in a
# finally)
# --------------------------------------------------------------------------
def _admin_cookies(page: Page) -> dict[str, str]:
return {
c["name"]: c["value"]
for c in page.context.cookies()
if "name" in c and "value" in c
}
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
assert r.status_code == 200
return r.json()["chats"]
def _auto_title(question: str) -> str:
"""The phase-50 auto-title convention: the first question,
whitespace-collapsed, capped at 120 chars."""
return " ".join(question.split())[:120]
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
return next((c for c in rows if c["title"] == title), None)
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
def _wait_saved_row(
app_url: str,
cookies: dict[str, str],
title: str,
messages: int = 2,
) -> dict[str, Any]:
"""Wait for the auto-saved row (phase 55 — auto-saves are SILENT,
so there is no status line to wait on). The upsert is
fire-and-forget from the UI's point of view, so poll the admin
list until the row with the conversation's auto-title appears with
the expected message count."""
deadline = time.monotonic() + 15
last: dict[str, Any] | None = None
while time.monotonic() < deadline:
last = _find_row(_chats(app_url, cookies), title)
if last is not None and last["message_count"] >= messages:
return last
time.sleep(0.2)
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
def _grant_clipboard(page: Page, app_url: str) -> None:
"""Grant the async-clipboard permissions on the admin context
(headless Chromium requires the grant before ``writeText``
resolves — without it the inline-link fallback fires)."""
page.context.grant_permissions(
["clipboard-read", "clipboard-write"], origin=app_url
)
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
"""Press the chat page's Share pill and pin the outcome on the
live region (the assertion branches on the API's availability —
the test_share_chat.py convention)."""
page.locator("#share-chat-btn").click()
if page.evaluate("() => !!navigator.clipboard"):
expect(page.locator("#send-status")).to_have_text(
"Share link copied.", timeout=15_000
)
else:
expect(page.locator("#send-status")).to_have_text(
"Share link ready — copy it from the field.", timeout=15_000
)
field = page.locator(".share-link-fallback")
expect(field).to_be_visible()
href = field.get_attribute("href")
assert href is not None
assert href.startswith(app_url)
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
# --------------------------------------------------------------------------
# 1. D1/D2: the completed turn renders the 7-element arrival-order
# timeline (the runs are separate blocks, not one lump)
# --------------------------------------------------------------------------
def test_timeline_order_after_completed_turn(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# D1: the pre-bubble segment sequence is EXACTLY the 7-element
# arrival-order timeline — thinking, tool, ×3 (one block per
# contiguous run; the plain ``use your tools`` flow's 3 calls land
# in 3 SEPARATE one-line disclosures, one per thinking run).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the pre-bubble segment sequence is not the timeline: {seq}"
)
# D1/D3: the four thinking blocks carry the DISTINCT ``Step N``
# markers — distinctness proves the runs are separate blocks, not
# one lump (the pre-phase single-block layout would put all four
# markers in one ``.thinking-text``).
texts = page.locator("#messages .msg.brain .thinking-text")
expect(texts).to_have_count(4)
for i, marker in enumerate(STEP_MARKERS):
expect(texts.nth(i)).to_contain_text(marker)
# D3: each tool run's disclosure summary is per-run — one line →
# "Tool call (1)" (the phase-117 singular form).
summaries = page.locator("#messages .msg.brain .tool-calls-summary")
expect(summaries).to_have_count(3)
for i in range(3):
expect(summaries.nth(i)).to_have_text("Tool call (1)")
# The answer bubble carries the ``Read <source/path>`` echo (the
# read document reached the model and landed in the answer).
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(ANSWER_PREFIX)
expect(bubble).to_contain_text(ANSWER_QUOTE)
# --------------------------------------------------------------------------
# 2. D5: the runs fold at rest (done settled every open block)
# --------------------------------------------------------------------------
def test_runs_fold_at_rest(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# D5: at rest NOTHING is open — the ``done`` settle (closeAllRuns)
# folded every run (a closed <details> has no ``open`` attribute).
expect(
page.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
page.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The collapsed summaries are still visible (the timeline stays
# readable at rest — 4 "Thinking" + 3 "Tool call (1)" lines).
expect(
page.locator("#messages .msg.brain details.thinking > summary").first
).to_be_visible()
expect(
page.locator("#messages .msg.brain .tool-calls-summary").first
).to_be_visible()
# --------------------------------------------------------------------------
# 3. D5: the active run's block stays OPEN while the turn streams
# --------------------------------------------------------------------------
def test_live_runs_open_while_streaming(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
page.fill("#message-input", TIMELINE_QUESTION)
page.click("#send-btn")
expect(page.locator(".msg.user .bubble").last).to_contain_text(TIMELINE_QUESTION)
# D5 (the live half): during the turn — the mock's pacing window
# (the 0.1 s tool frames + the thinking stream; the answer bubble
# is still EMPTY — the first delta has not landed) — the active
# run's block is ``open`` (the live feedback the owner keeps:
# thinking/tool lines stream into their OPEN blocks, folded only
# by the first delta / done / stop). The house deadline-poll
# pattern (the ``test_agent_document_tools.py`` ``_frames``
# convention): a 50 ms stride over the ~2 s turn comfortably
# samples the open windows.
seen_live_open = False
deadline = time.monotonic() + 30.0
while time.monotonic() < deadline:
if page.evaluate(LIVE_OPEN_JS):
seen_live_open = True
break
time.sleep(0.05)
assert seen_live_open, (
"no open thinking/tool run was observed while the answer "
"bubble was still empty (the D5 live-open window)"
)
# The turn settles (the poll caught the live window; the settle is
# the D5 fold — test 2's contract).
_wait_settled(page)
# --------------------------------------------------------------------------
# 4. The restore path (the same 7-element timeline, collapsed) + the
# record's ``segments`` array (D4 — the legacy keys ride along)
# --------------------------------------------------------------------------
def test_restored_timeline_and_record(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# RELOAD (same context) — the phase-14 record restores through the
# restore path (restoreTimeline — the segments take precedence).
page.reload()
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
# The restored message renders the SAME 7-element timeline, all
# COLLAPSED (D5 — the restore renders collapsed).
seq = page.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the restored pre-bubble sequence is not the timeline: {seq}"
)
expect(
page.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
page.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The restored answer (the restore is synchronous at boot).
expect(
page.locator(".msg.brain .bubble").last
).to_contain_text(ANSWER_PREFIX, timeout=30_000)
# D4: the localStorage record (``bor.chat.v1`` — phase 14) carries
# a ``segments`` array of EXACTLY 7 entries in the arrival-order
# pattern — 4 thinking runs (the ``Step N`` text) + 3 tool entries
# (the call records, in order).
rec = page.evaluate(READ_RECORD_JS)
assert rec is not None, "the phase-14 record must exist after the reload"
segments = rec.get("segments")
assert isinstance(segments, list), (
f"the record must carry a segments array, got {type(segments)}"
)
assert len(segments) == 7, f"expected EXACTLY 7 segments, got {len(segments)}"
assert [s.get("kind") for s in segments] == [
"thinking", "tool", "thinking", "tool", "thinking", "tool", "thinking"
], f"the kind sequence is not the arrival-order pattern: {segments}"
# The thinking entries carry the DISTINCT ``Step N`` runs.
for i, marker in enumerate(STEP_MARKERS):
text = segments[i * 2].get("text")
assert isinstance(text, str) and marker in text, (
f"thinking segment {i} must carry {marker!r}: {text!r}"
)
# The tool entries are the 3 call records, in call order.
assert segments[1].get("name") == "ls"
assert segments[1].get("argument") is None # the no-arg top-level ls
assert segments[3].get("name") == "ls"
assert segments[3].get("argument") == READ_SOURCE # the drill scope
assert segments[5].get("name") == "read"
assert segments[5].get("argument") == READ_SP # the combined identity
# D4 (the old-client graceful degradation): the legacy fallback
# fields are STILL present — the full ``thinking`` concatenation
# (feeds the history wire + the per-turn log) + every ``tools``
# record (the old-client render path).
thinking_full = rec.get("thinking")
assert isinstance(thinking_full, str) and thinking_full, (
"the full thinking concatenation must still be present"
)
for marker in STEP_MARKERS:
assert marker in thinking_full, (
f"the concatenated thinking must carry {marker!r}"
)
tools = rec.get("tools")
assert isinstance(tools, list) and len(tools) == 3, (
f"the tools record must still carry the 3 calls: {tools!r}"
)
# --------------------------------------------------------------------------
# 5. The shared page (phase-51 save + share flow): the same 7-element
# timeline, collapsed (the addSegments parity)
# --------------------------------------------------------------------------
def test_shared_page_timeline(
page: Page,
browser: Browser,
app_url: str,
mock_llm: int,
db_ready: None,
) -> None:
_reset_db(_seed)
page.set_default_timeout(30_000)
login(page, app_url, next="/")
expect(page).to_have_url(app_url + "/", timeout=30_000)
_submit(page, TIMELINE_QUESTION)
_wait_settled(page)
# The phase-51 flow: the conversation is AUTO-SAVED by the time the
# answer settles (phase 55) — the Share click shares the linked row.
cookies = _admin_cookies(page)
_grant_clipboard(page, app_url)
_wait_saved_row(app_url, cookies, _auto_title(TIMELINE_QUESTION))
share = page.locator("#share-chat-btn")
expect(share).to_be_visible()
_click_share_and_assert_status(page, app_url)
row = _find_row(_chats(app_url, cookies), _auto_title(TIMELINE_QUESTION))
assert row is not None and row.get("share_url"), (
"the shared row must carry a share_url"
)
share_url: str = row["share_url"]
assert SHARE_URL_RE.fullmatch(share_url), f"bad share_url shape: {share_url}"
created: str = row["id"]
anon_ctx: BrowserContext | None = None
try:
# A FRESH context: a separate session with no cookies — the
# guest's only credential is the token in the URL.
anon_ctx = browser.new_context()
anon = anon_ctx.new_page()
anon.set_default_timeout(30_000)
anon.goto(app_url + share_url)
# The title = the auto-title (the shared chat's h1).
expect(anon.locator("#shared-title")).to_have_text(_auto_title(TIMELINE_QUESTION))
# The SAME 7-element timeline renders (the ``addSegments``
# parity), all COLLAPSED (D5 — the shared page is a finished
# conversation; a closed <details> has no ``open`` attribute).
seq = anon.evaluate(BODY_SEQUENCE_JS)
assert seq == TIMELINE_SEQ, (
f"the shared pre-bubble sequence is not the timeline: {seq}"
)
expect(
anon.locator("#messages .msg.brain details.thinking[open]")
).to_have_count(0)
expect(
anon.locator(
"#messages .msg.brain details.tool-calls-disclosure[open]"
)
).to_have_count(0)
# The ``Step N`` markers are visible on a summary click (expand
# ONE — the native <summary> is a real focusable toggle).
anon.locator("#messages .msg.brain details.thinking > summary").first.click()
expect(
anon.locator("#messages .msg.brain details.thinking").first
).to_have_attribute("open")
first_text = anon.locator("#messages .msg.brain .thinking-text").first
expect(first_text).to_be_visible()
expect(first_text).to_contain_text(STEP_MARKERS[0])
# The answer bubble carries the ``Read <source/path>`` echo.
expect(
anon.locator(".msg.brain .bubble").first
).to_contain_text(ANSWER_PREFIX)
finally:
if anon_ctx is not None:
anon_ctx.close()
if created:
_delete_chat(app_url, cookies, created)
# --------------------------------------------------------------------------
# 6. The run-grouping contract for no-thinking turns (phase 117
# preserved): one disclosure, 3 lines, zero thinking blocks
# --------------------------------------------------------------------------
def test_no_thinking_turn_single_disclosure(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
page.set_default_timeout(30_000)
_reset_db(_seed)
login(page, app_url, next="/")
# The plain ``use your tools`` marker (NO thinking trigger — the
# mock's tool frames carry no ``reasoning_content``): the 3 calls
# are ONE contiguous tool run.
_submit(page, PLAIN_TOOLS_QUESTION)
_wait_settled(page)
# D1 (the grouping contract): no thinking frames → ONE tool run →
# EXACTLY ONE disclosure (the phase-117 behavior for no-thinking
# turns is preserved), with the 3 ``.tool-call`` lines and the
# plural summary.
expect(
page.locator("#messages .msg.brain .tool-calls-disclosure")
).to_have_count(1)
expect(page.locator("#messages .msg.brain .tool-call")).to_have_count(3)
expect(
page.locator("#messages .msg.brain .tool-calls-summary")
).to_have_text("Tool calls (3)")
# And ZERO thinking blocks (no thinking ran — no empty block).
expect(page.locator("#messages .msg.brain details.thinking")).to_have_count(0)
# Folded at rest (the first delta / done settle) + the answer.
expect(
page.locator("#messages .msg.brain .tool-calls-disclosure[open]")
).to_have_count(0)
expect(page.locator(".msg.brain .bubble").last).to_contain_text(ANSWER_PREFIX)