The owner refinement (2026-10-06): the per-run thinking and tool-call blocks inside the Reasoning bubble were individually collapsible details elements — a fold inside the fold. A tool frame closed the previous thinking block, so the visible thinking vanished mid-turn; and the pinned-tail follow died on the append of every run after the first (the pin was captured AFTER the new block's height growth contaminated the 32px reading). - app.js: the in-container runs are now plain divs — createThinkingRunEl (div.thinking + static .thinking-summary label + .thinking-text) and createToolRunEl (div.tool-calls-disclosure + static .tool-calls-summary label + the .tool-calls list); they never open or fold (the container is the ONE open/fold control, open through the turn, folded at settle — unchanged); the pin capture moves BEFORE ensureThinkingRun/ensureToolRun append (plus the first-frame edge: no container yet -> always pinned); closeCurrentRun just ends the run; closeAllRuns stays the defensive legacy guard; restoreTimeline/addSegments (shared.js) use the div builders; the legacy no-segments restore path keeps the old details-based blocks byte-identical - styles.css: the in-container runs are distinct by label + rail (brand-ink Thinking label on a brand-soft rail; accent-ink Tool call(s) (N) label on a neutral rail); the interactive treatment (cursor + chevron) is scoped to the legacy <summary>; the inner windows stay disabled (the container's .reasoning-body is the ONE live scroll surface) - unit + E2E: the D5-refinement contract is pinned — one details.reasoning disclosure, per-run blocks as plain divs, zero nested <details> inside the container, the container as the only open/fold control, the .reasoning-body window auto-follows the tail while streaming unit+integration 3738 passed (coverage 99%), ruff + pyright clean, all five affected E2E suites pass in isolation (reasoning_bubble 6, thinking_timeline 6, thinking_display 5, thinking_scroll 8, tool_call_compact 4), verified live: the follow holds through a long thinking stream, the tool-frame transition, and the second thinking run; restore renders the div structure without client-side errors LLM-Generated: true
876 lines
36 KiB
Python
876 lines
36 KiB
Python
"""Phase 158 E2E (Playwright, mock-only): the chronological thinking /
|
||
tool-call timeline (the owner's 2026-10-04 report — "the thinking block
|
||
happens first … and then went back to thinking. This is awkward. Then
|
||
the tool calling happens, and the thinking block reads strangely because
|
||
the tool calls are shown below the thinking block").
|
||
|
||
Source: owner report (live interactive-browser session, 2026-10-04,
|
||
shared page ``https://brain.reeseapps.com/shared/
|
||
95dd7a2a-8cda-478b-b694-53936070b9fb`` — a real ``turbo`` turn with 6
|
||
tool calls). Run in isolation (DB must be up:
|
||
``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov
|
||
|
||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate
|
||
is the deterministic marker flow in ``tests/e2e/mock_llm.py`` (user
|
||
message contains ``think out loud and use your tools``
|
||
(``THINK_TOOLS_TRIGGER``) **and** the system prompt carries the
|
||
``<tools>`` section of the HIGH prompt): the phase-37 single-read
|
||
shape (list → drill → read → the quoted answer) with a DISTINCT
|
||
``reasoning_content`` run streamed AHEAD of every tool call and ahead
|
||
of the final answer — the ``Step N`` markers make each run
|
||
individually assertable. The client renders each CONTIGUOUS run in its
|
||
own per-run block (D1) in arrival order (D2 — Option B,
|
||
owner-chosen) and persists the run order in the record's ``segments``
|
||
array (D4). Phase 161 (the documented supersession, owner TODO
|
||
direction 2026-10-05): the per-run blocks are NESTED inside ONE
|
||
``details.reasoning`` container (``.reasoning-body``) that sits
|
||
before the bubble — open during the turn, folded at done / on
|
||
restore / on the shared page. The D5 refinement (owner 2026-10-06):
|
||
the per-run blocks are plain divs — NOT collapsible (the container
|
||
is the ONE disclosure; a nested fold inside the fold used to hide
|
||
the live reasoning mid-turn), always visible while the turn is live.
|
||
The sequence pin is shape-unchanged, read from the container's
|
||
``.reasoning-body`` children (the container is the one
|
||
new parent — the pre-bubble ``.msg-body`` children are exactly
|
||
``["details.reasoning"]``).
|
||
|
||
KB fixture — mirrored from ``test_agent_document_tools.py`` (the
|
||
deterministic two-document pair): ``Deployments`` registered FIRST so
|
||
the mock's drill (first source of the top-level listing) — and
|
||
therefore the read — lands on the JSON file; the marker question
|
||
cosines ≈0.595 against the retrievable document (well past the E2E
|
||
0.30 threshold) plus FTS hits, so the turn is solidly grounded (HIGH
|
||
prompt → ``<tools>``).
|
||
|
||
Test → phase mapping (Playwright Mapping Rule):
|
||
1. ``test_timeline_order_after_completed_turn`` — D1/D2: after ``done``
|
||
the pre-bubble ``.msg-body`` children are EXACTLY the reasoning
|
||
container (the one new parent — phase 161), and its
|
||
``.reasoning-body`` children are EXACTLY the 7-element
|
||
arrival-order sequence (4 thinking blocks + 3 one-line
|
||
disclosures, alternating); the four ``.thinking-text``s carry the
|
||
DISTINCT ``Step N`` markers (distinctness = separate blocks, not
|
||
one lump); each disclosure summary is ``Tool call (1)``; the
|
||
answer bubble carries the ``Read <source/path>`` echo.
|
||
2. ``test_runs_fold_at_rest`` — D5 (phase 161 D2): after ``done``
|
||
the container has NO ``open`` attribute (the settle folded it) and
|
||
the inner runs are NOT collapsible (zero ``<details>`` inside the
|
||
container — the D5 refinement); the at-rest visible line is the
|
||
container's ``Reasoning`` summary (``7 steps``); opening it
|
||
reveals the always-visible run blocks.
|
||
3. ``test_live_runs_open_while_streaming`` — D5 (the live half):
|
||
DURING the turn (the mock's pacing window — the 0.1 s tool frames +
|
||
the thinking stream, the answer bubble still EMPTY) the container
|
||
is ``open`` with a run visible in its ``.reasoning-body`` (the
|
||
live feedback the owner keeps — the container is the ONE open
|
||
disclosure).
|
||
4. ``test_restored_timeline_and_record`` — the restore path + D4: a
|
||
same-context RELOAD re-renders the same 7-element sequence in ONE
|
||
collapsed reasoning container (``7 steps``, phase 161 D5/D6), and
|
||
the localStorage ``bor.chat.v1`` record (phase 14) carries a
|
||
``segments`` array of EXACTLY 7 entries in the
|
||
``thinking, tool, ×3`` pattern (the ``Step N`` text on the thinking
|
||
entries, the ``name``/``argument`` on the tool entries); the legacy
|
||
``thinking`` (full concatenation) + ``tools`` (3 records) keys are
|
||
STILL present (the old-client graceful degradation).
|
||
5. ``test_shared_page_timeline`` — the shared-page parity (phase 51
|
||
save + share flow): a FRESH context on ``/shared/<token>`` renders
|
||
the same 7-element sequence in ONE collapsed reasoning container
|
||
(``addSegments``), the ``Step N`` markers visible on container
|
||
click (the D5 refinement — the inner blocks are always visible
|
||
once the container opens), the answer bubble carries the
|
||
``Read <source/path>`` echo.
|
||
6. ``test_no_thinking_turn_single_disclosure`` — the run-grouping
|
||
contract for no-thinking turns (the phase-117 behavior preserved,
|
||
now wrapped — phase 161): a plain ``use your tools`` turn (NO
|
||
thinking trigger) renders ONE folded reasoning container (``1
|
||
steps``) wrapping EXACTLY ONE ``.tool-calls-disclosure`` inside its
|
||
``.reasoning-body`` (3 ``.tool-call`` lines, summary ``Tool calls
|
||
(3)``) and ZERO ``.thinking``.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import hashlib
|
||
import re
|
||
import time
|
||
from collections.abc import Callable
|
||
from datetime import UTC, datetime
|
||
from typing import Any
|
||
|
||
import httpx
|
||
from playwright.sync_api import Browser, BrowserContext, Page, expect
|
||
from sqlalchemy import text
|
||
from sqlalchemy.orm import Session
|
||
|
||
from app.db import SessionLocal
|
||
from app.models import Chunk, Document, GitSource
|
||
from e2e.auth_helpers import login
|
||
from tests.e2e.mock_llm import embed_text
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Fixture documents (deterministic, token-controlled — mirrored from
|
||
# test_agent_document_tools.py so the flow is byte-stable)
|
||
# --------------------------------------------------------------------------
|
||
|
||
SEED_SOURCE = "Homelab"
|
||
SEED_PATH = "aws-route53.md"
|
||
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
|
||
|
||
READ_SOURCE = "Deployments"
|
||
READ_PATH = "example-record-file.json"
|
||
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
|
||
|
||
#: The retrievable document: references the JSON file "for the exact
|
||
#: JSON shape of reeselink.json" but never includes it. The repeated
|
||
#: record-file lines carry the marker question's key tokens (aws,
|
||
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
|
||
#: ≈0.595 cosine against the mock's embeddings (E2E threshold 0.30)
|
||
#: plus FTS hits, so the turn is solidly grounded.
|
||
ROUTE53_CONTENT = (
|
||
"# AWS Route 53 Notes\n\n"
|
||
"## Record file\n\n"
|
||
+ (
|
||
"The aws route53 hosted zone for reeselink keeps every record in "
|
||
"reseelink.json — the exact JSON shape of reeselink.json is "
|
||
"documented in example-record-file.json.\n"
|
||
)
|
||
* 10
|
||
+ "\n## Sync job\n\n"
|
||
"A cron job pushes reeselink.json to the aws route53 hosted zone "
|
||
"every fifteen minutes; the diff is applied through the route53 api.\n"
|
||
)
|
||
|
||
#: The referenced document: the exact JSON shape. Its FIRST line is
|
||
#: longer than 80 chars, so the mock's first-80-chars quote content
|
||
#: part is newline-free (the rendered-text assertions below match it
|
||
#: after the date line). Pinned by the assert below.
|
||
RECORD_FILE_CONTENT = (
|
||
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
|
||
'JSON shape of reeselink.json",\n'
|
||
' "hosted_zone_id": "Z0RESEELINK01",\n'
|
||
' "record_sets": [\n'
|
||
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
|
||
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
|
||
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
|
||
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
|
||
" ]\n"
|
||
"}\n"
|
||
)
|
||
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
|
||
|
||
#: The phase-158 trigger question (``THINK_TOOLS_TRIGGER`` + the KB
|
||
#: topic — the ≈0.595 cosine keeps the turn grounded over the mock-
|
||
#: calibrated E2E threshold).
|
||
TIMELINE_QUESTION = (
|
||
"Think out loud and use your tools: what is the exact JSON shape of "
|
||
"reseelink.json for my aws route53 hosted zone?"
|
||
)
|
||
|
||
#: The NO-thinking control — the plain ``use your tools`` marker
|
||
#: (``test_agent_document_tools.py`` / ``test_tool_call_compact.py``'s
|
||
#: question, mirrored so the 3-call flow is the phase-37/117 one).
|
||
PLAIN_TOOLS_QUESTION = (
|
||
"Use your tools: what is the exact JSON shape of reeselink.json "
|
||
"for my aws route53 hosted zone?"
|
||
)
|
||
|
||
#: The mock's deterministic single-read answer —
|
||
#: "Read <source/path>. <first 80 chars of the read document>" (the
|
||
#: phase-37 shape). Phase 106 (D5): the read result's ``date:`` SECOND
|
||
#: line rides into the first-80-chars quote — the quote is the date
|
||
#: line (the fixture's fixed ``created_at`` UTC date part, 17 chars) +
|
||
#: the first 63 content chars (80 − 17).
|
||
ANSWER_PREFIX = f"Read {READ_SP}."
|
||
ANSWER_QUOTE = "date: 2024-06-15" + RECORD_FILE_CONTENT[:63]
|
||
|
||
#: The mock's four DISTINCT reasoning runs (the ``Step N`` markers —
|
||
#: their distinctness proves the runs are separate blocks, not one
|
||
#: lump).
|
||
STEP_MARKERS = ("Step 1", "Step 2", "Step 3", "Step 4")
|
||
|
||
#: The 7-element sequence the timeline renders in the reasoning
|
||
#: container's ``.reasoning-body`` children, in arrival order (D1:
|
||
#: alternating thinking/tool runs — one block per run; phase 161: the
|
||
#: sequence is shape-unchanged — the container is the one new parent)
|
||
#: — tests 1/4 (chat page) and 5 (shared page) pin it.
|
||
TIMELINE_SEQ = [
|
||
"div.thinking",
|
||
"div.tool-calls-disclosure",
|
||
"div.thinking",
|
||
"div.tool-calls-disclosure",
|
||
"div.thinking",
|
||
"div.tool-calls-disclosure",
|
||
"div.thinking",
|
||
]
|
||
|
||
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
|
||
|
||
# --------------------------------------------------------------------------
|
||
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _seed(db: Session) -> None:
|
||
"""The two-document pair (see the module docstring), plus the
|
||
phase-94 registry rows: both sources registered, ``Deployments``
|
||
FIRST — the mock's drill (first source of the top-level listing)
|
||
lands on the JSON file deterministically, independent of the
|
||
operator's ``BOR_GIT_SOURCES`` (a non-empty table ignores the env
|
||
fallback)."""
|
||
# COMMIT between the inserts (not flush): ``added_at`` is
|
||
# ``server_default now()`` — the transaction timestamp — and the
|
||
# tie-break is the random uuid ``id``, so one-transaction rows order
|
||
# nondeterministically.
|
||
db.add(GitSource(url=READ_SOURCE, kind="local"))
|
||
db.commit()
|
||
db.add(GitSource(url=SEED_SOURCE, kind="local"))
|
||
md = Document(
|
||
source=SEED_SOURCE,
|
||
path=SEED_PATH,
|
||
full_path=f"/tmp/{SEED_PATH}",
|
||
title="AWS Route 53 Notes",
|
||
content=ROUTE53_CONTENT,
|
||
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
|
||
indexed_at=datetime.now(UTC),
|
||
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
|
||
# (the mock's first-80-chars read quote carries the date line).
|
||
created_at=datetime(2024, 6, 15, tzinfo=UTC),
|
||
)
|
||
db.add(md)
|
||
db.flush()
|
||
# One chunk carrying the mock's own embedding → genuine token
|
||
# overlap between the marker question and this document.
|
||
db.add(
|
||
Chunk(
|
||
document_id=md.id,
|
||
position=0,
|
||
content=ROUTE53_CONTENT,
|
||
embedding=embed_text(ROUTE53_CONTENT),
|
||
)
|
||
)
|
||
# The referenced JSON: indexed, catalogued, readable — but NO
|
||
# chunks, so retrieval never puts it in context (the failure the
|
||
# tools fix).
|
||
db.add(
|
||
Document(
|
||
source=READ_SOURCE,
|
||
path=READ_PATH,
|
||
full_path=f"/tmp/{READ_PATH}",
|
||
title="Example Record File",
|
||
content=RECORD_FILE_CONTENT,
|
||
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
|
||
indexed_at=datetime.now(UTC),
|
||
created_at=datetime(2024, 6, 15, tzinfo=UTC),
|
||
)
|
||
)
|
||
|
||
|
||
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
|
||
"""Truncate the KB (plus the prompt-shaping tables and the
|
||
phase-55 auto-save rows), then re-seed — the E2E isolation
|
||
pattern. ``steering_notes`` / ``kb_overview`` are truncated too,
|
||
so the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
|
||
``<tools>`` regardless of leftovers — byte-stable prompts,
|
||
byte-stable answers."""
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, "
|
||
"steering_notes, kb_overview, saved_chats, git_sources"
|
||
)
|
||
)
|
||
db.commit()
|
||
if seed is not None:
|
||
seed(db)
|
||
db.commit()
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Page helpers
|
||
# --------------------------------------------------------------------------
|
||
|
||
#: The last ``.msg.brain`` wrap's ``.msg-body`` children BEFORE the
|
||
#: bubble, as ``tag.className`` strings. Phase 161 (D1): EXACTLY ONE
|
||
#: — the ``details.reasoning`` container (the one new anchor before
|
||
#: the bubble; the per-run blocks live inside it, not as direct
|
||
#: children; the ``.msg-meta`` row lands AFTER the bubble, so it
|
||
#: never enters the sequence).
|
||
PRE_BUBBLE_JS = """
|
||
() => {
|
||
const wrap = document.querySelector('#messages .msg.brain');
|
||
if (!wrap) return null;
|
||
const body = wrap.querySelector('.msg-body');
|
||
const out = [];
|
||
for (const child of body.children) {
|
||
if (child.classList.contains('bubble')) break;
|
||
out.push(child.tagName.toLowerCase() + '.' + child.className);
|
||
}
|
||
return out;
|
||
}
|
||
"""
|
||
|
||
#: The last ``.msg.brain`` wrap's reasoning container's
|
||
#: ``.reasoning-body`` children, as ``tag.className`` strings — the
|
||
#: timeline's order pin (phase 161: the 7-element arrival-order
|
||
#: sequence nested in the container — shape unchanged from phase 158).
|
||
BODY_SEQUENCE_JS = """
|
||
() => {
|
||
const wrap = document.querySelector('#messages .msg.brain');
|
||
if (!wrap) return null;
|
||
const body = wrap.querySelector('details.reasoning .reasoning-body');
|
||
if (!body) return null;
|
||
const out = [];
|
||
for (const child of body.children) {
|
||
out.push(child.tagName.toLowerCase() + '.' + child.className);
|
||
}
|
||
return out;
|
||
}
|
||
"""
|
||
|
||
#: D5's live predicate (the D5 refinement — the container is the ONE
|
||
#: open disclosure): the container is ``open`` with a per-run block
|
||
#: nested in its ``.reasoning-body`` while the answer bubble is STILL
|
||
#: EMPTY (the turn is in flight; the mock's pacing window — the 0.1 s
|
||
#: tool frames + the thinking stream — keeps the window comfortably
|
||
#: open for a 100 ms poll).
|
||
LIVE_OPEN_JS = """
|
||
() => {
|
||
const wrap = document.querySelector('#messages .msg.brain');
|
||
if (!wrap) return false;
|
||
const bubble = wrap.querySelector('.msg-body .bubble');
|
||
if (!bubble || bubble.textContent !== '') return false;
|
||
const cont = wrap.querySelector('details.reasoning');
|
||
if (!cont || !cont.hasAttribute('open')) return false;
|
||
return !!cont.querySelector(
|
||
'.reasoning-body div.thinking, ' +
|
||
'.reasoning-body div.tool-calls-disclosure');
|
||
}
|
||
"""
|
||
|
||
#: The phase-14 localStorage record's LAST brain message (the
|
||
#: ``bor.chat.v1`` — ``{ v: 1, chatId, messages: [...] }``; the
|
||
#: filter mirrors ``loadStoredRecord``'s well-formedness).
|
||
READ_RECORD_JS = """
|
||
() => {
|
||
const raw = localStorage.getItem('bor.chat.v1');
|
||
if (!raw) return null;
|
||
const data = JSON.parse(raw);
|
||
const brain = (data.messages || []).filter(
|
||
(m) => m && m.who === 'brain' && typeof m.text === 'string');
|
||
return brain.length ? brain[brain.length - 1] : null;
|
||
}
|
||
"""
|
||
|
||
|
||
def _submit(page: Page, question: str) -> None:
|
||
page.fill("#message-input", question)
|
||
page.click("#send-btn")
|
||
# The user bubble lands synchronously with the submit handler.
|
||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||
|
||
|
||
def _wait_settled(page: Page) -> None:
|
||
"""The turn is complete: answer text in the bubble, button
|
||
recovered (phase 48 — the in-flight button is the enabled Stop
|
||
control, so the label assertion carries the settle wait)."""
|
||
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Phase-51 save + share flow (the test_share_chat.py pattern — the
|
||
# auto-saved row is shared from the chat page; the row is deleted in a
|
||
# finally)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _admin_cookies(page: Page) -> dict[str, str]:
|
||
return {
|
||
c["name"]: c["value"]
|
||
for c in page.context.cookies()
|
||
if "name" in c and "value" in c
|
||
}
|
||
|
||
|
||
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
|
||
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
return r.json()["chats"]
|
||
|
||
|
||
def _auto_title(question: str) -> str:
|
||
"""The phase-50 auto-title convention: the first question,
|
||
whitespace-collapsed, capped at 120 chars."""
|
||
return " ".join(question.split())[:120]
|
||
|
||
|
||
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
|
||
return next((c for c in rows if c["title"] == title), None)
|
||
|
||
|
||
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
|
||
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
|
||
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
||
|
||
|
||
def _wait_saved_row(
|
||
app_url: str,
|
||
cookies: dict[str, str],
|
||
title: str,
|
||
messages: int = 2,
|
||
) -> dict[str, Any]:
|
||
"""Wait for the auto-saved row (phase 55 — auto-saves are SILENT,
|
||
so there is no status line to wait on). The upsert is
|
||
fire-and-forget from the UI's point of view, so poll the admin
|
||
list until the row with the conversation's auto-title appears with
|
||
the expected message count."""
|
||
deadline = time.monotonic() + 15
|
||
last: dict[str, Any] | None = None
|
||
while time.monotonic() < deadline:
|
||
last = _find_row(_chats(app_url, cookies), title)
|
||
if last is not None and last["message_count"] >= messages:
|
||
return last
|
||
time.sleep(0.2)
|
||
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
|
||
|
||
|
||
def _grant_clipboard(page: Page, app_url: str) -> None:
|
||
"""Grant the async-clipboard permissions on the admin context
|
||
(headless Chromium requires the grant before ``writeText``
|
||
resolves — without it the inline-link fallback fires)."""
|
||
page.context.grant_permissions(
|
||
["clipboard-read", "clipboard-write"], origin=app_url
|
||
)
|
||
|
||
|
||
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
|
||
"""Press the chat page's Share pill and pin the outcome on the
|
||
live region (the assertion branches on the API's availability —
|
||
the test_share_chat.py convention)."""
|
||
page.locator("#share-chat-btn").click()
|
||
if page.evaluate("() => !!navigator.clipboard"):
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link copied.", timeout=15_000
|
||
)
|
||
else:
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link ready — copy it from the field.", timeout=15_000
|
||
)
|
||
field = page.locator(".share-link-fallback")
|
||
expect(field).to_be_visible()
|
||
href = field.get_attribute("href")
|
||
assert href is not None
|
||
assert href.startswith(app_url)
|
||
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 1. D1/D2: the completed turn renders the 7-element arrival-order
|
||
# timeline (the runs are separate blocks, not one lump)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_timeline_order_after_completed_turn(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# Phase 161 (D1): the container is the ONE pre-bubble anchor —
|
||
# the per-run blocks are no longer direct ``.msg-body`` children.
|
||
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
|
||
"the pre-bubble children must be exactly the reasoning "
|
||
"container (phase 161 D1)"
|
||
)
|
||
# D1: the container's ``.reasoning-body`` sequence is EXACTLY the
|
||
# 7-element arrival-order timeline — thinking, tool, ×3 (one block
|
||
# per contiguous run; the plain ``use your tools`` flow's 3 calls
|
||
# land in 3 SEPARATE one-line disclosures, one per thinking run).
|
||
seq = page.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the in-container segment sequence is not the timeline: {seq}"
|
||
)
|
||
|
||
# D1/D3: the four thinking blocks carry the DISTINCT ``Step N``
|
||
# markers — distinctness proves the runs are separate blocks, not
|
||
# one lump (the pre-phase single-block layout would put all four
|
||
# markers in one ``.thinking-text``).
|
||
texts = page.locator("#messages .msg.brain .thinking-text")
|
||
expect(texts).to_have_count(4)
|
||
for i, marker in enumerate(STEP_MARKERS):
|
||
expect(texts.nth(i)).to_contain_text(marker)
|
||
|
||
# D3: each tool run's disclosure summary is per-run — one line →
|
||
# "Tool call (1)" (the phase-117 singular form).
|
||
summaries = page.locator("#messages .msg.brain .tool-calls-summary")
|
||
expect(summaries).to_have_count(3)
|
||
for i in range(3):
|
||
expect(summaries.nth(i)).to_have_text("Tool call (1)")
|
||
|
||
# The answer bubble carries the ``Read <source/path>`` echo (the
|
||
# read document reached the model and landed in the answer).
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(ANSWER_PREFIX)
|
||
expect(bubble).to_contain_text(ANSWER_QUOTE)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 2. D5: the runs fold at rest (done settled every open block)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_runs_fold_at_rest(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# Phase 161 (D2): the container folds at done — the at-rest
|
||
# visible line is its ``Reasoning`` summary (the D6 settled status
|
||
# is the run count — 7 runs → "7 steps").
|
||
container = page.locator("#messages .msg.brain details.reasoning")
|
||
expect(container).to_have_count(1)
|
||
expect(container).not_to_have_attribute("open")
|
||
expect(
|
||
page.locator("#messages .msg.brain .reasoning-summary").first
|
||
).to_be_visible()
|
||
expect(
|
||
page.locator("#messages .msg.brain .reasoning-status")
|
||
).to_have_text("7 steps")
|
||
|
||
# D5 refinement (owner 2026-10-06): the inner runs are NOT
|
||
# collapsible — ZERO <details> inside the container (the
|
||
# per-run blocks are plain divs, always visible while the
|
||
# container is open).
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning details")
|
||
).to_have_count(0)
|
||
|
||
# The inner runs stay in the DOM inside the fold — opening the
|
||
# container (the native toggle) reveals the 4 "Thinking" + 3
|
||
# "Tool call (1)" labels (the timeline stays readable at rest;
|
||
# the D5 refinement — no second toggle: the blocks are always
|
||
# visible once the container opens).
|
||
page.locator("#messages .msg.brain .reasoning-summary").click()
|
||
expect(container).to_have_attribute("open", "")
|
||
expect(
|
||
page.locator("#messages .msg.brain .thinking .thinking-summary").first
|
||
).to_be_visible()
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-summary").first
|
||
).to_be_visible()
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 3. D5: the active run's block stays OPEN while the turn streams
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_live_runs_open_while_streaming(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
page.fill("#message-input", TIMELINE_QUESTION)
|
||
page.click("#send-btn")
|
||
expect(page.locator(".msg.user .bubble").last).to_contain_text(TIMELINE_QUESTION)
|
||
|
||
# D5 (the live half): during the turn — the mock's pacing window
|
||
# (the 0.1 s tool frames + the thinking stream; the answer bubble
|
||
# is still EMPTY — the first delta has not landed) — the active
|
||
# run's block is ``open`` (the live feedback the owner keeps:
|
||
# thinking/tool lines stream into their OPEN blocks, folded only
|
||
# by the first delta / done / stop). The house deadline-poll
|
||
# pattern (the ``test_agent_document_tools.py`` ``_frames``
|
||
# convention): a 50 ms stride over the ~2 s turn comfortably
|
||
# samples the open windows.
|
||
seen_live_open = False
|
||
deadline = time.monotonic() + 30.0
|
||
while time.monotonic() < deadline:
|
||
if page.evaluate(LIVE_OPEN_JS):
|
||
seen_live_open = True
|
||
break
|
||
time.sleep(0.05)
|
||
assert seen_live_open, (
|
||
"no open thinking/tool run was observed while the answer "
|
||
"bubble was still empty (the D5 live-open window)"
|
||
)
|
||
|
||
# The turn settles (the poll caught the live window; the settle is
|
||
# the D5 fold — test 2's contract).
|
||
_wait_settled(page)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 4. The restore path (the same 7-element timeline, collapsed) + the
|
||
# record's ``segments`` array (D4 — the legacy keys ride along)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_restored_timeline_and_record(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# RELOAD (same context) — the phase-14 record restores through the
|
||
# restore path (restoreTimeline — the segments take precedence).
|
||
page.reload()
|
||
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
|
||
|
||
# Phase 161 (D5/D6): the restore renders ONE collapsed container
|
||
# around the timeline (the D6 at-rest step count). The pre-bubble
|
||
# children are exactly the container (the live pin, D1).
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning")
|
||
).to_have_count(1)
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning")
|
||
).not_to_have_attribute("open")
|
||
expect(
|
||
page.locator("#messages .msg.brain .reasoning-status")
|
||
).to_have_text("7 steps")
|
||
assert page.evaluate(PRE_BUBBLE_JS) == ["details.reasoning"], (
|
||
"the restored pre-bubble children must be exactly the "
|
||
"container (phase 161 D1)"
|
||
)
|
||
# The restored message renders the SAME 7-element timeline
|
||
# (in-container), all COLLAPSED (D5 — the restore renders
|
||
# collapsed).
|
||
seq = page.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the restored in-container sequence is not the timeline: {seq}"
|
||
)
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning details")
|
||
).to_have_count(0)
|
||
# The restored answer (the restore is synchronous at boot).
|
||
expect(
|
||
page.locator(".msg.brain .bubble").last
|
||
).to_contain_text(ANSWER_PREFIX, timeout=30_000)
|
||
|
||
# D4: the localStorage record (``bor.chat.v1`` — phase 14) carries
|
||
# a ``segments`` array of EXACTLY 7 entries in the arrival-order
|
||
# pattern — 4 thinking runs (the ``Step N`` text) + 3 tool entries
|
||
# (the call records, in order).
|
||
rec = page.evaluate(READ_RECORD_JS)
|
||
assert rec is not None, "the phase-14 record must exist after the reload"
|
||
segments = rec.get("segments")
|
||
assert isinstance(segments, list), (
|
||
f"the record must carry a segments array, got {type(segments)}"
|
||
)
|
||
assert len(segments) == 7, f"expected EXACTLY 7 segments, got {len(segments)}"
|
||
assert [s.get("kind") for s in segments] == [
|
||
"thinking", "tool", "thinking", "tool", "thinking", "tool", "thinking"
|
||
], f"the kind sequence is not the arrival-order pattern: {segments}"
|
||
# The thinking entries carry the DISTINCT ``Step N`` runs.
|
||
for i, marker in enumerate(STEP_MARKERS):
|
||
text = segments[i * 2].get("text")
|
||
assert isinstance(text, str) and marker in text, (
|
||
f"thinking segment {i} must carry {marker!r}: {text!r}"
|
||
)
|
||
# The tool entries are the 3 call records, in call order.
|
||
assert segments[1].get("name") == "ls"
|
||
assert segments[1].get("argument") is None # the no-arg top-level ls
|
||
assert segments[3].get("name") == "ls"
|
||
assert segments[3].get("argument") == READ_SOURCE # the drill scope
|
||
assert segments[5].get("name") == "read"
|
||
assert segments[5].get("argument") == READ_SP # the combined identity
|
||
|
||
# D4 (the old-client graceful degradation): the legacy fallback
|
||
# fields are STILL present — the full ``thinking`` concatenation
|
||
# (feeds the history wire + the per-turn log) + every ``tools``
|
||
# record (the old-client render path).
|
||
thinking_full = rec.get("thinking")
|
||
assert isinstance(thinking_full, str) and thinking_full, (
|
||
"the full thinking concatenation must still be present"
|
||
)
|
||
for marker in STEP_MARKERS:
|
||
assert marker in thinking_full, (
|
||
f"the concatenated thinking must carry {marker!r}"
|
||
)
|
||
tools = rec.get("tools")
|
||
assert isinstance(tools, list) and len(tools) == 3, (
|
||
f"the tools record must still carry the 3 calls: {tools!r}"
|
||
)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 5. The shared page (phase-51 save + share flow): the same 7-element
|
||
# timeline, collapsed (the addSegments parity)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_shared_page_timeline(
|
||
page: Page,
|
||
browser: Browser,
|
||
app_url: str,
|
||
mock_llm: int,
|
||
db_ready: None,
|
||
) -> None:
|
||
_reset_db(_seed)
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url, next="/")
|
||
expect(page).to_have_url(app_url + "/", timeout=30_000)
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# The phase-51 flow: the conversation is AUTO-SAVED by the time the
|
||
# answer settles (phase 55) — the Share click shares the linked row.
|
||
cookies = _admin_cookies(page)
|
||
_grant_clipboard(page, app_url)
|
||
_wait_saved_row(app_url, cookies, _auto_title(TIMELINE_QUESTION))
|
||
share = page.locator("#share-chat-btn")
|
||
expect(share).to_be_visible()
|
||
_click_share_and_assert_status(page, app_url)
|
||
row = _find_row(_chats(app_url, cookies), _auto_title(TIMELINE_QUESTION))
|
||
assert row is not None and row.get("share_url"), (
|
||
"the shared row must carry a share_url"
|
||
)
|
||
share_url: str = row["share_url"]
|
||
assert SHARE_URL_RE.fullmatch(share_url), f"bad share_url shape: {share_url}"
|
||
created: str = row["id"]
|
||
anon_ctx: BrowserContext | None = None
|
||
try:
|
||
# A FRESH context: a separate session with no cookies — the
|
||
# guest's only credential is the token in the URL.
|
||
anon_ctx = browser.new_context()
|
||
anon = anon_ctx.new_page()
|
||
anon.set_default_timeout(30_000)
|
||
anon.goto(app_url + share_url)
|
||
|
||
# The title = the auto-title (the shared chat's h1).
|
||
expect(anon.locator("#shared-title")).to_have_text(_auto_title(TIMELINE_QUESTION))
|
||
|
||
# Phase 161 (D5/D6): the shared page renders ONE collapsed
|
||
# container (the D6 at-rest step count) around the timeline.
|
||
expect(
|
||
anon.locator("#messages .msg.brain details.reasoning")
|
||
).to_have_count(1)
|
||
expect(
|
||
anon.locator("#messages .msg.brain details.reasoning")
|
||
).not_to_have_attribute("open")
|
||
expect(
|
||
anon.locator("#messages .msg.brain .reasoning-status")
|
||
).to_have_text("7 steps")
|
||
# The SAME 7-element timeline renders inside it (the
|
||
# ``addSegments`` parity), all COLLAPSED (D5 — the shared
|
||
# page is a finished conversation; a closed <details> has no
|
||
# ``open`` attribute).
|
||
seq = anon.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the shared in-container sequence is not the timeline: {seq}"
|
||
)
|
||
# D5 refinement (owner 2026-10-06): the inner runs are NOT
|
||
# collapsible — ZERO <details> inside the container (the
|
||
# per-run blocks are plain divs).
|
||
expect(
|
||
anon.locator("#messages .msg.brain details.reasoning details")
|
||
).to_have_count(0)
|
||
|
||
# The ``Step N`` markers are visible on the container's summary
|
||
# click (the D5 refinement — ONE toggle: the inner blocks are
|
||
# always visible once the container opens; no second toggle).
|
||
anon.locator("#messages .msg.brain .reasoning-summary").click()
|
||
first_text = anon.locator("#messages .msg.brain .thinking-text").first
|
||
expect(first_text).to_be_visible()
|
||
expect(first_text).to_contain_text(STEP_MARKERS[0])
|
||
|
||
# The answer bubble carries the ``Read <source/path>`` echo.
|
||
expect(
|
||
anon.locator(".msg.brain .bubble").first
|
||
).to_contain_text(ANSWER_PREFIX)
|
||
finally:
|
||
if anon_ctx is not None:
|
||
anon_ctx.close()
|
||
if created:
|
||
_delete_chat(app_url, cookies, created)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 6. The run-grouping contract for no-thinking turns (phase 117
|
||
# preserved): one disclosure, 3 lines, zero thinking blocks
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_no_thinking_turn_single_disclosure(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
# The plain ``use your tools`` marker (NO thinking trigger — the
|
||
# mock's tool frames carry no ``reasoning_content``): the 3 calls
|
||
# are ONE contiguous tool run.
|
||
_submit(page, PLAIN_TOOLS_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# Phase 161 (D1/D2): the tool frames create the container — it
|
||
# exists and folds at done; the D6 settled status is the single
|
||
# tool run ("1 steps").
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning")
|
||
).to_have_count(1)
|
||
expect(
|
||
page.locator("#messages .msg.brain details.reasoning")
|
||
).not_to_have_attribute("open")
|
||
expect(
|
||
page.locator("#messages .msg.brain .reasoning-status")
|
||
).to_have_text("1 steps")
|
||
# D1 (the grouping contract): no thinking frames → ONE tool run →
|
||
# EXACTLY ONE disclosure INSIDE THE CONTAINER (the phase-117
|
||
# behavior for no-thinking turns is preserved, now wrapped), with
|
||
# the 3 ``.tool-call`` lines and the plural summary.
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.reasoning .reasoning-body "
|
||
".tool-calls-disclosure"
|
||
)
|
||
).to_have_count(1)
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.reasoning .reasoning-body "
|
||
".tool-call"
|
||
)
|
||
).to_have_count(3)
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-summary")
|
||
).to_have_text("Tool calls (3)")
|
||
# And ZERO thinking blocks inside (no thinking ran — no empty
|
||
# block).
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.reasoning .reasoning-body "
|
||
"details.thinking"
|
||
)
|
||
).to_have_count(0)
|
||
|
||
# Folded at rest (the first delta / done settle) + the answer.
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.reasoning .reasoning-body "
|
||
".tool-calls-disclosure[open]"
|
||
)
|
||
).to_have_count(0)
|
||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(ANSWER_PREFIX)
|