All verification complete. Every gate is green and the docs check out. Final report:
## Phase 158 final verification pass — COMPLETE
**Verified (all tasks pre-completed; this pass re-ran the full gate):**
- Unit + integration: `uv run pytest --cov=app --cov-report=term-missing` → **3618 passed, 99% coverage on `app/`** (>90% gate)
- Phase E2E in isolation: `uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov` → **6/6 passed** (order pin, fold-at-rest, live-open, restore+`segments` array, shared page, no-thinking single disclosure)
- Regression E2E, each in isolation: `test_thinking_display` 5✓, `test_tool_call_compact` 4✓, `test_turn_progress_loader` 3✓ (per-run pin updated), `test_big_read_progress` 4✓, `test_agent_document_tools` 4✓, `test_share_chat` 4✓, `test_chat_persistence` 4✓, `test_smoke` 3✓
- Lint/types: `uv run ruff check . && uv run pyright` → **clean (0 errors)**
**Completion criteria:** all 9 met — timeline E2E pins ✓, restore/shared `segments` ✓, no-thinking single-disclosure + the three UNCHANGED regression suites green ✓, pre-phase byte-identical round-trip (integration pin: key absent, never null; >50 / >32 000 / empty text / unknown kind all 422) ✓, full suite + coverage + E2E + lint ✓, `git diff --stat` footprint limited to `app/schemas.py`, `frontend/assets/{app,shared}.js`, `tests/`, README, phase files (a pre-existing owner `TODO.md` edit is out of scope, not staged by the phase commit) ✓, D15→per-run supersession recorded in the updated pin ✓. (Commit + phase move are the harness's job; task dirs already in `complete/`.)
**Documentation:** README.md — Thinking section correctly describes the phase-158 timeline; config reference matches `.env.example` exactly; page table matches registered routes; no stale single-block wording. AGENTS.md — rules and quick-reference commands all accurate. **Verified consistent — no changes needed.**
**Decisions/deviations:** none. `test_tool_call_compact.py` carries a one-line docstring-only update (referenced the removed `closeToolCalls`); its test logic is byte-identical to the phase-117 contract and green.
**Next pending phase:** none — `todo/` contains only 158.
758 lines
31 KiB
Python
758 lines
31 KiB
Python
"""Phase 158 E2E (Playwright, mock-only): the chronological thinking /
|
||
tool-call timeline (the owner's 2026-10-04 report — "the thinking block
|
||
happens first … and then went back to thinking. This is awkward. Then
|
||
the tool calling happens, and the thinking block reads strangely because
|
||
the tool calls are shown below the thinking block").
|
||
|
||
Source: owner report (live interactive-browser session, 2026-10-04,
|
||
shared page ``https://brain.reeseapps.com/shared/
|
||
95dd7a2a-8cda-478b-b694-53936070b9fb`` — a real ``turbo`` turn with 6
|
||
tool calls). Run in isolation (DB must be up:
|
||
``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_thinking_timeline.py -v --no-cov
|
||
|
||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the story's gate
|
||
is the deterministic marker flow in ``tests/e2e/mock_llm.py`` (user
|
||
message contains ``think out loud and use your tools``
|
||
(``THINK_TOOLS_TRIGGER``) **and** the system prompt carries the
|
||
``<tools>`` section of the HIGH prompt): the phase-37 single-read
|
||
shape (list → drill → read → the quoted answer) with a DISTINCT
|
||
``reasoning_content`` run streamed AHEAD of every tool call and ahead
|
||
of the final answer — the ``Step N`` markers make each run
|
||
individually assertable. The client renders each CONTIGUOUS run in its
|
||
own ``details.thinking`` / ``details.tool-calls-disclosure`` (D1), all
|
||
above the answer bubble in arrival order (D2 — Option B,
|
||
owner-chosen), open while the run's frames stream and folded at rest
|
||
(D5), and persists the run order in the record's ``segments`` array
|
||
(D4).
|
||
|
||
KB fixture — mirrored from ``test_agent_document_tools.py`` (the
|
||
deterministic two-document pair): ``Deployments`` registered FIRST so
|
||
the mock's drill (first source of the top-level listing) — and
|
||
therefore the read — lands on the JSON file; the marker question
|
||
cosines ≈0.595 against the retrievable document (well past the E2E
|
||
0.30 threshold) plus FTS hits, so the turn is solidly grounded (HIGH
|
||
prompt → ``<tools>``).
|
||
|
||
Test → phase mapping (Playwright Mapping Rule):
|
||
1. ``test_timeline_order_after_completed_turn`` — D1/D2: after ``done``
|
||
the brain wrap's ``.msg-body`` children before the bubble are
|
||
EXACTLY the 7-element arrival-order sequence (4 thinking blocks +
|
||
3 one-line disclosures, alternating); the four ``.thinking-text``s
|
||
carry the DISTINCT ``Step N`` markers (distinctness = separate
|
||
blocks, not one lump); each disclosure summary is ``Tool call
|
||
(1)``; the answer bubble carries the ``Read <source/path>`` echo.
|
||
2. ``test_runs_fold_at_rest`` — D5: after ``done`` NO ``.thinking`` /
|
||
``.tool-calls-disclosure`` has the ``open`` attribute (the settle
|
||
folded the runs); the summaries are still visible.
|
||
3. ``test_live_runs_open_while_streaming`` — D5 (the live half):
|
||
DURING the turn (the mock's pacing window — the 0.1 s tool frames +
|
||
the thinking stream, the answer bubble still EMPTY) the active run's
|
||
block is ``open`` (the live feedback the owner keeps).
|
||
4. ``test_restored_timeline_and_record`` — the restore path + D4: a
|
||
same-context RELOAD re-renders the same 7-element sequence (all
|
||
collapsed), and the localStorage ``bor.chat.v1`` record (phase 14)
|
||
carries a ``segments`` array of EXACTLY 7 entries in the
|
||
``thinking, tool, ×3`` pattern (the ``Step N`` text on the thinking
|
||
entries, the ``name``/``argument`` on the tool entries); the legacy
|
||
``thinking`` (full concatenation) + ``tools`` (3 records) keys are
|
||
STILL present (the old-client graceful degradation).
|
||
5. ``test_shared_page_timeline`` — the shared-page parity (phase 51
|
||
save + share flow): a FRESH context on ``/shared/<token>`` renders
|
||
the same 7-element sequence, all collapsed (``addSegments``), the
|
||
``Step N`` markers visible on summary click (expand ONE), the answer
|
||
bubble carries the ``Read <source/path>`` echo.
|
||
6. ``test_no_thinking_turn_single_disclosure`` — the run-grouping
|
||
contract for no-thinking turns (the phase-117 behavior preserved): a
|
||
plain ``use your tools`` turn (NO thinking trigger) renders EXACTLY
|
||
ONE ``.tool-calls-disclosure`` (3 ``.tool-call`` lines, summary
|
||
``Tool calls (3)``) and ZERO ``.thinking``.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import hashlib
|
||
import re
|
||
import time
|
||
from collections.abc import Callable
|
||
from datetime import UTC, datetime
|
||
from typing import Any
|
||
|
||
import httpx
|
||
from playwright.sync_api import Browser, BrowserContext, Page, expect
|
||
from sqlalchemy import text
|
||
from sqlalchemy.orm import Session
|
||
|
||
from app.db import SessionLocal
|
||
from app.models import Chunk, Document, GitSource
|
||
from e2e.auth_helpers import login
|
||
from tests.e2e.mock_llm import embed_text
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Fixture documents (deterministic, token-controlled — mirrored from
|
||
# test_agent_document_tools.py so the flow is byte-stable)
|
||
# --------------------------------------------------------------------------
|
||
|
||
SEED_SOURCE = "Homelab"
|
||
SEED_PATH = "aws-route53.md"
|
||
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
|
||
|
||
READ_SOURCE = "Deployments"
|
||
READ_PATH = "example-record-file.json"
|
||
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
|
||
|
||
#: The retrievable document: references the JSON file "for the exact
|
||
#: JSON shape of reeselink.json" but never includes it. The repeated
|
||
#: record-file lines carry the marker question's key tokens (aws,
|
||
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
|
||
#: ≈0.595 cosine against the mock's embeddings (E2E threshold 0.30)
|
||
#: plus FTS hits, so the turn is solidly grounded.
|
||
ROUTE53_CONTENT = (
|
||
"# AWS Route 53 Notes\n\n"
|
||
"## Record file\n\n"
|
||
+ (
|
||
"The aws route53 hosted zone for reeselink keeps every record in "
|
||
"reseelink.json — the exact JSON shape of reeselink.json is "
|
||
"documented in example-record-file.json.\n"
|
||
)
|
||
* 10
|
||
+ "\n## Sync job\n\n"
|
||
"A cron job pushes reeselink.json to the aws route53 hosted zone "
|
||
"every fifteen minutes; the diff is applied through the route53 api.\n"
|
||
)
|
||
|
||
#: The referenced document: the exact JSON shape. Its FIRST line is
|
||
#: longer than 80 chars, so the mock's first-80-chars quote content
|
||
#: part is newline-free (the rendered-text assertions below match it
|
||
#: after the date line). Pinned by the assert below.
|
||
RECORD_FILE_CONTENT = (
|
||
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
|
||
'JSON shape of reeselink.json",\n'
|
||
' "hosted_zone_id": "Z0RESEELINK01",\n'
|
||
' "record_sets": [\n'
|
||
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
|
||
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
|
||
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
|
||
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
|
||
" ]\n"
|
||
"}\n"
|
||
)
|
||
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
|
||
|
||
#: The phase-158 trigger question (``THINK_TOOLS_TRIGGER`` + the KB
|
||
#: topic — the ≈0.595 cosine keeps the turn grounded over the mock-
|
||
#: calibrated E2E threshold).
|
||
TIMELINE_QUESTION = (
|
||
"Think out loud and use your tools: what is the exact JSON shape of "
|
||
"reseelink.json for my aws route53 hosted zone?"
|
||
)
|
||
|
||
#: The NO-thinking control — the plain ``use your tools`` marker
|
||
#: (``test_agent_document_tools.py`` / ``test_tool_call_compact.py``'s
|
||
#: question, mirrored so the 3-call flow is the phase-37/117 one).
|
||
PLAIN_TOOLS_QUESTION = (
|
||
"Use your tools: what is the exact JSON shape of reeselink.json "
|
||
"for my aws route53 hosted zone?"
|
||
)
|
||
|
||
#: The mock's deterministic single-read answer —
|
||
#: "Read <source/path>. <first 80 chars of the read document>" (the
|
||
#: phase-37 shape). Phase 106 (D5): the read result's ``date:`` SECOND
|
||
#: line rides into the first-80-chars quote — the quote is the date
|
||
#: line (the fixture's fixed ``created_at`` UTC date part, 17 chars) +
|
||
#: the first 63 content chars (80 − 17).
|
||
ANSWER_PREFIX = f"Read {READ_SP}."
|
||
ANSWER_QUOTE = "date: 2024-06-15" + RECORD_FILE_CONTENT[:63]
|
||
|
||
#: The mock's four DISTINCT reasoning runs (the ``Step N`` markers —
|
||
#: their distinctness proves the runs are separate blocks, not one
|
||
#: lump).
|
||
STEP_MARKERS = ("Step 1", "Step 2", "Step 3", "Step 4")
|
||
|
||
#: The 7-element pre-bubble segment sequence the timeline renders, in
|
||
#: arrival order (D1: alternating thinking/tool runs — one block per
|
||
#: run) — tests 1/4 (chat page) and 5 (shared page) pin it.
|
||
TIMELINE_SEQ = [
|
||
"details.thinking",
|
||
"details.tool-calls-disclosure",
|
||
"details.thinking",
|
||
"details.tool-calls-disclosure",
|
||
"details.thinking",
|
||
"details.tool-calls-disclosure",
|
||
"details.thinking",
|
||
]
|
||
|
||
SHARE_URL_RE = re.compile(r"^/shared/[0-9a-f-]{36}$")
|
||
|
||
# --------------------------------------------------------------------------
|
||
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _seed(db: Session) -> None:
|
||
"""The two-document pair (see the module docstring), plus the
|
||
phase-94 registry rows: both sources registered, ``Deployments``
|
||
FIRST — the mock's drill (first source of the top-level listing)
|
||
lands on the JSON file deterministically, independent of the
|
||
operator's ``BOR_GIT_SOURCES`` (a non-empty table ignores the env
|
||
fallback)."""
|
||
# COMMIT between the inserts (not flush): ``added_at`` is
|
||
# ``server_default now()`` — the transaction timestamp — and the
|
||
# tie-break is the random uuid ``id``, so one-transaction rows order
|
||
# nondeterministically.
|
||
db.add(GitSource(url=READ_SOURCE, kind="local"))
|
||
db.commit()
|
||
db.add(GitSource(url=SEED_SOURCE, kind="local"))
|
||
md = Document(
|
||
source=SEED_SOURCE,
|
||
path=SEED_PATH,
|
||
full_path=f"/tmp/{SEED_PATH}",
|
||
title="AWS Route 53 Notes",
|
||
content=ROUTE53_CONTENT,
|
||
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
|
||
indexed_at=datetime.now(UTC),
|
||
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
|
||
# (the mock's first-80-chars read quote carries the date line).
|
||
created_at=datetime(2024, 6, 15, tzinfo=UTC),
|
||
)
|
||
db.add(md)
|
||
db.flush()
|
||
# One chunk carrying the mock's own embedding → genuine token
|
||
# overlap between the marker question and this document.
|
||
db.add(
|
||
Chunk(
|
||
document_id=md.id,
|
||
position=0,
|
||
content=ROUTE53_CONTENT,
|
||
embedding=embed_text(ROUTE53_CONTENT),
|
||
)
|
||
)
|
||
# The referenced JSON: indexed, catalogued, readable — but NO
|
||
# chunks, so retrieval never puts it in context (the failure the
|
||
# tools fix).
|
||
db.add(
|
||
Document(
|
||
source=READ_SOURCE,
|
||
path=READ_PATH,
|
||
full_path=f"/tmp/{READ_PATH}",
|
||
title="Example Record File",
|
||
content=RECORD_FILE_CONTENT,
|
||
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
|
||
indexed_at=datetime.now(UTC),
|
||
created_at=datetime(2024, 6, 15, tzinfo=UTC),
|
||
)
|
||
)
|
||
|
||
|
||
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
|
||
"""Truncate the KB (plus the prompt-shaping tables and the
|
||
phase-55 auto-save rows), then re-seed — the E2E isolation
|
||
pattern. ``steering_notes`` / ``kb_overview`` are truncated too,
|
||
so the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
|
||
``<tools>`` regardless of leftovers — byte-stable prompts,
|
||
byte-stable answers."""
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, "
|
||
"steering_notes, kb_overview, saved_chats, git_sources"
|
||
)
|
||
)
|
||
db.commit()
|
||
if seed is not None:
|
||
seed(db)
|
||
db.commit()
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Page helpers
|
||
# --------------------------------------------------------------------------
|
||
|
||
#: The last ``.msg.brain`` wrap's ``.msg-body`` children BEFORE the
|
||
#: bubble, as ``tag.className`` strings — the pre-bubble segment
|
||
#: sequence (the timeline's order pin; the ``.msg-meta`` row lands
|
||
#: AFTER the bubble, so it never enters the sequence).
|
||
BODY_SEQUENCE_JS = """
|
||
() => {
|
||
const wrap = document.querySelector('#messages .msg.brain');
|
||
if (!wrap) return null;
|
||
const body = wrap.querySelector('.msg-body');
|
||
const out = [];
|
||
for (const child of body.children) {
|
||
if (child.classList.contains('bubble')) break;
|
||
out.push(child.tagName.toLowerCase() + '.' + child.className);
|
||
}
|
||
return out;
|
||
}
|
||
"""
|
||
|
||
#: D5's live-open predicate — the active run's block is ``open`` while
|
||
#: the answer bubble is STILL EMPTY (the turn is in flight; the mock's
|
||
#: pacing window — the 0.1 s tool frames + the thinking stream — keeps
|
||
#: the window comfortably open for a 100 ms poll).
|
||
LIVE_OPEN_JS = """
|
||
() => {
|
||
const wrap = document.querySelector('#messages .msg.brain');
|
||
if (!wrap) return false;
|
||
const bubble = wrap.querySelector('.msg-body .bubble');
|
||
if (!bubble || bubble.textContent !== '') return false;
|
||
return !!wrap.querySelector(
|
||
'details.thinking[open], details.tool-calls-disclosure[open]');
|
||
}
|
||
"""
|
||
|
||
#: The phase-14 localStorage record's LAST brain message (the
|
||
#: ``bor.chat.v1`` — ``{ v: 1, chatId, messages: [...] }``; the
|
||
#: filter mirrors ``loadStoredRecord``'s well-formedness).
|
||
READ_RECORD_JS = """
|
||
() => {
|
||
const raw = localStorage.getItem('bor.chat.v1');
|
||
if (!raw) return null;
|
||
const data = JSON.parse(raw);
|
||
const brain = (data.messages || []).filter(
|
||
(m) => m && m.who === 'brain' && typeof m.text === 'string');
|
||
return brain.length ? brain[brain.length - 1] : null;
|
||
}
|
||
"""
|
||
|
||
|
||
def _submit(page: Page, question: str) -> None:
|
||
page.fill("#message-input", question)
|
||
page.click("#send-btn")
|
||
# The user bubble lands synchronously with the submit handler.
|
||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||
|
||
|
||
def _wait_settled(page: Page) -> None:
|
||
"""The turn is complete: answer text in the bubble, button
|
||
recovered (phase 48 — the in-flight button is the enabled Stop
|
||
control, so the label assertion carries the settle wait)."""
|
||
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
|
||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Phase-51 save + share flow (the test_share_chat.py pattern — the
|
||
# auto-saved row is shared from the chat page; the row is deleted in a
|
||
# finally)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _admin_cookies(page: Page) -> dict[str, str]:
|
||
return {
|
||
c["name"]: c["value"]
|
||
for c in page.context.cookies()
|
||
if "name" in c and "value" in c
|
||
}
|
||
|
||
|
||
def _chats(app_url: str, cookies: dict[str, str]) -> list[dict[str, Any]]:
|
||
r = httpx.get(f"{app_url}/api/chats", timeout=10, cookies=cookies)
|
||
assert r.status_code == 200
|
||
return r.json()["chats"]
|
||
|
||
|
||
def _auto_title(question: str) -> str:
|
||
"""The phase-50 auto-title convention: the first question,
|
||
whitespace-collapsed, capped at 120 chars."""
|
||
return " ".join(question.split())[:120]
|
||
|
||
|
||
def _find_row(rows: list[dict[str, Any]], title: str) -> dict[str, Any] | None:
|
||
return next((c for c in rows if c["title"] == title), None)
|
||
|
||
|
||
def _delete_chat(app_url: str, cookies: dict[str, str], chat_id: str) -> None:
|
||
"""Best-effort row cleanup (a 404 — already deleted — is fine)."""
|
||
httpx.delete(f"{app_url}/api/chats/{chat_id}", timeout=10, cookies=cookies)
|
||
|
||
|
||
def _wait_saved_row(
|
||
app_url: str,
|
||
cookies: dict[str, str],
|
||
title: str,
|
||
messages: int = 2,
|
||
) -> dict[str, Any]:
|
||
"""Wait for the auto-saved row (phase 55 — auto-saves are SILENT,
|
||
so there is no status line to wait on). The upsert is
|
||
fire-and-forget from the UI's point of view, so poll the admin
|
||
list until the row with the conversation's auto-title appears with
|
||
the expected message count."""
|
||
deadline = time.monotonic() + 15
|
||
last: dict[str, Any] | None = None
|
||
while time.monotonic() < deadline:
|
||
last = _find_row(_chats(app_url, cookies), title)
|
||
if last is not None and last["message_count"] >= messages:
|
||
return last
|
||
time.sleep(0.2)
|
||
raise AssertionError(f"no auto-saved row for {title!r} (last: {last!r})")
|
||
|
||
|
||
def _grant_clipboard(page: Page, app_url: str) -> None:
|
||
"""Grant the async-clipboard permissions on the admin context
|
||
(headless Chromium requires the grant before ``writeText``
|
||
resolves — without it the inline-link fallback fires)."""
|
||
page.context.grant_permissions(
|
||
["clipboard-read", "clipboard-write"], origin=app_url
|
||
)
|
||
|
||
|
||
def _click_share_and_assert_status(page: Page, app_url: str) -> None:
|
||
"""Press the chat page's Share pill and pin the outcome on the
|
||
live region (the assertion branches on the API's availability —
|
||
the test_share_chat.py convention)."""
|
||
page.locator("#share-chat-btn").click()
|
||
if page.evaluate("() => !!navigator.clipboard"):
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link copied.", timeout=15_000
|
||
)
|
||
else:
|
||
expect(page.locator("#send-status")).to_have_text(
|
||
"Share link ready — copy it from the field.", timeout=15_000
|
||
)
|
||
field = page.locator(".share-link-fallback")
|
||
expect(field).to_be_visible()
|
||
href = field.get_attribute("href")
|
||
assert href is not None
|
||
assert href.startswith(app_url)
|
||
assert SHARE_URL_RE.fullmatch(href.removeprefix(app_url))
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 1. D1/D2: the completed turn renders the 7-element arrival-order
|
||
# timeline (the runs are separate blocks, not one lump)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_timeline_order_after_completed_turn(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# D1: the pre-bubble segment sequence is EXACTLY the 7-element
|
||
# arrival-order timeline — thinking, tool, ×3 (one block per
|
||
# contiguous run; the plain ``use your tools`` flow's 3 calls land
|
||
# in 3 SEPARATE one-line disclosures, one per thinking run).
|
||
seq = page.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the pre-bubble segment sequence is not the timeline: {seq}"
|
||
)
|
||
|
||
# D1/D3: the four thinking blocks carry the DISTINCT ``Step N``
|
||
# markers — distinctness proves the runs are separate blocks, not
|
||
# one lump (the pre-phase single-block layout would put all four
|
||
# markers in one ``.thinking-text``).
|
||
texts = page.locator("#messages .msg.brain .thinking-text")
|
||
expect(texts).to_have_count(4)
|
||
for i, marker in enumerate(STEP_MARKERS):
|
||
expect(texts.nth(i)).to_contain_text(marker)
|
||
|
||
# D3: each tool run's disclosure summary is per-run — one line →
|
||
# "Tool call (1)" (the phase-117 singular form).
|
||
summaries = page.locator("#messages .msg.brain .tool-calls-summary")
|
||
expect(summaries).to_have_count(3)
|
||
for i in range(3):
|
||
expect(summaries.nth(i)).to_have_text("Tool call (1)")
|
||
|
||
# The answer bubble carries the ``Read <source/path>`` echo (the
|
||
# read document reached the model and landed in the answer).
|
||
bubble = page.locator(".msg.brain .bubble").last
|
||
expect(bubble).to_contain_text(ANSWER_PREFIX)
|
||
expect(bubble).to_contain_text(ANSWER_QUOTE)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 2. D5: the runs fold at rest (done settled every open block)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_runs_fold_at_rest(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# D5: at rest NOTHING is open — the ``done`` settle (closeAllRuns)
|
||
# folded every run (a closed <details> has no ``open`` attribute).
|
||
expect(
|
||
page.locator("#messages .msg.brain details.thinking[open]")
|
||
).to_have_count(0)
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.tool-calls-disclosure[open]"
|
||
)
|
||
).to_have_count(0)
|
||
|
||
# The collapsed summaries are still visible (the timeline stays
|
||
# readable at rest — 4 "Thinking" + 3 "Tool call (1)" lines).
|
||
expect(
|
||
page.locator("#messages .msg.brain details.thinking > summary").first
|
||
).to_be_visible()
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-summary").first
|
||
).to_be_visible()
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 3. D5: the active run's block stays OPEN while the turn streams
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_live_runs_open_while_streaming(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
page.fill("#message-input", TIMELINE_QUESTION)
|
||
page.click("#send-btn")
|
||
expect(page.locator(".msg.user .bubble").last).to_contain_text(TIMELINE_QUESTION)
|
||
|
||
# D5 (the live half): during the turn — the mock's pacing window
|
||
# (the 0.1 s tool frames + the thinking stream; the answer bubble
|
||
# is still EMPTY — the first delta has not landed) — the active
|
||
# run's block is ``open`` (the live feedback the owner keeps:
|
||
# thinking/tool lines stream into their OPEN blocks, folded only
|
||
# by the first delta / done / stop). The house deadline-poll
|
||
# pattern (the ``test_agent_document_tools.py`` ``_frames``
|
||
# convention): a 50 ms stride over the ~2 s turn comfortably
|
||
# samples the open windows.
|
||
seen_live_open = False
|
||
deadline = time.monotonic() + 30.0
|
||
while time.monotonic() < deadline:
|
||
if page.evaluate(LIVE_OPEN_JS):
|
||
seen_live_open = True
|
||
break
|
||
time.sleep(0.05)
|
||
assert seen_live_open, (
|
||
"no open thinking/tool run was observed while the answer "
|
||
"bubble was still empty (the D5 live-open window)"
|
||
)
|
||
|
||
# The turn settles (the poll caught the live window; the settle is
|
||
# the D5 fold — test 2's contract).
|
||
_wait_settled(page)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 4. The restore path (the same 7-element timeline, collapsed) + the
|
||
# record's ``segments`` array (D4 — the legacy keys ride along)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_restored_timeline_and_record(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# RELOAD (same context) — the phase-14 record restores through the
|
||
# restore path (restoreTimeline — the segments take precedence).
|
||
page.reload()
|
||
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
|
||
|
||
# The restored message renders the SAME 7-element timeline, all
|
||
# COLLAPSED (D5 — the restore renders collapsed).
|
||
seq = page.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the restored pre-bubble sequence is not the timeline: {seq}"
|
||
)
|
||
expect(
|
||
page.locator("#messages .msg.brain details.thinking[open]")
|
||
).to_have_count(0)
|
||
expect(
|
||
page.locator(
|
||
"#messages .msg.brain details.tool-calls-disclosure[open]"
|
||
)
|
||
).to_have_count(0)
|
||
# The restored answer (the restore is synchronous at boot).
|
||
expect(
|
||
page.locator(".msg.brain .bubble").last
|
||
).to_contain_text(ANSWER_PREFIX, timeout=30_000)
|
||
|
||
# D4: the localStorage record (``bor.chat.v1`` — phase 14) carries
|
||
# a ``segments`` array of EXACTLY 7 entries in the arrival-order
|
||
# pattern — 4 thinking runs (the ``Step N`` text) + 3 tool entries
|
||
# (the call records, in order).
|
||
rec = page.evaluate(READ_RECORD_JS)
|
||
assert rec is not None, "the phase-14 record must exist after the reload"
|
||
segments = rec.get("segments")
|
||
assert isinstance(segments, list), (
|
||
f"the record must carry a segments array, got {type(segments)}"
|
||
)
|
||
assert len(segments) == 7, f"expected EXACTLY 7 segments, got {len(segments)}"
|
||
assert [s.get("kind") for s in segments] == [
|
||
"thinking", "tool", "thinking", "tool", "thinking", "tool", "thinking"
|
||
], f"the kind sequence is not the arrival-order pattern: {segments}"
|
||
# The thinking entries carry the DISTINCT ``Step N`` runs.
|
||
for i, marker in enumerate(STEP_MARKERS):
|
||
text = segments[i * 2].get("text")
|
||
assert isinstance(text, str) and marker in text, (
|
||
f"thinking segment {i} must carry {marker!r}: {text!r}"
|
||
)
|
||
# The tool entries are the 3 call records, in call order.
|
||
assert segments[1].get("name") == "ls"
|
||
assert segments[1].get("argument") is None # the no-arg top-level ls
|
||
assert segments[3].get("name") == "ls"
|
||
assert segments[3].get("argument") == READ_SOURCE # the drill scope
|
||
assert segments[5].get("name") == "read"
|
||
assert segments[5].get("argument") == READ_SP # the combined identity
|
||
|
||
# D4 (the old-client graceful degradation): the legacy fallback
|
||
# fields are STILL present — the full ``thinking`` concatenation
|
||
# (feeds the history wire + the per-turn log) + every ``tools``
|
||
# record (the old-client render path).
|
||
thinking_full = rec.get("thinking")
|
||
assert isinstance(thinking_full, str) and thinking_full, (
|
||
"the full thinking concatenation must still be present"
|
||
)
|
||
for marker in STEP_MARKERS:
|
||
assert marker in thinking_full, (
|
||
f"the concatenated thinking must carry {marker!r}"
|
||
)
|
||
tools = rec.get("tools")
|
||
assert isinstance(tools, list) and len(tools) == 3, (
|
||
f"the tools record must still carry the 3 calls: {tools!r}"
|
||
)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 5. The shared page (phase-51 save + share flow): the same 7-element
|
||
# timeline, collapsed (the addSegments parity)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_shared_page_timeline(
|
||
page: Page,
|
||
browser: Browser,
|
||
app_url: str,
|
||
mock_llm: int,
|
||
db_ready: None,
|
||
) -> None:
|
||
_reset_db(_seed)
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url, next="/")
|
||
expect(page).to_have_url(app_url + "/", timeout=30_000)
|
||
|
||
_submit(page, TIMELINE_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# The phase-51 flow: the conversation is AUTO-SAVED by the time the
|
||
# answer settles (phase 55) — the Share click shares the linked row.
|
||
cookies = _admin_cookies(page)
|
||
_grant_clipboard(page, app_url)
|
||
_wait_saved_row(app_url, cookies, _auto_title(TIMELINE_QUESTION))
|
||
share = page.locator("#share-chat-btn")
|
||
expect(share).to_be_visible()
|
||
_click_share_and_assert_status(page, app_url)
|
||
row = _find_row(_chats(app_url, cookies), _auto_title(TIMELINE_QUESTION))
|
||
assert row is not None and row.get("share_url"), (
|
||
"the shared row must carry a share_url"
|
||
)
|
||
share_url: str = row["share_url"]
|
||
assert SHARE_URL_RE.fullmatch(share_url), f"bad share_url shape: {share_url}"
|
||
created: str = row["id"]
|
||
anon_ctx: BrowserContext | None = None
|
||
try:
|
||
# A FRESH context: a separate session with no cookies — the
|
||
# guest's only credential is the token in the URL.
|
||
anon_ctx = browser.new_context()
|
||
anon = anon_ctx.new_page()
|
||
anon.set_default_timeout(30_000)
|
||
anon.goto(app_url + share_url)
|
||
|
||
# The title = the auto-title (the shared chat's h1).
|
||
expect(anon.locator("#shared-title")).to_have_text(_auto_title(TIMELINE_QUESTION))
|
||
|
||
# The SAME 7-element timeline renders (the ``addSegments``
|
||
# parity), all COLLAPSED (D5 — the shared page is a finished
|
||
# conversation; a closed <details> has no ``open`` attribute).
|
||
seq = anon.evaluate(BODY_SEQUENCE_JS)
|
||
assert seq == TIMELINE_SEQ, (
|
||
f"the shared pre-bubble sequence is not the timeline: {seq}"
|
||
)
|
||
expect(
|
||
anon.locator("#messages .msg.brain details.thinking[open]")
|
||
).to_have_count(0)
|
||
expect(
|
||
anon.locator(
|
||
"#messages .msg.brain details.tool-calls-disclosure[open]"
|
||
)
|
||
).to_have_count(0)
|
||
|
||
# The ``Step N`` markers are visible on a summary click (expand
|
||
# ONE — the native <summary> is a real focusable toggle).
|
||
anon.locator("#messages .msg.brain details.thinking > summary").first.click()
|
||
expect(
|
||
anon.locator("#messages .msg.brain details.thinking").first
|
||
).to_have_attribute("open")
|
||
first_text = anon.locator("#messages .msg.brain .thinking-text").first
|
||
expect(first_text).to_be_visible()
|
||
expect(first_text).to_contain_text(STEP_MARKERS[0])
|
||
|
||
# The answer bubble carries the ``Read <source/path>`` echo.
|
||
expect(
|
||
anon.locator(".msg.brain .bubble").first
|
||
).to_contain_text(ANSWER_PREFIX)
|
||
finally:
|
||
if anon_ctx is not None:
|
||
anon_ctx.close()
|
||
if created:
|
||
_delete_chat(app_url, cookies, created)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 6. The run-grouping contract for no-thinking turns (phase 117
|
||
# preserved): one disclosure, 3 lines, zero thinking blocks
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_no_thinking_turn_single_disclosure(
|
||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
_reset_db(_seed)
|
||
login(page, app_url, next="/")
|
||
|
||
# The plain ``use your tools`` marker (NO thinking trigger — the
|
||
# mock's tool frames carry no ``reasoning_content``): the 3 calls
|
||
# are ONE contiguous tool run.
|
||
_submit(page, PLAIN_TOOLS_QUESTION)
|
||
_wait_settled(page)
|
||
|
||
# D1 (the grouping contract): no thinking frames → ONE tool run →
|
||
# EXACTLY ONE disclosure (the phase-117 behavior for no-thinking
|
||
# turns is preserved), with the 3 ``.tool-call`` lines and the
|
||
# plural summary.
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-disclosure")
|
||
).to_have_count(1)
|
||
expect(page.locator("#messages .msg.brain .tool-call")).to_have_count(3)
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-summary")
|
||
).to_have_text("Tool calls (3)")
|
||
# And ZERO thinking blocks (no thinking ran — no empty block).
|
||
expect(page.locator("#messages .msg.brain details.thinking")).to_have_count(0)
|
||
|
||
# Folded at rest (the first delta / done settle) + the answer.
|
||
expect(
|
||
page.locator("#messages .msg.brain .tool-calls-disclosure[open]")
|
||
).to_have_count(0)
|
||
expect(page.locator(".msg.brain .bubble").last).to_contain_text(ANSWER_PREFIX)
|