Files
brain-of-reese/tests/e2e/test_tool_call_compact.py
T
ducoterra 690fd98036
Build and Push Containers / build-and-push-app (push) Successful in 1m43s
Build and Push Containers / build-and-push-db (push) Successful in 15s
fix(rag): stop the per-run blocks from folding — the container is the one disclosure
The owner refinement (2026-10-06): the per-run thinking and tool-call
blocks inside the Reasoning bubble were individually collapsible
details elements — a fold inside the fold. A tool frame closed the
previous thinking block, so the visible thinking vanished mid-turn;
and the pinned-tail follow died on the append of every run after the
first (the pin was captured AFTER the new block's height growth
contaminated the 32px reading).

- app.js: the in-container runs are now plain divs —
  createThinkingRunEl (div.thinking + static .thinking-summary label +
  .thinking-text) and createToolRunEl (div.tool-calls-disclosure +
  static .tool-calls-summary label + the .tool-calls list); they never
  open or fold (the container is the ONE open/fold control, open
  through the turn, folded at settle — unchanged); the pin capture
  moves BEFORE ensureThinkingRun/ensureToolRun append (plus the
  first-frame edge: no container yet -> always pinned);
  closeCurrentRun just ends the run; closeAllRuns stays the defensive
  legacy guard; restoreTimeline/addSegments (shared.js) use the div
  builders; the legacy no-segments restore path keeps the old
  details-based blocks byte-identical
- styles.css: the in-container runs are distinct by label + rail
  (brand-ink Thinking label on a brand-soft rail; accent-ink
  Tool call(s) (N) label on a neutral rail); the interactive
  treatment (cursor + chevron) is scoped to the legacy <summary>; the
  inner windows stay disabled (the container's .reasoning-body is the
  ONE live scroll surface)
- unit + E2E: the D5-refinement contract is pinned — one
  details.reasoning disclosure, per-run blocks as plain divs, zero
  nested <details> inside the container, the container as the only
  open/fold control, the .reasoning-body window auto-follows the
  tail while streaming

unit+integration 3738 passed (coverage 99%), ruff + pyright clean, all
five affected E2E suites pass in isolation (reasoning_bubble 6,
thinking_timeline 6, thinking_display 5, thinking_scroll 8,
tool_call_compact 4), verified live: the follow holds through a long
thinking stream, the tool-frame transition, and the second thinking
run; restore renders the div structure without client-side errors

LLM-Generated: true
2026-10-06 14:44:41 -04:00

480 lines
21 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Phase 117 E2E (Playwright, mock-only): compact, well-wrapped tool-call lines.
Source: owner visual-glitch report (2026-09-15, mobile viewport,
``https://brain.reeseapps.com``) — "how much space the tool calls take
up, and the tool call text is spit and wrapped poorly": one completed
answer stacked 6+ full-width bordered tool-call cards (one per ``tool``
SSE frame), each a complete card (accent left border + surface
background + radius + a mono path chip inside), with the label flex
item breaking mid-word (``Rea``/``ding``) and the path wrapping to
three lines indented to the right of that broken label.
Phase 117's fix (frontend-only, D1–D6 in ``00_phase.md``): the
per-call lines ride in ONE compact "Tool calls (N)" disclosure, open
while the turn is live and FOLDED when the answer begins (the first
``delta``), on ``done``, on stop/abort, and on restore — and the lines
themselves are deboxed inline-flow text (no card, no flex: the label
+ ``<code>`` run as one continuous sentence and the path wraps to the
left edge). Phase 161 (D2): the disclosure sits INSIDE the ONE
reasoning container (``details.reasoning``), which is the only
collapsible. The D5 refinement (owner 2026-10-06) goes one step
further: the per-run disclosure is now a PLAIN div (no ``<details>``)
— always visible while the container is open; the container is the
ONE disclosure (open while the turn is live, folded at done/stop/
restore).
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_tool_call_compact.py -v --no-cov
MOCK-ONLY suite: the marker question drives the SAME deterministic
3-call flow ``test_agent_document_tools.py`` pins — ``ls`` →
``ls(scoped)`` → ``read`` (``tests/e2e/mock_llm.py``: the user message
contains ``use your tools`` and the system prompt carries the
``<tools>`` section of the HIGH prompt) — over the byte-identical
two-document KB fixture (mirrored from that suite so the flow is
deterministic). No slow proxy: the fold is about a COMPLETED turn (it
folds on the first delta), so the fast mock is the deterministic
driver.
Completion marker: the mock's single-read flow answers
``Read <source/path>. <first 80 chars of the read document>`` — the
deterministic quote below (``MOCK_ANSWER_MARKER``), NOT the plain
grounded answer's "Deterministic mock answer for E2E" tail (the tool
flow never reaches that branch). Its presence in the bubble proves the
first delta has landed — the turn is COMPLETE and the disclosure has
already folded (phase 117, D3: the fold rides the first delta).
Test → phase mapping (Playwright Mapping Rule):
1. ``test_tool_calls_fold_to_one_line_after_turn`` — pin 1: a
completed 3-call turn renders ONE visible "Tool calls (3)" summary
(revealed by opening the container — the ONE disclosure, folded at
rest); the in-container runs are plain divs (zero ``<details>``
nested in the container — the D5 refinement); the three
``.tool-call`` lines are present in the DOM but hidden (the
container is folded); the answer is present.
2. ``test_summary_click_expands_the_calls`` — pin 2: tapping the
container's native ``<summary>`` (the ONE real focusable toggle,
AA) opens the container; the three lines become visible, in order,
with the phase-37/94 pinned text.
3. ``test_expanded_line_is_deboxed_inline_flow`` — pin 3: the
expanded "Reading" line's computed ``display`` is NOT ``flex``
(the label + path are one inline run, not two flex items — the
debox), and the label runs directly into the path in the same run.
4. ``test_restored_turn_renders_folded`` — pin 4: a same-context
RELOAD (the phase-14/50 persisted conversation restores) renders
the turn FOLDED — folded container, plain div runs (zero nested
``<details>``), "Tool calls (3)" summary (revealed by opening the
container), three lines in the DOM, first line hidden — no
auto-expand on load.
"""
from __future__ import annotations
import hashlib
from collections.abc import Callable
from datetime import UTC, datetime
from playwright.sync_api import Page, expect
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import Chunk, Document, GitSource
from e2e.auth_helpers import login
from tests.e2e.mock_llm import embed_text
# --------------------------------------------------------------------------
# KB fixture — byte-identical to the phase-37 suite (the mirror)
# --------------------------------------------------------------------------
SEED_SOURCE = "Homelab"
SEED_PATH = "aws-route53.md"
SEED_SP = f"{SEED_SOURCE}/{SEED_PATH}"
READ_SOURCE = "Deployments"
READ_PATH = "example-record-file.json"
READ_SP = f"{READ_SOURCE}/{READ_PATH}"
#: The retrievable document: references the JSON file "for the exact
#: JSON shape of reeselink.json" but never includes it. The repeated
#: record-file lines carry the marker question's key tokens (aws,
#: route53, hosted, zone, reeselink, json, exact, shape) — verified
#: ≈0.69 cosine against the mock's embeddings (E2E threshold 0.30)
#: plus FTS hits, so the turn is solidly grounded (HIGH prompt →
#: ``<tools>``).
ROUTE53_CONTENT = (
"# AWS Route 53 Notes\n\n"
"## Record file\n\n"
+ (
"The aws route53 hosted zone for reeselink keeps every record in "
"reseelink.json — the exact JSON shape of reeselink.json is "
"documented in example-record-file.json.\n"
)
* 10
+ "\n## Sync job\n\n"
"A cron job pushes reeselink.json to the aws route53 hosted zone "
"every fifteen minutes; the diff is applied through the route53 api.\n"
)
#: The referenced document: the exact JSON shape, seeded WITHOUT chunks
#: (indexed + catalogued + readable, but never a retrieval candidate).
#: Its ``(source, path)`` sorts FIRST in the catalog — the line the
#: mock's drill + read land on.
RECORD_FILE_CONTENT = (
'{ "version": 3, "comment": "ReeseLink hosted zone records — the exact '
'JSON shape of reeselink.json",\n'
' "hosted_zone_id": "Z0RESEELINK01",\n'
' "record_sets": [\n'
' { "name": "www.reeselink.example", "type": "A", "ttl": 300,\n'
' "resource_records": [ { "value": "10.0.0.20" } ] },\n'
' { "name": "api.reeselink.example", "type": "CNAME", "ttl": 300,\n'
' "resource_records": [ { "value": "www.reeselink.example" } ] }\n'
" ]\n"
"}\n"
)
assert "\n" not in RECORD_FILE_CONTENT[:63] # the quote's content part stays one line
#: The SAME marker that drives the mock's 3-tool flow (mirrored from
#: ``test_agent_document_tools.py`` — deterministic ``ls`` →
#: ``ls(scoped)`` → ``read``).
MARKER_QUESTION = (
"Use your tools: what is the exact JSON shape of reeselink.json "
"for my aws route53 hosted zone?"
)
#: The mock's deterministic single-read answer —
#: "Read <source/path>. <first 80 chars of the read document>"
#: (the phase-37 shape). This suite's completion marker: its presence
#: in the bubble proves the FIRST DELTA has landed — the turn is
#: COMPLETE and the disclosure has already folded (phase 117, D3).
MOCK_ANSWER_MARKER = f"Read {READ_SP}."
#: The phase-117 selectors (scoped to the conversation column).
SUMMARY = "#messages .msg.brain .tool-calls-summary"
DISCLOSURE = "#messages .msg.brain .tool-calls-disclosure"
LINES = "#messages .msg.brain .tool-call"
#: Phase 161 (D1/D2): the ONE reasoning container — created on the
#: turn's first tool frame, OPEN live, folded at the settle (D2).
REASONING = "#messages .msg.brain details.reasoning"
# --------------------------------------------------------------------------
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
# --------------------------------------------------------------------------
def _seed(db: Session) -> None:
"""The phase-37 two-document pair, byte-identical (see the module
docstring), plus the phase-94 registry rows: both sources
registered, ``Deployments`` FIRST — the mock's drill (first
source of the top-level listing) lands on the JSON file
deterministically, independent of the operator's
``BOR_GIT_SOURCES`` (a non-empty table ignores the env fallback)."""
# COMMIT between the inserts (not flush): ``added_at`` is
# ``server_default now()`` — the transaction timestamp — and the
# tie-break is the random uuid ``id``, so one-transaction rows order
# nondeterministically.
db.add(GitSource(url=READ_SOURCE, kind="local"))
db.commit()
db.add(GitSource(url=SEED_SOURCE, kind="local"))
md = Document(
source=SEED_SOURCE,
path=SEED_PATH,
full_path=f"/tmp/{SEED_PATH}",
title="AWS Route 53 Notes",
content=ROUTE53_CONTENT,
content_hash=hashlib.sha256(ROUTE53_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
# Phase 106 (D5): explicit dates — byte-stable prompts/quotes
# (the mock's first-80-chars read quote carries the date line).
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
db.add(md)
db.flush()
# One chunk carrying the mock's own embedding → genuine token
# overlap between the marker question and this document.
db.add(
Chunk(
document_id=md.id,
position=0,
content=ROUTE53_CONTENT,
embedding=embed_text(ROUTE53_CONTENT),
)
)
# The referenced JSON: indexed, catalogued, readable — but NO
# chunks, so retrieval never puts it in context.
db.add(
Document(
source=READ_SOURCE,
path=READ_PATH,
full_path=f"/tmp/{READ_PATH}",
title="Example Record File",
content=RECORD_FILE_CONTENT,
content_hash=hashlib.sha256(RECORD_FILE_CONTENT.encode()).hexdigest(),
indexed_at=datetime.now(UTC),
created_at=datetime(2024, 6, 15, tzinfo=UTC),
)
)
def _reset_db(seed: Callable[[Session], None] | None = None) -> None:
"""Truncate the KB (plus the prompt-shaping tables and the
phase-55 auto-save rows), then re-seed — the E2E isolation
pattern. ``steering_notes`` / ``kb_overview`` are truncated too, so
the HIGH prompt is exactly ``<relevance>`` + ``<documents>`` +
``<tools>`` regardless of leftovers — byte-stable prompts,
byte-stable answers."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, "
"steering_notes, kb_overview, saved_chats, git_sources"
)
)
db.commit()
if seed is not None:
seed(db)
db.commit()
# --------------------------------------------------------------------------
# Page helpers
# --------------------------------------------------------------------------
def _container_open(page: Page) -> bool:
"""The container's open state: a closed ``<details>`` has NO
``open`` attribute (``get_attribute`` → ``None``); an open one
carries it (any value — the native boolean attribute is empty).
Phase 161 (D2): the container is the ONE disclosure — the
per-run disclosure is a plain div (the D5 refinement)."""
return page.get_attribute(REASONING, "open") is not None
def _submit_tools_turn(page: Page, app_url: str) -> None:
"""Log in as admin, submit the marker question, and wait for the
answer bubble — i.e. the turn is COMPLETE (the disclosure has
already folded on the first delta).
Auth: the phase-37 agent-tools E2E's pattern (the mock flow + the
house pattern for this flow win)."""
login(page, app_url, next="/")
page.fill("#message-input", MARKER_QUESTION)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(MARKER_QUESTION)
# The answer's deterministic quote — its presence proves the first
# delta has landed (the fold rides that frame, phase 117 D3)…
expect(
page.locator(".msg.brain .bubble").last
).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# …and the turn settled: button recovered (phase-48 — the in-flight
# button is the enabled Stop control, so the label assertion carries
# the settle wait).
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
def _expand_disclosure(page: Page) -> None:
"""Tap the container's summary — the native ``<summary>`` is a
real focusable toggle (AA), so a plain click is the user
contract. The D5 refinement: ONE toggle — the per-run disclosure
is a plain div, always visible while the container is open."""
page.locator(f"{REASONING} > summary").click()
# --------------------------------------------------------------------------
# 1. A completed 3-call turn folds to one "Tool calls (3)" line
# --------------------------------------------------------------------------
def test_tool_calls_fold_to_one_line_after_turn(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""Pin 1 (the space fix): after the deterministic ``ls`` →
``ls(scoped)`` → ``read`` turn COMPLETES, the record is ONE
visible summary line — the disclosure is CLOSED (no ``open``
attribute: the first answer delta folded it, D3), the three
``.tool-call`` lines are still in the DOM (the permanent record)
but hidden (the first line is not visible), and the answer bubble
is present. Phase 161 (D2): the at-rest record is wrapped in the
FOLDED reasoning container — the at-rest visible line is the
container's "Reasoning" summary; the disclosure's "Tool calls (3)"
count is the in-container record, revealed by opening it."""
page.set_default_timeout(30_000)
_reset_db(_seed)
_submit_tools_turn(page, app_url)
# Phase 161 (D2): the container exists (the tool frames created
# it) and folds at done — the at-rest visible line is its summary.
container = page.locator(REASONING)
expect(container).to_have_count(1)
expect(container).not_to_have_attribute("open")
# Folded at rest: the record's lines are hidden (the D5 refinement —
# the at-rest fold is the container's; the per-run runs are plain
# divs that render on reveal).
expect(page.locator(LINES)).to_have_count(3)
expect(page.locator(LINES).first).not_to_be_visible()
page.locator(f"{REASONING} > summary").click()
expect(container).to_have_attribute("open", "")
# One compact summary line, visible, with the live count.
summary = page.locator(SUMMARY)
expect(summary).to_be_visible()
expect(summary).to_contain_text("Tool calls (3)")
# The D5 refinement: the in-container runs are plain divs — the
# per-run disclosure is NOT a <details> (the container is the ONE
# disclosure).
expect(page.locator(f"{REASONING} details")).to_have_count(0)
# The lines are present in the DOM (the permanent record) and
# render on reveal — always visible inside the open container (D5).
expect(page.locator(LINES)).to_have_count(3)
expect(page.locator(LINES).first).to_be_visible()
# The answer is present (the mock's deterministic single-read quote).
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
MOCK_ANSWER_MARKER
)
# --------------------------------------------------------------------------
# 2. Tapping the summary expands the calls
# --------------------------------------------------------------------------
def test_summary_click_expands_the_calls(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""Pin 2 (the expand contract): clicking the native ``<summary>``
opens the disclosure (the ``open`` attribute appears) and all
three ``.tool-call`` lines become visible, in order, with the
phase-37/94 pinned text (the drill line is "Listing documents in
<source>" — the top level lists sources only)."""
page.set_default_timeout(30_000)
_reset_db(_seed)
_submit_tools_turn(page, app_url)
assert not _container_open(page) # folded after the turn (pin 1)
_expand_disclosure(page)
# The native boolean attribute is now present (any value — it is
# empty in the DOM; "true" is the attribute-present shorthand).
assert _container_open(page)
# All three lines, in order, visible with their pinned text.
lines = page.locator(LINES)
expect(lines).to_have_count(3)
expect(lines.first).to_be_visible()
expect(lines.nth(0)).to_be_visible()
expect(lines.nth(0)).to_contain_text("Listing documents")
expect(lines.nth(1)).to_be_visible()
expect(lines.nth(1)).to_contain_text("Listing documents in")
expect(lines.nth(1)).to_contain_text(READ_SOURCE)
expect(lines.nth(2)).to_be_visible()
expect(lines.nth(2)).to_contain_text("Reading ")
expect(lines.nth(2)).to_contain_text(READ_SP)
# --------------------------------------------------------------------------
# 3. An expanded line is deboxed inline-flow text (not two flex items)
# --------------------------------------------------------------------------
def test_expanded_line_is_deboxed_inline_flow(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""Pin 3 (the wrapping fix): on the expanded "Reading" line the
computed ``display`` is NOT ``flex`` — the pre-phase-117 card had
``display: flex; align-items: baseline``, which made the label text
node and the ``<code>`` path two separate flex items (the label
broke mid-word, the path wrapped indented). Deboxed (D4), the label
+ inline ``<code>`` flow as one continuous run: the path wraps to
the left edge like a normal sentence."""
page.set_default_timeout(30_000)
_reset_db(_seed)
_submit_tools_turn(page, app_url)
_expand_disclosure(page)
# The visible "Reading" line (the third of the three).
line = page.locator(LINES).nth(2)
expect(line).to_be_visible()
expect(line).to_contain_text("Reading ")
# The debox: NOT a flex item pair — one inline run. (As a flex item
# of the .tool-calls column its used display blockifies to "block";
# the pre-phase-117 card computed "flex" — the discriminator.)
display = page.evaluate(
"""() => {
const lines = document.querySelectorAll('#messages .msg.brain .tool-call');
const el = Array.from(lines).find(
(l) => l.textContent.includes('Reading '));
return el ? getComputedStyle(el).display : null;
}"""
)
assert display is not None and display != "flex", (
"the expanded .tool-call line must be deboxed (label + path one "
f"inline run, not two flex items) — computed display: {display!r}"
)
# --------------------------------------------------------------------------
# 4. A reload restores the turn FOLDED (no auto-expand on load)
# --------------------------------------------------------------------------
def test_restored_turn_renders_folded(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""Pin 4 (the restore path): after the turn completes (the
phase-50/55 auto-save persisted it — the local ``bor.chat.v1``
record is written at the save points, so the reload sees the full
conversation), a same-context RELOAD re-renders the persisted
record through the restore path (``renderStoredMessage`` →
``restoreTimeline`` → the after-loop fold): the disclosure is
present, CLOSED (no ``open`` attribute — no auto-expand on load),
the summary reads "Tool calls (3)", the three lines are in the DOM
and the first is hidden (folded on load). Phase 161 (D2/D5): the
restored record is wrapped in the FOLDED reasoning container —
the at-rest visible line is the container's "Reasoning" summary;
the disclosure's "Tool calls (3)" count is the in-container
record, revealed by opening the container (the same two-native-
toggle user flow as the live at-rest pins)."""
page.set_default_timeout(30_000)
_reset_db(_seed)
_submit_tools_turn(page, app_url)
page.reload()
expect(page.locator("#empty-state")).to_be_hidden(timeout=30_000)
# The restored brain message's answer (the restore is synchronous at
# boot — the persisted record is byte-stable for this fixture).
expect(
page.locator(".msg.brain .bubble").last
).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# Folded on load (phase 161, D2/D5): the record is wrapped in the
# FOLDED reasoning container — the at-rest visible line is the
# container's "Reasoning" summary.
container = page.locator(REASONING)
expect(container).to_have_count(1)
expect(container).not_to_have_attribute("open")
expect(container.locator("> summary")).to_be_visible()
expect(container.locator("> summary")).to_contain_text("Reasoning")
# No auto-expand on load (the container is folded) — and the
# in-container record is revealed by opening the ONE toggle.
assert not _container_open(page)
# The D5 refinement: the restored runs are plain divs (no nested
# <details>).
expect(page.locator(f"{REASONING} details")).to_have_count(0)
page.locator(f"{REASONING} > summary").click()
expect(page.locator(SUMMARY)).to_be_visible()
expect(page.locator(SUMMARY)).to_contain_text("Tool calls (3)")
# The three lines are in the DOM (the permanent record) and render
# on reveal — always visible inside the open container (D5).
expect(page.locator(LINES)).to_have_count(3)
expect(page.locator(LINES).first).to_be_visible()