phase: 109_turn_progress_loader
All gates green. Final report: **Phase 109 — final verification pass (all 3 tasks were already complete; no re-implementation needed)** - Verified D15 (thinking-handler re-open `block.open = true` after `ensureThinkingBlock`; delta keeps closing; "never reopens" narrative removed; restore path still collapses) and D16 (single static `#turn-loader` in composer row, `aria-hidden`, sole `turnLoader.hidden = !inFlight` write in `setUiState`; CSS reuses `typing` keyframes + reduced-motion override after the main rule + provenance; global `[hidden]` guard intact; mock `TURN_PROGRESS_TRIGGER` repro marker; `app/` untouched — pure UI phase). - Unit pins: `tests/unit/test_frontend_turn_loader.py` (9 tests) — green via full run. - `uv run pytest tests/e2e/test_turn_progress_loader.py -v --no-cov` → **3 passed** (isolation, DB up): loader visible send→tool-gap→final-thinking, hidden after done; re-opened scratchpad carries `marker-thought-42`/`marker-final-thought-42`; `#send-status` carries state text, back to empty idle; reduced-motion context stills dots (computed `animation-name: none`) with loader still visible. - Regressions in isolation: `test_thinking_display` 5 passed, `test_stop_generation` 3 passed, `test_big_read_progress` 4 passed, `test_loading_feedback` 5 passed. - `uv run pytest` → **2314 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90%); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings. - All terminal paths (done→idle L2554, stream error→error L2543, timeout→error L2247) funnel through `setUiState` — never-stale by construction. - No defects found; no code changes made in this pass. Commit + phase-dir move left to the harness per pipeline rules (working tree carries all changes). - Next pending phase: `108_history_wire_check` (still in `todo/`; out of scope here).
This commit is contained in:
@@ -357,6 +357,44 @@ Implements just enough of the aipi surface:
|
||||
client never sends tool calls/results) — so every existing marker
|
||||
flow (which classifies statelessly from TOOL results and the LAST
|
||||
user message) is unaffected by the now-always-present history.
|
||||
- user message containing ``answer first, then list, then think``
|
||||
(``TURN_PROGRESS_TRIGGER``, phase 109 task 03 — the never-frozen-
|
||||
turn story's dedicated suite
|
||||
``tests/e2e/test_turn_progress_loader.py``) **and** the system
|
||||
prompt carries the ``<tools>`` section -> the deterministic
|
||||
REPORTED-REPRO turn (TODO.md L3: "the model responds, calls a
|
||||
tool, then continues thinking without re-expanding the thinking
|
||||
block"): the owner's exact sequence, deterministic, with baked-in
|
||||
delays (mid-turn windows of ≥1 s each — the ``slow_llm.py``
|
||||
pacing precedent) so the story E2E's window assertions are
|
||||
race-free. Discriminated statelessly from the messages (streaming
|
||||
only):
|
||||
* request 1 (``tools`` offered, no tool results yet): ~2 s pre-
|
||||
delay (model latency — the loader's start-state window), a
|
||||
short ``delta.content`` stream (3 chunks, NO reasoning — the
|
||||
answer starts FIRST), then the ``ls`` tool call (synthetic id
|
||||
``call_0``, no arguments — the phase-37 pattern),
|
||||
``finish_reason: "tool_calls"``;
|
||||
* request 2 (a ``tool``-role result in the messages — the server
|
||||
ran the ``ls``): a ~7 s frameless gap (long enough that the
|
||||
phase-87 tool-line ``(Ns)`` counter — 5 s+ after the line's
|
||||
own arm — appears and ticks BEFORE the first ``thinking``
|
||||
frame settles the line), then the ``reasoning_content`` stream
|
||||
(10 × 0.3 s), a ``delta.content`` stream (3 chunks ending in
|
||||
the distinctive final sentence carrying
|
||||
``marker-progress-42``), and the FINAL ``reasoning_content``
|
||||
chunks (3 × 0.3 s — thinking AFTER the answer, the reported
|
||||
repro), ``finish_reason: "stop"``.
|
||||
The server is position-independent over the wire (each
|
||||
``reasoning_content`` chunk → a ``thinking`` SSE frame, each
|
||||
``content`` chunk → a ``delta`` frame — ``app/rag/llm.py``; the
|
||||
tool call materializes after its request's stream — the
|
||||
content-before-tools convention), so the turn's SSE is exactly
|
||||
``delta → tool → thinking → delta → thinking → done``.
|
||||
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
|
||||
phrases — the phase-71/72/94 ordering convention); verified
|
||||
2026-09-16: no existing E2E question or fixture file contains the
|
||||
phrase, so every other suite is unaffected.
|
||||
|
||||
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
|
||||
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
|
||||
@@ -1597,6 +1635,207 @@ def _folder_summary_incident(body: dict[str, Any]) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 109 (task 03, the never-frozen-turn story suite): the
|
||||
# deterministic REPORTED-REPRO turn (delta → tool →
|
||||
# thinking-after-delta) — see the module docstring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A user message containing this substring (case-insensitive) —
|
||||
#: combined with the ``<tools>`` section in the system prompt — drives
|
||||
#: the deterministic REPORTED-REPRO turn (TODO.md L3): the scripted
|
||||
#: first answer (no reasoning), the no-arg ``ls``, then the post-tool
|
||||
#: thinking → answer → FINAL thinking round — the owner's exact
|
||||
#: "the model responds, calls a tool, then continues thinking" sequence
|
||||
#: with baked-in delays (mid-turn windows of ≥1 s each — the
|
||||
#: ``slow_llm.py`` pacing precedent) so the story E2E's window
|
||||
#: assertions are race-free. Checked BEFORE the plain ``TOOLS_TRIGGER``
|
||||
#: flow (disjoint trigger phrases — the phase-71/72/94 ordering
|
||||
#: convention); verified 2026-09-16: no existing E2E question or
|
||||
#: fixture file contains the phrase, so every other suite is
|
||||
#: unaffected.
|
||||
TURN_PROGRESS_TRIGGER = "answer first, then list, then think"
|
||||
|
||||
#: Request 1's pre-delay (s) — model latency: the loader's start-state
|
||||
#: window (the turn is in flight, NO frame has arrived yet).
|
||||
TURN_PROGRESS_PREDELAY_S = 2.0
|
||||
|
||||
#: Request 2's frameless gap (s) — the silence after the ``tool`` frame
|
||||
#: before the first ``thinking`` frame. Deliberately past the phase-87
|
||||
#: tool-line counter's 5 s gate (``TOOL_LINE_ELAPSED_AFTER_MS`` in
|
||||
#: app.js) with ≥2 s of margin: the line's ``(Ns)`` suffix appears
|
||||
#: (5 s tick) and ticks (6 s) BEFORE the first thinking frame settles
|
||||
#: the line, so the story E2E can pin the counter in the tool gap
|
||||
#: without a race.
|
||||
TURN_PROGRESS_TOOL_GAP_S = 7.0
|
||||
|
||||
#: Per-chunk delays (s) for the two streams of request 2.
|
||||
TURN_PROGRESS_THINK_DELAY_S = 0.3 # reasoning chunks (~10 × 0.3 s, ~3 × 0.3 s)
|
||||
TURN_PROGRESS_CONTENT_DELAY_S = 0.05 # the short content deltas (3 + 3 chunks)
|
||||
|
||||
#: Chunk sizes (chars): the content deltas stay SHORT (2-3 chunks each),
|
||||
#: the reasoning streams run 10 × 30 and 3 × 30.
|
||||
TURN_PROGRESS_CONTENT_CHUNK = 40
|
||||
TURN_PROGRESS_THINK_CHUNK = 30
|
||||
|
||||
#: Request 1's answer (byte-stable): 3 × 40-char content chunks, NO
|
||||
#: reasoning — the answer starts FIRST (the repro's "the model
|
||||
#: responds"). ``marker-progress-41`` is the sentinel the story E2E
|
||||
#: matches to prove call 1's content landed in the bubble.
|
||||
TURN_PROGRESS_FIRST_ANSWER = (
|
||||
"Checking the listing first — opening answer: the kubernetes setup, "
|
||||
"short and sweet. marker-progress-41."
|
||||
)
|
||||
|
||||
#: Request 2's scratchpad (byte-stable): 10 × 30-char reasoning chunks —
|
||||
#: thinking AFTER the answer, the reported repro. ``marker-thought-42``
|
||||
#: is the sentinel the story E2E matches in the re-opened block.
|
||||
TURN_PROGRESS_THINKING = (
|
||||
"The listing just landed — now I can see which documents exist, so the "
|
||||
"answer can anchor to the kubernetes file first and the deployments note "
|
||||
"second, citing each fact by the exact path it came from. Hosts, "
|
||||
"versions, and ports stay exactly as the notes write them. "
|
||||
"marker-thought-42."
|
||||
)
|
||||
|
||||
#: Request 2's answer (byte-stable): 3 × 40-char content chunks ending in
|
||||
#: the DISTINCTIVE final sentence the story E2E matches on to pin the
|
||||
#: post-delta state (``marker-progress-42`` — the repro's second answer).
|
||||
TURN_PROGRESS_FINAL_ANSWER = (
|
||||
"Here is the plan after the listing: step one, step two, step three — "
|
||||
"that is the whole of it. marker-progress-42."
|
||||
)
|
||||
|
||||
#: Request 2's FINAL reasoning (byte-stable): 3 × 30-char chunks — the
|
||||
#: thinking that arrives AFTER the answer's last delta. The story E2E
|
||||
#: waits for the re-opened block after the final sentence landed and
|
||||
#: matches ``marker-final-thought-42`` to prove the last round's
|
||||
#: thinking rendered in the scratchpad.
|
||||
TURN_PROGRESS_FINAL_THINKING = (
|
||||
"Final check — hosts and ports are verbatim. marker-final-thought-42."
|
||||
)
|
||||
|
||||
|
||||
def _turn_progress_flow(body: dict[str, Any]) -> str | None:
|
||||
"""Classify a phase-109 reported-repro request (see the module
|
||||
docstring). Stateless over the messages, like the other marker
|
||||
flows:
|
||||
|
||||
* ``"first"`` — ``tools`` are offered and no ``tool``-role result
|
||||
is in the messages yet: request 1 — the ~2 s pre-delay (model
|
||||
latency — the loader's start-state window), the short
|
||||
``delta.content`` stream (3 chunks, NO reasoning — the answer
|
||||
starts first), then the ``ls`` tool call (synthetic id
|
||||
``call_0``, no arguments), ``finish_reason: "tool_calls"``.
|
||||
* ``"second"`` — a ``tool``-role result is in the messages (the
|
||||
server ran the ``ls``): request 2 — the ~7 s frameless gap
|
||||
(the phase-87 window), the ``reasoning_content`` stream (10 ×
|
||||
0.3 s), the ``delta.content`` stream (3 chunks ending in the
|
||||
distinctive final sentence), and the FINAL
|
||||
``reasoning_content`` chunks (3 × 0.3 s) — thinking AFTER the
|
||||
answer — then ``finish_reason: "stop"``.
|
||||
* ``None`` — not the flow: the trigger is absent, the ``<tools>``
|
||||
section is missing (deflected turns never carry it), or
|
||||
``tools`` are not offered and no tool result is in the messages
|
||||
yet (e.g. ``agent_max_rounds=0``).
|
||||
"""
|
||||
if TURN_PROGRESS_TRIGGER not in _user(body).lower():
|
||||
return None
|
||||
if "<tools>" not in _system(body):
|
||||
return None
|
||||
if _tool_results(body):
|
||||
return "second"
|
||||
if not body.get("tools"):
|
||||
return None
|
||||
return "first"
|
||||
|
||||
|
||||
def _turn_progress_stream(step: str) -> Any:
|
||||
"""SSE frames for one phase-109 reported-repro request.
|
||||
|
||||
``"first"`` (model call 1): the ~2 s pre-delay, the short
|
||||
``delta.content`` stream (3 chunks — the answer starts first, NO
|
||||
reasoning), the ``ls`` tool-call partial (synthetic id ``call_0``,
|
||||
no arguments — the phase-37 ``_tool_call_stream`` first-partial
|
||||
shape), and the ``finish_reason: "tool_calls"`` frame. The app is
|
||||
position-independent over the wire (``app/rag/llm.py``): the
|
||||
content chunks stream as ``delta`` SSE frames as they arrive and
|
||||
the tool call materializes AFTER the stream (the content-before-
|
||||
tools convention), so the request's SSE is exactly ``delta →
|
||||
tool``.
|
||||
|
||||
``"second"`` (model call 2 — after the server ran the ``ls``): the
|
||||
~7 s frameless gap (the phase-87 window — the tool line's
|
||||
``(Ns)`` counter appears and ticks before the first frame settles
|
||||
it), the ``reasoning_content`` stream (10 × 30 chars, 0.3 s each),
|
||||
the ``delta.content`` stream (3 chunks ending in the distinctive
|
||||
final sentence), the FINAL ``reasoning_content`` chunks (3 × 30
|
||||
chars, 0.3 s each — thinking AFTER the answer, the reported
|
||||
repro), and the ``finish_reason: "stop"`` frame.
|
||||
"""
|
||||
model = "turbo"
|
||||
chunk_id = f"chatcmpl-{uuid.uuid4()}"
|
||||
|
||||
def frame(delta: dict[str, Any], finish: str | None = None) -> str:
|
||||
return (
|
||||
"data: "
|
||||
+ json_dumps(
|
||||
{
|
||||
"id": chunk_id,
|
||||
"object": "chat.completion.chunk",
|
||||
"created": int(time.time()),
|
||||
"model": model,
|
||||
"choices": [{"index": 0, "delta": delta, "finish_reason": finish}],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
)
|
||||
|
||||
if step == "first":
|
||||
time.sleep(TURN_PROGRESS_PREDELAY_S)
|
||||
for piece in re.findall(
|
||||
rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FIRST_ANSWER, re.S
|
||||
):
|
||||
yield frame({"content": piece})
|
||||
time.sleep(TURN_PROGRESS_CONTENT_DELAY_S)
|
||||
yield frame(
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 0,
|
||||
"id": "call_0",
|
||||
"type": "function",
|
||||
"function": {"name": "ls", "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
time.sleep(0.1)
|
||||
yield frame({}, "tool_calls")
|
||||
yield "data: [DONE]\n\n"
|
||||
return
|
||||
|
||||
time.sleep(TURN_PROGRESS_TOOL_GAP_S)
|
||||
for piece in re.findall(
|
||||
rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_THINKING, re.S
|
||||
):
|
||||
yield frame({"reasoning_content": piece})
|
||||
time.sleep(TURN_PROGRESS_THINK_DELAY_S)
|
||||
for piece in re.findall(
|
||||
rf".{{1,{TURN_PROGRESS_CONTENT_CHUNK}}}", TURN_PROGRESS_FINAL_ANSWER, re.S
|
||||
):
|
||||
yield frame({"content": piece})
|
||||
time.sleep(TURN_PROGRESS_CONTENT_DELAY_S)
|
||||
for piece in re.findall(
|
||||
rf".{{1,{TURN_PROGRESS_THINK_CHUNK}}}", TURN_PROGRESS_FINAL_THINKING, re.S
|
||||
):
|
||||
yield frame({"reasoning_content": piece})
|
||||
time.sleep(TURN_PROGRESS_THINK_DELAY_S)
|
||||
yield frame({}, "stop")
|
||||
yield "data: [DONE]\n\n"
|
||||
|
||||
|
||||
def compose_answer(body: dict[str, Any]) -> str:
|
||||
system = _system(body)
|
||||
user = _user(body)
|
||||
@@ -2222,6 +2461,23 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 109 (task 03, never-frozen-turn story suite): the
|
||||
# deterministic REPORTED-REPRO turn (delta → tool →
|
||||
# thinking-after-delta — TODO.md L3): the scripted first answer,
|
||||
# the no-arg ls, then the post-tool thinking → answer → FINAL
|
||||
# thinking round — the owner's exact repro with baked-in delays
|
||||
# (see the module docstring). Checked BEFORE the plain
|
||||
# TOOLS_TRIGGER flow (disjoint trigger phrases — the
|
||||
# phase-71/72/94 ordering convention; the trigger needs the
|
||||
# ``<tools>`` section, so deflected turns never hit it).
|
||||
turn_progress = _turn_progress_flow(body)
|
||||
if turn_progress is not None:
|
||||
stream = _turn_progress_stream(turn_progress)
|
||||
return StreamingResponse(
|
||||
stream,
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
flow = _tool_flow(body)
|
||||
if flow is not None:
|
||||
if flow[0] == "list":
|
||||
|
||||
@@ -0,0 +1,321 @@
|
||||
"""Phase 109 E2E (Playwright): the never-frozen turn — the reported
|
||||
repro (delta → tool → thinking-after-delta) replayed deterministically.
|
||||
|
||||
Story: n/a — owner request 2026-09-16 (TODO.md L3): "Thinking can
|
||||
happen after the model starts responding. This sometimes results in a
|
||||
the chat appearing 'frozen' because the model responds, calls a tool,
|
||||
then continues thinking without re-expanding the thinking block. There
|
||||
should be a visual that the chat is still progressing regardless of
|
||||
what state it's in (some kind of loader will do)."
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_turn_progress_loader.py -v --no-cov
|
||||
|
||||
Marker contract: the mock's ``answer first, then list, then think``
|
||||
marker (``TURN_PROGRESS_TRIGGER`` — see the ``mock_llm.py`` module
|
||||
docstring) drives the deterministic reported repro. Model call 1
|
||||
streams a short content delta (3 chunks, NO reasoning) after a ~2 s
|
||||
pre-delay (model latency — the loader's start-state window), then the
|
||||
no-arg ``ls`` tool call; model call 2 (after the server ran the ``ls``)
|
||||
streams a ~7 s frameless gap (past the phase-87 tool-line ``(Ns)``
|
||||
counter's 5 s gate), then a ``reasoning_content`` stream (10 × 0.3 s),
|
||||
a content delta (3 chunks ending in the distinctive final sentence
|
||||
carrying ``marker-progress-42``), and a FINAL ``reasoning_content``
|
||||
chunk group (3 × 0.3 s). The server is position-independent over the
|
||||
wire (``app/rag/llm.py`` — each ``reasoning_content`` chunk → a
|
||||
``thinking`` SSE frame, each ``content`` chunk → a ``delta`` frame;
|
||||
the tool call materializes after its request's stream), so the turn's
|
||||
SSE is exactly ``delta → tool → thinking → delta → thinking → done`` —
|
||||
the owner's repro, and every window is deterministic (the mock's
|
||||
baked delays, the ``slow_llm.py`` pacing precedent).
|
||||
|
||||
Each test sends the marker question in its OWN fresh conversation
|
||||
(one full turn per test — the ``page`` fixture is a fresh browser
|
||||
context, ``_reset_db`` reseeds the KB deterministically):
|
||||
|
||||
* ``test_loader_visible_from_send_through_the_tool_gap`` — the loader
|
||||
is visible right after the send (the thinking state, no frame yet —
|
||||
the start-state window), is STILL visible when the ``.tool-call``
|
||||
line lands (the turn is provably in flight), the tool line carries
|
||||
the phase-87 elapsed counter in the tool gap, and ``#send-status``
|
||||
carries a state text (not empty) throughout.
|
||||
* ``test_thinking_block_reopens_after_delta_with_visible_loader`` —
|
||||
THE reported symptom's state: the thinking block re-opens after the
|
||||
answer has started (open + non-empty ``.thinking-text`` while the
|
||||
bubble already carries call 1's content) with the loader STILL
|
||||
visible (the frozen window is gone); the terminal state is clean
|
||||
(loader hidden, the scratchpad carries both thinking rounds, the
|
||||
bubble carries BOTH answers, the send button reads "Send" not
|
||||
"Stop", ``#send-status`` back to the idle shape).
|
||||
* ``test_loader_a11y_and_reduced_motion`` — after a full turn: the
|
||||
loader element is ``aria-hidden="true"`` in the DOM and
|
||||
``#send-status`` (the sole ``aria-live`` announcer) is back to the
|
||||
``SEND_STATUS`` idle shape (empty — not stuck on a mid-turn label);
|
||||
then, in a context with ``reducedMotion: "reduce"`` (the Playwright
|
||||
context option), a second turn shows the loader STILL visible
|
||||
mid-turn — the reduced-motion variant stills the dots (computed
|
||||
``animation-name: none``), it does not HIDE the cue: visibility is
|
||||
owned by the JS ``hidden`` attribute in ``setUiState`` (D16), not by
|
||||
CSS.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
from pathlib import Path
|
||||
from threading import Thread
|
||||
from typing import Any
|
||||
|
||||
from playwright.sync_api import Browser, Page, expect
|
||||
from sqlalchemy import text
|
||||
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient
|
||||
from e2e.auth_helpers import login
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
FIXTURES = REPO / "tests" / "fixtures" / "docs"
|
||||
|
||||
#: The marker question — the mock's ``TURN_PROGRESS_TRIGGER`` phrase +
|
||||
#: an on-topic tail (HIGH gate → the ``<tools>`` section, so the
|
||||
#: scripted ``ls`` round is reachable — the phase-37/70 convention).
|
||||
QUESTION = "answer first, then list, then think — how is my Kubernetes cluster set up?"
|
||||
|
||||
#: The mock's byte-stable sentinels (``mock_llm.py``): call 1's
|
||||
#: content, call 2's final sentence, call 2's scratchpad, call 2's
|
||||
#: FINAL scratchpad (the thinking after the answer's last delta).
|
||||
SENTINEL_FIRST = "marker-progress-41"
|
||||
SENTINEL_FINAL = "marker-progress-42"
|
||||
SENTINEL_THOUGHT = "marker-thought-42"
|
||||
SENTINEL_FINAL_THOUGHT = "marker-final-thought-42"
|
||||
|
||||
|
||||
async def _import_fixtures(mock_port: int) -> ImportSummary:
|
||||
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
|
||||
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
||||
return await import_sources([FIXTURES], LLMClient(settings))
|
||||
|
||||
|
||||
def _run_in_thread(coro: Any) -> Any:
|
||||
"""Run a coroutine on a worker thread.
|
||||
|
||||
Playwright's sync API keeps an asyncio loop running on the test
|
||||
thread, so ``asyncio.run`` cannot be called directly from a test
|
||||
body.
|
||||
"""
|
||||
box: dict[str, Any] = {}
|
||||
|
||||
def runner() -> None:
|
||||
try:
|
||||
box["value"] = asyncio.run(coro)
|
||||
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
|
||||
box["error"] = e
|
||||
|
||||
t = Thread(target=runner)
|
||||
t.start()
|
||||
t.join()
|
||||
if "error" in box:
|
||||
raise box["error"]
|
||||
return box["value"]
|
||||
|
||||
|
||||
def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
|
||||
"""Truncate the KB (+ query log + steering notes — deterministic
|
||||
mock answers), then optionally re-import fixtures."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE chunks, documents, query_log, steering_notes"))
|
||||
db.commit()
|
||||
if not seed:
|
||||
return None
|
||||
return _run_in_thread(_import_fixtures(mock_port))
|
||||
|
||||
|
||||
def _send(page: Page) -> None:
|
||||
"""Send the marker question (one full reported-repro turn)."""
|
||||
page.fill("#message-input", QUESTION)
|
||||
page.click("#send-btn")
|
||||
|
||||
|
||||
def _answer_bubble(page: Page) -> Any:
|
||||
"""The answer bubble (the typing indicator's bubble is ``.typing``
|
||||
— scoped out)."""
|
||||
return page.locator(".msg.brain .bubble:not(.typing)")
|
||||
|
||||
|
||||
def test_loader_visible_from_send_through_the_tool_gap(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
summary = _reset_db(mock_llm, seed=True)
|
||||
assert summary is not None and summary.added == 13 # A9 formats (phase 47 added quadlet+j2)
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
_send(page)
|
||||
|
||||
# Right after the send — inside call 1's ~2 s pre-delay window:
|
||||
# the turn is in flight (the thinking state, NO frame has arrived
|
||||
# yet) and the loader is ALREADY visible — the start-state cue.
|
||||
# ``setUiState(thinking)`` runs synchronously in the submit path,
|
||||
# so this is race-free: the button is the Stop control, the
|
||||
# loader is out, and no frame-derived surface exists yet (no
|
||||
# answer bubble, no thinking block, no tool line — only the
|
||||
# pre-delta typing bubble).
|
||||
loader = page.locator("#turn-loader")
|
||||
expect(loader).to_be_visible()
|
||||
expect(page.locator("#send-label")).to_have_text("Stop")
|
||||
expect(page.locator("#typing-indicator")).to_be_visible()
|
||||
expect(_answer_bubble(page)).to_have_count(0)
|
||||
expect(page.locator(".msg.brain .thinking")).to_have_count(0)
|
||||
expect(page.locator(".tool-call")).to_have_count(0)
|
||||
|
||||
# The ``.tool-call`` line lands (call 1's content + the ls): the
|
||||
# turn is provably in flight — and the loader is STILL visible.
|
||||
expect(page.locator(".tool-call")).to_be_visible(timeout=30_000)
|
||||
expect(loader).to_be_visible()
|
||||
expect(_answer_bubble(page)).to_contain_text(SENTINEL_FIRST)
|
||||
|
||||
# The tool gap (call 2's ~7 s frameless delay — deliberately past
|
||||
# the phase-87 counter's 5 s gate): the tool line's elapsed
|
||||
# counter appears and ticks BEFORE the first thinking frame
|
||||
# settles the line ...
|
||||
elapsed = page.locator(".tool-call .tool-elapsed")
|
||||
expect(elapsed).to_be_visible(timeout=20_000)
|
||||
expect(elapsed).to_have_text(re.compile(r"\(\d+s\)"))
|
||||
# ... and the loader + the #send-status state text hold through
|
||||
# the whole gap (the status line carries the streaming state's
|
||||
# text — it is never empty while a frame is in flight).
|
||||
expect(loader).to_be_visible()
|
||||
expect(page.locator("#send-status")).not_to_be_empty()
|
||||
|
||||
|
||||
def test_thinking_block_reopens_after_delta_with_visible_loader(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
summary = _reset_db(mock_llm, seed=True)
|
||||
assert summary is not None and summary.added == 13 # A9 formats (phase 47 added quadlet+j2)
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
_send(page)
|
||||
|
||||
bubble = _answer_bubble(page)
|
||||
block = page.locator(".msg.brain .thinking")
|
||||
text_el = page.locator(".msg.brain .thinking-text")
|
||||
loader = page.locator("#turn-loader")
|
||||
|
||||
# THE reported symptom's state (D15): the thinking block is open
|
||||
# with non-empty text WHILE the answer bubble already carries
|
||||
# call 1's content — the post-delta re-open (call 2's thinking
|
||||
# frames re-opened the block the first delta had closed).
|
||||
page.wait_for_function(
|
||||
"""() => {
|
||||
const b = document.querySelector('.msg.brain .thinking');
|
||||
const t = b ? b.querySelector('.thinking-text') : null;
|
||||
return !!b && b.open && !!t && t.textContent.trim().length > 0;
|
||||
}""",
|
||||
timeout=30_000,
|
||||
)
|
||||
expect(block).to_have_count(1)
|
||||
expect(block).to_have_attribute("open", "")
|
||||
expect(text_el).to_contain_text(SENTINEL_THOUGHT)
|
||||
# The re-open is genuinely AFTER a delta: call 1's content is
|
||||
# already in the bubble ...
|
||||
expect(bubble).to_contain_text(SENTINEL_FIRST)
|
||||
# ... and the loader is STILL visible — the frozen window is gone.
|
||||
expect(loader).to_be_visible()
|
||||
|
||||
# Terminal approach: call 2's distinctive final sentence landed in
|
||||
# the bubble ...
|
||||
expect(bubble).to_contain_text(SENTINEL_FINAL, timeout=30_000)
|
||||
# ... and the FINAL thinking round (the last frames before done —
|
||||
# thinking AFTER the answer's last delta) re-opened the block
|
||||
# AGAIN: the scratchpad carries the last round's text and the
|
||||
# block is open (the `thinking` handler opens it before it
|
||||
# renders — D15), with the loader still visible (the turn is
|
||||
# still in flight — no terminal frame yet).
|
||||
expect(text_el).to_contain_text(SENTINEL_FINAL_THOUGHT, timeout=30_000)
|
||||
expect(block).to_have_attribute("open", "")
|
||||
expect(loader).to_be_visible()
|
||||
|
||||
# The turn settles (done → idle through setUiState): the loader is
|
||||
# HIDDEN (its sole owner — D16, no per-handler cleanup) ...
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
expect(loader).to_be_hidden()
|
||||
# ... the scratchpad text is intact (BOTH thinking rounds) ...
|
||||
expect(text_el).to_contain_text(SENTINEL_THOUGHT)
|
||||
expect(text_el).to_contain_text(SENTINEL_FINAL_THOUGHT)
|
||||
# ... the bubble carries BOTH answers (call 1 + call 2) ...
|
||||
expect(bubble).to_contain_text(SENTINEL_FIRST)
|
||||
expect(bubble).to_contain_text(SENTINEL_FINAL)
|
||||
# ... the send button is back to "Send" (asserted above via the
|
||||
# wait) and NOT the Stop treatment ...
|
||||
expect(page.locator("#send-btn")).not_to_have_class(re.compile(r"is-stop"))
|
||||
# ... and #send-status is back to the idle shape (not stuck on a
|
||||
# mid-turn label).
|
||||
expect(page.locator("#send-status")).to_have_text("")
|
||||
|
||||
|
||||
def test_loader_a11y_and_reduced_motion(
|
||||
page: Page, browser: Browser, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
summary = _reset_db(mock_llm, seed=True)
|
||||
assert summary is not None and summary.added == 13 # A9 formats (phase 47 added quadlet+j2)
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
_send(page)
|
||||
# The full turn lands (the distinctive final sentence + done →
|
||||
# the Send button back).
|
||||
expect(_answer_bubble(page)).to_contain_text(SENTINEL_FINAL, timeout=60_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
# The a11y split: the loader element is ``aria-hidden="true"`` in
|
||||
# the DOM (decorative — it never carries meaning) ...
|
||||
expect(page.locator("#turn-loader")).to_have_attribute("aria-hidden", "true")
|
||||
# ... and #send-status (the sole ``aria-live`` announcer) is back
|
||||
# to the SEND_STATUS idle shape after done — EMPTY, not stuck on a
|
||||
# mid-turn label ("… is thinking" / "… is answering").
|
||||
status = page.locator("#send-status")
|
||||
expect(status).to_have_attribute("aria-live", "polite")
|
||||
expect(status).to_have_text("")
|
||||
# Control (default-motion context): the loader's dots run the
|
||||
# typing animation — the reduced-motion check below stills it.
|
||||
assert (
|
||||
page.evaluate(
|
||||
"getComputedStyle(document.querySelector('#turn-loader')).animationName"
|
||||
)
|
||||
== "typing"
|
||||
)
|
||||
|
||||
# Reduced motion: a FRESH context with ``reducedMotion: "reduce"``
|
||||
# (the Playwright context option) — send a second turn and pin
|
||||
# that the loader is STILL visible mid-turn: the reduced-motion
|
||||
# variant stills the dots (the CSS rule changes — computed
|
||||
# ``animation-name: none``), but visibility is owned by the JS
|
||||
# ``hidden`` attribute in setUiState (D16), not by CSS.
|
||||
rm_context = browser.new_context(
|
||||
reduced_motion="reduce", viewport={"width": 1280, "height": 800}
|
||||
)
|
||||
try:
|
||||
rm_page = rm_context.new_page()
|
||||
rm_page.set_default_timeout(30_000)
|
||||
login(rm_page, app_url, next="/")
|
||||
_send(rm_page)
|
||||
# Mid-turn (the tool gap — call 2's ~7 s frameless delay): the
|
||||
# loader is still the visible cue ...
|
||||
expect(rm_page.locator(".tool-call")).to_be_visible(timeout=30_000)
|
||||
expect(rm_page.locator("#turn-loader")).to_be_visible()
|
||||
# ... and the reduced-motion CSS variant IS applied: static
|
||||
# dots, no pulse (the animation is gone from the computed
|
||||
# style — the element itself is unchanged).
|
||||
assert (
|
||||
rm_page.evaluate(
|
||||
"getComputedStyle(document.querySelector('#turn-loader')).animationName"
|
||||
)
|
||||
== "none"
|
||||
)
|
||||
finally:
|
||||
rm_context.close()
|
||||
Reference in New Issue
Block a user