phase: 120_failed_turn_retry
All verification complete. Final report: **Phase 120 — Failed-turn retry: verification pass (all 3 tasks were done; final verification + 1 regression fix)** **Verified:** `ChatMessage.failed`/`error` (≤500, `extra="forbid"` intact); `finalizeFailedTurn` funnel on the 3 failure paths (catch-else, stream-drop guard, zero-frame fallback) with `failed: true` + capped detail + `markLastRetryable`; `appendFailedNote` restore branch (Save-as-doc/Tune excluded); `showErrorBanner`/`retryLastTurn` byte-pinned untouched; only the three paths persist `failed: true` (grep + unit pin); no test asserts the old broken behavior. **Defect found & fixed (rule 7):** a real navigate-away mid-turn let the browser's teardown fetch rejection (TypeError, not AbortError) leak into the failed funnel, persisting a phantom failed brain record — `test_sources_midstream_bug.py::test_no_orphan_brain_message_when_navigated_before_first_token` failed (2 `.msg` after reload) and violated the phase-20 navigate-away convention. Fixed: turn-scoped `leftThePage` flag (set unconditionally on `pagehide`, reset in `runTurn`) skips the funnel in the catch-else branch; pinned by new unit test `test_navigate_away_is_not_a_failed_turn`. No phase-overview/PLAN/todo/complete files touched; no commits made. **Gates (exact):** - `uv run pytest` → 2577 passed - `uv run pytest --cov=app --cov-report=term-missing` → TOTAL 4271 stmts, 99% (>90%) - `uv run pytest tests/e2e/test_failed_turn_retry.py -v --no-cov` → 4 passed (isolated) - `uv run ruff check . && uv run pyright` → clean (0 errors) - Regression E2E, isolated: `test_sources_midstream_bug.py` 6/6 (was 5/6); `test_llm_retry`/`test_tool_scaffolding_guardrails`/`test_stop_generation`/`test_navbar_refresh` 17/17 **Completion criteria:** (1) network error → banner + in-bubble Retry, re-ask without re-typing ✅ (E2E A); (2) refresh restores failed bubble + working Retry, no "new chat" ✅ (E2E C); (3) stopped/successful turns byte-identical ✅ (negative E2E, stop suite, byte-identity units); (4) pytest/coverage/lint/types ✅; (5) commit + phase move — left to the harness per pass rules. **Notable:** deviation = the regression fix above (a navigation is not a failed turn; phase-20 partial-persist convention restored). Next pending phase: `121_git_source_tokens`.
This commit is contained in:
@@ -521,6 +521,36 @@ test_llm_retry.py``). The mock is single-conversation per e2e server, so
|
||||
strings are this suite's own folder names, so no other E2E can hit
|
||||
them (they seed different trees).
|
||||
|
||||
Failure injection (phase 120, failed-turn retry, TODO.md L3–4) —
|
||||
deterministic failures for the failed-turn E2E suite
|
||||
(``tests/e2e/test_failed_turn_retry.py``), reusing the phase-67
|
||||
counter machinery:
|
||||
- user message containing ``fail first turn``
|
||||
(``FAIL_FIRST_TURN_TRIGGER``): EVERY app-level attempt of the FIRST
|
||||
turn responds 500 — the full forced-default budget (conftest pins
|
||||
``BOR_LLM_RETRIES`` to the code default 3 → 4 app-level attempts =
|
||||
12 POSTs, ``FAIL_FIRST_TURN_DEAD_ATTEMPTS`` ×
|
||||
``_HTTPS_PER_DEAD_ATTEMPT``) — the retry-budget exhaustion with
|
||||
ZERO frames (the terminal ``error`` before any delta) — and the
|
||||
SECOND turn's first request streams the normal composed answer (the
|
||||
banner/in-bubble Retry's re-ask). The sequence resets after that
|
||||
success, so a second question carrying the trigger re-drives the
|
||||
failure from zero (the phase-67 convention).
|
||||
- user message containing ``partial then fail``
|
||||
(``PARTIAL_FAIL_TRIGGER``): the FIRST matching streaming request
|
||||
sends ``PARTIAL_FAIL_TEXT`` in normal 12-char ``delta.content``
|
||||
chunks and then the generator RAISES — a genuine mid-stream
|
||||
connection reset (the body ends without ``finish_reason`` /
|
||||
``[DONE]``; a mid-stream failure never re-POSTs — the body was
|
||||
already flowing). The app's ``chat_stream_retried`` re-raises the
|
||||
wrapped ``LLMError`` (a piece already emitted — the locked
|
||||
retry-before-first-frame rule) and the chat endpoint settles the
|
||||
turn with the terminal SSE ``error`` frame AFTER the partial
|
||||
deltas (the partial-then-error wire: the partial bubble keeps its
|
||||
text + the error note + the Retry). The SECOND request streams the
|
||||
normal composed answer (the retry's re-ask) and resets the
|
||||
sequence.
|
||||
|
||||
``max_tokens`` is honored deterministically (token ≈ whitespace word),
|
||||
like a real endpoint: an answer longer than the cap is truncated. This
|
||||
is what makes the phase-11 truncation regression observable.
|
||||
@@ -750,6 +780,46 @@ ALWAYS_FAIL_TRIGGER = "always fail"
|
||||
#: bag-of-words vector — the endpoint's pre-stream embedding retry loop.
|
||||
EMBED_FAIL_TRIGGER = "embed fail once"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 120 (failed-turn retry, TODO.md L3–4): deterministic failure
|
||||
# injections for the failed-turn E2E suite (tests/e2e/
|
||||
# test_failed_turn_retry.py) — see the module docstring
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A user message containing this substring (case-insensitive) dies for
|
||||
#: the ENTIRE first turn — every app-level attempt 500s (the
|
||||
#: retry-budget exhaustion, the zero-frame terminal ``error``) — and
|
||||
#: streams the normal composed answer from the SECOND turn on (the
|
||||
#: banner/in-bubble Retry's re-ask, phase 111/120). The dead window is
|
||||
#: exactly ONE turn: ``FAIL_FIRST_TURN_DEAD_ATTEMPTS`` = the e2e
|
||||
#: forced-default budget (conftest pins ``BOR_LLM_RETRIES`` to the code
|
||||
#: default 3 → 4 app-level attempts = 12 POSTs at
|
||||
#: ``_HTTPS_PER_DEAD_ATTEMPT``). Each sequence resets after the success
|
||||
#: it guards, so a second question carrying the trigger re-drives the
|
||||
#: failure from zero (the phase-67 ``_fail_posts`` convention).
|
||||
FAIL_FIRST_TURN_TRIGGER = "fail first turn"
|
||||
#: One full turn under the forced-default budget (3 retries → 4
|
||||
#: attempts): the whole first turn dies, the second turn's first
|
||||
#: attempt streams (the Retry's re-ask).
|
||||
FAIL_FIRST_TURN_DEAD_ATTEMPTS = 4
|
||||
|
||||
#: A user message containing this substring (case-insensitive) streams
|
||||
#: ``PARTIAL_FAIL_TEXT`` as ordinary ``delta.content`` chunks on its
|
||||
#: FIRST matching request and then the mock's generator RAISES (a
|
||||
#: mid-stream connection reset — the SSE body ends without
|
||||
#: ``finish_reason``/``[DONE]``; a mid-stream failure never re-POSTs).
|
||||
#: The app's ``chat_stream_retried`` re-raises the wrapped ``LLMError``
|
||||
#: (a piece already emitted — the locked retry-before-first-frame rule)
|
||||
#: and the chat endpoint settles with the terminal SSE ``error`` frame
|
||||
#: AFTER the partial deltas. The SECOND request streams the normal
|
||||
#: composed answer (the retry's re-ask) and resets the sequence.
|
||||
PARTIAL_FAIL_TRIGGER = "partial then fail"
|
||||
#: Matching requests that partial-then-die (one POST — see above).
|
||||
PARTIAL_FAIL_DEAD_REQUESTS = 1
|
||||
#: The deterministic partial answer (byte-stable — the E2E asserts the
|
||||
#: bubble KEEPS exactly this text, with the error note appended).
|
||||
PARTIAL_FAIL_TEXT = "Here is the start of the answer that never finished."
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 71 (tool-scaffolding guardrails, 2026-09-03 incident):
|
||||
# deterministic raw-markup flows — see the module docstring
|
||||
@@ -987,6 +1057,32 @@ def _chat_dead(key: str, dead_attempts: int) -> bool:
|
||||
return _bump_fail(key) <= dead_attempts * _HTTPS_PER_DEAD_ATTEMPT
|
||||
|
||||
|
||||
def _partial_fail_stream() -> Any:
|
||||
"""The phase-120 partial-then-fail stream (``PARTIAL_FAIL_TRIGGER``):
|
||||
``PARTIAL_FAIL_TEXT`` in 12-char ``delta.content`` chunks (the
|
||||
mock's default cadence — NO ``finish_reason``, NO ``[DONE]``), then
|
||||
a RAISE that ends the SSE body abruptly (a connection reset
|
||||
mid-stream — the client SDK sees a truncated body, the app wraps it
|
||||
in ``LLMError`` after the pieces already emitted, and the chat
|
||||
endpoint's terminal ``error`` frame lands AFTER the partial deltas
|
||||
on the client's stream)."""
|
||||
model = "turbo"
|
||||
chunk_id = f"chatcmpl-{uuid.uuid4()}"
|
||||
for piece in re.findall(r".{1,12}", PARTIAL_FAIL_TEXT, re.S):
|
||||
payload = {
|
||||
"id": chunk_id,
|
||||
"object": "chat.completion.chunk",
|
||||
"created": int(time.time()),
|
||||
"model": model,
|
||||
"choices": [
|
||||
{"index": 0, "delta": {"content": piece}, "finish_reason": None}
|
||||
],
|
||||
}
|
||||
yield f"data: {json_dumps(payload)}\n\n"
|
||||
time.sleep(0.02)
|
||||
raise RuntimeError("e2e mid-stream failure (phase 120 partial-then-fail)")
|
||||
|
||||
|
||||
#: The agent's ``read`` tool-result prefix (app.rag.agent
|
||||
#: ``_execute_tool``): ``"Document <source/path>:\n<content>"``.
|
||||
_READ_RESULT_PREFIX = "Document "
|
||||
@@ -2471,6 +2567,22 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
if _chat_dead(RETRY_TRIGGER, RETRY_DEAD_ATTEMPTS):
|
||||
return _llm_500(RETRY_TRIGGER)
|
||||
_fail_posts[RETRY_TRIGGER] = 0 # the answer streamed — restart
|
||||
# Phase 120 (failed-turn retry): the failed-turn suite's two
|
||||
# injections — the whole-first-turn death (the zero-frame
|
||||
# exhaustion whose Retry's re-ask the mock then answers) and the
|
||||
# partial-then-die (the partial deltas + terminal error frame).
|
||||
if FAIL_FIRST_TURN_TRIGGER in user_lower:
|
||||
if _chat_dead(FAIL_FIRST_TURN_TRIGGER, FAIL_FIRST_TURN_DEAD_ATTEMPTS):
|
||||
return _llm_500(FAIL_FIRST_TURN_TRIGGER)
|
||||
_fail_posts[FAIL_FIRST_TURN_TRIGGER] = 0 # the answer streamed — restart
|
||||
if PARTIAL_FAIL_TRIGGER in user_lower:
|
||||
if _bump_fail(PARTIAL_FAIL_TRIGGER) <= PARTIAL_FAIL_DEAD_REQUESTS:
|
||||
return StreamingResponse(
|
||||
_partial_fail_stream(),
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
_fail_posts[PARTIAL_FAIL_TRIGGER] = 0 # the answer streamed — restart
|
||||
# Phase 71 (tool-scaffolding guardrails): the deterministic raw-
|
||||
# markup flow — checked BEFORE the search/tool marker flows (the
|
||||
# trigger is independent of the ``<tools>`` marker, so both
|
||||
|
||||
@@ -0,0 +1,566 @@
|
||||
"""Phase 120 E2E (Playwright): failed-turn retry — network errors and
|
||||
refresh survive a failed turn.
|
||||
|
||||
Source: ``TODO.md`` L3–4 — "Retry doesn't seem to work on network
|
||||
error" + "Refreshing the page after an error shows only the chat
|
||||
message you sent and no options to retry the message …".
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_failed_turn_retry.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the failure
|
||||
shapes are the deterministic injections in ``tests/e2e/mock_llm.py``
|
||||
(the phase-67 counter machinery, the phase-120 triggers):
|
||||
|
||||
* ``fail first turn`` (``FAIL_FIRST_TURN_TRIGGER``): the WHOLE first
|
||||
turn dies — every app-level attempt of the forced-default budget
|
||||
(conftest pins ``BOR_LLM_RETRIES`` to the code default 3 → 4
|
||||
attempts) 500s, ZERO frames — the retry-budget exhaustion
|
||||
(``tests/e2e/test_llm_retry.py``'s ``always fail`` past its budget) —
|
||||
and the second turn's first request streams the normal answer (the
|
||||
Retry's re-ask).
|
||||
* ``partial then fail`` (``PARTIAL_FAIL_TRIGGER``): the first request
|
||||
streams ``PARTIAL_FAIL_TEXT`` in ordinary delta chunks, then the
|
||||
mock's generator RAISES — a genuine mid-stream connection reset: the
|
||||
app's ``chat_stream_retried`` re-raises (a piece already emitted —
|
||||
the locked retry-before-first-frame rule) and the chat endpoint
|
||||
settles with the terminal SSE ``error`` frame AFTER the partial
|
||||
deltas. The second request streams the normal answer.
|
||||
|
||||
The e2e app server boots with ``BOR_LLM_RETRY_DELAY=0`` (conftest) so
|
||||
the dead attempts are instant; ``BOR_LLM_RETRIES`` is forced to the
|
||||
real default (3) so the exhaustion turn burns the REAL budget — 4
|
||||
attempts, the last ``retry`` frame reads "(4 of 4)".
|
||||
|
||||
KB seed (the ``test_llm_retry.py`` direct-seed pattern): ONE fixture
|
||||
document (``homelab/kubernetes.md``) — the "How is my Kubernetes
|
||||
cluster set up?" questions FTS-hit it → HIGH → the grounded path (the
|
||||
re-asked turn streams the grounded ``MOCK_ANSWER_MARKER`` answer).
|
||||
|
||||
Test → contract mapping (locked A1: a failed turn persists as a brain
|
||||
record with the ``failed`` marker + the capped ``error`` detail; the
|
||||
banner Retry stays as-is — ``lastBrainWrap`` simply EXISTS on the error
|
||||
paths now):
|
||||
|
||||
1. ``test_network_error_shows_banner_retry_and_failed_bubble`` —
|
||||
A (zero-frame network error): the banner (role=alert) is visible
|
||||
WITH the phase-111 Retry button (its ``lastBrainWrap`` precondition
|
||||
finally holds), a failed bubble with the fixed honest line + the
|
||||
in-bubble error note (the detail) + the in-bubble Retry exists, the
|
||||
record persists ``failed: true`` + ``error`` in localStorage, and
|
||||
the wire is retry×3 + terminal error with NO delta/done. Clicking
|
||||
the BANNER Retry re-asks WITHOUT re-typing (no second user
|
||||
bubble), the mock now answers, the grounded answer streams in
|
||||
place, and the failed bubble is gone (redo-in-place).
|
||||
2. ``test_partial_then_error_keeps_partial_with_note_and_retry`` —
|
||||
B (SSE error frame after partial deltas): the partial bubble KEEPS
|
||||
its streamed text, gains the in-bubble error note (the detail) +
|
||||
the in-bubble Retry, the record persists the RAW partial +
|
||||
``failed: true`` + ``error``, and the wire is deltas-then-terminal-
|
||||
error with no done. Clicking the IN-BUBBLE Retry re-asks in place
|
||||
(the failed record is replaced by the fresh grounded answer — no
|
||||
re-typing, no failed note).
|
||||
3. ``test_failed_turn_survives_a_refresh`` — C (the refresh case):
|
||||
after a zero-frame failure, ``page.reload()`` restores the question
|
||||
+ the failed bubble (error detail visible) WITH a working Retry
|
||||
button on it — no "new chat" required — and clicking it re-asks
|
||||
(the grounded answer replaces the failed record).
|
||||
4. ``test_stopped_turn_is_not_a_failed_turn`` — negative (phase 48
|
||||
unchanged): a user-stopped turn restores with the "Stopped" note
|
||||
and NOT a failed note — the two markers are mutually exclusive by
|
||||
construction (the stop path never touches the failed funnel).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import Page, expect
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document
|
||||
from e2e.auth_helpers import login
|
||||
from tests.e2e.mock_llm import PARTIAL_FAIL_TEXT, embed_text
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Seed + questions (see the module docstring for the gate notes)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
SEED_SOURCE = "docs"
|
||||
SEED_PATH = "homelab/kubernetes.md"
|
||||
KUB_CONTENT = (REPO / "tests" / "fixtures" / "docs" / SEED_PATH).read_text()
|
||||
|
||||
#: A (zero-frame network error) + C (refresh) — HIGH turn (FTS-hits the
|
||||
#: seed) so the re-asked turn streams the GROUNDED mock answer.
|
||||
FAIL_Q = "How is my Kubernetes cluster set up? fail first turn"
|
||||
#: B (partial deltas + the terminal error frame) — HIGH turn, same
|
||||
#: re-ask contract.
|
||||
PARTIAL_Q = "How is my Kubernetes cluster set up? partial then fail"
|
||||
#: Negative: the mock's ~8 s long answer (12 chars / 0.02 s) — the
|
||||
#: comfortable stop window (the phase-48 suite's phrasing).
|
||||
STOP_Q = "How is my Kubernetes cluster set up? write a long answer"
|
||||
|
||||
#: The app's terminal LLM-failure copy (app/api/chat.py's LLMError
|
||||
#: frame) — the error detail the note/banner/record all carry.
|
||||
ERROR_COPY = "The chat model dropped the connection — try again?"
|
||||
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
||||
|
||||
#: The conftest forces BOR_LLM_RETRIES to its default (3) → 4 attempts
|
||||
#: total; the retry-frame math in the assertions is fixed by that.
|
||||
MAX_ATTEMPTS = 4
|
||||
|
||||
STORAGE_KEY = "bor.chat.v1"
|
||||
|
||||
|
||||
def _js_const(name: str) -> str:
|
||||
"""A frontend string constant, read from app.js — the E2E asserts
|
||||
against the SAME text the page renders (no JS/Python drift)."""
|
||||
js = (REPO / "frontend" / "assets" / "app.js").read_text(encoding="utf-8")
|
||||
m = re.search(rf'const {name}\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m, f"const {name} not found in app.js"
|
||||
return m.group(1)
|
||||
|
||||
|
||||
FAILED_TURN_TEXT = _js_const("FAILED_TURN_TEXT")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# DB seeding (TRUNCATE-then-seed, cf. test_llm_retry.py)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed(db: Session) -> None:
|
||||
"""The single fixture document (see the module docstring)."""
|
||||
md = Document(
|
||||
source=SEED_SOURCE,
|
||||
path=SEED_PATH,
|
||||
full_path=f"/tmp/{SEED_PATH}",
|
||||
title="Kubernetes",
|
||||
content=KUB_CONTENT,
|
||||
content_hash=hashlib.sha256(KUB_CONTENT.encode()).hexdigest(),
|
||||
indexed_at=datetime.now(UTC),
|
||||
)
|
||||
db.add(md)
|
||||
db.flush()
|
||||
# One chunk carrying the mock's own embedding → genuine token overlap
|
||||
# for the grounded questions (the FTS path carries them to HIGH).
|
||||
db.add(
|
||||
Chunk(
|
||||
document_id=md.id,
|
||||
position=0,
|
||||
content=KUB_CONTENT,
|
||||
embedding=embed_text(KUB_CONTENT),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _reset_db() -> None:
|
||||
"""Truncate the KB (plus the prompt-shaping tables), then re-seed.
|
||||
|
||||
``steering_notes`` / ``kb_overview`` are truncated too, so the
|
||||
prompts are byte-stable regardless of leftovers from other suites.
|
||||
"""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
|
||||
)
|
||||
db.commit()
|
||||
_seed(db)
|
||||
db.commit()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_chats() -> Iterator[None]:
|
||||
"""The send auto-saves a ``saved_chats`` row per turn (phase 55) —
|
||||
truncate around every test so the suite starts from (and leaves)
|
||||
an empty deployment (the phase-80/103 isolation pattern)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
yield
|
||||
# The auto-save is fire-and-forget (the browser never awaits the
|
||||
# PUT): a short margin lets the last turn's PUT land server-side
|
||||
# before the TRUNCATE, so the teardown never 500s an in-flight
|
||||
# upsert (the "Could not refresh instance" race). This runs AFTER
|
||||
# the page closes (LIFO finalization), so the PUT is either done
|
||||
# or aborted — never in flight.
|
||||
time.sleep(0.75)
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Page hooks (the SSE capture — the phase-37/67 pattern from
|
||||
# test_llm_retry.py) + localStorage reads
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
SSE_HOOK = """
|
||||
() => {
|
||||
if (window.__sseInstalled) return;
|
||||
window.__sseInstalled = true;
|
||||
window.__sseFrames = [];
|
||||
const origFetch = window.fetch;
|
||||
window.fetch = async function (...args) {
|
||||
const res = await origFetch.apply(this, args);
|
||||
try {
|
||||
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
|
||||
if (url.includes('/api/chat')) {
|
||||
res.clone().text().then((bodyText) => {
|
||||
for (const block of bodyText.split('\\n\\n')) {
|
||||
const line = block.trim();
|
||||
if (line.startsWith('data: ')) {
|
||||
window.__sseFrames.push(line.slice(6));
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (e) { /* non-clonable responses: ignored */ }
|
||||
return res;
|
||||
};
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _install_sse_hook(page: Page) -> None:
|
||||
page.evaluate(SSE_HOOK)
|
||||
|
||||
|
||||
def _frames(page: Page, terminal: str = "done") -> list[dict]:
|
||||
"""The captured SSE frames of the CURRENT turn, once the *terminal*
|
||||
frame lands (``error`` for the failure scenarios)."""
|
||||
deadline = time.monotonic() + 30.0
|
||||
while True:
|
||||
raw = page.evaluate("() => window.__sseFrames || []")
|
||||
parsed = [json.loads(line) for line in raw if line]
|
||||
if any(f.get("type") == terminal for f in parsed):
|
||||
return parsed
|
||||
if time.monotonic() > deadline:
|
||||
raise AssertionError(
|
||||
f"SSE hook captured no `{terminal}` frame (frames so far: "
|
||||
f"{len(parsed)}) — hook install failed?"
|
||||
)
|
||||
time.sleep(0.05)
|
||||
|
||||
|
||||
def _stored(page: Page) -> dict:
|
||||
"""The persisted ``bor.chat.v1`` conversation (localStorage — the
|
||||
phase-14 store the restore reads on refresh)."""
|
||||
raw = page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
|
||||
assert raw is not None, "the conversation key must exist in localStorage"
|
||||
return json.loads(raw)
|
||||
|
||||
|
||||
def _submit(page: Page, question: str) -> None:
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
# The user bubble lands synchronously with the submit handler.
|
||||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||||
|
||||
|
||||
def _assert_failed_bubble(
|
||||
page: Page, *, expected_text: str, note_detail: str
|
||||
) -> None:
|
||||
"""The failed-bubble contract shared by the scenarios: ONE brain
|
||||
bubble with *expected_text*, the in-bubble error note (the "Failed"
|
||||
label + *note_detail*), and the in-bubble Retry button
|
||||
(``markLastRetryable`` on the failed bubble — the last brain
|
||||
wrap)."""
|
||||
brain = page.locator(".msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(expected_text)
|
||||
note = brain.locator(".failed-note")
|
||||
expect(note).to_have_count(1)
|
||||
expect(note).to_contain_text("Failed")
|
||||
expect(note).to_contain_text(note_detail)
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
# A note, not an answer: no Save-as-doc button on the failed bubble.
|
||||
expect(brain.locator(".save-as-doc-btn")).to_have_count(0)
|
||||
|
||||
|
||||
def _assert_settled_composer(page: Page) -> None:
|
||||
"""The failed turn settles to idle: the Send button recovered
|
||||
(never a zombified Stop — the §7.4 never-stale contract)."""
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
|
||||
def _assert_redo_succeeded(page: Page) -> None:
|
||||
"""After a Retry re-ask: exactly question + the fresh GROUNDED
|
||||
answer (the mock now answers) — the failed record is REPLACED in
|
||||
place (no second user bubble, no failed note, no failed marker on
|
||||
the new record)."""
|
||||
expect(page.locator(".msg.user .bubble")).to_have_count(1)
|
||||
expect(page.locator(".msg.brain")).to_have_count(1)
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
||||
expect(page.locator(".failed-note")).to_have_count(0)
|
||||
# The turn has FULLY settled: the ``done`` frame is processed (the
|
||||
# answer record persisted through rememberBrainTurn — synchronously
|
||||
# in the frame handler) BEFORE the finally's idle state flips the
|
||||
# label to Send. The label is the settle signal, so the local
|
||||
# storage read below is race-free (the marker can be visible a few
|
||||
# frames before ``done`` — the last deltas carry it).
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
|
||||
last = stored["messages"][-1]
|
||||
assert last.get("failed") in (None, False), "the fresh answer is not failed"
|
||||
assert MOCK_ANSWER_MARKER in last["text"]
|
||||
# The fresh grounded record's text is the rendered answer (no
|
||||
# failed-marker leftovers in the conversation).
|
||||
assert FAILED_TURN_TEXT not in last["text"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# A — zero-frame network error: the banner WITH a working Retry + the
|
||||
# failed bubble; clicking the banner Retry re-asks without re-typing
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_network_error_shows_banner_retry_and_failed_bubble(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, FAIL_Q)
|
||||
|
||||
# After the 4th dead attempt (the forced-default budget) the turn
|
||||
# dies with ZERO frames: the existing terminal error banner
|
||||
# (role=alert, the "dropped the connection" copy) — NOW with its
|
||||
# Retry button revealed (the phase-111 condition
|
||||
# ``opts.retryable && lastBrainWrap`` finally holds: the failed
|
||||
# bubble's wrap exists before the error state).
|
||||
banner = page.locator("#kb-banner")
|
||||
expect(banner).to_have_attribute("role", "alert", timeout=60_000)
|
||||
expect(banner).to_contain_text(ERROR_COPY)
|
||||
expect(page.locator("#banner-retry")).to_be_visible(timeout=10_000)
|
||||
_assert_settled_composer(page)
|
||||
|
||||
# The zero-frame failed bubble: the fixed honest line (NOT the raw
|
||||
# detail — that rides the note), the in-bubble error note carrying
|
||||
# the detail, and the in-bubble Retry.
|
||||
_assert_failed_bubble(page, expected_text=FAILED_TURN_TEXT, note_detail=ERROR_COPY)
|
||||
|
||||
# Persisted: the question's user record + the failed brain record —
|
||||
# ``failed: true`` + the capped detail (the phase-48 ``stopped``
|
||||
# precedent in localStorage; the phase-55 auto-save rides the same
|
||||
# call, so the server-side saved chat carries it too).
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
|
||||
assert stored["messages"][0]["text"] == FAIL_Q
|
||||
failed = stored["messages"][-1]
|
||||
assert failed["text"] == FAILED_TURN_TEXT
|
||||
assert failed["failed"] is True
|
||||
assert failed["error"] == ERROR_COPY
|
||||
|
||||
# Wire: the three retry frames (attempts 2–4 of 4 — the real
|
||||
# budget), then the terminal error frame LAST — no delta, no done
|
||||
# (ZERO frames reached the client).
|
||||
frames = _frames(page, terminal="error")
|
||||
assert [f for f in frames if f["type"] == "retry"] == [
|
||||
{"type": "retry", "attempt": a, "max_attempts": MAX_ATTEMPTS}
|
||||
for a in (2, 3, 4)
|
||||
], frames
|
||||
assert frames[-1]["type"] == "error"
|
||||
assert ERROR_COPY in frames[-1]["detail"]
|
||||
assert not [f for f in frames if f.get("type") in ("done", "delta")], frames
|
||||
|
||||
# The BANNER Retry: re-asks WITHOUT re-typing (no second user
|
||||
# bubble) — the mock now answers (the sequence resets after its
|
||||
# guarded success) and the grounded answer streams in place of the
|
||||
# failed bubble (redo-in-place — the failed record is popped).
|
||||
page.click("#banner-retry")
|
||||
_assert_redo_succeeded(page)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# B — SSE error frame after partial deltas: the partial keeps its text
|
||||
# + the error note + the in-bubble Retry; clicking it re-asks in
|
||||
# place (the failed record is replaced by the fresh answer)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_partial_then_error_keeps_partial_with_note_and_retry(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, PARTIAL_Q)
|
||||
|
||||
# The partial streams, then the mock's generator dies mid-stream:
|
||||
# the app settles the turn with the terminal error banner AFTER the
|
||||
# partial deltas — and the partial bubble KEEPS its text.
|
||||
banner = page.locator("#kb-banner")
|
||||
expect(banner).to_have_attribute("role", "alert", timeout=60_000)
|
||||
expect(banner).to_contain_text(ERROR_COPY)
|
||||
_assert_settled_composer(page)
|
||||
|
||||
# The failed partial: the streamed text kept verbatim + the
|
||||
# in-bubble error note (the detail) + the in-bubble Retry.
|
||||
_assert_failed_bubble(
|
||||
page, expected_text=PARTIAL_FAIL_TEXT, note_detail=ERROR_COPY
|
||||
)
|
||||
|
||||
# Persisted: the RAW partial (what the user saw is what is stored —
|
||||
# the phase-17/20 convention, now with the failed marker + detail).
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
|
||||
failed = stored["messages"][-1]
|
||||
assert failed["text"] == PARTIAL_FAIL_TEXT
|
||||
assert failed["failed"] is True
|
||||
assert failed["error"] == ERROR_COPY
|
||||
|
||||
# Wire: the delta frames (the partial, in order) precede the
|
||||
# terminal error frame — no done (the stream never settled), no
|
||||
# retry (a piece already emitted — the locked retry-before-first-
|
||||
# frame rule means the failure is terminal, never redone).
|
||||
frames = _frames(page, terminal="error")
|
||||
deltas = [f for f in frames if f["type"] == "delta"]
|
||||
assert deltas, "no delta frames before the error"
|
||||
assert "".join(d["text"] for d in deltas) == PARTIAL_FAIL_TEXT
|
||||
assert not [f for f in frames if f.get("type") == "retry"], frames
|
||||
assert frames[-1]["type"] == "error"
|
||||
assert not [f for f in frames if f.get("type") == "done"], frames
|
||||
|
||||
# The IN-BUBBLE Retry: redo-in-place — the failed partial record is
|
||||
# popped + re-asked (no re-typing), and the fresh grounded answer
|
||||
# (the mock now answers — the sequence resets) replaces it.
|
||||
page.click(".msg.brain .retry-btn")
|
||||
_assert_redo_succeeded(page)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# C — the refresh case: reload restores the failed bubble WITH a
|
||||
# working Retry; clicking it re-asks (no "new chat" required)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_failed_turn_survives_a_refresh(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_sse_hook(page)
|
||||
|
||||
_submit(page, FAIL_Q)
|
||||
|
||||
# The zero-frame failure lands (same shape as scenario A).
|
||||
banner = page.locator("#kb-banner")
|
||||
expect(banner).to_have_attribute("role", "alert", timeout=60_000)
|
||||
expect(banner).to_contain_text(ERROR_COPY)
|
||||
expect(page.locator("#banner-retry")).to_be_visible(timeout=10_000)
|
||||
_assert_failed_bubble(
|
||||
page, expected_text=FAILED_TURN_TEXT, note_detail=ERROR_COPY
|
||||
)
|
||||
|
||||
# The refresh: the page restores from localStorage (the failed
|
||||
# record persisted by the funnel) — the question + the failed
|
||||
# bubble with its error detail, and the Retry button on the failed
|
||||
# bubble (the restore loop's lastBrainWrap + markLastRetryable land
|
||||
# it on the LAST restored brain bubble — the failed one).
|
||||
page.reload()
|
||||
expect(page.locator("#messages > .msg")).to_have_count(2)
|
||||
expect(page.locator(".msg.user .bubble")).to_have_text(FAIL_Q)
|
||||
_assert_failed_bubble(
|
||||
page, expected_text=FAILED_TURN_TEXT, note_detail=ERROR_COPY
|
||||
)
|
||||
# The error state itself does NOT restore (the banner is a live-
|
||||
# turn state) — the recovery affordance is the in-bubble Retry.
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
_assert_settled_composer(page)
|
||||
|
||||
# The restored record still carries the marker (the restore is
|
||||
# lossless: text + detail + marker).
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
|
||||
assert stored["messages"][-1]["failed"] is True
|
||||
assert stored["messages"][-1]["error"] == ERROR_COPY
|
||||
|
||||
# The restored Retry WORKS: clicking it re-asks the question that
|
||||
# precedes the failed record (retryLastTurn, unchanged) — the
|
||||
# grounded answer streams and replaces the failed record.
|
||||
page.click(".msg.brain .retry-btn")
|
||||
_assert_redo_succeeded(page)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Negative — a STOPPED turn (phase 48) is NOT a failed turn: the stop
|
||||
# path restores with the "Stopped" note and never the failed note
|
||||
# (mutually exclusive by construction — the stop branch is the catch's
|
||||
# own, the failed funnel is the catch's else)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_stopped_turn_is_not_a_failed_turn(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
# The ~8 s long answer — a comfortable stop window (phase 48).
|
||||
_submit(page, STOP_Q)
|
||||
answer = page.locator(".msg.brain .bubble:not(.typing)")
|
||||
answer.wait_for(state="visible", timeout=30_000)
|
||||
partial = ""
|
||||
deadline = time.monotonic() + 15
|
||||
while time.monotonic() < deadline:
|
||||
partial = answer.inner_text()
|
||||
if len(partial.split()) >= 8:
|
||||
break
|
||||
time.sleep(0.05)
|
||||
assert len(partial.split()) >= 8, "no answer deltas before the stop"
|
||||
|
||||
expect(page.locator("#send-label")).to_have_text("Stop")
|
||||
page.click("#send-btn") # the in-flight button IS the Stop control
|
||||
|
||||
# Settled to idle: no error banner (the stop path never shows one),
|
||||
# the partial kept + the "Stopped" note — and NO failed note.
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=5_000)
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
expect(page.locator(".msg.brain .stopped-note")).to_have_count(1)
|
||||
expect(page.locator(".msg.brain .stopped-note")).to_contain_text("Stopped")
|
||||
expect(page.locator(".msg.brain .failed-note")).to_have_count(0)
|
||||
stopped_text = answer.inner_text()
|
||||
assert stopped_text.strip()
|
||||
|
||||
# Persisted: the stopped marker, NOT the failed marker (the two are
|
||||
# mutually exclusive by construction).
|
||||
stored = _stored(page)
|
||||
last = stored["messages"][-1]
|
||||
assert last["stopped"] is True
|
||||
assert last.get("failed") in (None, False), "a stopped turn is not a failed turn"
|
||||
assert "error" not in last or last["error"] is None
|
||||
|
||||
# The refresh: the stopped partial restores with the "Stopped"
|
||||
# note and NOT the failed note — phase 48's restore is unchanged.
|
||||
page.reload()
|
||||
expect(page.locator("#messages > .msg")).to_have_count(2)
|
||||
expect(page.locator(".msg.user .bubble")).to_have_text(STOP_Q)
|
||||
expect(page.locator(".msg.brain .bubble:not(.typing)")).to_have_text(
|
||||
stopped_text
|
||||
)
|
||||
expect(page.locator(".msg.brain .stopped-note")).to_have_count(1)
|
||||
expect(page.locator(".msg.brain .failed-note")).to_have_count(0)
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
stored = _stored(page)
|
||||
assert stored["messages"][-1]["stopped"] is True
|
||||
assert stored["messages"][-1].get("failed") in (None, False)
|
||||
@@ -60,6 +60,11 @@ Test → source mapping:
|
||||
appears (role=alert, the "dropped the connection" copy), the last
|
||||
retrying status is the highest attempt — "(4 of 4)…" — and the send
|
||||
button re-enables (the banner path settles the state machine).
|
||||
Phase 120 (locked A1): the zero-frame turn ALSO renders the
|
||||
retryable failed bubble (the fixed honest line + the in-bubble
|
||||
error note with the detail + the in-bubble Retry) and reveals the
|
||||
banner Retry (its ``lastBrainWrap`` precondition finally holds on
|
||||
the error path) — a network error is retryable.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -115,6 +120,21 @@ MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
||||
DEFLECT_PHRASE = r"haven't done anything like that"
|
||||
ERROR_COPY = "The chat model dropped the connection — try again?"
|
||||
|
||||
|
||||
def _js_const(name: str) -> str:
|
||||
"""A frontend string constant, read from app.js (no JS/Python
|
||||
drift — the E2E asserts against the SAME text the page renders)."""
|
||||
js = (REPO / "frontend" / "assets" / "app.js").read_text(encoding="utf-8")
|
||||
m = re.search(rf'const {name}\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m, f"const {name} not found in app.js"
|
||||
return m.group(1)
|
||||
|
||||
|
||||
#: Phase 120: the fixed bubble text of a ZERO-frame failed turn (the
|
||||
#: exhaustion case below) — read from app.js so this suite's assertion
|
||||
#: tracks the page's constant.
|
||||
FAILED_TURN_TEXT = _js_const("FAILED_TURN_TEXT")
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# DB seeding (TRUNCATE-then-seed, cf. test_agent_document_tools.py)
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -501,7 +521,18 @@ def test_exhaustion_lands_on_the_error_banner(
|
||||
assert not [f for f in frames if f.get("type") == "done"]
|
||||
assert not [f for f in frames if f.get("type") == "delta"]
|
||||
|
||||
# No answer bubble was ever rendered (no frame ever streamed a
|
||||
# token) — the user bubble is the only message in the DOM.
|
||||
expect(page.locator("#messages > .msg.brain")).to_have_count(0)
|
||||
# Phase 120 (locked A1): the zero-frame turn is no longer a bare
|
||||
# question — it persists as a FAILED brain record (the phase-48
|
||||
# ``stopped`` precedent) and renders the retryable error bubble:
|
||||
# the fixed honest line (no frame ever streamed a token, so the
|
||||
# bubble is NOT the empty-answer fallback), the in-bubble error
|
||||
# note carrying the detail, the in-bubble Retry, and the banner
|
||||
# Retry revealed too (its ``lastBrainWrap`` precondition finally
|
||||
# holds on the error path — a network error is retryable).
|
||||
brain = page.locator("#messages > .msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(FAILED_TURN_TEXT)
|
||||
expect(brain.locator(".failed-note")).to_contain_text(ERROR_COPY)
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
expect(page.locator("#banner-retry")).to_be_visible()
|
||||
expect(page.locator(".msg.user .bubble")).to_have_count(1)
|
||||
|
||||
@@ -628,7 +628,7 @@ def test_api_created_chats_carry_the_bor_chat_v1_shape(
|
||||
# brain records (accepted by the ChatMessage schema — its earlier
|
||||
# absence 422'd the done-time auto-save, phase-118 verification fix).
|
||||
allowed = {"who", "text", "sources", "related", "deflected", "suggestions",
|
||||
"thinking", "tools", "stopped"}
|
||||
"thinking", "tools", "stopped", "failed", "error"}
|
||||
assert all({"who", "text"} <= set(m) <= allowed for m in body["messages"])
|
||||
# The History row renders it (the Open link's text IS the title).
|
||||
page.click("#nav-history")
|
||||
|
||||
@@ -48,10 +48,13 @@ Test → phase mapping (Playwright Mapping Rule):
|
||||
2. ``test_terminal_scaffolding_lands_on_dedicated_error_app_stays_usable``
|
||||
— the terminal case: the existing error state renders with the
|
||||
dedicated copy ("The model returned a malformed reply — please try
|
||||
again."), no raw tokens in the DOM, no answer bubble, no ``done``
|
||||
and NO ``query_log`` row (the existing terminal-error semantics) —
|
||||
and the app stays usable: a follow-up plain question in the same
|
||||
session gets a normal deflected answer and the banner clears.
|
||||
again."), no raw tokens in the DOM, no answer bubble (phase 120:
|
||||
the zero-frame turn renders the FAILED note bubble — the fixed
|
||||
honest line + the error note + the Retry, NOT an answer), no
|
||||
``done`` and NO ``query_log`` row (the existing terminal-error
|
||||
semantics) — and the app stays usable: a follow-up plain question
|
||||
in the same session gets a normal deflected answer and the banner
|
||||
clears.
|
||||
3. ``test_plain_turn_never_recovers_and_streams_byte_clean`` — no
|
||||
false positive: a plain deflected question streams its
|
||||
first-request answer byte-clean (the concatenated delta text is
|
||||
@@ -64,6 +67,7 @@ from __future__ import annotations
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from playwright.sync_api import Page, expect
|
||||
from sqlalchemy import select, text
|
||||
@@ -73,6 +77,24 @@ from app.models import QueryLog
|
||||
from e2e.auth_helpers import login
|
||||
from tests.e2e.mock_llm import SCAFFOLD_RECOVERY_ANSWER, compose_answer
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _js_const(name: str) -> str:
|
||||
"""A frontend string constant, read from app.js (no JS/Python
|
||||
drift — the E2E asserts against the SAME text the page renders)."""
|
||||
js = (REPO / "frontend" / "assets" / "app.js").read_text(encoding="utf-8")
|
||||
m = re.search(rf'const {name}\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m, f"const {name} not found in app.js"
|
||||
return m.group(1)
|
||||
|
||||
|
||||
#: Phase 120: the fixed bubble text of a ZERO-frame failed turn (the
|
||||
#: terminal scaffolding case below) — read from app.js so this suite's
|
||||
#: assertion tracks the page's constant.
|
||||
FAILED_TURN_TEXT = _js_const("FAILED_TURN_TEXT")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Questions + the mock's deterministic expectations
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -364,9 +386,20 @@ def test_terminal_scaffolding_lands_on_dedicated_error_app_stays_usable(
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
# No answer bubble was ever rendered (every streamed frame was
|
||||
# No ANSWER bubble was ever rendered (every streamed frame was
|
||||
# stripped server-side) and no raw token is anywhere in the DOM.
|
||||
expect(page.locator("#messages > .msg.brain")).to_have_count(0)
|
||||
# Phase 120 (locked A1): the zero-frame turn persists as a FAILED
|
||||
# brain record (the phase-48 ``stopped`` precedent) and renders the
|
||||
# retryable error bubble — the fixed honest line (NOT an answer),
|
||||
# the in-bubble error note carrying the dedicated copy, the
|
||||
# in-bubble Retry, and the banner Retry revealed too (its
|
||||
# ``lastBrainWrap`` precondition finally holds on the error path).
|
||||
brain = page.locator("#messages > .msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(FAILED_TURN_TEXT)
|
||||
expect(brain.locator(".failed-note")).to_contain_text(MALFORMED_ERROR_COPY)
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
expect(page.locator("#banner-retry")).to_be_visible()
|
||||
_assert_no_raw_tokens(page)
|
||||
|
||||
# Wire: the terminal error frame is LAST — no done, and NO delta
|
||||
|
||||
Reference in New Issue
Block a user