phase: 164_chat_error_honesty
All gates green. Report: - **Verified (phase 164, all 4 tasks already in `complete/`)**: `HistoryTurn`/`ChatMessage`/`ChatThinkingSegment` caps → 600 000, `history`/saved-messages lists → 500, new `SavedChatCreate/Update` total-8 000 000-char validator; exported `errorDetailText` at the two previously unguarded chat paths (pre-stream `!res.ok` + SSE `error` frame); `FAILED_REQUEST_TEXT` + `bubbleText` threading + phase-114 banner hint; dedicated E2E `tests/e2e/test_chat_error_honesty.py`. - **Final-pass additions**: `tests/integration/test_chat_api.py::test_history_rejects_one_char_over_the_text_cap` (endpoint-level >600k → 422 with the pydantic list-of-objects `msg` — the criterion's upper half was only unit-pinned); README env rows for `BOR_APP_VERSION` + `BOR_SESSION_COOKIE`; restored `test_document_viewer.py`'s tracked screenshot artifact dirtied by the E2E run (via read-only `git show`, no index/history touched). - **Tests/lint (exact commands)**: `uv run pytest --cov=app --cov-report=term-missing` → **3837 passed**, `app/` coverage **99%** (>90%); `uv run pytest tests/e2e/test_chat_error_honesty.py -v --no-cov` → **4 passed in isolation**; per-file loop over **all 148** E2E files (`uv run pytest tests/e2e/<f> -q --no-cov`) → **148 × exit 0**, incl. `test_failed_turn_retry`, `test_chat_save_payload_limits`, `test_edit_and_resend`, `test_llm_history`, `test_smoke`; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - **Completion criteria**: pre-stream 422 shows the distinct line + verbatim `msg`, `[object Object]` unrepresentable + no `input` in DOM — **met** (unit totality/source pins + E2E asserts 1–2); 525k history accepted (200 settles) / >600k → 422 / saved chat 201 + 8M bound — **met**; transport + SSE-frame settles keep `FAILED_TURN_TEXT`, stop/abort untouched, §7.4 re-enables — **met**; budget trim, `message` 4000, the 7 guarded detail-readers and all SSE shapes byte-identical (diff touches only caps + the two chat paths) — **met**; README/AGENTS full review + proxy deploy note — **met**; commit/`complete/` move = pipeline gate. - **Docs**: README reviewed in full (env tables cross-checked programmatically: 74 `BOR_*` rows vs `Settings` defaults — no mismatches; every `.env.example` var documented; referenced scripts/pages/endpoints/`query_log` columns exist) — found & fixed the 2 undocumented real knobs above; the phase's payload-limits section, failed-turn copy and `client_max_body_size ≥ 16 MB` deploy note verified accurate. `AGENTS.md` verified consistent (rules, gitignore list, quick-reference commands, referenced paths all correct) — no changes needed. - **Deviations**: `ChatThinkingSegment.text` raised 32 000 → 600 000 too (not itemized in A1, but its own docstring mirrors `HistoryTurn.thinking`; without it a long-thinking turn 422s its own auto-save — same mirror invariant). No live-infra changes made (the proxy limit lives in the separate deployment repo; README note only). - **Next pending phase**: none — `164_chat_error_honesty` was the only entry in `todo/`.
This commit is contained in:
@@ -0,0 +1,540 @@
|
||||
"""Phase 164 E2E (Playwright): chat failure honesty — a pre-stream HTTP
|
||||
rejection states what ACTUALLY happened, and a full-length answer flows
|
||||
through the whole chain.
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_chat_error_honesty.py -v --no-cov
|
||||
|
||||
The live finding (owner report, 2026-10-07): after an answer hit the
|
||||
length cap, the NEXT turn's bubble read "My answer didn't make it — the
|
||||
connection dropped. Use Retry to ask again. Failed [object Object]."
|
||||
The chain: the long answer rode back as ``history`` on the next
|
||||
``POST /api/chat``; the old 32 000-char ``HistoryTurn.text`` cap
|
||||
rejected it with a 422 whose ``detail`` is a LIST OF OBJECTS; the
|
||||
unguarded ``body.detail`` assignment stringified the array to
|
||||
"[object Object]" inside the connection-dropped bubble — a lie (the
|
||||
server WAS reachable; it answered 422).
|
||||
|
||||
The phase's remedies, proven here against the REAL ``app.js`` send
|
||||
path (the stubs ride the browser's own network layer via
|
||||
``page.route`` — the house pattern from test_navbar_refresh /
|
||||
test_archive_upload_sources — so every assert drives the shipped
|
||||
pre-stream handler, serializer, and failed-turn funnel):
|
||||
|
||||
1. ``test_prestream_422_renders_honest_line_with_serialized_detail`` —
|
||||
(the live bug) ``POST /api/chat`` is stubbed to the EXACT FastAPI
|
||||
422 shape for a long-history rejection. The bubble reads
|
||||
``FAILED_REQUEST_TEXT`` verbatim (NOT the connection line), the
|
||||
``.failed-note`` carries the pydantic ``msg`` verbatim, NO
|
||||
"[object Object]" and NO rejected ``input`` text reach the DOM, the
|
||||
banner shows the phase-114 hint, Retry + Edit land, and the
|
||||
composer re-enables (§7.4).
|
||||
2. ``test_non_json_500_settles_on_the_http_status_fallback`` — a 500
|
||||
with a NON-JSON body settles on the ``HTTP 500`` default string
|
||||
(the serializer's fallback path) under the same honest pre-stream
|
||||
bubble — no crash.
|
||||
3. ``test_full_length_answer_history_is_accepted`` — (the D1 fix,
|
||||
REAL server, no stub) a seeded ``bor.chat.v1`` conversation whose
|
||||
last brain record carries a ~525k-char answer (the live full-cap
|
||||
size) sends a follow-up that is ACCEPTED — the response is 200, the
|
||||
mock's deterministic echo settles the turn, the saved-chat
|
||||
auto-save chain stays 2xx — where pre-phase this was the 422 that
|
||||
killed every follow-up. The locked budget trim (A4) may drop the
|
||||
giant turn WHOLE from the prompt; this suite asserts ACCEPTANCE and
|
||||
settle, never prompt content.
|
||||
4. ``test_sse_error_frame_with_array_detail_serializes_into_note`` —
|
||||
(client tolerance) a valid SSE stream whose terminal ``error``
|
||||
frame carries an ARRAY ``detail``: the note serializes the ``msg``,
|
||||
the zero-frame settle keeps ``FAILED_TURN_TEXT`` byte-identically
|
||||
(a model-side failure is NOT the pre-stream line), and the
|
||||
persisted record carries the serialized detail.
|
||||
|
||||
The bubble texts are read from ``app.js`` itself (the phase-120
|
||||
``_js_const`` convention) so the Python pins can never drift from the
|
||||
shipped copy.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import Page, Route, expect
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document
|
||||
from e2e.auth_helpers import login
|
||||
from tests.e2e.mock_llm import embed_text
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Copy constants, read from app.js (no JS/Python drift — the phase-120
|
||||
# convention). The unit suite pins the strings themselves once; the E2E
|
||||
# asserts the PAGE renders the SAME text.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _js_const(name: str) -> str:
|
||||
js = (REPO / "frontend" / "assets" / "app.js").read_text(encoding="utf-8")
|
||||
m = re.search(rf'const {name}\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m, f"const {name} not found in app.js"
|
||||
return m.group(1)
|
||||
|
||||
|
||||
FAILED_TURN_TEXT = _js_const("FAILED_TURN_TEXT")
|
||||
FAILED_REQUEST_TEXT = _js_const("FAILED_REQUEST_TEXT")
|
||||
|
||||
|
||||
def _js_prestream_hint() -> str:
|
||||
"""The pre-stream throw's banner hint (phase 164 D3) — the ONLY
|
||||
``err.hint`` assignment in app.js that carries a string literal (the
|
||||
SSE-frame one reads ``ev.hint``), so the match is unique."""
|
||||
js = (REPO / "frontend" / "assets" / "app.js").read_text(encoding="utf-8")
|
||||
matches = re.findall(r'err\.hint =\s*\n?\s*"([^"]*)"', js)
|
||||
assert len(matches) == 1, f"expected exactly one literal err.hint, got {matches}"
|
||||
return matches[0]
|
||||
|
||||
|
||||
PRESTREAM_HINT = _js_prestream_hint()
|
||||
|
||||
#: The pydantic 83 boundary rejection for HistoryTurn.text (600 000
|
||||
#: since phase 164, locked A1) — the EXACT live message the note must
|
||||
#: render verbatim.
|
||||
PYDANTIC_MSG = "String should have at most 600000 characters"
|
||||
|
||||
#: A probe from the rejected 422 entry's `input` (the whole rejected
|
||||
#: turn — it must NEVER reach the DOM). A 100-char run of "x" cannot
|
||||
#: occur naturally in the serialized page.
|
||||
INPUT_PROBE = "x" * 100
|
||||
|
||||
STORAGE_KEY = "bor.chat.v1"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Assert 3 fixtures: the seeded full-length answer + the grounded echo
|
||||
# question (the test_failed_turn_retry / test_llm_history seed pattern).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SEED_SOURCE = "docs"
|
||||
SEED_PATH = "homelab/kubernetes.md"
|
||||
KUB_CONTENT = (REPO / "tests" / "fixtures" / "docs" / SEED_PATH).read_text()
|
||||
|
||||
#: The seeded conversation (the live repro's shape): a question and a
|
||||
#: brain answer of the FULL-CAP size — 131 072 tokens ≈ 524 300 chars
|
||||
#: at ~4 chars/token (the 2026-10-07 live report). Over the OLD 32 000
|
||||
#: ceiling (the pre-phase 422), well under the new 600 000.
|
||||
SEED_QUESTION = "Write everything you know about my Kubernetes cluster?"
|
||||
BIG_ANSWER_LEN = 525_000
|
||||
BIG_ANSWER = ("A kubernetes cluster note. " * (BIG_ANSWER_LEN // 27 + 1))[
|
||||
:BIG_ANSWER_LEN
|
||||
]
|
||||
|
||||
#: The follow-up that died with the 422 live. `echo my history` is the
|
||||
#: mock's HISTORY_TRIGGER (fires whatever the honesty gate says) and
|
||||
#: "kubernetes cluster" keeps it on-topic for the one-doc KB, so the
|
||||
#: turn settles with the byte-stable deterministic echo.
|
||||
FOLLOWUP_Q = "echo my history about my kubernetes cluster"
|
||||
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB seeding (the test_failed_turn_retry pattern): TRUNCATE-then-seed
|
||||
# one fixture document so the follow-up's gate is deterministic.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed(db: Session) -> None:
|
||||
md = Document(
|
||||
source=SEED_SOURCE,
|
||||
path=SEED_PATH,
|
||||
full_path=f"/tmp/{SEED_PATH}",
|
||||
title="Kubernetes",
|
||||
content=KUB_CONTENT,
|
||||
content_hash=hashlib.sha256(KUB_CONTENT.encode()).hexdigest(),
|
||||
indexed_at=datetime.now(UTC),
|
||||
)
|
||||
db.add(md)
|
||||
db.flush()
|
||||
db.add(
|
||||
Chunk(
|
||||
document_id=md.id,
|
||||
position=0,
|
||||
content=KUB_CONTENT,
|
||||
embedding=embed_text(KUB_CONTENT),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _reset_db() -> None:
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text("TRUNCATE chunks, documents, query_log, steering_notes, kb_overview")
|
||||
)
|
||||
db.commit()
|
||||
_seed(db)
|
||||
db.commit()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_chats() -> Iterator[None]:
|
||||
"""The send auto-saves a ``saved_chats`` row per turn (phase 55) —
|
||||
truncate around every test (the phase-80/103/120 isolation
|
||||
pattern, including the teardown margin for the fire-and-forget
|
||||
PUT)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
yield
|
||||
time.sleep(0.75)
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE saved_chats"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Page helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _submit(page: Page, question: str) -> None:
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||||
|
||||
|
||||
def _stored(page: Page) -> dict:
|
||||
raw = page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
|
||||
assert raw is not None, "the conversation key must exist in localStorage"
|
||||
return json.loads(raw)
|
||||
|
||||
|
||||
def _assert_settled_composer(page: Page) -> None:
|
||||
"""§7.4 never-stale: after the failure the composer is fully usable
|
||||
again (Send enabled, label back — nothing zombified)."""
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
|
||||
|
||||
|
||||
def _assert_dom_is_honest(page: Page, *, forbidden_input_probe: str | None) -> None:
|
||||
"""The [object Object] / input-leak rule, page-wide: neither the
|
||||
garbage stringification nor a probe of the rejected `input` may
|
||||
appear anywhere in the DOM."""
|
||||
html = page.content()
|
||||
assert "[object Object]" not in html
|
||||
if forbidden_input_probe:
|
||||
assert forbidden_input_probe not in html
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1 — the live bug: a pre-stream 422 (FastAPI's EXACT validation shape)
|
||||
# renders the distinct honest line + the serialized msg, never
|
||||
# "[object Object]", never the rejected input
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _stub_long_history_422(route: Route) -> None:
|
||||
"""The EXACT FastAPI 422 body for a long-history rejection (a
|
||||
``string_too_long`` entry whose ``input`` carries the whole
|
||||
rejected answer — the payload whose array ``detail`` produced the
|
||||
live "[object Object]")."""
|
||||
route.fulfill(
|
||||
status=422,
|
||||
content_type="application/json",
|
||||
body=json.dumps(
|
||||
{
|
||||
"detail": [
|
||||
{
|
||||
"type": "string_too_long",
|
||||
"loc": ["body", "history", 5, "text"],
|
||||
"msg": PYDANTIC_MSG,
|
||||
"input": "x" * 1000,
|
||||
}
|
||||
]
|
||||
}
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def test_prestream_422_renders_honest_line_with_serialized_detail(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
page.route("**/api/chat", _stub_long_history_422)
|
||||
with page.expect_response(
|
||||
lambda r: urlparse(r.url).path == "/api/chat"
|
||||
) as info:
|
||||
_submit(page, "How is my Kubernetes cluster set up? (phase 164 e2e)")
|
||||
# The stub answered through the browser's real network layer — this
|
||||
# is the pre-stream HTTP settle, not a transport failure.
|
||||
assert info.value.status == 422
|
||||
|
||||
# The bubble: the DISTINCT honest pre-stream line (phase 164 D3) —
|
||||
# NOT the connection-dropped copy (the server answered deliberately;
|
||||
# the connection held).
|
||||
brain = page.locator(".msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(FAILED_REQUEST_TEXT)
|
||||
|
||||
# The note: the pydantic `msg` VERBATIM (task 02's serializer), and
|
||||
# nowhere on the page the array garbage or the rejected `input`.
|
||||
note = brain.locator(".failed-note")
|
||||
expect(note).to_have_count(1)
|
||||
expect(note).to_contain_text("Failed")
|
||||
expect(note).to_contain_text(PYDANTIC_MSG)
|
||||
_assert_dom_is_honest(page, forbidden_input_probe=INPUT_PROBE)
|
||||
|
||||
# The banner (role=alert): the same detail plus the phase-114 hint
|
||||
# channel (D3) — "the app answered, nothing was lost but this send".
|
||||
banner = page.locator("#kb-banner")
|
||||
expect(banner).to_have_attribute("role", "alert", timeout=30_000)
|
||||
expect(banner).to_be_visible()
|
||||
expect(banner).to_contain_text(PYDANTIC_MSG)
|
||||
expect(banner).to_contain_text(PRESTREAM_HINT)
|
||||
|
||||
# Recovery affordances: Retry on the failed bubble (phase 120) +
|
||||
# Edit on the question (phase 159), and the composer fully usable
|
||||
# again (§7.4).
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
expect(page.locator(".msg.user .edit-btn")).to_have_count(1)
|
||||
_assert_settled_composer(page)
|
||||
|
||||
# Persisted (the phase-120 contract extended to the new text):
|
||||
# what the user saw is what is stored — the new bubble line +
|
||||
# failed: true + the serialized detail in the capped `error`.
|
||||
stored = _stored(page)
|
||||
assert [m["who"] for m in stored["messages"]] == ["user", "brain"]
|
||||
failed = stored["messages"][-1]
|
||||
assert failed["text"] == FAILED_REQUEST_TEXT
|
||||
assert failed["failed"] is True
|
||||
assert failed["error"] == PYDANTIC_MSG
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2 — the fallback: a 500 with a NON-JSON body settles on the
|
||||
# "HTTP 500" default string under the same honest pre-stream line
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _stub_html_500(route: Route) -> None:
|
||||
"""An outage page: text/html where JSON was expected — the
|
||||
serializer's `res.json()` catch keeps the per-status default."""
|
||||
route.fulfill(
|
||||
status=500,
|
||||
content_type="text/html",
|
||||
body="<html><head><title>500</title></head><body>gateway trouble</body></html>",
|
||||
)
|
||||
|
||||
|
||||
def test_non_json_500_settles_on_the_http_status_fallback(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
page.route("**/api/chat", _stub_html_500)
|
||||
with page.expect_response(
|
||||
lambda r: urlparse(r.url).path == "/api/chat"
|
||||
) as info:
|
||||
_submit(page, "How is my Kubernetes cluster set up? (phase 164 e2e 500)")
|
||||
assert info.value.status == 500
|
||||
|
||||
# Still the pre-stream settle (the server answered 500) — the
|
||||
# honest request line, with the `HTTP 500` default string on the
|
||||
# note (the body was not JSON; the serializer fell back).
|
||||
brain = page.locator(".msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(FAILED_REQUEST_TEXT)
|
||||
note = brain.locator(".failed-note")
|
||||
expect(note).to_have_count(1)
|
||||
expect(note).to_contain_text("Failed")
|
||||
expect(note.locator(".failed-detail")).to_have_text(
|
||||
"Brain's API answered with HTTP 500."
|
||||
)
|
||||
_assert_dom_is_honest(page, forbidden_input_probe=None)
|
||||
|
||||
# No crash, and every control usable again (§7.4).
|
||||
banner = page.locator("#kb-banner")
|
||||
expect(banner).to_be_visible()
|
||||
expect(banner).to_contain_text(PRESTREAM_HINT)
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
_assert_settled_composer(page)
|
||||
|
||||
stored = _stored(page)
|
||||
failed = stored["messages"][-1]
|
||||
assert failed["text"] == FAILED_REQUEST_TEXT
|
||||
assert failed["failed"] is True
|
||||
assert failed["error"] == "Brain's API answered with HTTP 500."
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3 — the D1 fix, REAL server: a history turn carrying a ~525k-char
|
||||
# brain answer is ACCEPTED (the live "every follow-up dies after a
|
||||
# long answer" failure mode is gone) and the turn settles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_full_length_answer_history_is_accepted(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
assert len(BIG_ANSWER) == BIG_ANSWER_LEN
|
||||
assert len(BIG_ANSWER) > 32_000, "the repro must exceed the OLD 32 000 cap"
|
||||
|
||||
# Seed the live repro's shape into bor.chat.v1 (the saved-chat
|
||||
# suites' seed pattern): question + a FULL-CAP brain answer, the
|
||||
# brain record LAST — exactly the record the next send turns into
|
||||
# `history`.
|
||||
seed = {
|
||||
"v": 1,
|
||||
"chatId": None,
|
||||
"messages": [
|
||||
{"who": "user", "text": SEED_QUESTION},
|
||||
{"who": "brain", "text": BIG_ANSWER},
|
||||
],
|
||||
}
|
||||
page.evaluate(
|
||||
"([key, value]) => localStorage.setItem(key, value)",
|
||||
[STORAGE_KEY, json.dumps(seed)],
|
||||
)
|
||||
|
||||
# Track the saved-chat chain too: the phase-55 auto-save carries
|
||||
# the SAME 525k-char record to POST/PUT /api/chats — every one of
|
||||
# those must stay 2xx (the mirror invariant, live).
|
||||
chat_saves: list[tuple[str, int]] = []
|
||||
page.on(
|
||||
"response",
|
||||
lambda r: chat_saves.append((r.request.method, r.status))
|
||||
if urlparse(r.url).path.startswith("/api/chats")
|
||||
else None,
|
||||
)
|
||||
|
||||
page.reload()
|
||||
# The restore renders the seeded conversation — the giant bubble
|
||||
# proves the client actually holds the full-length answer the send
|
||||
# will put on the wire.
|
||||
expect(page.locator("#messages > .msg")).to_have_count(2)
|
||||
expect(page.locator(".msg.user .bubble")).to_have_text(SEED_QUESTION)
|
||||
restored = page.locator(".msg.brain .bubble").last.inner_text()
|
||||
assert len(restored) > 32_000, "the seeded long answer must restore"
|
||||
|
||||
with page.expect_response(
|
||||
lambda r: urlparse(r.url).path == "/api/chat"
|
||||
) as info:
|
||||
_submit(page, FOLLOWUP_Q)
|
||||
|
||||
# THE fix: the follow-up of the full-length answer is ACCEPTED —
|
||||
# no 422 (pre-phase the 32 000 ceiling rejected exactly this
|
||||
# request and killed every follow-up).
|
||||
assert info.value.status == 200, (
|
||||
f"the long-history follow-up must be accepted, got {info.value.status}"
|
||||
)
|
||||
|
||||
# The turn settles normally on the deterministic echo (the locked
|
||||
# budget trim may have dropped the giant turn WHOLE from the
|
||||
# prompt — A4 — so the echo's COUNT is never asserted, only the
|
||||
# settle). No failed bubble, no banner, composer usable (§7.4).
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=60_000)
|
||||
expect(bubble).to_contain_text("prior messages")
|
||||
expect(page.locator(".failed-note")).to_have_count(0)
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
_assert_settled_composer(page)
|
||||
|
||||
# The record chain held: the question's user record and the fresh
|
||||
# answer are persisted, the answer is not marked failed.
|
||||
stored = _stored(page)
|
||||
last = stored["messages"][-1]
|
||||
assert last["who"] == "brain"
|
||||
assert MOCK_ANSWER_MARKER in last["text"]
|
||||
assert last.get("failed") in (None, False)
|
||||
|
||||
# The saved-chat auto-save of the SAME full-length conversation
|
||||
# was accepted too (201/200 — the ChatMessage mirror + the total
|
||||
# budget; any 4xx here is the phase-113 quiet-failure class).
|
||||
time.sleep(1.0) # the fire-and-forget save margin
|
||||
assert chat_saves, "the auto-save never fired"
|
||||
assert all(status < 400 for _, status in chat_saves), chat_saves
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4 — client tolerance: an SSE terminal `error` frame whose `detail` is
|
||||
# an ARRAY serializes into the note, and the zero-frame settle
|
||||
# keeps FAILED_TURN_TEXT (a model-side failure is NOT the
|
||||
# pre-stream line)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _stub_sse_array_error(route: Route) -> None:
|
||||
"""A valid SSE stream ending in an `error` frame with an ARRAY
|
||||
detail (server-built frames carry a plain string — pinned in the
|
||||
task-02 units; this is the tolerant half). The `input` again must
|
||||
never reach the DOM."""
|
||||
frame = {
|
||||
"type": "error",
|
||||
"detail": [
|
||||
{
|
||||
"type": "value_error",
|
||||
"loc": ["body"],
|
||||
"msg": "mock stream-side rejection",
|
||||
"input": "q" * 1000,
|
||||
}
|
||||
],
|
||||
}
|
||||
route.fulfill(
|
||||
status=200,
|
||||
content_type="text/event-stream",
|
||||
body=f"data: {json.dumps(frame)}\n\n",
|
||||
)
|
||||
|
||||
|
||||
def test_sse_error_frame_with_array_detail_serializes_into_note(
|
||||
page: Page, app_url: str, mock_llm: int, db_ready: None
|
||||
) -> None:
|
||||
_reset_db()
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/")
|
||||
|
||||
page.route("**/api/chat", _stub_sse_array_error)
|
||||
_submit(page, "How is my Kubernetes cluster set up? (phase 164 e2e sse)")
|
||||
|
||||
# Zero frames + the terminal error frame: the MODEL-side settle —
|
||||
# FAILED_TURN_TEXT byte-identically (phase 120; the pre-stream
|
||||
# bubbleText is NEVER set on an SSE-frame throw), with the array
|
||||
# detail serialized into the note instead of stringified garbage.
|
||||
brain = page.locator(".msg.brain")
|
||||
expect(brain).to_have_count(1)
|
||||
expect(brain.locator(".bubble")).to_have_text(FAILED_TURN_TEXT)
|
||||
note = brain.locator(".failed-note")
|
||||
expect(note).to_have_count(1)
|
||||
expect(note).to_contain_text("Failed")
|
||||
expect(note).to_contain_text("mock stream-side rejection")
|
||||
_assert_dom_is_honest(page, forbidden_input_probe="q" * 100)
|
||||
|
||||
# The persisted record: the same text the user saw + the serialized
|
||||
# detail (the phase-120 persistence contract, unchanged shape).
|
||||
stored = _stored(page)
|
||||
failed = stored["messages"][-1]
|
||||
assert failed["text"] == FAILED_TURN_TEXT
|
||||
assert failed["failed"] is True
|
||||
assert failed["error"] == "mock stream-side rejection"
|
||||
|
||||
# Recovery + re-enable (§7.4).
|
||||
expect(brain.locator(".retry-btn")).to_have_count(1)
|
||||
expect(page.locator(".msg.user .edit-btn")).to_have_count(1)
|
||||
_assert_settled_composer(page)
|
||||
@@ -11,12 +11,16 @@ PUBLIC write surface ``POST/PUT /api/chats``; the caps live in
|
||||
``app/schemas.py`` and FastAPI rejects before the handler runs, A2):
|
||||
|
||||
* **Oversized save 422s, anonymously** — the exact audit vector (one
|
||||
40_000-char message, NO session, driven through the page's
|
||||
``page.request`` context — the browser's own network layer) is
|
||||
rejected at the schema boundary with a 422 and NOTHING is stored
|
||||
(the admin list carries no row for the probe's distinctive
|
||||
message PAST the per-message cap — 600 001 chars since phase 164
|
||||
raised it to admit a full-length answer, NO session, driven through
|
||||
the page's ``page.request`` context — the browser's own network
|
||||
layer) is rejected at the schema boundary with a 422 and NOTHING is
|
||||
stored (the admin list carries no row for the probe's distinctive
|
||||
auto-title — the shared e2e DB may hold other suites' rows, so the
|
||||
"nothing stored" proof is title-scoped, the house convention);
|
||||
the same suite's shape stays boundary-tied to ``app/schemas.py``:
|
||||
re-cut the probe whenever the cap moves, it must always be ONE past
|
||||
the current 600 000;
|
||||
* **Small save still 201s** — a normal in-cap save through the same
|
||||
anonymous network layer lands 201 with a valid ``id``: the boundary
|
||||
tightened the DoS surface without breaking the real (in-cap) flow
|
||||
@@ -66,10 +70,13 @@ def test_anonymous_oversized_save_422s_and_stores_nothing(
|
||||
page.set_default_timeout(30_000)
|
||||
_settled_anonymous(page, app_url)
|
||||
|
||||
text = PROBE_MARKER + " " + "x" * (40_000 - len(PROBE_MARKER) - 1)
|
||||
assert len(text) == 40_000
|
||||
# The audit vector, end-to-end and anonymous: one 40_000-char
|
||||
# message through the page's request context (no session cookie).
|
||||
# ONE past the per-message cap (phase 164, locked A1: 32 000 →
|
||||
# 600 000, so a full-length answer saves — the boundary moves with
|
||||
# the schema cap, this probe must always sit one char over it).
|
||||
text = PROBE_MARKER + " " + "x" * (600_001 - len(PROBE_MARKER) - 1)
|
||||
assert len(text) == 600_001
|
||||
# The audit vector, end-to-end and anonymous: one over-cap message
|
||||
# through the page's request context (no session cookie).
|
||||
r = page.request.post(
|
||||
app_url + "/api/chats",
|
||||
data={
|
||||
|
||||
@@ -2204,18 +2204,101 @@ def test_history_rejects_unknown_who(client, db) -> None:
|
||||
assert r.status_code == 422
|
||||
|
||||
|
||||
def test_history_rejects_more_than_100_entries(client, db) -> None:
|
||||
"""Schema pin: the DoS sanity ceiling is 100 turns — 101 is a 422
|
||||
def test_history_rejects_more_than_500_entries(client, db) -> None:
|
||||
"""Schema pin: the DoS sanity ceiling is 500 turns (phase 164, locked
|
||||
A1 — raised from 100 because the client sends the WHOLE
|
||||
``bor.chat.v1`` record minus the current question) — 501 is a 422
|
||||
(the config budgets do the real trimming; this only keeps a
|
||||
pathological body from wasting the mapper's work)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={
|
||||
"message": "hi",
|
||||
"history": [{"who": "user", "text": f"q{i}"} for i in range(101)],
|
||||
"history": [{"who": "user", "text": f"q{i}"} for i in range(501)],
|
||||
},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
assert r.json()["detail"][0]["loc"] == ["body", "history"]
|
||||
|
||||
|
||||
def test_history_rejects_one_char_over_the_text_cap(client, db) -> None:
|
||||
"""The upper half of the phase-164 boundary at the ENDPOINT (task 01,
|
||||
locked A1): one character past the 600 000 per-message cap is a 422 —
|
||||
and the ``detail`` FastAPI hands the browser is the pydantic LIST OF
|
||||
OBJECTS (``loc``/``msg``), the exact shape the client's serializer
|
||||
must read ``msg`` from (the live "[object Object]"). The 4000-char
|
||||
``message`` cap is unchanged (the composer mirrors it client-side)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={"message": "hi", "history": [{"who": "brain", "text": "a" * 600_001}]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
(entry,) = r.json()["detail"]
|
||||
assert entry["loc"] == ["body", "history", 0, "text"]
|
||||
assert entry["type"] == "string_too_long"
|
||||
assert entry["msg"] == "String should have at most 600000 characters"
|
||||
|
||||
|
||||
def test_history_accepts_a_full_length_prior_answer(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""Phase 164 task 01 — the live 422. The turn AFTER a full-length
|
||||
answer (525k chars ≈ a 131 072-token generation) sends that answer
|
||||
back as ``history``; the 32 000 ceiling rejected it, so the
|
||||
follow-up died with "[object Object]". It is now ACCEPTED: the
|
||||
stream opens (200) and settles with ``done``, no 422 anywhere.
|
||||
|
||||
What the model SEES is the locked budget trim's business (A4, not
|
||||
touched here): a turn over ``history_max_chars`` is dropped WHOLE,
|
||||
so this giant turn trims itself out of the prompt and
|
||||
``history_msgs=0`` — the follow-up works with a trimmed window.
|
||||
The acceptance is what this pin owns."""
|
||||
long_answer = "a" * 525_000
|
||||
history = [
|
||||
{"who": "user", "text": "Write me the whole k3s runbook."},
|
||||
{"who": "brain", "text": long_answer},
|
||||
]
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
frames = _stream_chat_with_history(client, "And the rollback part?", history)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["type"] == "done" # accepted: the turn settled normally
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "history_msgs=0" in lines[-1]
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
assert messages[1] == {"role": "user", "content": "And the rollback part?"}
|
||||
|
||||
|
||||
def test_history_accepts_a_full_length_answer_bounded_by_the_char_budget(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
) -> None:
|
||||
"""The same request with the giant answer NOT the newest turn: the
|
||||
newer small turns fit ``history_max_chars`` and reach the model,
|
||||
while the 525k turn is dropped WHOLE (locked A3/A4 — never cut,
|
||||
never a 422). Proves the raise changed what ARRIVES, not what the
|
||||
trim keeps."""
|
||||
history = [
|
||||
{"who": "brain", "text": "a" * 525_000},
|
||||
{"who": "user", "text": "Short follow-up question."},
|
||||
]
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
_stream_chat_with_history(client, "And the rollback part?", history)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
# The giant turn is gone whole; the fitting newer turn is kept.
|
||||
assert {"role": "user", "content": "Short follow-up question."} in messages
|
||||
assert all(len(m["content"]) < 525_000 for m in messages)
|
||||
|
||||
|
||||
def test_done_event_serializes_column_maximum_source_refs() -> None:
|
||||
|
||||
@@ -1642,18 +1642,20 @@ def test_page_route_serves_shared_html_when_present(
|
||||
# handler runs — no route change, nothing ever lands in the JSONB) ----------
|
||||
|
||||
|
||||
def test_post_rejects_text_over_32000_and_stores_nothing(
|
||||
def test_post_rejects_text_over_600000_and_stores_nothing(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The audit vector (SEC-05): one 32_001-char message 422s at the
|
||||
boundary (``ChatMessage.text`` mirrors ``HistoryTurn``'s 32 000
|
||||
cap) and NOTHING is stored — the admin list is unchanged (the happy
|
||||
path for in-cap bodies stays the existing guest/admin pins)."""
|
||||
"""The audit vector (SEC-05): one 600_001-char message 422s at the
|
||||
boundary (``ChatMessage.text`` mirrors ``HistoryTurn``'s 600 000 cap
|
||||
— raised TOGETHER in phase 164, locked A1) and NOTHING is stored —
|
||||
the admin list is unchanged (the happy path for in-cap bodies stays
|
||||
the existing guest/admin pins, incl. the full-cap-answer save
|
||||
below)."""
|
||||
baseline = admin_client.get("/api/chats").json()["chats"]
|
||||
|
||||
r = client.post(
|
||||
"/api/chats",
|
||||
json={"messages": [_user("a" * 32_001), _user("follow-up")]},
|
||||
json={"messages": [_user("a" * 600_001), _user("follow-up")]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
|
||||
@@ -1661,16 +1663,87 @@ cap) and NOTHING is stored — the admin list is unchanged (the happy
|
||||
assert listing == baseline, "an oversized POST must not store a row"
|
||||
|
||||
|
||||
def test_post_rejects_more_than_200_messages_and_stores_nothing(
|
||||
def test_post_accepts_a_full_cap_answer_and_round_trips(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The unbounded message list is the second DoS lever (SEC-05):
|
||||
201 minimally-valid messages 422 (``SavedChatCreate.messages``
|
||||
``max_items=200``) and nothing lands."""
|
||||
"""Phase 164 task 01 (the live report's save side): a conversation
|
||||
carrying ONE full-length brain answer (525k chars — a
|
||||
``BOR_MAX_OUTPUT_TOKENS=131_072`` answer at ~4 chars/token)
|
||||
saves 201 and round-trips byte-for-byte through the JSONB — the
|
||||
whole ``done``-time auto-save of a long answer must not 422 into
|
||||
the phase-113 quiet-failure. No column change: ``messages`` is raw
|
||||
JSONB, so the only gate is the schema boundary."""
|
||||
answer = "a" * 525_000
|
||||
r = client.post(
|
||||
"/api/chats",
|
||||
json={
|
||||
"messages": [
|
||||
_user("Write the whole runbook"),
|
||||
{"who": "brain", "text": answer},
|
||||
_user("And the rollback?"),
|
||||
{"who": "brain", "text": "Short rollback."},
|
||||
]
|
||||
},
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
chat_id = r.json()["id"]
|
||||
assert r.json()["messages"][1]["text"] == answer
|
||||
|
||||
stored = admin_client.get(f"/api/chats/{chat_id}").json()
|
||||
assert stored["messages"][1]["text"] == answer
|
||||
assert len(stored["messages"]) == 4
|
||||
|
||||
|
||||
def test_post_rejects_over_total_char_budget_and_stores_nothing(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The ONE new anonymous bound (phase 164, locked A1): messages that
|
||||
are individually legal but total over 8 000 000 characters 422 with
|
||||
the budget named in the ``detail`` (the model-level validator, so
|
||||
``loc`` is empty) — and nothing lands. 16 full-cap answers
|
||||
= 8 388 800 chars."""
|
||||
baseline = len(admin_client.get("/api/chats").json()["chats"])
|
||||
|
||||
r = client.post(
|
||||
"/api/chats", json={"messages": [_user(f"question {i}") for i in range(201)]}
|
||||
"/api/chats",
|
||||
json={"messages": [{"who": "brain", "text": "a" * 524_300} for _ in range(16)]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
assert "8,000,000-character budget" in json.dumps(r.json())
|
||||
|
||||
assert len(admin_client.get("/api/chats").json()["chats"]) == baseline
|
||||
|
||||
|
||||
def test_put_rejects_over_total_char_budget_and_leaves_row_unchanged(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The re-Save path carries the SAME validator (phase 164): an
|
||||
over-budget replacement 422s and the row keeps its original
|
||||
payload."""
|
||||
created = client.post("/api/chats", json={"messages": _simple_conversation()}).json()
|
||||
|
||||
r = client.put(
|
||||
f"/api/chats/{created['id']}",
|
||||
json={"messages": [{"who": "brain", "text": "a" * 524_300} for _ in range(16)]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
assert "8,000,000-character budget" in json.dumps(r.json())
|
||||
|
||||
kept = admin_client.get(f"/api/chats/{created['id']}").json()
|
||||
assert kept["messages"] == _expect(_simple_conversation())
|
||||
|
||||
|
||||
def test_post_rejects_more_than_500_messages_and_stores_nothing(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The unbounded message list is the second DoS lever (SEC-05):
|
||||
501 minimally-valid messages 422 (``SavedChatCreate.messages``
|
||||
``max_items=500`` — phase 164 raised it WITH
|
||||
:attr:`ChatRequest.history`) and nothing lands."""
|
||||
baseline = len(admin_client.get("/api/chats").json()["chats"])
|
||||
|
||||
r = client.post(
|
||||
"/api/chats", json={"messages": [_user(f"question {i}") for i in range(501)]}
|
||||
)
|
||||
assert r.status_code == 422
|
||||
|
||||
@@ -1707,7 +1780,7 @@ def test_put_rejects_oversized_message_and_leaves_row_unchanged(
|
||||
client: TestClient, admin_client: TestClient
|
||||
) -> None:
|
||||
"""The re-Save path is gated by the SAME caps (A1: create AND update
|
||||
carry the bounds): a 32_001-char message 422s and the row keeps its
|
||||
carry the bounds): a 600_001-char message 422s and the row keeps its
|
||||
original payload byte-for-byte (the stored shape is untouched — the
|
||||
rejected body never reaches the JSONB)."""
|
||||
created = client.post(
|
||||
@@ -1716,7 +1789,7 @@ def test_put_rejects_oversized_message_and_leaves_row_unchanged(
|
||||
|
||||
r = client.put(
|
||||
f"/api/chats/{created['id']}",
|
||||
json={"messages": [_user("b" * 32_001)]},
|
||||
json={"messages": [_user("b" * 600_001)]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
|
||||
@@ -1976,9 +2049,10 @@ def test_segments_over_50_entries_rejected(admin_client: TestClient) -> None:
|
||||
|
||||
|
||||
def test_thinking_segment_text_caps_rejected(admin_client: TestClient) -> None:
|
||||
"""The thinking entry's ``text`` cap (HistoryTurn's 32 000, phase
|
||||
83): 32 001 chars 422; the ``min_length=1`` bound — an EMPTY text
|
||||
(the zero-char run the client never stores) 422s too."""
|
||||
"""The thinking entry's ``text`` cap (HistoryTurn's 600 000 — the
|
||||
phase-164 raise, mirrored so a long-thinking turn never 422s its own
|
||||
auto-save): 600 001 chars 422; the ``min_length=1`` bound — an EMPTY
|
||||
text (the zero-char run the client never stores) 422s too."""
|
||||
over = admin_client.post(
|
||||
"/api/chats",
|
||||
json={
|
||||
@@ -1986,7 +2060,7 @@ def test_thinking_segment_text_caps_rejected(admin_client: TestClient) -> None:
|
||||
{
|
||||
"who": "brain",
|
||||
"text": "ok",
|
||||
"segments": [{"kind": "thinking", "text": "x" * 32_001}],
|
||||
"segments": [{"kind": "thinking", "text": "x" * 600_001}],
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -376,7 +376,7 @@ def test_failed_funnel_note_and_retry_join_the_actions_row() -> None:
|
||||
header = (
|
||||
"function finalizeFailedTurn(\n"
|
||||
" detail,\n"
|
||||
" { acc, thinking, tools, segments, wrap, leavePartialIndex }\n"
|
||||
" { acc, thinking, tools, segments, wrap, leavePartialIndex, bubbleText }\n"
|
||||
") {"
|
||||
)
|
||||
body = _fn_body(js, header)
|
||||
|
||||
@@ -924,9 +924,21 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
let detail = `Brain's API answered with HTTP ${res.status}.`;
|
||||
try {
|
||||
const body = await res.json();
|
||||
if (body.detail) detail = body.detail;
|
||||
// Phase 164 (D2): the 422 `detail` is a LIST OF OBJECTS — the
|
||||
// shared serializer yields the `msg` text, never
|
||||
// "[object Object]" and never the rejected `input`.
|
||||
detail = errorDetailText(body.detail, detail);
|
||||
} catch { /* non-JSON error body */ }
|
||||
throw new Error(detail);
|
||||
// Phase 164 (D3, locked A3/A4): a pre-stream HTTP rejection gets
|
||||
// its own honest bubble (the server ANSWERED — it did not drop),
|
||||
// and the phase-114 hint channel: the banner reassures that only
|
||||
// this send died. runTurn's catch threads bubbleText into
|
||||
// finalizeFailedTurn; hint rides setUiState's opts unchanged.
|
||||
const err = new Error(detail);
|
||||
err.bubbleText = FAILED_REQUEST_TEXT;
|
||||
err.hint =
|
||||
"The app answered — it rejected the request before the model ran. Nothing was lost but this send.";
|
||||
throw err;
|
||||
}
|
||||
await readSSE(res, (ev) => {
|
||||
if (aborted) return;
|
||||
@@ -1321,7 +1333,7 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
// Phase 114 (TODO L6): carry the frame's optional hint (the
|
||||
// "question too long" frame has one) through the throw — the
|
||||
// banner shows it in place of the default reachability hint.
|
||||
const err = new Error(ev.detail || "Something went wrong on my side.");
|
||||
const err = new Error(errorDetailText(ev.detail, "Something went wrong on my side."));
|
||||
err.hint = ev.hint;
|
||||
throw err;
|
||||
}
|
||||
@@ -1453,6 +1465,11 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
|
||||
segments: buildSegments(segRuns), // phase 158 (D4): the partial's timeline
|
||||
wrap,
|
||||
leavePartialIndex,
|
||||
// Phase 164 (D3): the pre-stream HTTP rejection carries its own
|
||||
// honest bubble line (FAILED_REQUEST_TEXT); every other throw
|
||||
// (transport, SSE error frame) leaves it undefined → the
|
||||
// zero-frame settle keeps FAILED_TURN_TEXT byte-identically.
|
||||
bubbleText: err?.bubbleText,
|
||||
});
|
||||
}
|
||||
// Phase 114 (TODO L6): the SSE error frame's optional hint flows to
|
||||
@@ -1511,8 +1528,15 @@ def test_run_turn_is_pinned_at_phase_159_state() -> None:
|
||||
moves BEFORE the block append — the append's height growth must
|
||||
not leak into the "pre" reading; the first-frame edge case — a
|
||||
fresh window is always pinned; the legacy block-level pin
|
||||
capture stays, no-op'ing on the divs): the full source must
|
||||
match the pin — the ``reask`` path, the turn-local resets, the
|
||||
capture stays, no-op'ing on the divs) and phase 164 (tasks 02+03)'s
|
||||
sanctioned changes (the pre-stream ``!res.ok`` detail read goes
|
||||
through the shared ``errorDetailText`` serializer — the live 422
|
||||
list-of-objects no longer stringifies to ``[object Object]``; the
|
||||
task-03 pre-stream throw carries ``bubbleText`` (FAILED_REQUEST_TEXT)
|
||||
+ the phase-114 ``hint``, and the catch forwards ``bubbleText:
|
||||
err?.bubbleText`` into the failed-turn funnel — locked A3): the
|
||||
full source must match the pin — the ``reask`` path, the turn-local
|
||||
resets, the
|
||||
SSE contract, the save points, and the sanctioned additions (the
|
||||
task-01 ``lastUserWrap`` assignment + the idle-settle mark
|
||||
pass, the task-03 aborted-settle mark pair, the phase-160
|
||||
|
||||
@@ -0,0 +1,362 @@
|
||||
"""Unit: the shared ``errorDetailText`` serializer (phase 164, task 02).
|
||||
|
||||
Behavior-pins the exported helper under node (the ``_run_node`` house
|
||||
pattern — the function is pure, so its extracted source runs standalone)
|
||||
across the full shape matrix, led by the EXACT live FastAPI 422 body of
|
||||
2026-10-07: a pydantic list-of-objects ``detail`` must serialize to the
|
||||
``msg`` verbatim — never ``"[object Object]"``, never the rejected
|
||||
``input`` text (the entries carry the full conversation — it must never
|
||||
reach the DOM). Source pins: the two chat call sites now read through
|
||||
the helper, the helper never ``JSON.stringify``-s anything, and the
|
||||
SEVEN existing guarded detail-readers (tuning.js, theme.js,
|
||||
document.js, git-sources.js, doc-edit.js, app.js's tuning form and its
|
||||
image-upload guard) stay byte-identical — no opportunistic refactor.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||||
ASSETS = FRONTEND / "assets"
|
||||
APP_JS = ASSETS / "app.js"
|
||||
CHAT_PY = Path(__file__).resolve().parents[2] / "app" / "api" / "chat.py"
|
||||
|
||||
HAVE_NODE = shutil.which("node") is not None
|
||||
|
||||
|
||||
def _js() -> str:
|
||||
return APP_JS.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _asset(name: str) -> str:
|
||||
return (ASSETS / name).read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _run_node(script: str) -> str:
|
||||
proc = subprocess.run(["node", "-e", script], capture_output=True, text=True, timeout=60)
|
||||
assert proc.returncode == 0, f"node failed: {proc.stderr}"
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def _fn(js: str, name: str) -> str:
|
||||
"""The full source of ``export function name(…) {…}`` via
|
||||
balanced-brace counting (the test_frontend_* convention — the
|
||||
``export`` prefix still matches the ``function name(`` search)."""
|
||||
start = js.find(f"function {name}(")
|
||||
assert start != -1, f"{name}() must exist in app.js"
|
||||
depth = 0
|
||||
for i in range(js.find("{", start), len(js)):
|
||||
if js[i] == "{":
|
||||
depth += 1
|
||||
elif js[i] == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return js[start : i + 1]
|
||||
raise AssertionError(f"unbalanced braces in {name}()")
|
||||
|
||||
|
||||
#: Sentinel serialized to the JS literal ``undefined`` (json has none).
|
||||
class _Undef:
|
||||
pass
|
||||
|
||||
|
||||
UNDEFINED = _Undef()
|
||||
|
||||
|
||||
def _lit(v: object) -> str:
|
||||
if isinstance(v, _Undef):
|
||||
return "undefined"
|
||||
return json.dumps(v)
|
||||
|
||||
|
||||
def _serialize(cases: list[tuple[str, object, object]]) -> dict[str, str]:
|
||||
"""Run the REAL exported ``errorDetailText`` under node over
|
||||
``[(name, value, fallback), …]`` and return ``{name: result}``."""
|
||||
script = (
|
||||
_fn(_js(), "errorDetailText")
|
||||
+ "\nconst out = [];\n"
|
||||
+ "".join(
|
||||
f"out.push([{_lit(name)}, errorDetailText({_lit(value)}, {_lit(fallback)})]);\n"
|
||||
for name, value, fallback in cases
|
||||
)
|
||||
+ "for (const row of out) console.log(JSON.stringify(row));\n"
|
||||
)
|
||||
rows = (json.loads(line) for line in _run_node(script).splitlines() if line)
|
||||
return {row[0]: row[1] for row in rows}
|
||||
|
||||
|
||||
#: The EXACT live 422 body (phase 164 overview): the follow-up turn of
|
||||
#: a >32k-char answer, rejected by the old HistoryTurn.text cap. The
|
||||
#: `input` is the whole rejected conversation — here a marked stand-in
|
||||
#: that must NEVER appear in the serialized result.
|
||||
LIVE_422_DETAIL = [
|
||||
{
|
||||
"type": "string_too_long",
|
||||
"loc": ["body", "history", 5, "text"],
|
||||
"msg": "String should have at most 600000 characters",
|
||||
"input": f"CANARY-INPUT {'x' * 4000}",
|
||||
}
|
||||
]
|
||||
|
||||
FB = "FALLBACK-LINE"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Behavior — the shape matrix under node (the exported, REAL function)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_live_422_list_detail_serializes_the_msg_only() -> None:
|
||||
"""The EXACT live 422 body: the entry's ``msg`` verbatim, and the
|
||||
result carries NO ``input`` text and no ``"[object Object]"`` —
|
||||
the 2026-10-07 failure is unrepresentable (locked A2)."""
|
||||
got = _serialize([("live422", LIVE_422_DETAIL, FB)])
|
||||
assert got["live422"] == "String should have at most 600000 characters"
|
||||
assert "CANARY" not in got["live422"], "the rejected input must never serialize"
|
||||
assert "[object Object]" not in got["live422"]
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_array_entries_join_with_semicolon_space() -> None:
|
||||
"""Multi-entry arrays join their usable texts with ``"; "``:
|
||||
several object ``msg``s, and mixed object + plain-string entries
|
||||
(the generalization of the seven guarded copies)."""
|
||||
got = _serialize(
|
||||
[
|
||||
(
|
||||
"multi_obj",
|
||||
[{"msg": "first problem"}, {"msg": "second problem"}],
|
||||
FB,
|
||||
),
|
||||
("mixed", [{"msg": "obj msg"}, "plain string"], FB),
|
||||
("plain_only", ["a", "b"], FB),
|
||||
("number_entry", [42], FB),
|
||||
]
|
||||
)
|
||||
assert got["multi_obj"] == "first problem; second problem"
|
||||
assert got["mixed"] == "obj msg; plain string"
|
||||
assert got["plain_only"] == "a; b"
|
||||
assert got["number_entry"] == "42"
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_empty_array_entries_drop_and_all_empty_falls_back() -> None:
|
||||
"""Entries with neither a usable ``msg`` nor a non-empty string
|
||||
drop (objects without msg, empty msgs, nullish, empty strings); an
|
||||
all-empty array — and the bare ``[]`` — fall back."""
|
||||
got = _serialize(
|
||||
[
|
||||
("drop_and_keep", [{"nomsg": 1}, {"msg": ""}, None, "", {"msg": "kept"}], FB),
|
||||
("all_empty", [{"nomsg": 1}, None, ""], FB),
|
||||
("bare_empty", [], FB),
|
||||
]
|
||||
)
|
||||
assert got["drop_and_keep"] == "kept"
|
||||
assert got["all_empty"] == FB
|
||||
assert got["bare_empty"] == FB
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_object_detail_msg_or_fallback() -> None:
|
||||
"""A plain object with a non-empty ``msg`` serializes it verbatim;
|
||||
without one — even carrying a huge rejected ``input`` — it falls
|
||||
back (never stringified)."""
|
||||
got = _serialize(
|
||||
[
|
||||
("obj_msg", {"type": "string_too_long", "msg": "only the msg"}, FB),
|
||||
("obj_nomsg", {"input": "CANARY " * 500}, FB),
|
||||
("obj_empty_msg", {"msg": ""}, FB),
|
||||
]
|
||||
)
|
||||
assert got["obj_msg"] == "only the msg"
|
||||
assert got["obj_nomsg"] == FB
|
||||
assert got["obj_empty_msg"] == FB
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_string_verbatim_incl_server_fixed_copy_otherwise_fallback() -> None:
|
||||
"""A non-empty string serializes VERBATIM — the pinned server-built
|
||||
frame copies ("Question too long…", "The chat model dropped the
|
||||
connection…") must reach the user byte-identical. ``""``, null,
|
||||
undefined, a number, and a boolean all fall back."""
|
||||
server_copy_1 = "Question too long — trim it and re-ask."
|
||||
server_copy_2 = "The chat model dropped the connection — try again?"
|
||||
# The server-side half of the pin: these ARE the ChatErrorEvent copies.
|
||||
chat_py = CHAT_PY.read_text(encoding="utf-8")
|
||||
assert f'detail="{server_copy_1}"' in chat_py
|
||||
assert f'detail="{server_copy_2}"' in chat_py
|
||||
got = _serialize(
|
||||
[
|
||||
("server_copy_1", server_copy_1, FB),
|
||||
("server_copy_2", server_copy_2, FB),
|
||||
("plain", "some string", FB),
|
||||
("empty", "", FB),
|
||||
("null", None, FB),
|
||||
("undefined", UNDEFINED, FB),
|
||||
("number", 42, FB),
|
||||
("boolean", True, FB),
|
||||
]
|
||||
)
|
||||
assert got["server_copy_1"] == server_copy_1
|
||||
assert got["server_copy_2"] == server_copy_2
|
||||
assert got["plain"] == "some string"
|
||||
for name in ("empty", "null", "undefined", "number", "boolean"):
|
||||
assert got[name] == FB, name
|
||||
|
||||
|
||||
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
|
||||
def test_no_shape_yields_object_object() -> None:
|
||||
"""TOTALITY sweep: across every shape the helper can meet, no
|
||||
result is ever ``"[object Object]"`` (or contains it) — the string
|
||||
is unrepresentable at the serializer level."""
|
||||
shapes = [
|
||||
LIVE_422_DETAIL,
|
||||
[{"type": "x", "loc": [1], "msg": "m", "input": "i"}] * 3,
|
||||
[{"a": {"b": 1}}],
|
||||
{"deep": {"nested": [1, 2]}},
|
||||
[[{"msg": "nested"}]],
|
||||
"s",
|
||||
0,
|
||||
None,
|
||||
UNDEFINED,
|
||||
]
|
||||
got = _serialize([(f"s{i}", v, FB) for i, v in enumerate(shapes)])
|
||||
for name, res in got.items():
|
||||
assert isinstance(res, str) and res, name
|
||||
assert "[object Object]" not in res, name
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Source pins — the export, the two call sites, the untouched copies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_error_detail_text_is_exported_for_unit_pinning() -> None:
|
||||
"""The helper is an EXPORTED function (the TURN_TIMEOUT_MS /
|
||||
THINKING_NEAR_BOTTOM_PX export-for-unit-pin idiom), declared
|
||||
exactly once."""
|
||||
js = _js()
|
||||
assert re.search(r"^export function errorDetailText\(value, fallback\) \{", js, re.M), (
|
||||
"errorDetailText must be an exported top-level function"
|
||||
)
|
||||
assert js.count("function errorDetailText(") == 1
|
||||
|
||||
|
||||
def test_helper_never_stringifies_never_reads_input_loc_ctx() -> None:
|
||||
"""LOCKED A2 inside the helper: NO ``JSON.stringify`` anywhere in
|
||||
its source, and the words ``input``/``loc``/``ctx`` never appear —
|
||||
only ``msg`` and plain strings serialize (the docstring names the
|
||||
rule but the FUNCTION body is what this pins)."""
|
||||
fn = _fn(_js(), "errorDetailText")
|
||||
assert "JSON.stringify" not in fn
|
||||
for forbidden in ("input", "loc", "ctx"):
|
||||
assert re.search(rf"\b{forbidden}\b", fn) is None, (
|
||||
f"the helper must never touch `{forbidden}` (the 422 entries "
|
||||
"carry the full rejected conversation)"
|
||||
)
|
||||
# The docstring above the helper names the live failure + the rule.
|
||||
js = _js()
|
||||
doc = js[: js.find("function errorDetailText(")][-2500:]
|
||||
assert "2026-10-07" in doc and "[object Object]" in doc
|
||||
assert "input" in doc, "the docstring names the input-leak rule"
|
||||
|
||||
|
||||
def test_prestream_call_site_reads_through_the_helper() -> None:
|
||||
"""The pre-stream ``!res.ok || !res.body`` handler: the unguarded
|
||||
``if (body.detail) detail = body.detail`` is GONE, the assignment
|
||||
reads ``errorDetailText(body.detail, …)`` with the per-status
|
||||
string as the fallback, and the try-around-res.json + non-JSON
|
||||
catch keep their shape."""
|
||||
js = _js()
|
||||
assert "if (body.detail) detail = body.detail;" not in js
|
||||
site = js.index('if (!res.ok || !res.body) {')
|
||||
block_end = js.index("await readSSE(", site)
|
||||
block = js[site:block_end]
|
||||
assert "let detail = `Brain's API answered with HTTP ${res.status}.`;" in block
|
||||
assert "detail = errorDetailText(body.detail, detail);" in block, (
|
||||
"the per-status default is the serializer's fallback"
|
||||
)
|
||||
assert "} catch { /* non-JSON error body */ }" in block
|
||||
# Phase 164 task 03 decorates THIS error (bubbleText + hint) before
|
||||
# the throw — the thrown message stays the serialized detail.
|
||||
assert "const err = new Error(detail);" in block
|
||||
assert "throw err;" in block
|
||||
# The fallback is exactly the status line the bubble must show for
|
||||
# a non-JSON 500 (phase 164 E2E case b).
|
||||
assert "Brain's API answered with HTTP" in block
|
||||
|
||||
|
||||
def test_sse_error_frame_call_site_reads_through_the_helper() -> None:
|
||||
"""The SSE ``error``-frame throw: ``new Error(errorDetailText(ev.detail,
|
||||
"Something went wrong on my side."))`` — and the phase-114
|
||||
``err.hint`` line stays byte-identical right after it."""
|
||||
js = _js()
|
||||
assert (
|
||||
'const err = new Error(errorDetailText(ev.detail, "Something went wrong on my side."));'
|
||||
in js
|
||||
)
|
||||
site = js.index('const err = new Error(errorDetailText(ev.detail,')
|
||||
tail = js[site : site + 200]
|
||||
assert tail.splitlines()[1].strip() == "err.hint = ev.hint;", (
|
||||
"the err.hint channel is untouched"
|
||||
)
|
||||
assert 'new Error(ev.detail ||' not in js, "the old unguarded read is gone"
|
||||
|
||||
|
||||
#: One guarded 422-reader block per existing site (the seven copies,
|
||||
#: D2 "stay byte-identical — no opportunistic refactor"): the exact
|
||||
#: idiom lines, file → (array-msg line, string line).
|
||||
GUARDED_COPIES: dict[str, tuple[str, str]] = {
|
||||
"tuning.js": (
|
||||
" if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {",
|
||||
' if (typeof data.detail === "string" && data.detail) return data.detail;',
|
||||
),
|
||||
"theme.js": (
|
||||
" if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {",
|
||||
' if (typeof data.detail === "string" && data.detail) return data.detail;',
|
||||
),
|
||||
"document.js": (
|
||||
" if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {",
|
||||
' if (typeof data.detail === "string" && data.detail) return data.detail;',
|
||||
),
|
||||
"git-sources.js": (
|
||||
" if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {",
|
||||
' if (typeof data.detail === "string" && data.detail) return data.detail;',
|
||||
),
|
||||
"doc-edit.js": (
|
||||
" if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {",
|
||||
' if (typeof data.detail === "string" && data.detail) return data.detail;',
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", sorted(GUARDED_COPIES))
|
||||
def test_existing_guarded_readers_are_byte_identical(name: str) -> None:
|
||||
"""The five external guarded readers keep their EXACT lines
|
||||
(indented, verbatim) — the phase touches only the two unguarded
|
||||
chat paths."""
|
||||
src = _asset(name)
|
||||
array_line, string_line = GUARDED_COPIES[name]
|
||||
assert array_line in src, f"{name} lost its guarded array-msg line"
|
||||
assert string_line in src, f"{name} lost its guarded string line"
|
||||
|
||||
|
||||
def test_app_js_guarded_copies_are_untouched() -> None:
|
||||
"""app.js's own guarded copies — the tuning-form reader (the
|
||||
three-branch idiom) and the image-upload string guard — stay
|
||||
byte-identical, and the tuning form was NOT opportunistically
|
||||
rewritten onto the new helper (exactly one Array-guard block in
|
||||
app.js: the tuning form)."""
|
||||
js = _js()
|
||||
guard = "if (Array.isArray(data.detail) && data.detail[0] && data.detail[0].msg) {"
|
||||
assert js.count(guard) == 1
|
||||
assert "detail = String(data.detail[0].msg);" in js
|
||||
assert "} else if (typeof data.detail === \"string\" && data.detail) {" in js
|
||||
assert 'if (typeof body?.detail === "string") detail = body.detail;' in js
|
||||
@@ -137,9 +137,10 @@ def test_failed_turn_text_is_a_distinct_constant() -> None:
|
||||
assert failed_text != m_empty.group(1), (
|
||||
"FAILED_TURN_TEXT must be DISTINCT from EMPTY_ANSWER_FALLBACK"
|
||||
)
|
||||
# The zero-frame branch uses the constant, not a copy of the
|
||||
# fallback.
|
||||
assert 'addMessage("brain", FAILED_TURN_TEXT);' in js
|
||||
# The zero-frame branch uses the constant as its DEFAULT (phase 164
|
||||
# D3: ``bubbleText || FAILED_TURN_TEXT`` — only the pre-stream HTTP
|
||||
# rejection overrides it), never a copy of the fallback.
|
||||
assert "bubbleText || FAILED_TURN_TEXT" in js
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -160,8 +161,12 @@ def test_finalize_failed_turn_persists_failed_in_both_shapes() -> None:
|
||||
# …the detail trimmed + capped at 500 before persistence (the
|
||||
# schema's ChatMessage.error bound is the backstop)…
|
||||
assert '(detail || "").trim().slice(0, 500)' in body
|
||||
# …the zero-frame shape creates the FAILED_TURN_TEXT bubble…
|
||||
assert 'addMessage("brain", FAILED_TURN_TEXT);' in body
|
||||
# …the zero-frame shape creates the failed bubble (phase 164 D3:
|
||||
# ``bubbleText || FAILED_TURN_TEXT`` — the SAME value feeds the
|
||||
# bubble AND the record: stored-what-you-saw)…
|
||||
assert "const fwrapText = bubbleText || FAILED_TURN_TEXT;" in body
|
||||
assert 'addMessage("brain", fwrapText);' in body
|
||||
assert "rememberBrainTurn(\n fwrapText," in body
|
||||
assert "appendFailedNote(fwrap, error);" in body
|
||||
# …and the partial shape settles every open run closed (phase 158,
|
||||
# D5 — the stop-finalize pattern) and adds the note.
|
||||
@@ -448,6 +453,158 @@ def test_retry_last_turn_is_byte_unchanged() -> None:
|
||||
assert _fn_source(_js(), "retryLastTurn") == PINNED_RETRY_LAST_TURN
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 164 task 03 — the distinct PRE-STREAM failure line (locked A3)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The owner-locked bubble copy (D3) — pinned VERBATIM, exactly once.
|
||||
FAILED_REQUEST_TEXT_COPY = (
|
||||
"I couldn't take that — my API rejected the request. Use Retry to ask again."
|
||||
)
|
||||
|
||||
#: The phase-114 banner hint the pre-stream throw carries (D3).
|
||||
PRESTREAM_HINT_COPY = (
|
||||
"The app answered — it rejected the request before the model ran. "
|
||||
"Nothing was lost but this send."
|
||||
)
|
||||
|
||||
|
||||
def _prestream_block(js: str) -> str:
|
||||
"""The ``if (!res.ok || !res.body) { … }`` pre-stream handler block."""
|
||||
site = js.index("if (!res.ok || !res.body) {")
|
||||
return js[site : js.index("await readSSE(", site)]
|
||||
|
||||
|
||||
def _catch_else_body(js: str) -> str:
|
||||
"""``runTurn``'s catch: the plain ``else`` branch (non-abort,
|
||||
non-stop — the failed-turn funnel site)."""
|
||||
stop_branch = js.index('} else if (stoppedByUser || err?.name === "AbortError") {')
|
||||
else_branch = js.index("} else {", stop_branch)
|
||||
return js[else_branch : js.index("} finally {", else_branch)]
|
||||
|
||||
|
||||
def test_failed_request_text_is_owner_locked_and_distinct() -> None:
|
||||
"""``FAILED_REQUEST_TEXT`` is its OWN string literal with the exact
|
||||
owner-locked copy (D3), declared exactly once, DISTINCT from both
|
||||
``FAILED_TURN_TEXT`` (the connection line — a lie for a 4xx/5xx the
|
||||
server answered deliberately) and ``EMPTY_ANSWER_FALLBACK``."""
|
||||
js = _js()
|
||||
m_req = re.search(r'const FAILED_REQUEST_TEXT\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m_req, "const FAILED_REQUEST_TEXT must be a string literal"
|
||||
assert m_req.group(1) == FAILED_REQUEST_TEXT_COPY, (
|
||||
"the pre-stream copy is owner-locked — verbatim or not at all"
|
||||
)
|
||||
assert js.count("const FAILED_REQUEST_TEXT") == 1
|
||||
m_turn = re.search(r'const FAILED_TURN_TEXT\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
m_empty = re.search(r'const EMPTY_ANSWER_FALLBACK\s*=\s*\n?\s*"([^"]*)"', js)
|
||||
assert m_turn and m_empty
|
||||
assert m_req.group(1) != m_turn.group(1)
|
||||
assert m_req.group(1) != m_empty.group(1)
|
||||
# The docstring names the finding + the not-for-transport boundary.
|
||||
doc = js[: js.index("const FAILED_REQUEST_TEXT")][-1200:]
|
||||
assert "2026-10-07" in doc and "PRE-STREAM" in doc
|
||||
|
||||
|
||||
def test_prestream_throw_carries_bubble_text_and_hint() -> None:
|
||||
"""The pre-stream ``!res.ok`` throw decorates the Error with
|
||||
``bubbleText = FAILED_REQUEST_TEXT`` AND the phase-114 ``hint``
|
||||
before throwing — the thrown message stays the serialized detail
|
||||
(task 02), so the note + banner keep the real reason while the
|
||||
bubble reads the honest pre-stream line."""
|
||||
block = _prestream_block(_js())
|
||||
assert "const err = new Error(detail);" in block
|
||||
assert "err.bubbleText = FAILED_REQUEST_TEXT;" in block
|
||||
assert f'"{PRESTREAM_HINT_COPY}"' in block, (
|
||||
"the banner hint copy is locked (D3) — verbatim"
|
||||
)
|
||||
# Both decorations ride the SAME thrown Error (set before the throw).
|
||||
throw = block.index("throw err;")
|
||||
assert 0 < block.index("err.bubbleText = FAILED_REQUEST_TEXT;") < throw
|
||||
assert 0 < block.index("err.hint") < throw
|
||||
assert "throw new Error(detail);" not in block
|
||||
|
||||
|
||||
def test_run_turn_catch_forwards_bubble_text_to_the_funnel() -> None:
|
||||
"""``runTurn``'s catch forwards ``bubbleText: err?.bubbleText``
|
||||
into ``finalizeFailedTurn`` (the ONE option addition) — optional
|
||||
chaining so a transport ``TypeError`` (no key, not even a custom
|
||||
Error) reads ``undefined`` and the settle keeps ``FAILED_TURN_TEXT``.
|
||||
The detail/hint/banner sequencing after the call is untouched."""
|
||||
else_body = _catch_else_body(_js())
|
||||
assert "bubbleText: err?.bubbleText," in else_body
|
||||
funnel = else_body.index("finalizeFailedTurn(detail, {")
|
||||
bubble = else_body.index("bubbleText: err?.bubbleText,")
|
||||
banner = else_body.index("setUiState(UI_STATE.error, detail,")
|
||||
assert funnel < bubble < banner, (
|
||||
"the key rides the funnel call, before the error-state settle "
|
||||
"(the phase-120 ordering contract)"
|
||||
)
|
||||
|
||||
|
||||
def test_finalize_reads_bubble_text_only_in_the_zero_frame_branch() -> None:
|
||||
"""``finalizeFailedTurn`` destructures ``bubbleText`` and reads it
|
||||
ONLY in the zero-frame branch (the partial-stream settle ignores
|
||||
the key — a pre-stream throw has no ``wrap`` by construction). The
|
||||
zero-frame settle puts the SAME ``text`` value into the bubble AND
|
||||
the persisted record (stored-what-you-saw, phase-120 contract
|
||||
extended), and touches no ``innerHTML``."""
|
||||
js = _js()
|
||||
src = _fn_source(js, "finalizeFailedTurn")
|
||||
assert "{ acc, thinking, tools, segments, wrap, leavePartialIndex, bubbleText }" in src
|
||||
body = _fn_body(js, "finalizeFailedTurn")
|
||||
zero = body.index("} else {")
|
||||
assert "bubbleText" not in body[:zero], (
|
||||
"the partial branch must never read bubbleText"
|
||||
)
|
||||
# Code-level (comment-stripped) reads of the key: exactly one.
|
||||
code = "\n".join(
|
||||
line
|
||||
for line in body.splitlines()
|
||||
if not line.strip().startswith(("//", "*", "/*"))
|
||||
)
|
||||
assert code.count("bubbleText") == 1
|
||||
zero_part = body[zero:]
|
||||
assert "const fwrapText = bubbleText || FAILED_TURN_TEXT;" in zero_part
|
||||
assert 'addMessage("brain", fwrapText);' in zero_part
|
||||
assert "rememberBrainTurn(\n fwrapText," in zero_part, (
|
||||
"the bubble text and the record text are the SAME value"
|
||||
)
|
||||
assert "innerHTML" not in zero_part, "the failed bubble is text, never HTML"
|
||||
|
||||
|
||||
def test_sse_error_frame_throw_sets_no_bubble_text() -> None:
|
||||
"""The SSE ``error``-frame throw (a model-side failure AFTER the
|
||||
stream opened) decorates ``err.hint`` only — NO ``bubbleText``: a
|
||||
zero-frame error-frame settle keeps ``FAILED_TURN_TEXT``
|
||||
byte-identically (locked A3)."""
|
||||
js = _js()
|
||||
site = js.index('} else if (ev.type === "error") {')
|
||||
frame = js[site : js.index("throw err;", site) + len("throw err;")]
|
||||
assert "const err = new Error(errorDetailText(ev.detail," in frame, (
|
||||
"the frame throw reads through the serializer (task 02)"
|
||||
)
|
||||
assert '"Something went wrong on my side."));' in frame
|
||||
assert "err.hint = ev.hint;" in frame
|
||||
assert "bubbleText" not in frame
|
||||
|
||||
|
||||
def test_hint_reaches_banner_opts_for_both_throw_kinds() -> None:
|
||||
"""The phase-114 hint channel reaches the banner the SAME way for
|
||||
both throw kinds: the catch passes ``{ hint: err.hint }`` into
|
||||
``setUiState(UI_STATE.error, …)`` — the pre-stream throw sets a
|
||||
literal hint, the SSE frame throw copies ``ev.hint`` (undefined →
|
||||
showErrorBanner's default). The stream-drop guard's banner call
|
||||
keeps its optionless shape."""
|
||||
js = _js()
|
||||
assert (
|
||||
"setUiState(UI_STATE.error, detail, err instanceof Error ? { hint: err.hint } : {});"
|
||||
in _catch_else_body(js)
|
||||
)
|
||||
guard = js.index("if (!sawDone && !aborted && (acc || thinkingAcc)) {")
|
||||
zero_frame = js.index("if (!aborted && !wrap) {", guard)
|
||||
assert "setUiState(UI_STATE.error, detail);" in js[guard:zero_frame]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CSS — the in-bubble error line (the .stopped-note family, error color)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -323,7 +323,7 @@ def test_continue_button_is_not_added_on_the_other_settle_paths() -> None:
|
||||
js,
|
||||
"function finalizeFailedTurn(\n"
|
||||
" detail,\n"
|
||||
" { acc, thinking, tools, segments, wrap, leavePartialIndex }\n"
|
||||
" { acc, thinking, tools, segments, wrap, leavePartialIndex, bubbleText }\n"
|
||||
") {",
|
||||
)
|
||||
assert "appendContinueButton" not in funnel, (
|
||||
|
||||
+221
-3
@@ -7,6 +7,20 @@ past it raises a pydantic ``ValidationError`` naming the offending field
|
||||
Plus the regression pin: a realistic ``bor.chat.v1`` payload validates
|
||||
cleanly and round-trips ``model_dump()`` (the stored-shape contract —
|
||||
the caps added value bounds only, no key/shape change).
|
||||
|
||||
Phase 164 (task 01, locked A1) re-cuts the long-answer chain —
|
||||
``HistoryTurn.text``/``.thinking``, ``ChatMessage.text``/``.thinking``
|
||||
and the thinking-segment cap to 600 000 (a FULL-cap answer at
|
||||
``BOR_MAX_OUTPUT_TOKENS=131_072``, ≈ 524 300 chars), the history list
|
||||
to 500 and the saved-messages list to 500, and adds the ONE new bound:
|
||||
the ``SavedChatCreate``/``SavedChatUpdate`` total-character budget
|
||||
(8 000 000) that keeps the anonymous SEC-05 surface bounded now that a
|
||||
single message may hold a full-length answer. The chat-path caps are
|
||||
pinned here too (``HistoryTurn`` + ``ChatRequest.history``) — same
|
||||
boundaries as the saved-chat mirror, and the 422 ``detail`` SHAPE a cap
|
||||
rejection produces is pinned as the server-side half of the phase-164
|
||||
serializer contract (a LIST OF OBJECTS — the live ``[object Object]``
|
||||
report).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -14,11 +28,13 @@ import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from app.schemas import (
|
||||
SAVED_CHAT_TOTAL_CHARS_BUDGET,
|
||||
ChatMessage,
|
||||
ChatRequest,
|
||||
ChatSegment,
|
||||
ChatThinkingSegment,
|
||||
ChatToolSegment,
|
||||
HistoryTurn,
|
||||
SavedChatCreate,
|
||||
SavedChatUpdate,
|
||||
SourceRef,
|
||||
@@ -27,7 +43,14 @@ from app.schemas import (
|
||||
|
||||
# --- boundary constants (phase 83, A1 — mirror sources in app/schemas.py) ---
|
||||
|
||||
TEXT_CAP = 32_000 # mirrors HistoryTurn.text / .thinking
|
||||
#: Phase 164 (locked A1): 32 000 → 600 000. ONE constant for the whole
|
||||
#: mirror — ``HistoryTurn.text``/``.thinking`` (the chat path),
|
||||
#: ``ChatMessage.text``/``.thinking`` and the ``segments`` thinking
|
||||
#: entry: the identical-boundary test IS the proof the phase-83 mirror
|
||||
#: invariant survived the raise (raise one side and these pins fail).
|
||||
#: 600 000 ≈ a full-cap answer at ``BOR_MAX_OUTPUT_TOKENS=131_072``
|
||||
#: (≈ 524 300 chars at ~4 chars/token) with margin.
|
||||
TEXT_CAP = 600_000
|
||||
SOURCES_CAP = 20
|
||||
SUGGESTIONS_CAP = 50
|
||||
CHIP_CAP = 200
|
||||
@@ -37,8 +60,15 @@ PATH_CAP = 1000 # documents.path String(1000)
|
||||
TITLE_CAP = 500 # documents.title String(500)
|
||||
NAME_CAP = 100 # ToolCall.name
|
||||
ARGUMENT_CAP = 2000 # ToolCall.argument
|
||||
MESSAGES_CAP = 200 # SavedChatCreate/Update.messages
|
||||
MESSAGES_CAP = 500 # SavedChatCreate/Update.messages (phase 164: was 200)
|
||||
HISTORY_CAP = 500 # ChatRequest.history (phase 164: was 100)
|
||||
QUESTION_CAP = 4_000 # ChatRequest.message — the cap the composer mirrors (phase 104)
|
||||
#: The phase-164 anonymous-surface bound, imported from the source (not
|
||||
#: restated) so the schema and the suite cannot drift apart.
|
||||
TOTAL_BUDGET = SAVED_CHAT_TOTAL_CHARS_BUDGET
|
||||
#: A realistic full-cap answer length (131 072 output tokens at ~4
|
||||
#: chars/token) — the payload the whole chain must now carry.
|
||||
FULL_CAP_ANSWER = 524_300
|
||||
|
||||
|
||||
def _source_ref() -> dict:
|
||||
@@ -207,6 +237,103 @@ def test_chat_request_message_one_over_cap_rejects() -> None:
|
||||
_failed_loc(exc.value, "message")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HistoryTurn + ChatRequest.history — the CHAT path of the long-answer
|
||||
# chain (phase 164, task 01, locked A1). The live 422 this phase fixes:
|
||||
# a >32k-char brain answer rode back as ``history`` on the NEXT turn and
|
||||
# rejected it, so the follow-up of any long answer died with
|
||||
# "[object Object]". Same boundaries as the saved-chat mirror above.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_history_turn_text_at_cap_validates() -> None:
|
||||
"""600 000 chars — a full-cap answer (≈ 524 300 at
|
||||
``BOR_MAX_OUTPUT_TOKENS=131_072``) flows back as ``history``:
|
||||
the exact request shape the browser sends on the turn AFTER a long
|
||||
answer."""
|
||||
turn = HistoryTurn(who="brain", text="a" * TEXT_CAP)
|
||||
assert len(turn.text) == TEXT_CAP
|
||||
# And the realistic full-cap answer, not just the boundary value.
|
||||
assert len(HistoryTurn(who="brain", text="a" * FULL_CAP_ANSWER).text) == FULL_CAP_ANSWER
|
||||
|
||||
|
||||
def test_history_turn_text_one_over_cap_rejects() -> None:
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
HistoryTurn(who="brain", text="a" * (TEXT_CAP + 1))
|
||||
_failed_loc(exc.value, "text")
|
||||
|
||||
|
||||
def test_history_turn_thinking_at_cap_validates() -> None:
|
||||
turn = HistoryTurn(who="brain", text="ok", thinking="t" * TEXT_CAP)
|
||||
assert len(turn.thinking or "") == TEXT_CAP
|
||||
|
||||
|
||||
def test_history_turn_thinking_one_over_cap_rejects() -> None:
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
HistoryTurn(who="brain", text="ok", thinking="t" * (TEXT_CAP + 1))
|
||||
_failed_loc(exc.value, "thinking")
|
||||
|
||||
|
||||
def test_history_turn_caps_match_the_saved_chat_mirror() -> None:
|
||||
"""The phase-83 mirror invariant, verified by identical boundaries
|
||||
(not by comment): a payload at the cap validates on BOTH models and
|
||||
one past it fails on BOTH — ``HistoryTurn`` (the chat path) and
|
||||
``ChatMessage`` (the saved-chat path). Raise one side in
|
||||
``app/schemas.py`` and this pin fails."""
|
||||
for length, rejected in ((TEXT_CAP, False), (TEXT_CAP + 1, True)):
|
||||
payload = {"who": "brain", "text": "a" * length, "thinking": "t" * length}
|
||||
for model in (HistoryTurn, ChatMessage):
|
||||
if rejected:
|
||||
with pytest.raises(ValidationError):
|
||||
model.model_validate(payload)
|
||||
else:
|
||||
model.model_validate(payload)
|
||||
|
||||
|
||||
def test_chat_request_history_at_cap_validates() -> None:
|
||||
"""500 turns pass — the client sends the WHOLE ``bor.chat.v1``
|
||||
record minus the current question, so the ceiling is a DoS sanity
|
||||
bound only (the config budgets trim for real, locked A4)."""
|
||||
req = ChatRequest.model_validate(
|
||||
{"message": "hi", "history": [{"who": "user", "text": f"q{i}"} for i in range(HISTORY_CAP)]}
|
||||
)
|
||||
assert len(req.history) == HISTORY_CAP
|
||||
|
||||
|
||||
def test_chat_request_history_one_over_cap_rejects() -> None:
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
ChatRequest.model_validate(
|
||||
{
|
||||
"message": "hi",
|
||||
"history": [
|
||||
{"who": "user", "text": f"q{i}"} for i in range(HISTORY_CAP + 1)
|
||||
],
|
||||
}
|
||||
)
|
||||
_failed_loc(exc.value, "history")
|
||||
|
||||
|
||||
def test_cap_rejection_detail_is_a_list_of_objects() -> None:
|
||||
"""The server-side half of the phase-164 contract (task 02's input):
|
||||
a cap rejection is the pydantic 422 ``detail`` — a LIST OF OBJECTS
|
||||
(``loc``/``msg``/``type``), never a plain string. This is exactly the
|
||||
shape the client turned into ``[object Object]``; the serializer must
|
||||
read ``msg`` and never the ``input`` (which carries the whole
|
||||
conversation)."""
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
HistoryTurn(who="brain", text="a" * (TEXT_CAP + 1))
|
||||
detail = exc.value.errors()
|
||||
assert isinstance(detail, list) and len(detail) == 1
|
||||
(entry,) = detail
|
||||
assert entry["loc"] == ("text",)
|
||||
assert entry["type"] == "string_too_long"
|
||||
# The message the UI shows verbatim in the failed-turn note.
|
||||
assert entry["msg"] == f"String should have at most {TEXT_CAP} characters"
|
||||
# The raw input DOES ride the error object server-side — which is why
|
||||
# the client serializer only ever reads ``msg``.
|
||||
assert entry["input"].startswith("aaaa")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SourceRef (documents column-length mirrors)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -322,6 +449,97 @@ def test_saved_chat_update_messages_one_over_cap_rejects() -> None:
|
||||
_failed_loc(exc.value, "messages")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The phase-164 total-character budget (locked A1) — the ONE new bound on
|
||||
# the anonymous saved-chat surface. The per-message caps went to 600 000,
|
||||
# so the LIST is bounded by total characters instead of by an item count
|
||||
# that a full-cap answer would blow through: 8 000 000 keeps the worst
|
||||
# case at the order the phase-83 SEC-05 audit accepted (200 × 64k
|
||||
# ≈ 12.8 MB) while fitting ~15 full-cap answers.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _full_cap_messages(count: int, *, size: int = FULL_CAP_ANSWER) -> list[dict]:
|
||||
"""``count`` brain answers at the realistic full-cap length."""
|
||||
return [{"who": "brain", "text": "a" * size} for _ in range(count)]
|
||||
|
||||
|
||||
def test_saved_chat_within_total_budget_of_full_cap_answers_validates() -> None:
|
||||
"""~15 full-cap answers (15 × 524 300 = 7 864 500 chars, the biggest
|
||||
whole-count fit under 8 000 000) validate — the budget is a sanity
|
||||
bound, not a practical limit on a real conversation."""
|
||||
fits = TOTAL_BUDGET // FULL_CAP_ANSWER
|
||||
assert fits == 15
|
||||
payload = SavedChatCreate.model_validate({"messages": _full_cap_messages(fits)})
|
||||
assert len(payload.messages) == fits
|
||||
|
||||
|
||||
def test_saved_chat_over_total_budget_rejects_with_the_budget_message() -> None:
|
||||
"""One full-cap answer past the fit → ValidationError whose ``msg``
|
||||
NAMES the budget (the 422 the anonymous caller sees), on BOTH the
|
||||
create and the update model — the same validator on both bodies."""
|
||||
over = _full_cap_messages(TOTAL_BUDGET // FULL_CAP_ANSWER + 1)
|
||||
for model in (SavedChatCreate, SavedChatUpdate):
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
model.model_validate({"messages": over})
|
||||
(error,) = exc.value.errors()
|
||||
assert error["loc"] == () # a model-level validator, not a field
|
||||
assert "8,000,000-character budget" in error["msg"]
|
||||
|
||||
|
||||
def test_saved_chat_one_full_cap_answer_plus_normal_turns_validates() -> None:
|
||||
"""The realistic case the live report blocked: a saved chat carrying
|
||||
ONE 525k-char brain answer alongside ordinary turns."""
|
||||
payload = SavedChatCreate.model_validate(
|
||||
{
|
||||
"messages": [
|
||||
{"who": "user", "text": "Write me the whole runbook."},
|
||||
{"who": "brain", "text": "a" * 525_000},
|
||||
{"who": "user", "text": "And the rollback?"},
|
||||
{"who": "brain", "text": "Short rollback."},
|
||||
]
|
||||
}
|
||||
)
|
||||
assert len(payload.messages) == 4
|
||||
|
||||
|
||||
def test_saved_chat_total_budget_counts_thinking() -> None:
|
||||
"""``thinking`` counts against the budget exactly like ``text`` (the
|
||||
history budget's own accounting, mirrored): 15 turns of 280 000 text
|
||||
+ 280 000 thinking = 8 400 000 REJECTS even though the text alone is
|
||||
only 4 200 000 — a text-only accounting would have waved it
|
||||
through."""
|
||||
turns = 15
|
||||
size = 280_000
|
||||
assert turns * size <= TOTAL_BUDGET < turns * (size * 2)
|
||||
heavy = [
|
||||
{"who": "brain", "text": "a" * size, "thinking": "t" * size}
|
||||
for _ in range(turns)
|
||||
]
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
SavedChatCreate.model_validate({"messages": heavy})
|
||||
assert "8,000,000-character budget" in exc.value.errors()[0]["msg"]
|
||||
# One turn fewer fits (14 × 560 000 = 7 840 000).
|
||||
assert len(SavedChatCreate.model_validate({"messages": heavy[:-1]}).messages) == 14
|
||||
|
||||
|
||||
def test_saved_chat_at_exactly_the_total_budget_validates() -> None:
|
||||
"""The boundary is INCLUSIVE: exactly 8 000 000 characters across the
|
||||
messages passes (13 at the 600 000 cap + one of 200 000), one
|
||||
character more rejects."""
|
||||
|
||||
def _conversation(last_size: int) -> list[dict]:
|
||||
return [{"who": "brain", "text": "a" * TEXT_CAP} for _ in range(13)] + [
|
||||
{"who": "brain", "text": "b" * last_size}
|
||||
]
|
||||
|
||||
at = SavedChatCreate.model_validate({"messages": _conversation(200_000)})
|
||||
assert sum(len(m.text) for m in at.messages) == TOTAL_BUDGET
|
||||
with pytest.raises(ValidationError) as exc:
|
||||
SavedChatCreate.model_validate({"messages": _conversation(200_001)})
|
||||
assert "8,000,000-character budget" in exc.value.errors()[0]["msg"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JSON-shape regression: same KEYS as before, value bounds only
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -747,7 +965,7 @@ def test_segments_absent_by_default_and_omitted_in_dump() -> None:
|
||||
|
||||
def test_segments_thinking_entry_round_trips_and_caps() -> None:
|
||||
"""The thinking entry: ``min_length=1`` (the zero-char run the
|
||||
client never stores) + ``max_length=32_000`` (HistoryTurn's cap,
|
||||
client never stores) + ``max_length=600_000`` (HistoryTurn's cap,
|
||||
phase 83) — at the cap validates, one past raises naming the
|
||||
offender; a missing ``text`` too."""
|
||||
at_cap = ChatMessage.model_validate(
|
||||
|
||||
Reference in New Issue
Block a user