phase: 164_chat_error_honesty

All gates green. Report:

- **Verified (phase 164, all 4 tasks already in `complete/`)**: `HistoryTurn`/`ChatMessage`/`ChatThinkingSegment` caps → 600 000, `history`/saved-messages lists → 500, new `SavedChatCreate/Update` total-8 000 000-char validator; exported `errorDetailText` at the two previously unguarded chat paths (pre-stream `!res.ok` + SSE `error` frame); `FAILED_REQUEST_TEXT` + `bubbleText` threading + phase-114 banner hint; dedicated E2E `tests/e2e/test_chat_error_honesty.py`.
- **Final-pass additions**: `tests/integration/test_chat_api.py::test_history_rejects_one_char_over_the_text_cap` (endpoint-level >600k → 422 with the pydantic list-of-objects `msg` — the criterion's upper half was only unit-pinned); README env rows for `BOR_APP_VERSION` + `BOR_SESSION_COOKIE`; restored `test_document_viewer.py`'s tracked screenshot artifact dirtied by the E2E run (via read-only `git show`, no index/history touched).
- **Tests/lint (exact commands)**: `uv run pytest --cov=app --cov-report=term-missing` → **3837 passed**, `app/` coverage **99%** (>90%); `uv run pytest tests/e2e/test_chat_error_honesty.py -v --no-cov` → **4 passed in isolation**; per-file loop over **all 148** E2E files (`uv run pytest tests/e2e/<f> -q --no-cov`) → **148 × exit 0**, incl. `test_failed_turn_retry`, `test_chat_save_payload_limits`, `test_edit_and_resend`, `test_llm_history`, `test_smoke`; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- **Completion criteria**: pre-stream 422 shows the distinct line + verbatim `msg`, `[object Object]` unrepresentable + no `input` in DOM — **met** (unit totality/source pins + E2E asserts 1–2); 525k history accepted (200 settles) / >600k → 422 / saved chat 201 + 8M bound — **met**; transport + SSE-frame settles keep `FAILED_TURN_TEXT`, stop/abort untouched, §7.4 re-enables — **met**; budget trim, `message` 4000, the 7 guarded detail-readers and all SSE shapes byte-identical (diff touches only caps + the two chat paths) — **met**; README/AGENTS full review + proxy deploy note — **met**; commit/`complete/` move = pipeline gate.
- **Docs**: README reviewed in full (env tables cross-checked programmatically: 74 `BOR_*` rows vs `Settings` defaults — no mismatches; every `.env.example` var documented; referenced scripts/pages/endpoints/`query_log` columns exist) — found & fixed the 2 undocumented real knobs above; the phase's payload-limits section, failed-turn copy and `client_max_body_size ≥ 16 MB` deploy note verified accurate. `AGENTS.md` verified consistent (rules, gitignore list, quick-reference commands, referenced paths all correct) — no changes needed.
- **Deviations**: `ChatThinkingSegment.text` raised 32 000 → 600 000 too (not itemized in A1, but its own docstring mirrors `HistoryTurn.thinking`; without it a long-thinking turn 422s its own auto-save — same mirror invariant). No live-infra changes made (the proxy limit lives in the separate deployment repo; README note only).
- **Next pending phase**: none — `164_chat_error_honesty` was the only entry in `todo/`.

app/schemas.py

@@ -13,6 +13,7 @@ from pydantic import (
     SerializerFunctionWrapHandler,
     field_validator,
     model_serializer,
+    model_validator,
 )
 
 
@@ -42,20 +43,32 @@ class HistoryTurn(BaseModel):
     assistant message (the wire convention :mod:`app.rag.llm` already
     documents for the response side) — only when non-empty (A4).
     ``text`` mirrors :attr:`ChatMessage.text`'s answer shape; long
-    answers (and the scratchpads that ride along as ``thinking``) both
-    run to tens of kilobytes of text, so both share the same loose
-    boundary cap. These are boundary sanity caps only — the real
-    trimming budget is the settings pair ``history_max_turns`` /
+    answers (and the scratchpads that ride along as ``thinking``) run to
+    hundreds of kilobytes, so both share the same loose boundary
+    cap. These are boundary sanity caps only — the real trimming
+    budget is the settings pair ``history_max_turns`` /
     ``history_max_chars`` (``app.config``, A3: a capped-out turn is
-    dropped whole, never truncated). The old 4000-char text cap was
-    stricter than the 24_000-char default total budget and rejected
-    any second turn in a chat whose history held a long answer (422 —
-    found by the phase-42 E2E suite on the phase-76 shell).
+    dropped whole, never truncated) — that locked budget trim stays the
+    ONLY gate on what reaches the model, and it never 422s.
+
+    The cap is 600 000 since phase 164 (locked A1). History: the old
+    4000-char text cap was stricter than the 24_000-char default total
+    budget and rejected any second turn in a chat whose history held a
+    long answer (422 — found by the phase-42 E2E suite on the phase-76
+    shell), so it went to 32 000 — and that ceiling broke AGAIN the
+    moment an answer could exceed it (the 2026-10-07 live report: the
+    FOLLOW-UP turn of a long answer 422'd, the same failure class).
+    600 000 admits a FULL-cap answer with margin: ``max_output_tokens``
+    runs to ``BOR_MAX_OUTPUT_TOKENS`` (the owner raised it to
+    131 072), ≈ 524 300 chars at the ~4 chars/token rule of thumb, so
+    the ceiling must clear that — and a turn longer than the prompt
+    budget simply trims ITSELF out of later prompts (dropped whole by
+    ``history_to_messages``) instead of rejecting the send.
     """
 
     who: Literal["user", "brain"]
-    text: str = Field(min_length=1, max_length=32_000)
-    thinking: str | None = Field(default=None, max_length=32000)
+    text: str = Field(min_length=1, max_length=600_000)
+    thinking: str | None = Field(default=None, max_length=600_000)
 
 
 class ChatRequest(BaseModel):
@@ -68,14 +81,19 @@ class ChatRequest(BaseModel):
     ``bor.chat.v1`` record minus the current question); the mapper
     (:func:`app.rag.prompts.history_to_messages`) trims it newest-first
     against the settings budgets and maps it to model messages. The
-    schema-level ``max_length=100`` is a DoS sanity ceiling only — the
-    config budgets do the real trimming (A3). Absent or empty keeps the
-    request byte-identical to pre-phase-74: the model sees exactly the
+    schema-level ``max_length`` (500 since phase 164, locked A1) is a
+    DoS sanity ceiling ONLY — the config budgets do the real trimming
+    (A3). It must clear a LONG conversation, because the client sends
+    the whole ``bor.chat.v1`` record minus the current question (not a
+    window) — the turn count, not the char budget, is what a chatty
+    session runs into first, and hitting a sanity ceiling with a 422 is
+    never the right answer. Absent or empty keeps the request
+    byte-identical to pre-phase-74: the model sees exactly the
     two-message ``[system, user]`` request.
     """
 
     message: str = Field(min_length=1, max_length=4000)
-    history: list[HistoryTurn] = Field(default_factory=list, max_length=100)
+    history: list[HistoryTurn] = Field(default_factory=list, max_length=500)
     #: Phase 123 (TODO L6, LOCKED A5): the STORED PATH of the question's
     #: attached image — ``/api/chat-images/<uuid4-hex>.<ext>`` exactly as
     #: ``POST /api/chat-images`` returns it. NEVER a raw data URL: the
@@ -1143,14 +1161,18 @@ class ChatThinkingSegment(BaseModel):
     EMPTY run stores NOTHING (a zero-char run — e.g. a
     ``BOR_STREAM_THINKING``-suppressed turn) is absent from the
     record, so a stored entry is always non-empty. ``max_length=
-    32_000`` mirrors :class:`HistoryTurn.thinking`'s cap (phase 83,
+    600_000`` mirrors :class:`HistoryTurn.thinking`'s cap (phase 83,
     SEC-05 — a single turn longer than that is already rejected on
     the chat path, so a saved chat can never legitimately carry
-    more).
+    more), RAISED with it in phase 164 (locked A1): these runs are the
+    pieces of :attr:`ChatMessage.thinking` — the same text,
+    concatenated — so the piece cap has to clear what the whole
+    accepts, or a long-thinking turn would 422 its own auto-save
+    (the phase-113 quiet-failure).
     """
 
     kind: Literal["thinking"]
-    text: str = Field(min_length=1, max_length=32_000)
+    text: str = Field(min_length=1, max_length=600_000)
 
 
 class ChatToolSegment(BaseModel):
@@ -1246,15 +1268,23 @@ class ChatMessage(BaseModel):
 
     Phase 83 (SEC-05) bounds the anonymous write surface (``POST/PUT
     /api/chats`` is public — the row id is the credential, phase 55 A1):
-    ``text`` / ``thinking`` carry :class:`HistoryTurn`'s 32 000 caps
-    (a single message longer than that is already rejected on the chat
-    path, so a saved chat can never legitimately carry more), and the
-    nested lists get length caps (``max_length``) sized to the realistic
-    ``bor.chat.v1`` record the UI produces (``sources`` ≤ 20 — top-N docs
+    ``text`` / ``thinking`` carry :class:`HistoryTurn`'s caps (raised to
+    600 000 TOGETHER with them in phase 164, locked A1 — the invariant
+    is "a saved chat can never legitimately carry MORE than the chat
+    path accepts", and it survives the raise only if BOTH sides move:
+    raise one and every done-time auto-save of a full-length answer 422s
+    into the phase-113 quiet-failure), and the nested lists get length
+    caps (``max_length``) sized to the realistic ``bor.chat.v1`` record
+    the UI produces (``sources`` ≤ 20 — top-N docs
     + agent reads; ``related`` ≤ 20 — the related-doc tier, same ref
     shape; ``suggestions`` ≤ 50 chips of ≤ 200 chars; ``tools``
     ≤ 50 — one entry per tool call, the round cap is 10). Only value
-    bounds were added — the accepted/rejected KEYS are unchanged.
+    bounds were added — the accepted/rejected KEYS are unchanged. The
+    per-conversation TOTAL is bounded on that same anonymous surface by
+    :class:`SavedChatCreate` / :class:`SavedChatUpdate`'s character
+    budget (phase 164): the caps above bind per message, so the
+    list-level validator is what keeps the worst case at the audited
+    order.
 
     Phase 120 (task 01, locked A1): a FAILED turn (network error, SSE
     ``error`` frame, stream drop — the client-side failure paths) stores
@@ -1300,12 +1330,12 @@ class ChatMessage(BaseModel):
     model_config = ConfigDict(extra="forbid")
 
     who: Literal["user", "brain"]
-    text: str = Field(min_length=1, max_length=32_000)
+    text: str = Field(min_length=1, max_length=600_000)
     sources: list[SourceRef] | None = Field(default=None, max_length=20)
     related: list[SourceRef] | None = Field(default=None, max_length=20)
     deflected: bool | None = None
     suggestions: list[_Chip] | None = Field(default=None, max_length=50)
-    thinking: str | None = Field(default=None, max_length=32_000)
+    thinking: str | None = Field(default=None, max_length=600_000)
     tools: list[ToolCall] | None = Field(default=None, max_length=50)
     # Phase 158 (task 01, D4/D6 — owner direction 2026-10-04): the
     # turn's CHRONOLOGICAL run order — one ``thinking`` entry per
@@ -1414,6 +1444,36 @@ class ChatMessage(BaseModel):
         return data
 
 
+#: Phase 164 (locked A1): the TOTAL character budget one saved chat may
+#: carry — ``len(text) + len(thinking or "")`` summed over every
+#: message. The per-message caps raised to 600 000 would otherwise
+#: multiply into an unbounded anonymous write surface (``POST/PUT
+#: /api/chats`` is public — the row id is the credential, phase 55 A1);
+#: 8 000 000 keeps the worst case at the order the phase-83 SEC-05 audit
+#: accepted (200 messages × 64k ≈ 12.8 MB) while fitting ~15 full-cap
+#: answers (≈ 524 300 chars each). Enforced on both the create and the
+#: update body — the framework's standard 422, boundary-only (the
+#: phase-83 A2 style: NO route change, nothing reaches the JSONB).
+SAVED_CHAT_TOTAL_CHARS_BUDGET = 8_000_000
+
+
+def _require_within_total_budget(messages: list[ChatMessage]) -> list[ChatMessage]:
+    """The saved-chat total-characters check shared by create and update
+    (phase 164): ``len(text) + len(thinking or "")`` over every message
+    must stay within :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET`.
+
+    Raises :class:`ValueError` naming the budget, which Pydantic turns
+    into the ordinary 422 validation response (``loc`` = the model
+    itself, ``msg`` carrying the budget wording) — the same boundary-
+    only style as every other phase-83/164 cap."""
+    total = sum(len(m.text) + len(m.thinking or "") for m in messages)
+    if total > SAVED_CHAT_TOTAL_CHARS_BUDGET:
+        raise ValueError(
+            f"saved chat exceeds the {SAVED_CHAT_TOTAL_CHARS_BUDGET:,}-character budget"
+        )
+    return messages
+
+
 class SavedChatCreate(BaseModel):
     """``POST /api/chats`` body (phase 50, task 02; ``share``, phase 51
     task 02).
@@ -1422,9 +1482,16 @@ class SavedChatCreate(BaseModel):
     row (the first user message's text, whitespace-collapsed, truncated
     to 120 chars — the owner-locked convention). ``messages`` must be
     non-empty — a saved chat with nothing to restore is meaningless —
-    and, since phase 83 (SEC-05), at most 200 items: well past any realistic
-    conversation (the chat history budget itself is 40 turns) and far
-    below a DoS-sized list on the anonymous write surface.
+    and bounded on the anonymous write surface two ways: at most 500
+    items (phase 83 SEC-05 set 200; phase 164 locked A1 raised it WITH
+    :attr:`ChatRequest.history`, which the client fills from the same
+    ``bor.chat.v1`` record — a save must carry what a chat turn accepts)
+    AND a TOTAL of at most :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET`
+    characters summed over every message's ``text`` + ``thinking`` (the
+    bound that keeps the SEC-05 worst case at the audited order now that
+    ONE message may hold a full-cap answer). Both rejections are the
+    framework's standard 422 (A2 boundary-only: no route change, nothing
+    stored).
 
     ``share`` (phase 51, owner-locked 2026-08-29): when true, the row is
     shared in the SAME commit — ``share_token = uuid.uuid4()`` is set on
@@ -1436,9 +1503,18 @@ class SavedChatCreate(BaseModel):
     """
 
     title: str | None = Field(default=None, max_length=500)
-    messages: list[ChatMessage] = Field(min_length=1, max_length=200)
+    messages: list[ChatMessage] = Field(min_length=1, max_length=500)
     share: bool = False
 
+    @model_validator(mode="after")
+    def _within_total_char_budget(self) -> SavedChatCreate:
+        """Phase 164 (locked A1): the conversation as a whole stays
+        inside :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET` (see
+        :func:`_require_within_total_budget`) — the anonymous-surface
+        bound the raised per-message caps make necessary."""
+        _require_within_total_budget(self.messages)
+        return self
+
 
 class SavedChatUpdate(BaseModel):
     """``PUT /api/chats/{chat_id}`` body (phase 50, task 02).
@@ -1446,13 +1522,23 @@ class SavedChatUpdate(BaseModel):
     ``messages`` is a full replacement (the re-Save upsert semantics —
     re-Saving the same conversation updates the same row, never a new
     one) and carries the same bounds as :class:`SavedChatCreate.messages`
-    (phase 83, SEC-05: non-empty, ≤ 200 items). ``title`` is replaced
-    only when supplied — an absent (or blank) ``title`` keeps the row's
-    current title.
+    (phase 83, SEC-05: non-empty; phase 164 locked A1: ≤ 500 items AND
+    the :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET` total — the re-Save path
+    IS the same anonymous write surface, so the same validator gates
+    it). ``title`` is replaced only when supplied — an absent (or
+    blank) ``title`` keeps the row's current title.
     """
 
     title: str | None = Field(default=None, max_length=500)
-    messages: list[ChatMessage] = Field(min_length=1, max_length=200)
+    messages: list[ChatMessage] = Field(min_length=1, max_length=500)
+
+    @model_validator(mode="after")
+    def _within_total_char_budget(self) -> SavedChatUpdate:
+        """Phase 164 (locked A1): identical to
+        :meth:`SavedChatCreate._within_total_char_budget` — create and
+        update carry the same bounds."""
+        _require_within_total_budget(self.messages)
+        return self
 
 
 def _drop_image_fields(model: KbTreeFile, handler: SerializerFunctionWrapHandler) -> Any: