phase: 182_verbatim_show_text
**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**
- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
+124
-7
@@ -182,6 +182,47 @@ class SourceRef(BaseModel):
|
||||
return data
|
||||
|
||||
|
||||
class VerbatimBlockRef(BaseModel):
|
||||
"""One SERVER-resolved verbatim block of a chat answer (phase 182).
|
||||
|
||||
The item of :attr:`ChatDoneEvent.answer_text` and of the persisted
|
||||
:class:`ChatMessage.answer_text` — the display block the
|
||||
``show_text`` tool or a `` ```bor `` locator fence asked for, with
|
||||
the exact bytes read out of ``documents.content`` by
|
||||
:func:`app.rag.verbatim.block_for`.
|
||||
|
||||
**``text`` is SERVER-supplied bytes, never model output** (LOCKED
|
||||
A4/A7): the model only ever names the location, so what the user
|
||||
reads and copies cannot differ from the indexed file by a character
|
||||
— the phase's whole reason for existing. The field carries the same
|
||||
600 000 cap as the phase-83/164 answer text (deliberately generous
|
||||
above the 12 000 default of ``BOR_SHOW_TEXT_MAX_CHARS``, so an
|
||||
operator raising that budget never hits the wire limit instead).
|
||||
|
||||
``start_line`` / ``end_line`` are **1-based and INCLUSIVE** and
|
||||
describe exactly the lines ``text`` contains AFTER any character cap
|
||||
— they are the SERVER's resolved extent, never the model's claim
|
||||
(LOCKED A2), which is what the panel's caption prints. The caps on
|
||||
``source`` / ``path`` / ``title`` are :class:`SourceRef`'s (the same
|
||||
``documents`` column lengths, phase 83 SEC-05); ``locator`` is the
|
||||
canonical matching key (phase 182 D7 — ``source/path`` or
|
||||
``source/path#12-40``) the client matches its rendered placeholder
|
||||
against.
|
||||
"""
|
||||
|
||||
source: str = Field(max_length=120)
|
||||
path: str = Field(max_length=1000)
|
||||
title: str = Field(max_length=500)
|
||||
start_line: int = Field(ge=0)
|
||||
end_line: int = Field(ge=0)
|
||||
total_lines: int = Field(ge=0)
|
||||
#: The verbatim bytes themselves (see the class docstring) — the
|
||||
#: phase-83 answer-text cap, generous above the per-block budget.
|
||||
text: str = Field(max_length=600_000)
|
||||
truncated: bool = False
|
||||
locator: str = Field(max_length=2100)
|
||||
|
||||
|
||||
class ChatThinkingEvent(BaseModel):
|
||||
"""SSE thinking event: one chunk of the model's reasoning (phase 17).
|
||||
|
||||
@@ -342,6 +383,23 @@ class ChatDoneEvent(BaseModel):
|
||||
byte-identical to pre-phase (the byte-identical-off contract,
|
||||
PLAN §13; the :attr:`answer_image` / :class:`SourceRef.image_url`
|
||||
omission precedent).
|
||||
|
||||
Phase 182 (task 05, LOCKED A1/A4/A9): ``answer_text`` — the answer's
|
||||
VERBATIM display blocks, the text sibling of :attr:`answer_image`
|
||||
with two deliberate differences. (1) It is a LIST: one answer may
|
||||
display several blocks (the ``show_text`` calls and/or the
|
||||
`` ```bor `` locator fences the answer itself wrote — D1), in
|
||||
DISPLAY order (answer order first, then the tool blocks the answer
|
||||
never located — :func:`app.rag.verbatim.resolve_answer_blocks`).
|
||||
(2) Each item carries the TEXT itself rather than a route: the bytes
|
||||
ride inline (A4 — no new bytes route, no new anonymous surface, so a
|
||||
shared page renders from its own record exactly like the phase-123
|
||||
chat image), and every byte of it comes from ``documents.content``,
|
||||
never from the model (A7). ``None`` / ``[]`` — every turn without
|
||||
blocks, every deflected turn (D9 resolves nothing on a disclaimer),
|
||||
every ``answer_text=False`` turn, and the whole pre-phase — is
|
||||
DROPPED on serialization: ABSENT, never ``null``, never ``[]``, so
|
||||
those frames stay byte-identical to pre-phase (the same contract).
|
||||
"""
|
||||
|
||||
type: str = "done"
|
||||
@@ -357,25 +415,36 @@ class ChatDoneEvent(BaseModel):
|
||||
#: ONLY on the cap-reached turn; ``False`` (every other turn)
|
||||
#: serializes WITHOUT the key — ABSENT, never ``false``.
|
||||
budget_exhausted: bool = False
|
||||
#: Phase 182 (task 05) — see the class docstring. ``None``/``[]``
|
||||
#: (every turn without verbatim blocks) serializes WITHOUT the key —
|
||||
#: ABSENT, never ``null``, never ``[]``.
|
||||
answer_text: list[VerbatimBlockRef] | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The phase-124 answer-image omission rule: ``answer_image``
|
||||
``None`` (every deflected / image-less / pre-phase turn)
|
||||
serializes WITHOUT the key — ABSENT, never ``null`` — so those
|
||||
done frames stay byte-identical to pre-phase (the
|
||||
"""The additive-key omission rules: ``answer_image`` ``None``
|
||||
(every deflected / image-less / pre-phase turn) serializes
|
||||
WITHOUT the key — ABSENT, never ``null`` — so those done frames
|
||||
stay byte-identical to pre-phase (the
|
||||
:class:`SourceRef.image_url` / phase-123 ``image`` omission
|
||||
precedents). A shown ref keeps all its fields (its own
|
||||
serializer drops a ``None`` ``image_url`` — a shown doc is an
|
||||
image doc, so in practice it always carries it). Phase 160
|
||||
(task 03): ``budget_exhausted`` ``False`` (every non-cap turn)
|
||||
serializes WITHOUT the key too — ABSENT, never ``false`` — the
|
||||
same byte-identical-off contract."""
|
||||
same byte-identical-off contract. Phase 182 (task 05):
|
||||
``answer_text`` ``None`` OR EMPTY (a turn whose answer resolved
|
||||
no block, a deflected turn, the whole ``answer_text=False``
|
||||
configuration, and the pre-phase) serializes WITHOUT the key —
|
||||
ABSENT, never ``null`` and never ``[]``, so every block-free
|
||||
frame keeps its exact pre-phase bytes."""
|
||||
data = handler(self)
|
||||
if self.answer_image is None:
|
||||
data.pop("answer_image", None)
|
||||
if not self.budget_exhausted:
|
||||
data.pop("budget_exhausted", None)
|
||||
if not self.answer_text:
|
||||
data.pop("answer_text", None)
|
||||
return data
|
||||
|
||||
|
||||
@@ -1478,6 +1547,28 @@ class ChatMessage(BaseModel):
|
||||
# exactly like ``stopped`` — the phase-50 byte-identical
|
||||
# round-trip contract covers the new key automatically.
|
||||
budget_exhausted: bool | None = None
|
||||
# Phase 182 (task 05, LOCKED A4/A9): the answer's VERBATIM display
|
||||
# blocks — the persisted twin of the done frame's ``answer_text``
|
||||
# (the ``show_text`` tool's blocks and/or the ones a `` ```bor ``
|
||||
# locator fence resolved), on the BRAIN record only. Like
|
||||
# ``answer_image`` the field EXISTS so the ``extra="forbid"``
|
||||
# boundary does NOT 422 the done-time auto-save carrying it (the
|
||||
# phase-113 lesson, applied up front), and the restore path plus the
|
||||
# anonymous shared page re-render the panels from it — the bytes are
|
||||
# part of the chat's own content, so they ride the same
|
||||
# credential-is-the-row trust model and need NO new route (A4).
|
||||
# No migration (``saved_chats.messages`` is raw JSONB, the phase-50
|
||||
# / 160 precedent). ``None``/``[]`` serializes WITHOUT the key (the
|
||||
# :meth:`_drop_image_when_absent` serializer below), so a turn with
|
||||
# no blocks — and every pre-phase record — round-trips
|
||||
# byte-identically (A9). The per-block ``text`` cap is
|
||||
# :class:`VerbatimBlockRef`'s own; the list cap matches the
|
||||
# ``tools``/``segments`` cap (50), far past the
|
||||
# ``show_text_max_blocks`` default of 5 — and the block bytes count
|
||||
# toward :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET`, so the display blocks
|
||||
# never become a second, unbounded channel on the anonymous save
|
||||
# surface (the phase-83 SEC-05 invariant).
|
||||
answer_text: list[VerbatimBlockRef] | None = Field(default=None, max_length=50)
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _drop_image_when_absent(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
@@ -1492,12 +1583,21 @@ class ChatMessage(BaseModel):
|
||||
record WITH one keeps it — the user bubble, the refreshed page,
|
||||
and the shared chat all render the image from the served route
|
||||
(the image is part of the chat's content, so it rides the same
|
||||
public/credential-is-the-id trust model)."""
|
||||
public/credential-is-the-id trust model).
|
||||
|
||||
Phase 182 (task 05) extends the same rule to ``answer_text``:
|
||||
``None`` OR an empty list (every record without verbatim blocks,
|
||||
and every pre-phase-182 record) serializes WITHOUT the key —
|
||||
ABSENT, never ``null``, never ``[]`` (A9), so those records
|
||||
round-trip byte-identically while a record WITH blocks keeps the
|
||||
server's bytes for the restore path and the shared page."""
|
||||
data = handler(self)
|
||||
if self.image is None:
|
||||
data.pop("image", None)
|
||||
if self.answer_image is None:
|
||||
data.pop("answer_image", None)
|
||||
if not self.answer_text:
|
||||
data.pop("answer_text", None)
|
||||
# Phase 158 (task 01, D4): the timeline is ABSENT (never
|
||||
# null) when there is no run to render — the pre-phase /
|
||||
# no-thinking-no-tools shape stays byte-identical (the
|
||||
@@ -1529,11 +1629,28 @@ def _require_within_total_budget(messages: list[ChatMessage]) -> list[ChatMessag
|
||||
(phase 164): ``len(text) + len(thinking or "")`` over every message
|
||||
must stay within :data:`SAVED_CHAT_TOTAL_CHARS_BUDGET`.
|
||||
|
||||
Phase 182 (task 05) counts the VERBATIM BLOCKS into the same sum: a
|
||||
block's ``text`` carries the phase-83 answer-text cap (600 000) so an
|
||||
operator raising ``BOR_SHOW_TEXT_MAX_CHARS`` never hits the wire limit,
|
||||
and the anonymous write surface (``POST/PUT /api/chats`` is public —
|
||||
phase 55 A1) must not gain a second, UNBOUNDED byte channel next to
|
||||
the bounded one (the SEC-05 invariant: the total is what keeps the
|
||||
audited worst case at the audited order). A legitimate turn carries at
|
||||
most ``BOR_SHOW_TEXT_MAX_BLOCKS`` × ``BOR_SHOW_TEXT_MAX_CHARS`` bytes
|
||||
(5 × 12 000 = 60 KB by default), orders of magnitude below the budget,
|
||||
so nothing the server produces is affected — the bound only rejects a
|
||||
client stuffing blocks.
|
||||
|
||||
Raises :class:`ValueError` naming the budget, which Pydantic turns
|
||||
into the ordinary 422 validation response (``loc`` = the model
|
||||
itself, ``msg`` carrying the budget wording) — the same boundary-
|
||||
only style as every other phase-83/164 cap."""
|
||||
total = sum(len(m.text) + len(m.thinking or "") for m in messages)
|
||||
total = sum(
|
||||
len(m.text)
|
||||
+ len(m.thinking or "")
|
||||
+ sum(len(b.text) for b in m.answer_text or [])
|
||||
for m in messages
|
||||
)
|
||||
if total > SAVED_CHAT_TOTAL_CHARS_BUDGET:
|
||||
raise ValueError(
|
||||
f"saved chat exceeds the {SAVED_CHAT_TOTAL_CHARS_BUDGET:,}-character budget"
|
||||
|
||||
Reference in New Issue
Block a user