phase: 192_verbatim_inline_midturn
Build and Push Containers / build-and-push-app (push) Successful in 1m56s
Build and Push Containers / build-and-push-db (push) Successful in 14s

Phase 192 — final verification pass complete.

- **Verified (all 4 tasks already complete):** additive `verbatim_block` SSE frame (`VerbatimBlockPiece` → `ChatVerbatimBlockEvent` → chat dispatch, after the round's `tool`/`tool_result`, before the next round); `hydrateVerbatimNow` matched-only mid-turn pass + app.js branch; `dropPlacedTwin` per-locator done/restore dedupe; the dedicated E2E suite.
- **Root cause of the rejected attempt:** my own concurrent background `pytest` run deadlocked the harness validation run on the shared Postgres (`psycopg.errors.DeadlockDetected`). No code defect — re-ran every gate **exclusively**.
- `uv run pytest -q --cov=app --cov-report=term-missing` → **exit 0**, `TOTAL 7181 16 99%` (>90%)
- `uv run pytest tests/e2e/test_verbatim_inline_midturn.py --no-cov` → **6 passed** (isolated)
- `uv run pytest tests/e2e/test_verbatim_show_text.py tests/e2e/test_chat_history.py --no-cov` → **18 passed** (`test_chat_history.py` byte-unedited; the phase-182 suite's "no eighth type" pin was tightened, not loosened, to seven + `verbatim_block` with frame-order asserts)
- `uv run ruff check .` → All checks passed (exit 0) · `uv run pyright` → **0 errors**
- **Completion criteria:** single frame per success + zero per refusal ✅ · `BOR_ANSWER_TEXT=0` wire byte-identical (integration spelled baseline + E2E App B) ✅ · fence hydrates in place / unfenced lands at the call moment, before `done` ✅ · append+fence → 1 panel, two ranges → 2 panels ✅ · restore 1 panel/locator, `done.answer_text` + record key + `verbatim_blocks=N` unchanged ✅ · suite green, coverage 99%, lint/types clean ✅ · no-show_text turn byte-identical ✅ · commit → left to the harness (no git index/history commands run).
- **Docs:** README reviewed line-by-line (2399 lines) + AGENTS.md in full; mechanically verified all 86 `BOR_*` settings, every `.env.example` var, table defaults, internal anchors (0 broken), ports 8123/8901, tree depth 1–8, pagination 25/1–100, 17 palette colors, `0029`/`0030` migrations, and every cited test file. Fixed 1 inconsistency: the verbatim "Parity" bullet claimed the live turn/reload/shared page share "the same three globals" — now names the settle-path trio and states that `hydrateVerbatimNow` is the ONE chat-only addition (a shared page has no stream).
- **Notable:** `run_agent`'s four repeated piece unions were collapsed into the ONE exported `AgentPiece` alias, so a future piece type is spelled in one place; `test_frontend_edit_message.py`'s `runTurn` byte-pin was regenerated, not weakened.
- **Next pending phase:** `193_sync_remaining_count`.
This commit is contained in:
2026-10-19 12:19:51 -04:00
parent 4f20c256d7
commit fb17d15fbe
49 changed files with 17313 additions and 102 deletions
+252 -11
View File
@@ -51,11 +51,19 @@ retries (a dead-then-recovered round restarts before its first piece
with a ``RetryPiece``; a mid-stream drop stays terminal — locked A2;
the forced final no-tools call retries too; ``llm_retries=0`` is one
plain attempt; retries are invisible to the round cap; consumer abandon
mid-retry-sleep leaks nothing).
mid-retry-sleep leaks nothing), and the phase-192 (task 01) mid-turn
``VerbatimBlockPiece``: exactly ONE piece per SUCCESSFUL ``show_text`` —
after the round's ``tool`` / ``tool_result`` pieces and before the next
model round, the piece's ``block`` being the holder's own
``shown_texts`` entry — and ZERO pieces on every refusal path (missing
``path``, the ``answer_text=False`` posture, no document, an ``is_image``
row, the spent ``show_text_max_blocks`` budget, the already-shown dedupe)
or on the ``BOR_ANSWER_TEXT=0`` turn where the tool is not even offered.
"""
from __future__ import annotations
import asyncio
import dataclasses
import json
import logging
import uuid
@@ -82,7 +90,9 @@ from app.rag.agent import (
SHOWN_TEXT_TRUNCATION_NOTICE,
UNKNOWN_TOOL,
AgentHolder,
AgentPiece,
MalformedReplyError,
VerbatimBlockPiece,
agent_tools,
run_agent,
)
@@ -96,6 +106,7 @@ from app.rag.llm import (
)
from app.rag.prompts import TOOLS_SECTION, _base, build_deflect_prompt, build_high_prompt
from app.rag.retriever import TRUNCATION_MARKER
from app.rag.verbatim import VerbatimBlock
if TYPE_CHECKING:
from app.rag.scaffolding import ScaffoldingFilter
@@ -201,12 +212,14 @@ async def _run(
seed_docs: list[Document] | None = None,
history: Sequence[dict[str, Any]] = (),
db_factory: Callable[[], Session] | None = None,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one ``run_agent`` turn; *history* (phase 74) is the
client's prior turns spliced between system and user (default
``()`` — the pre-phase-74 two-message request). Phase 95: the loop
may also yield a ``ToolResultPiece`` (a truncated ``read``)."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
may also yield a ``ToolResultPiece`` (a truncated ``read``), and
phase 192 a ``VerbatimBlockPiece`` (a successful ``show_text``) —
both members of the one :data:`app.rag.agent.AgentPiece` union."""
out: list[AgentPiece] = []
factory = db_factory or (lambda: _mock_session())
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5143,8 +5156,8 @@ def test_round_failure_after_first_piece_is_terminal(monkeypatch: pytest.MonkeyP
)
sleeps = _record_sleeps(monkeypatch)
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async def drain() -> list[AgentPiece]:
out: list[AgentPiece] = []
with pytest.raises(LLMError, match="mid-stream drop"):
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5205,8 +5218,8 @@ def test_zero_retries_is_one_plain_attempt(monkeypatch: pytest.MonkeyPatch) -> N
llm = FailingLLM([([], LLMError("connection refused"))])
sleeps = _record_sleeps(monkeypatch)
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async def drain() -> list[AgentPiece]:
out: list[AgentPiece] = []
with pytest.raises(LLMError, match="connection refused"):
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5255,9 +5268,7 @@ def test_abandon_mid_retry_sleep_leaks_nothing(monkeypatch: pytest.MonkeyPatch)
holder=holder,
)
async def consumer() -> list[
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
]:
async def consumer() -> list[AgentPiece]:
return [p async for p in gen]
task = asyncio.ensure_future(consumer())
@@ -6149,3 +6160,233 @@ def test_show_text_scripted_turn_returns_the_slice_to_the_model(
assert len(holder.shown_texts) == 1
assert holder.shown_texts[0].locator == f"{_SHOW_IDENTITY}#3-5"
assert holder.tool_calls == 1
# ---------------------------------------------------------------------------
# Phase 192 (task 01, A1) — the mid-turn ``VerbatimBlockPiece``
#
# The block a successful ``show_text`` appended to ``holder.shown_texts``
# used to reach the client only on the ``done`` frame (phase 182 task 05),
# so every verbatim panel popped in AFTER the model finished talking about
# the bytes. The loop now also yields ONE ``VerbatimBlockPiece`` the moment
# the block exists — after the round's ``tool`` / ``tool_result`` pieces and
# before the next model round — and the API layer turns it into the additive
# ``verbatim_block`` SSE frame (pinned in ``tests/integration/test_chat_api.py``
# and on the client in task 02). Success-only: every refusal path returns
# before the ``shown_texts.append``, so it yields NOTHING.
# ---------------------------------------------------------------------------
def _patch_two_docs(
monkeypatch: pytest.MonkeyPatch, *docs: Document
) -> None:
"""Resolve several fixture rows by ``(source, path)`` (the multi-document
variant of :func:`_patch_show_doc` — a truncated ``read`` doc AND a
showable doc in one turn)."""
def _find(_db: Session, source: str, path: str) -> Document | None:
for doc in docs:
if (source, path) == (doc.source, doc.path):
return doc
return None
monkeypatch.setattr(agent, "find_document", _find)
_DEFAULT_SHOW_DOC = object() # "the resolvable fixture row" sentinel
def _show_turn(
monkeypatch: pytest.MonkeyPatch,
arguments: dict[str, Any],
settings: Settings | None = None,
doc: Document | None | object = _DEFAULT_SHOW_DOC,
) -> tuple[AgentHolder, ScriptedLLM, list[AgentPiece]]:
"""One scripted turn whose model calls ``show_text`` with *arguments*
(the default resolves the fixture row; pass ``None`` for the
no-document refusal, or an ``is_image`` row for the A6 refusal)."""
_patch_show_doc(
monkeypatch, _show_doc() if doc is _DEFAULT_SHOW_DOC else cast("Document | None", doc)
)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="show_text", arguments=arguments)],
[StreamPiece("content", "ans")],
)
return holder, llm, asyncio.run(_run(llm, holder, settings or _settings()))
def test_verbatim_block_piece_is_one_frozen_block() -> None:
"""The piece's shape (the :class:`app.rag.llm.ToolResultPiece` pattern):
a FROZEN dataclass with exactly one field, ``block``, holding the
server's :class:`app.rag.verbatim.VerbatimBlock` — an immutable wire
value, never a re-derivation."""
block = VerbatimBlock(
source=_SHOW_SRC,
path=_SHOW_PATH,
title="Gufo quadlet",
start_line=2,
end_line=3,
total_lines=5,
text="L2-two\nL3-three",
)
piece = VerbatimBlockPiece(block=block)
assert [f.name for f in dataclasses.fields(piece)] == ["block"]
assert piece.block is block
with pytest.raises(dataclasses.FrozenInstanceError):
piece.block = None # pyright: ignore[reportAttributeAccessIssue]
def test_show_text_success_yields_one_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The success pin: a scripted ``show_text`` yields EXACTLY ONE
``VerbatimBlockPiece``, AFTER the round's ``ToolCallPiece`` (the ``tool``
frame) and BEFORE the next model round's first piece (the answer's
``StreamPiece``). Its ``block`` IS the ``holder.shown_texts`` entry — the
same object, so the mid-turn frame and the done frame can never
disagree."""
_patch_show_doc(monkeypatch, _show_doc())
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2}
)
],
[StreamPiece("content", "here"), StreamPiece("content", " it is")],
)
out = asyncio.run(_run(llm, holder, _settings()))
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
assert len(pieces) == 1
piece = pieces[0]
# Order: the tool call, the block, THEN the next round's answer pieces.
assert isinstance(out[0], ToolCallPiece) and out[0].name == "show_text"
assert out[1] is piece
assert all(isinstance(p, StreamPiece) for p in out[2:])
# The block is the holder's own entry (identity, not a copy).
assert piece.block is holder.shown_texts[0]
assert piece.block.locator == f"{_SHOW_IDENTITY}#2-5"
assert piece.block.text == "L2-two\nL3-three\nL4-four\nL5-five"
# The done-frame bookkeeping is untouched by the piece (task 05's path).
assert holder.tool_calls == 1
assert len(holder.shown_texts) == 1
assert len(llm.requests) == 2 # the tool round + the answer round
def test_verbatim_block_piece_lands_after_a_truncation_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The emission-order contract with BOTH derived pieces in one turn: a
truncated ``read`` (round 1 → ``ToolResultPiece``) then a
``show_text`` (round 2 → ``VerbatimBlockPiece``) stream
``tool, tool_result, tool, verbatim_block`` and only THEN the answer —
every derived piece stays inside its own round."""
big = _doc("S", "big.md", "Big", "\n".join(f"line {i}" for i in range(1, 30)))
_patch_two_docs(monkeypatch, big, _show_doc())
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/big.md"})],
[ToolCallPiece(id="c2", name="show_text", arguments={"path": _SHOW_IDENTITY})],
[StreamPiece("content", "ans")],
)
out = asyncio.run(_run(llm, holder, _settings(read_max_chars=20)))
kinds = [
(
"tool:" + p.name
if isinstance(p, ToolCallPiece)
else "trunc"
if isinstance(p, ToolResultPiece)
else "block"
if isinstance(p, VerbatimBlockPiece)
else "retry"
if isinstance(p, RetryPiece)
else p.kind
)
for p in out
]
assert kinds == [
"tool:read",
"trunc",
"tool:show_text",
"block",
"content",
]
assert len([p for p in out if isinstance(p, VerbatimBlockPiece)]) == 1
@pytest.mark.parametrize(
"case",
["missing_args", "flag_off", "no_document", "image_doc", "block_cap"],
)
def test_show_text_refusals_yield_no_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch, case: str
) -> None:
"""Every refusal path yields ZERO pieces — the single success gate is
the ``shown_texts.append`` (the mid-turn frame is success-only by
construction, and the refusal itself is byte-identical to phase 182):
the missing-``path`` refusal, the ``answer_text=False`` UNKNOWN_TOOL
posture, the no-document teaching, the ``is_image`` refusal (LOCKED A6)
and the spent ``show_text_max_blocks`` budget."""
if case == "missing_args":
arguments: dict[str, Any] = {}
settings, doc = None, _DEFAULT_SHOW_DOC
elif case == "flag_off":
arguments = {"path": _SHOW_IDENTITY}
settings, doc = _settings(answer_text=False), _DEFAULT_SHOW_DOC
elif case == "no_document":
arguments = {"path": "nosuch/file.md"}
settings, doc = None, None
elif case == "image_doc":
arguments = {"path": _SHOW_IDENTITY}
settings, doc = None, _show_doc(is_image=True)
else:
arguments = {"path": _SHOW_IDENTITY}
settings, doc = _settings(show_text_max_blocks=0), _DEFAULT_SHOW_DOC
holder, _llm, out = _show_turn(monkeypatch, arguments, settings, doc)
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
_assert_refusal(holder) # counts in nothing, displays nothing
def test_show_text_already_shown_dedupe_yields_no_second_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The canonical-locator dedupe is a REFUSAL (D7), so the second,
identical show adds no holder entry and yields no second piece —
exactly ONE ``VerbatimBlockPiece`` across both rounds (the client can
therefore never be told to paint the same block twice)."""
_patch_show_doc(monkeypatch, _show_doc())
holder = AgentHolder()
call = ToolCallPiece(
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2, "limit": 2}
)
llm = ScriptedLLM(
[call],
[ToolCallPiece(id="c2", name="show_text", arguments=call.arguments)],
[StreamPiece("content", "ans")],
)
out = asyncio.run(_run(llm, holder, _settings()))
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
assert len(pieces) == 1
assert pieces[0].block.locator == f"{_SHOW_IDENTITY}#2-3"
assert len(holder.shown_texts) == 1 # the dedupe refused the second show
def test_kill_switch_turn_yields_no_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""``BOR_ANSWER_TEXT=0`` (the phase-182 D8 posture): the tool is NOT
offered — so no call can succeed — and a scripted call anyway yields
zero pieces. The turn's piece sequence is byte-identical to pre-phase
(the wire pin lives in ``tests/integration/test_chat_api.py``)."""
holder, llm, out = _show_turn(
monkeypatch, {"path": _SHOW_IDENTITY}, _settings(answer_text=False)
)
offered = [t["function"]["name"] for t in (llm.requests[0][1] or [])]
assert "show_text" not in offered
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
_assert_refusal(holder)
+4 -10
View File
@@ -19,14 +19,8 @@ from sqlalchemy.orm import Session
from app.config import Settings
from app.models import Document
from app.rag import agent
from app.rag.agent import AgentHolder, run_agent
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
)
from app.rag.agent import AgentHolder, AgentPiece, run_agent
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
#: Fixture document creation date (phase 106, D5).
_FIXTURE_CREATED_AT = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
@@ -131,9 +125,9 @@ async def _consume(
holder: AgentHolder,
settings: Settings,
seed_docs: list[Document] | None = None,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one ``run_agent`` turn."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
out: list[AgentPiece] = []
async for piece in run_agent(
cast("LLMClient", llm),
db_factory,
+34 -2
View File
@@ -1471,6 +1471,30 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
}
}
}
} else if (ev.type === "verbatim_block") {
// Phase 192 (task 02, A2): the show_text panel lands at the moment
// the call succeeds — the fence it names hydrates IN PLACE; an
// unfenced block lands at the answer position now, not at done.
// The done pass re-runs the same functions and can never double this
// panel: hydrateVerbatim is idempotent (only .verbatim-pending is
// touched), it DROPS this appended twin when the answer fences the
// same locator later (dropPlacedTwin, phase 192 A3), and
// appendUnlocatedVerbatim skips an already-placed locator. NO
// client-side answer_text gate: the server's gate is the only one
// (a BOR_ANSWER_TEXT=0 turn sends no frame, so this branch never
// fires — a second, client-side gate would only drift).
// `wrap` always exists here (the round's `tool` frame created the
// live bubble) but the optional chain keeps a frame that arrives
// without one a silent no-op, like the `tool_result` branch.
const n = hydrateVerbatimNow(
wrap?.querySelector(".bubble"),
ev,
VERBATIM_LINK_OPTS
);
// The append is the ZERO-match path only: a fence already on screen
// keeps its panel IN PLACE (the model's order), and only a block no
// placeholder names lands at the answer position.
if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);
} else if (ev.type === "error") {
// Phase 182 (task 07): an `error` frame is terminal — the `done`
// hydrate will never run, so a placeholder that streamed in stays at
@@ -1701,14 +1725,22 @@ def test_run_turn_is_pinned_at_phase_159_state() -> None:
Copy buttons BEFORE ``appendSources``; the ``answer_text`` meta key rides
``rememberBrainTurn`` with ``|| undefined``; and the terminal paths — the
``error`` frame, the 300 s guard, its own abort surface, and the Stop
branch — settle any leftover placeholder to the unresolved state):
branch — settle any leftover placeholder to the unresolved state)
and phase 192 (task 02)'s sanctioned addition (the ONE new SSE branch:
``ev.type === "verbatim_block"`` settles the pending fence the shown
block names IN PLACE through ``hydrateVerbatimNow`` and appends the block
through ``appendUnlocatedVerbatim`` ONLY when that pass matched nothing —
the mid-turn landing, so a panel appears the moment the ``show_text``
call succeeds instead of at ``done``; no client-side gate, the server's
``BOR_ANSWER_TEXT`` gate is the only one):
the full source must match the pin — the ``reask`` path, the turn-local
resets, the
SSE contract, the save points, and the sanctioned additions (the
task-01 ``lastUserWrap`` assignment + the idle-settle mark
pass, the task-03 aborted-settle mark pair, the phase-160
done-settle pair, the phase-161 live-container additions, the
D5 refinement, and the phase-182 verbatim settle/hydrate pair) included."""
D5 refinement, the phase-182 verbatim settle/hydrate pair and the
phase-192 mid-turn branch) included."""
assert _fn_source(_js(), "runTurn") == PINNED_RUN_TURN
+846
View File
@@ -27,6 +27,17 @@ Why these particular pins (the phase's drift risks, in order):
call the SAME globals, in the SAME order (hydrate → append the unlocated →
settle the code blocks, all inside the ``answer_image`` band and BEFORE the
source clusters), so one answer cannot render three different ways.
* **The mid-turn pass settles ONLY what it matches** (phase 192, task 02):
the additive ``verbatim_block`` frame carries ONE block, and
``hydrateVerbatimNow`` may replace only the pending placeholders that block
matches — every other placeholder STAYS ``.verbatim-pending`` for the
``done`` / failure path, because burning a fence for a block that has not
arrived yet is how a resolved panel turns into "I could not verify this".
* **One canonical locator, one panel** (phase 192, A3): a mid-turn block that
no fence named yet is appended at the answer position, and when the answer
fences it later the hydration drops that appended twin (``dropPlacedTwin``,
bounded by the ``.msg`` wrap) — the scan is PER LOCATOR, so two honest
ranges of one file stay two panels.
* **No panel is ever left lying.** Every terminal path — the ``error`` frame,
the 300 s guard, the Stop action, the failed-turn funnel — settles the
leftover placeholders; nothing reads "Retrieving the exact text…" after the
@@ -43,6 +54,7 @@ import re
import shutil
import subprocess
from pathlib import Path
from typing import Any
import pytest
@@ -80,6 +92,7 @@ COPY_ARIA = "Copy ${shown} lines from ${identity}"
PUBLIC_GLOBALS = (
"buildVerbatimPanel",
"hydrateVerbatim",
"hydrateVerbatimNow", # phase 192 (task 02): the MID-TURN pass
"appendUnlocatedVerbatim",
"settleVerbatimPlaceholders",
"enhanceAnswerCodeBlocks",
@@ -112,6 +125,28 @@ def _body(js: str, name: str) -> str:
return _fn(js, name)
def _doc(js: str, name: str) -> str:
"""The ``/* … */`` doc comment that sits directly above
``function <name>`` — the house docstring-pinning convention for the
clients: the prose IS the contract record, so a refactor that "simplifies"
the rule away has to delete the sentence in the same change."""
start = js.find(f"\nfunction {name}(")
assert start != -1, f"function {name}() must exist"
open_ = js.rfind("/*", 0, start)
close = js.find("*/", open_)
assert open_ != -1 and close != -1 and close < start, (
f"{name}() must carry the doc comment above it"
)
return js[open_ : close]
def _flow(text: str) -> str:
"""Comment prose as one line — the pins quote the SENTENCE, not its
wrapping (the `` * `` line leaders come out with the wrap)."""
lines = [re.sub(r"^\s*\*+\s?", " ", line) for line in text.splitlines()]
return " ".join(" ".join(lines).split())
# ---------------------------------------------------------------------------
# the client ships, and ships in the right shape
# ---------------------------------------------------------------------------
@@ -346,6 +381,134 @@ def test_hydration_is_idempotent_and_never_throws_on_garbage() -> None:
assert "Array.isArray(blocks)" in guard, "a missing frame key is not an error"
def test_midturn_hydration_settles_only_the_placeholders_one_block_matches() -> None:
"""Phase 192 (task 02, A2) — the heart of the mid-turn pass. The additive
``verbatim_block`` frame carries ONE block, and ``hydrateVerbatimNow``:
* walks the SAME pending set as the done pass
(``.verbatim-pending[data-bor-locator]``, the attribute ``markdown.js``
normalized — no second grammar);
* matches through the ONE rule (``matchVerbatimBlock`` with a one-block
list) so the mid-turn pass and the ``done`` pass can never disagree
about which fence this block owns;
* replaces ONLY the matched nodes (``buildVerbatimPanel``) — the unmatched
stay ``.verbatim-pending``, NEVER the honest unresolved state, which is
the done/failure path's job (``settleVerbatimPlaceholders``);
* returns the number settled (``0`` is the caller's signal to append);
* is total without a ``try`` — a missing root, a root that is not an
element, or a frame that is not a block all return ``0``."""
js = _text(VERBATIM_JS)
fn = _fn(js, "hydrateVerbatimNow")
assert '.verbatim-pending[data-bor-locator]"' in fn, (
"the mid-turn walk is the done path's pending set"
)
assert 'node.getAttribute("data-bor-locator")' in fn, (
"the key is the attribute markdown.js wrote — never re-derived"
)
assert "matchVerbatimBlock(" in fn, "the ONE match rule, not a mid-turn variant"
assert "const list = [block]" in fn, "the one-block frame rides the done list shape"
assert "replaceWith(buildVerbatimPanel(" in fn, "only a MATCH builds a panel"
assert "buildVerbatimUnresolved" not in fn, (
"an unmatched fence stays pending mid-turn — settling is the terminal path's job"
)
assert "if (!matched) return" in fn, "the unmatched node is left in place"
assert "settled += 1" in fn and "return settled" in fn, (
"the count is the caller's append/no-append signal"
)
assert 'typeof root.querySelectorAll !== "function"' in fn, "a missing root is 0"
assert "isVerbatimBlock(block)" in fn, "a malformed frame settles NOTHING"
assert "try {" not in fn, "the guards make it total — no exception path"
assert "innerHTML" not in fn
def test_hydration_drops_the_midturn_twin_and_never_an_unresolved_one() -> None:
"""Phase 192 (A3): a block that arrived mid-turn before its fence was
appended at the answer position; when its fence hydrates, that twin is
removed so ONE locator keeps ONE panel and the panel sits where the model
fenced it. The shape of the scan is the contract:
* bounded by the message wrap (the appended panel lives in ``.msg-body``,
beside the bubble — a bubble-local walk would miss it);
* ``.verbatim-pending`` and ``.verbatim-unresolved`` are NOT placements
(the ``appendUnlocatedVerbatim`` exclusion, shared);
* the key is the ONE ``verbatimPlacedKey`` derivation (the same one the
append-side scan reads — two derivations is how the dedupe drifts);
* it compares the canonical LOCATOR (never an identity — two ranges of one
file are two panels) and never removes the placeholder being hydrated."""
js = _text(VERBATIM_JS)
twin = _fn(js, "dropPlacedTwin")
assert 'host.querySelectorAll(".verbatim[data-bor-locator]")' in twin
assert 'el.classList.contains("verbatim-pending")' in twin
assert 'el.classList.contains("verbatim-unresolved")' in twin
assert 'el.getAttribute("data-bor-locator") !== placedKey' in twin, (
"PER LOCATOR — an identity scan would collapse two honest ranges"
)
assert "verbatimSplitLocator" not in twin and "verbatimIdentity(" not in twin, (
"no identity fallback in the twin scan"
)
assert "if (el === keep) return" in twin, "the hydrating placeholder is never the twin"
assert "el.remove()" in twin
hydrate = _fn(js, "hydrateVerbatim")
assert 'root.closest(".msg")' in hydrate, "the scan is bounded by the message wrap"
assert "if (block) dropPlacedTwin(host, block, node);" in hydrate, (
"only a MATCHED placeholder drops a twin"
)
# The ONE key derivation, read by both dedupe sides.
assert js.count("function verbatimPlacedKey(") == 1
assert "verbatimPlacedKey(block)" in _fn(js, "appendUnlocatedVerbatim")
#: The sentences the phase-192 (A3) prose must keep carrying. They are the
#: only place a reader of ``verbatim.js`` learns the dedupe's two rules, and
#: the fastest way to lose a rule is a refactor that drops its documentation
#: first and its code second.
TWIN_DOC_SENTENCES = (
# PER LOCATOR, never per identity — the two-ranges-stay clause.
"PER LOCATOR, never per identity: two different ranges of one file are"
" two honest panels",
# The in-text panel wins over the mid-turn appended twin.
"The IN-TEXT panel wins",
)
def test_the_twin_dedupe_docstring_keeps_the_per_locator_clause() -> None:
"""Phase 192 (task 03, A3): the prose is part of the contract. The
``dropPlacedTwin`` docstring names the per-LOCATOR rule (and the
two-honest-ranges clause that follows from it) and why the in-text panel
wins; ``hydrateVerbatim``'s docstring says the matched placeholder drops
that twin ("one locator, one panel, and it sits where the model fenced
it"); and the module header states the same rule for whoever reads the
file top-down."""
js = _text(VERBATIM_JS)
twin = _flow(_doc(js, "dropPlacedTwin"))
for sentence in TWIN_DOC_SENTENCES:
assert sentence in twin, f"the docstring lost: {sentence}"
assert "appendUnlocatedVerbatim" in twin, (
"the placement exclusion is documented as SHARED with the append side"
)
hydrate = _flow(_doc(js, "hydrateVerbatim"))
assert "dropPlacedTwin" in hydrate
assert "one locator, one panel, and it sits where the model fenced it" in hydrate
header = _flow(js[: js.find("*/")])
assert "ONE CANONICAL LOCATOR, ONE PANEL (phase 192)" in header
assert "two honest ranges of one file stay two panels" in header
def test_the_twin_scan_bounds_to_the_msg_wrap_or_the_root() -> None:
"""The scan's host is the message wrap (the appended twin lives in
``.msg-body``, beside the bubble), and when a root has no ``.msg``
ancestor the ROOT bounds it — a missing wrap narrows the scan, it never
switches the dedupe off."""
hydrate = _fn(_text(VERBATIM_JS), "hydrateVerbatim")
assert 'root.closest(".msg")' in hydrate
assert 'root.closest(".msg") : null) || root' in hydrate, (
"no .msg ancestor → the root bounds the scan"
)
assert "document.querySelectorAll" not in hydrate, (
"never a page-wide scan — one message's twin, one message's panels"
)
def test_unlocated_blocks_land_in_the_answer_image_band() -> None:
"""The blocks no placeholder matched (a ``show_text`` call the prose never
named) append to ``.msg-body`` — the same band as the answer image, below
@@ -458,6 +621,50 @@ def test_the_done_handler_hydrates_before_the_source_clusters() -> None:
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatim(") :][:200]
def _midturn_branch(js: str) -> str:
start = js.index('ev.type === "verbatim_block"')
end = js.index('ev.type === "error"', start)
return js[start:end]
def test_the_midturn_frame_handler_hydrates_in_place_and_appends_only_on_zero() -> None:
"""Phase 192 (task 02, A2): the live turn's ``verbatim_block`` branch.
The shape IS the behaviour: hydrate the pending fence FIRST, and reach
for ``appendUnlocatedVerbatim`` ONLY on the zero-match path — a fence that
is already on screen keeps its panel where the model put it, and only a
block no placeholder names lands at the answer position (the
``answer_image`` band). The branch sits in the same ``readSSE`` chain, after
``tool_result`` (the server's emission order) and before ``error``.
And there is NO client-side kill switch: the server's
``BOR_ANSWER_TEXT`` gate is the only gate, so a disabled turn simply sends
no frame (a second, client-side settings check is a drift risk, exactly
the phase's reasoning for keeping the gate server-side)."""
js = _text(APP_JS)
branch = _midturn_branch(js)
assert "hydrateVerbatimNow(" in branch, "the mid-turn pass settles the fence"
assert "appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS)" in branch, (
"the unmatched block lands at the answer position, ONE opts object"
)
assert re.search(r"if \(\w+ === 0\)\s*appendUnlocatedVerbatim\(", branch), (
"the append is the ZERO-match path only — never alongside an in-place panel"
)
order = [branch.index("hydrateVerbatimNow("), branch.index("appendUnlocatedVerbatim(")]
assert order == sorted(order), "hydrate in place FIRST, append only if nothing matched"
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatimNow(") :][:200], (
"the mid-turn panel navigates exactly like the done panel (chip parity)"
)
# No second gate: nothing in the branch reads settings / a flag / the
# done frame's key — the arrival of the frame is the whole condition.
code = _strip_comments(branch)
for gate in ("answer_text", "answerText", "settings", "await ", "fetch("):
assert gate not in code, f"{gate} looks like a client-side gate (server-only)"
# The frame type is the ONE new thing the live turn knows (the server
# sends it between `tool_result` and the next round).
assert js.index('ev.type === "tool_result"') < js.index('ev.type === "verbatim_block"')
def test_the_answer_renderers_opt_in_and_nothing_else_does() -> None:
"""The per-delta write and the ``done``-time re-render both pass
``{ verbatim: true }`` (the third site is the restore renderer, pinned in
@@ -865,3 +1072,642 @@ def test_malformed_blocks_are_dropped_and_never_throw(tmp_dir: Path) -> None:
assert out["identity"] == {"ok": True, "value": "homelab/runbooks/a.md"}
assert out["unresolved_copy"] == {"ok": True, "value": UNRESOLVED_COPY}
assert out["badge_title"] == {"ok": True, "value": BADGE_TITLE}
# ---------------------------------------------------------------------------
# node: the MID-TURN pass (phase 192, task 02), run against the real file on
# a minimal DOM — the marker pins above prove the SHAPE, these prove the
# behaviour the E2E (task 04) will show in a browser: matched fences settle in
# place, everything else stays pending, a bad frame settles nothing.
# ---------------------------------------------------------------------------
#: The minimal DOM + the fixtures (``pendingBubble`` / ``shownBlock`` /
#: ``snapshot``). ``querySelectorAll`` understands the ONE compound selector
#: the client walks (``.a.b[attr]``) and throws on anything else, so a
#: selector drift in the client fails loudly instead of matching nothing.
_DOM_LIB = """
class El {
constructor(tag) {
this.tag = tag;
this.children = [];
this.parent = null;
this.attrs = {};
this.dataset = {};
this.classes = new Set();
this._own = "";
}
get className() { return [...this.classes].join(" "); }
set className(v) { this.classes = new Set(String(v).split(/\\s+/).filter(Boolean)); }
get classList() {
return { contains: (c) => this.classes.has(c), add: (c) => this.classes.add(c) };
}
setAttribute(k, v) { this.attrs[k] = String(v); }
addEventListener(ev, fn) { (this.listeners = this.listeners || {})[ev] = fn; }
getAttribute(k) {
if (Object.prototype.hasOwnProperty.call(this.attrs, k)) return this.attrs[k];
if (k.startsWith("data-")) {
const camel = k.slice(5).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
if (Object.prototype.hasOwnProperty.call(this.dataset, camel)) {
return String(this.dataset[camel]);
}
}
return null;
}
appendChild(c) { c.parent = this; this.children.push(c); return c; }
append(...cs) { for (const c of cs) this.appendChild(c); }
insertBefore(box, ref) {
const i = this.children.indexOf(ref);
this.children.splice(i < 0 ? this.children.length : i, 0, box);
box.parent = this;
return box;
}
replaceWith(node) {
if (!this.parent) throw new Error("replaceWith on a detached node");
const i = this.parent.children.indexOf(this);
this.parent.children[i] = node;
node.parent = this.parent;
this.parent = null;
}
remove() {
if (!this.parent) return;
const i = this.parent.children.indexOf(this);
if (i >= 0) this.parent.children.splice(i, 1);
this.parent = null;
}
closest(sel) {
let cur = this;
while (cur) {
if (matches(cur, sel)) return cur;
cur = cur.parent;
}
return null;
}
querySelectorAll(sel) { return descendants(this).filter((el) => matches(el, sel)); }
querySelector(sel) { return descendants(this).find((el) => matches(el, sel)) || null; }
get textContent() {
if (!this.children.length) return this._own;
return this.children.map((c) => c.textContent).join("");
}
set textContent(v) { this.children = []; this._own = String(v); }
}
function descendants(el, out) {
const list = out || [];
for (const c of el.children) { list.push(c); descendants(c, list); }
return list;
}
function matches(el, sel) {
const parts = sel.match(/\\.[A-Za-z0-9_-]+|\\[[^[\\]]+\\]/g) || [];
if (!parts.length) throw new Error("unsupported selector: " + sel);
return parts.every((p) =>
p.startsWith(".")
? el.classes.has(p.slice(1))
: el.getAttribute(p.slice(1, -1)) !== null
);
}
globalThis.document = {
createElement: (t) => new El(t),
createTextNode: (t) => {
const node = new El("#text");
node._own = String(t);
return node;
},
querySelectorAll: () => [],
body: new El("body"),
};
function pendingBubble(locators) {
const bubble = new El("div");
bubble.classes.add("bubble");
for (const loc of locators) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = loc;
bubble.appendChild(node);
}
return bubble;
}
function shownBlock(over) {
return Object.assign({
source: "homelab",
path: "runbooks/a.md",
title: "Runbook A",
start_line: 12,
end_line: 40,
total_lines: 210,
text: "line twelve\\nline thirteen",
truncated: false,
locator: "homelab/runbooks/a.md#12-40",
}, over || {});
}
function pendingWrap(locators) {
const msg = new El("div");
msg.className = "msg brain";
const body = new El("div");
body.className = "msg-body";
const bubble = new El("div");
bubble.className = "bubble";
msg.appendChild(body);
body.appendChild(bubble);
for (const loc of locators) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = loc;
bubble.appendChild(node);
}
return msg;
}
function findClass(root, cls) {
return descendants(root).find((el) => el.classes.has(cls)) || null;
}
function inBubble(el) {
let cur = el.parent;
while (cur) {
if (cur.classes.has("bubble")) return true;
cur = cur.parent;
}
return false;
}
/* The RESOLVED panels in document order — a pending or unresolved node is
* NOT a placement (appendUnlocatedVerbatim's own convention), which is what
* makes this the "one locator, one panel" reading. */
function placed(msg) {
return descendants(msg)
.filter((el) => el.classes.has("verbatim"))
.filter((el) => !el.classes.has("verbatim-pending"))
.filter((el) => !el.classes.has("verbatim-unresolved"))
.map((el) => ({
locator: el.getAttribute("data-bor-locator"),
where: inBubble(el) ? "bubble" : "body",
}));
}
/* The fence the model writes in a LATER round (the mid-turn frame arrived
* before it): a fresh pending node inside the bubble. */
function addFence(msg, locator) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = locator;
msg.querySelector(".bubble").appendChild(node);
return node;
}
/* The same later-round fence inside a BARE bubble (no `.msg` wrap) — the arm
* that proves the ROOT bounds the twin scan when there is no message wrap. */
function bareFence(bubble, locator) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = locator;
bubble.appendChild(node);
return node;
}
function snapshot(root) {
return descendants(root)
.filter((el) => el.classes.has("verbatim"))
.map((el) => {
const caption = findClass(el, "verbatim-caption");
const pre = findClass(el, "verbatim-text");
const link = findClass(el, "verbatim-open");
return {
pending: el.classes.has("verbatim-pending"),
unresolved: el.classes.has("verbatim-unresolved"),
locator: el.getAttribute("data-bor-locator"),
caption: caption ? caption.textContent : null,
text: pre ? pre.textContent : null,
href: link ? link.href : null,
};
});
}
module.exports = {
El, pendingBubble, pendingWrap, shownBlock, snapshot, placed, addFence,
bareFence, findClass, descendants,
};
"""
_DOM_HARNESS = """
const fs = require("fs");
const src = fs.readFileSync(process.argv[2], "utf8");
const dom = require(process.argv[3]); // installs globalThis.document
const cases = JSON.parse(fs.readFileSync(process.argv[4], "utf8"));
const panel = new Function(
src +
"\\nreturn { hydrateVerbatimNow, hydrateVerbatim, appendUnlocatedVerbatim," +
" buildVerbatimPanel, isVerbatimBlock };"
)();
const scope = Object.assign({}, panel, dom);
const out = {};
for (const [name, body] of Object.entries(cases)) {
try {
const fn = new Function("api", "with (api) { return (" + body + "); }");
out[name] = { ok: true, value: fn(scope) };
} catch (err) {
out[name] = { ok: false, error: String(err && err.message ? err.message : err) };
}
}
process.stdout.write(JSON.stringify(out));
"""
#: The block a mid-turn ``verbatim_block`` frame carries in these cases.
SHOWN_BLOCK_LOCATOR = "homelab/runbooks/a.md#12-40"
#: The OTHER fence on screen mid-answer — the one this frame must NOT touch.
OTHER_PENDING_LOCATOR = "homelab/runbooks/b.md#1-5"
#: The caption the panel must carry in every arm: the SERVER's extent (the
#: phase-182 contract), never the range the model typed into its fence.
CAPTION_SERVER_EXTENT = "homelab/runbooks/a.md \u00b7 lines 12\u201340 of 210"
def _node_dom(cases: dict[str, str], tmp_dir: Path) -> dict[str, Any]:
"""Run *cases* (JS expressions with the panel API + the DOM fixtures in
scope) under node against the real ``verbatim.js`` on the minimal DOM.
``Any`` on purpose: the DOM cases return nested objects/arrays, and the
assertions below walk them (``value["nodes"][0]["caption"]``)."""
lib = tmp_dir / "bor_dom.cjs"
lib.write_text(_DOM_LIB, encoding="utf-8")
script = tmp_dir / "bor_dom_harness.cjs"
script.write_text(_DOM_HARNESS, encoding="utf-8")
case_file = tmp_dir / "dom_cases.json"
case_file.write_text(json.dumps(dict(cases)), encoding="utf-8")
proc = subprocess.run(
["node", str(script), str(VERBATIM_JS), str(lib), str(case_file)],
capture_output=True,
text=True,
timeout=60,
check=False,
)
assert proc.returncode == 0, proc.stderr
return json.loads(proc.stdout)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_pass_settles_the_matched_fence_and_leaves_the_rest(tmp_dir: Path) -> None:
"""The behaviour of ``hydrateVerbatimNow`` against the real file: the fence
whose canonical locator the frame carries settles IN PLACE with the
server's caption and bytes; the next fence in the same answer is NOT
touched (no panel, no unresolved sentence — it stays a promise the ``done``
frame still has to keep)."""
out = _node_dom(
{
"exact": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"wrong_range_uses_the_servers_extent": """(() => {
const bubble = pendingBubble(["homelab/runbooks/a.md#99-120"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {});
return { n, nodes: snapshot(bubble) };
})()""",
"two_fences_one_locator": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, pending: snapshot(bubble).filter((x) => x.pending).length }};
}})()""",
"opts_thread_through": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{
documentHref: (s, p) => "HOOK/" + s + "/" + p,
}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
},
tmp_dir,
)
exact = out["exact"]
assert exact["ok"] is True, exact
assert exact["value"]["n"] == 1, "exactly the matching placeholder settles"
settled, untouched = exact["value"]["nodes"]
assert settled == {
"pending": False,
"unresolved": False,
"locator": SHOWN_BLOCK_LOCATOR,
"caption": CAPTION_SERVER_EXTENT,
"text": "line twelve\nline thirteen",
"href": "/document.html?source=homelab&path=runbooks%2Fa.md",
}, "the panel is the done path's panel, built from the SERVER's bytes"
assert untouched == {
"pending": True,
"unresolved": False,
"locator": OTHER_PENDING_LOCATOR,
"caption": None,
"text": None,
"href": None,
}, "a fence this block does not name STAYS pending"
# The model's range was wrong: the identity fallback still applies, and
# the caption / data attribute carry the SERVER's extent (mid-turn and
# done can never disagree — the phase-182 contract).
fallback = out["wrong_range_uses_the_servers_extent"]
assert fallback["ok"] is True, fallback
assert fallback["value"]["n"] == 1
assert fallback["value"]["nodes"][0]["locator"] == SHOWN_BLOCK_LOCATOR
assert fallback["value"]["nodes"][0]["caption"] == CAPTION_SERVER_EXTENT
dupes = out["two_fences_one_locator"]
assert dupes["ok"] is True, dupes
assert dupes["value"]["n"] == 2 and dupes["value"]["pending"] == 0
hooked = out["opts_thread_through"]
assert hooked["ok"] is True, hooked
assert hooked["value"]["n"] == 1
assert hooked["value"]["nodes"][0]["href"] == "HOOK/homelab/runbooks/a.md", (
"opts (the chat's modal hook) reach the mid-turn panel"
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_pass_is_total_on_garbage_and_no_match(tmp_dir: Path) -> None:
"""Nothing a mid-turn frame can carry settles a fence the block does not
name \u2014 and nothing throws: no match, a malformed frame (the ``isVerbatimBlock``
guard), a null root, a root that is not an element, no root at all. The
pending node stays ``.verbatim-pending`` in every arm (NEVER the honest
unresolved sentence \u2014 that is the terminal path's verdict, not this one)."""
out = _node_dom(
{
"other_file": f"""(() => {{
const bubble = pendingBubble(["{OTHER_PENDING_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"malformed_no_source": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, {{ path: "runbooks/a.md", text: "x" }}, {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"malformed_not_an_object": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, "verbatim", {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"null_root": "hydrateVerbatimNow(null, shownBlock(), {})",
"plain_object_root": "hydrateVerbatimNow({}, shownBlock(), {})",
"undefined_root": "hydrateVerbatimNow(undefined, shownBlock(), undefined)",
},
tmp_dir,
)
for name in ("other_file", "malformed_no_source", "malformed_not_an_object"):
case = out[name]
assert case["ok"] is True, (name, case)
assert case["value"]["n"] == 0, name
node = case["value"]["nodes"][0]
assert node["pending"] is True and node["unresolved"] is False, (
f"{name}: the fence stays a promise, never a burned placeholder"
)
assert node["text"] is None, f"{name}: no panel, so no bytes on screen"
for name in ("null_root", "plain_object_root", "undefined_root"):
assert out[name] == {"ok": True, "value": 0}, name
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_panel_survives_the_done_pass(tmp_dir: Path) -> None:
"""The parity that keeps the phase additive: the mid-turn pass runs, then
the ``done`` frame arrives with the SAME block. ``hydrateVerbatim`` only
touches ``.verbatim-pending``, so the panel the mid-turn pass built is the
SAME node afterwards (its ``Copied`` state and scroll position survive),
and the unrelated fence settles to the honest sentence."""
out = _node_dom(
{
"then_done": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
const n1 = hydrateVerbatimNow(bubble, shownBlock(), {{}});
const midturnPanel = findClass(bubble, "verbatim");
const n2 = hydrateVerbatim(bubble, [shownBlock()], {{}});
const nodes = snapshot(bubble);
return {{
n1,
n2,
sameNode: findClass(bubble, "verbatim") === midturnPanel,
nodes,
}};
}})()""",
},
tmp_dir,
)
case = out["then_done"]
assert case["ok"] is True, case
value = case["value"]
assert value["n1"] == 1 and value["n2"] == 1
assert value["sameNode"] is True, (
"the done pass re-runs over the mid-turn panel without rebuilding it"
)
settled, unresolved = value["nodes"]
assert settled["pending"] is False and settled["locator"] == SHOWN_BLOCK_LOCATOR
assert unresolved["pending"] is False and unresolved["unresolved"] is True, (
"the fence no frame ever resolved lands on the honest state at done"
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_one_locator_one_panel_across_the_midturn_and_done_passes(tmp_dir: Path) -> None:
"""The duplicate the mid-turn landing would otherwise leave behind (the
phase's A3): a ``show_text`` block arrives BEFORE the answer fences it, so
the mid-turn pass appends it at the answer position; the fence lands in a
later round and hydrates at ``done``. Exactly ONE panel remains and it sits
IN THE BUBBLE, where the model put the fence.
The two arms that must NOT collapse: two different ranges of one file are
two honest panels (the scan is per canonical locator), and a block the
answer never fences keeps its single appended panel."""
out = _node_dom(
{
"fence_arrives_later": """(() => {
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
const mid = hydrateVerbatimNow(bubble, shownBlock(), {});
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
addFence(msg, "homelab/runbooks/a.md#99-120");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { mid, appended, settled, panels: placed(msg) };
})()""",
"two_ranges_one_file": """(() => {
const b1 = shownBlock();
const b2 = shownBlock({ start_line: 10, end_line: 14,
locator: "homelab/runbooks/a.md#10-14" });
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
appendUnlocatedVerbatim(msg, [b1], {});
addFence(msg, "homelab/runbooks/a.md#10-14");
addFence(msg, "homelab/runbooks/a.md#12-40");
hydrateVerbatim(bubble, [b1, b2], {});
return { panels: placed(msg) };
})()""",
"never_fenced_keeps_its_panel": """(() => {
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
const again = appendUnlocatedVerbatim(msg, [shownBlock()], {});
return { appended, settled, again, panels: placed(msg) };
})()""",
"repeated_fence_shows_the_bytes_once": """(() => {
const msg = pendingWrap(["homelab/runbooks/a.md#12-40",
"homelab/runbooks/a.md#12-40"]);
const bubble = msg.querySelector(".bubble");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { settled, panels: placed(msg) };
})()""",
"unresolved_twin_is_not_a_placement": """(() => {
const msg = pendingWrap(["homelab/runbooks/a.md#1-2"]);
const bubble = msg.querySelector(".bubble");
hydrateVerbatim(bubble, [], {}); // the honest state, same locator
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
return { appended, panels: placed(msg) };
})()""",
},
tmp_dir,
)
fenced = out["fence_arrives_later"]
assert fenced["ok"] is True, fenced
assert fenced["value"] == {
"mid": 0,
"appended": 1,
"settled": 1,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
}, "the fence wins: one panel, in the answer, the mid-turn twin gone"
ranges = out["two_ranges_one_file"]
assert ranges["ok"] is True, ranges
assert ranges["value"]["panels"] == [
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
], "two ranges of one file are TWO panels — the scan is per locator"
unfenced = out["never_fenced_keeps_its_panel"]
assert unfenced["ok"] is True, unfenced
assert unfenced["value"] == {
"appended": 1,
"settled": 0,
"again": 0,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}],
}, "a block no fence names keeps exactly its one answer-position panel"
repeated = out["repeated_fence_shows_the_bytes_once"]
assert repeated["ok"] is True, repeated
assert repeated["value"]["settled"] == 2
assert repeated["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
], "the same fence twice shows those bytes once"
unresolved = out["unresolved_twin_is_not_a_placement"]
assert unresolved["ok"] is True, unresolved
assert unresolved["value"]["appended"] == 1, (
"an unresolved node is not a placement, so the block still lands"
)
assert unresolved["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
]
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_the_twin_scan_bounds_to_the_message_it_hydrates(tmp_dir: Path) -> None:
"""The dedupe's BOUND, run against the real file (task 03): the scan reads
the message the hydration belongs to.
* a bare bubble with no ``.msg`` ancestor still dedupes — the ``|| root``
fallback means "no wrap" narrows the scan to the root, it never turns
the dedupe off (two panels for one locator is the bug either way);
* a second range of the same file keeps its own panel under that fallback
too (per locator, never per identity — the fallback is not a wider scan);
* a twin that belongs to ANOTHER message survives: hydrating one answer
never reaches into the next one's panels."""
out = _node_dom(
{
"bare_bubble_dedupes_too": """(() => {
const bubble = pendingBubble([]);
const appended = appendUnlocatedVerbatim(bubble, [shownBlock()], {});
bareFence(bubble, "homelab/runbooks/a.md#12-40");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { appended, settled, panels: placed(bubble) };
})()""",
"bare_bubble_two_ranges_stay_two": """(() => {
const other = shownBlock({ start_line: 10, end_line: 14,
locator: "homelab/runbooks/a.md#10-14" });
const bubble = pendingBubble([]);
appendUnlocatedVerbatim(bubble, [shownBlock()], {});
bareFence(bubble, "homelab/runbooks/a.md#10-14");
const settled = hydrateVerbatim(bubble, [shownBlock(), other], {});
return { settled, panels: placed(bubble) };
})()""",
"another_message_is_out_of_bounds": """(() => {
const here = pendingWrap(["homelab/runbooks/a.md#12-40"]);
const neighbour = pendingWrap([]);
appendUnlocatedVerbatim(neighbour, [shownBlock()], {});
const settled = hydrateVerbatim(
here.querySelector(".bubble"), [shownBlock()], {});
return { settled, here: placed(here), neighbour: placed(neighbour) };
})()""",
},
tmp_dir,
)
bare = out["bare_bubble_dedupes_too"]
assert bare["ok"] is True, bare
assert bare["value"] == {
"appended": 1,
"settled": 1,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
}, "no .msg wrap still means ONE panel for one locator"
ranges = out["bare_bubble_two_ranges_stay_two"]
assert ranges["ok"] is True, ranges
assert ranges["value"]["settled"] == 1
assert ranges["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
], "the fallback bounds the scan — it does not widen it past the locator"
neighbour = out["another_message_is_out_of_bounds"]
assert neighbour["ok"] is True, neighbour
assert neighbour["value"]["settled"] == 1
assert neighbour["value"]["here"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
], "this answer hydrates to one panel"
assert neighbour["value"]["neighbour"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
], "the next answer's panel is not this scan's to remove"
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_a_stored_record_restores_one_panel_per_locator(tmp_dir: Path) -> None:
"""The restore/shared parity of the dedupe (task 03): both call sites run
the SAME two passes in the SAME order — hydrate the record's fences, then
append the blocks the answer never located. So a stored answer that repeats
one locator twice and additionally carries an unlocated block comes back as
exactly TWO panels: the locator once, IN the text, and the unlocated block
once, in the answer band. One reload, one render — the duplicate the
mid-turn frame could leave behind never reaches the record's reader."""
out = _node_dom(
{
"restore": f"""(() => {{
const msg = pendingWrap(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
const bubble = msg.querySelector(".bubble");
const record = [
shownBlock(),
shownBlock({{
source: "homelab", path: "runbooks/c.md", title: "Runbook C",
start_line: 1, end_line: 3, total_lines: 9,
text: "c one\\nc two", locator: "homelab/runbooks/c.md#1-3",
}}),
];
const settled = hydrateVerbatim(bubble, record, {{}});
const appended = appendUnlocatedVerbatim(msg, record, {{}});
return {{ settled, appended, panels: placed(msg) }};
}})()""",
},
tmp_dir,
)
case = out["restore"]
assert case["ok"] is True, case
assert case["value"]["settled"] == 2, "every fence settles — nothing is left pending"
assert case["value"]["appended"] == 1, (
"only the block the answer never located lands in the answer band"
)
assert case["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
{"locator": "homelab/runbooks/c.md#1-3", "where": "body"},
], "one locator, one panel — the same answer on every surface"
+114
View File
@@ -1401,3 +1401,117 @@ def test_saved_chat_total_budget_counts_verbatim_block_text() -> None:
assert "8,000,000-character budget" in exc.value.errors()[0]["msg"]
# 14 × 560 000 = 7 840 000 fits (plus the small text, still inside).
assert len(SavedChatCreate.model_validate({"messages": heavy[:-1]}).messages) == 14
# ---------------------------------------------------------------------------
# Phase 192 (task 01, A1): ChatVerbatimBlockEvent — the mid-turn twin of the
# done frame's ``answer_text`` item.
# ---------------------------------------------------------------------------
def test_chat_verbatim_block_event_shape() -> None:
"""The mid-turn frame's wire shape: ``type`` FIRST (the sibling events'
convention) then the exact :class:`VerbatimBlockRef` field set, the
``verbatim_block`` default, ``truncated`` defaulting False (the block was
not capped), and no ``model_config`` — the phase-95
:class:`ChatToolResultEvent` convention mirrored: an SSE frame is
server-built, so unknown keys are ignored rather than forbidden."""
from app.schemas import ChatVerbatimBlockEvent
ev = ChatVerbatimBlockEvent(**_verbatim_block())
dumped = ev.model_dump()
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
assert dumped == {"type": "verbatim_block", **_verbatim_block()}
minimal = ChatVerbatimBlockEvent(
source="Homelab",
path="a.md",
title="a.md",
start_line=1,
end_line=1,
total_lines=1,
text="one line",
locator="Homelab/a.md",
).model_dump()
assert minimal["truncated"] is False
assert minimal["type"] == "verbatim_block"
# The sibling convention mirrored EXACTLY: an SSE event model declares no
# ``extra="forbid"`` (the frame is server-built, so an unknown key is
# ignored rather than rejected — like ``ChatToolResultEvent``).
from app.schemas import ChatToolResultEvent
assert ChatVerbatimBlockEvent.model_config.get("extra") is None
assert ChatToolResultEvent.model_config.get("extra") is None
def test_chat_verbatim_block_event_field_set_matches_the_done_item() -> None:
"""The drift pin: the mid-turn frame carries EXACTLY the done frame's
item fields (plus ``type``) with the SAME caps — the one client
(``isVerbatimBlock`` + ``buildVerbatimPanel``) hydrates both from one
shape, so a field added to :class:`VerbatimBlockRef` without its twin
here fails this test."""
from app.schemas import ChatVerbatimBlockEvent
assert set(ChatVerbatimBlockEvent.model_fields) == set(
VerbatimBlockRef.model_fields
) | {"type"}
for name, ref_field in VerbatimBlockRef.model_fields.items():
mine = ChatVerbatimBlockEvent.model_fields[name]
assert mine.annotation == ref_field.annotation, name
assert mine.is_required() == ref_field.is_required(), name
assert mine.metadata == ref_field.metadata, name # the caps, verbatim
# The caps fire (the same limits, enforced on this model too).
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(text="x" * 600_001))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(source="s" * 121))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(path="p" * 1001))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(title="t" * 501))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(locator="l" * 2101))
for field in ("start_line", "end_line", "total_lines"):
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(**{field: -1}))
# ``isVerbatimBlock`` (frontend/assets/verbatim.js) needs exactly these
# three: a non-empty ``source`` string, a non-empty ``path`` string and a
# ``text`` string — so the mid-turn event is accepted as-is.
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
assert isinstance(dumped["source"], str) and dumped["source"]
assert isinstance(dumped["path"], str) and dumped["path"]
assert isinstance(dumped["text"], str)
def test_chat_verbatim_block_event_dump_keys_are_the_client_contract() -> None:
"""Phase 192 (task 03, A4): the key-set pin the ONE panel client leans on.
``frontend/assets/verbatim.js`` has no schema of its own: the
``isVerbatimBlock`` guard is a shape check and ``buildVerbatimPanel``
reads the fields by name — so the mid-turn frame is only renderable while its
``model_dump()`` key set is EXACTLY :class:`VerbatimBlockRef`'s key set
plus the ``type`` discriminator. A field added to one model and not the
other, a rename, or an ``exclude=`` on the dump all fail here (the client
would silently lose the caption, the extent, or the whole block).
"""
from app.schemas import ChatVerbatimBlockEvent
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
assert set(dumped) == set(VerbatimBlockRef.model_fields) | {"type"}
# ``type`` FIRST and the ref's fields in declaration order — the sibling
# SSE events' convention, unchanged by the dump.
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
# The three keys ``isVerbatimBlock`` cannot live without, plus the keys
# ``buildVerbatimPanel`` prints (the caption's extent and its open link).
for key in ("source", "path", "text", "title", "start_line", "end_line",
"total_lines", "truncated", "locator"):
assert key in dumped, f"the client reads {key} off both frames"
# The done item round-trips the SAME keys: the mid-turn frame is a copy of
# a done item, so the restore/shared hydration needs no second reader.
assert (
VerbatimBlockRef.model_validate(
{k: v for k, v in dumped.items() if k != "type"}
).model_dump()
== {k: v for k, v in dumped.items() if k != "type"}
)