phase: 192_verbatim_inline_midturn
Build and Push Containers / build-and-push-app (push) Successful in 1m56s
Build and Push Containers / build-and-push-db (push) Successful in 14s

Phase 192 — final verification pass complete.

- **Verified (all 4 tasks already complete):** additive `verbatim_block` SSE frame (`VerbatimBlockPiece` → `ChatVerbatimBlockEvent` → chat dispatch, after the round's `tool`/`tool_result`, before the next round); `hydrateVerbatimNow` matched-only mid-turn pass + app.js branch; `dropPlacedTwin` per-locator done/restore dedupe; the dedicated E2E suite.
- **Root cause of the rejected attempt:** my own concurrent background `pytest` run deadlocked the harness validation run on the shared Postgres (`psycopg.errors.DeadlockDetected`). No code defect — re-ran every gate **exclusively**.
- `uv run pytest -q --cov=app --cov-report=term-missing` → **exit 0**, `TOTAL 7181 16 99%` (>90%)
- `uv run pytest tests/e2e/test_verbatim_inline_midturn.py --no-cov` → **6 passed** (isolated)
- `uv run pytest tests/e2e/test_verbatim_show_text.py tests/e2e/test_chat_history.py --no-cov` → **18 passed** (`test_chat_history.py` byte-unedited; the phase-182 suite's "no eighth type" pin was tightened, not loosened, to seven + `verbatim_block` with frame-order asserts)
- `uv run ruff check .` → All checks passed (exit 0) · `uv run pyright` → **0 errors**
- **Completion criteria:** single frame per success + zero per refusal ✅ · `BOR_ANSWER_TEXT=0` wire byte-identical (integration spelled baseline + E2E App B) ✅ · fence hydrates in place / unfenced lands at the call moment, before `done` ✅ · append+fence → 1 panel, two ranges → 2 panels ✅ · restore 1 panel/locator, `done.answer_text` + record key + `verbatim_blocks=N` unchanged ✅ · suite green, coverage 99%, lint/types clean ✅ · no-show_text turn byte-identical ✅ · commit → left to the harness (no git index/history commands run).
- **Docs:** README reviewed line-by-line (2399 lines) + AGENTS.md in full; mechanically verified all 86 `BOR_*` settings, every `.env.example` var, table defaults, internal anchors (0 broken), ports 8123/8901, tree depth 1–8, pagination 25/1–100, 17 palette colors, `0029`/`0030` migrations, and every cited test file. Fixed 1 inconsistency: the verbatim "Parity" bullet claimed the live turn/reload/shared page share "the same three globals" — now names the settle-path trio and states that `hydrateVerbatimNow` is the ONE chat-only addition (a shared page has no stream).
- **Notable:** `run_agent`'s four repeated piece unions were collapsed into the ONE exported `AgentPiece` alias, so a future piece type is spelled in one place; `test_frontend_edit_message.py`'s `runTurn` byte-pin was regenerated, not weakened.
- **Next pending phase:** `193_sync_remaining_count`.
This commit is contained in:
2026-10-19 12:19:51 -04:00
parent 4f20c256d7
commit fb17d15fbe
49 changed files with 17313 additions and 102 deletions
+195
View File
@@ -750,6 +750,22 @@ Implements just enough of the aipi surface:
for, so the client must land on its honest unresolved state (no badge,
no body, none of the model's bytes). Checked BEFORE the
``DEFLECT_MODE`` branch, same convention.
- user message containing ``walk through the stored text as you go``
(``MIDTURN_TRIGGER``, phase 192 task 04 — the mid-turn panel suite
``tests/e2e/test_verbatim_inline_midturn.py``) -> the MID-TURN turn the
phase-182 flow cannot produce: a content delta (the intro, plus the
closed fence in the fence-first arm) THEN the ``show_text`` call, and
the final answer only after :data:`MIDTURN_GAP_S` of silence — the
window in which the ``verbatim_block`` frame's panel is the only thing
on screen. Arm markers, the two ranges of one file, the sentinel token
and the byte-exact answer constants are documented at the definitions
(``MIDTURN_*``). NOT clause-gated on purpose: the kill-switch arm boots
a ``BOR_ANSWER_TEXT=0`` app with the SAME question, so the attempt
reaches the server-side ``Unknown tool.`` refusal and the wire can be
pinned byte-identical to pre-phase. Checked BEFORE the phase-182
``SHOW_TEXT_TRIGGER`` flow (disjoint phrases — the ordering
convention); no existing E2E question or fixture file carries the
phrase.
- user message containing ``which diagram is in your notes``
(``DIAGRAM_TRIGGER``, phase 124 task 07 — the BUG-FIX-ONLY
trigger, same suite) -> the deterministic ZERO-READ answer: the
@@ -1619,6 +1635,157 @@ INLINE_ANSWER = (
"And the inline markdown ends here — a plain closing line."
)
# ---------------------------------------------------------------------------
# Phase 192 (task 04): the MID-TURN show_text flow (the phase's own E2E suite)
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) drives the
#: deterministic phase-192 MID-TURN turn for
#: ``tests/e2e/test_verbatim_inline_midturn.py`` — the shape the phase-182
#: flow CANNOT produce, because that one streams the tool call with NO
#: content in front of it and answers with NO gap behind it:
#:
#: round 1 ``delta``(the intro prose — and, in the fence-first arm, the
#: closed ```bor fence itself) → a ``show_text`` tool_calls delta
#: round 2 (the server has by now sent ``tool`` + the task-01
#: ``verbatim_block`` frame) → a deliberate
#: ``MIDTURN_GAP_S`` silence → the answer's ``delta`` frames,
#: the LAST of them carrying ``MIDTURN_SENTINEL`` → ``done``
#:
#: The gap is the test's observation window: the panel the frame paints is
#: on screen while the answer has NOT streamed a single token, so
#: "the panel landed mid-turn" is pinned by the DOM itself, not by a poll.
#: Arms, discriminated by markers in the question (all disjoint from every
#: other trigger, ordering convention intact):
#
#: * default — no fence anywhere: the mid-turn block is APPENDED at the
#: answer position (``appendUnlocatedVerbatim``).
#: * ``MIDTURN_FENCE_BEFORE_MARKER`` — the fence rides round 1, so the
#: placeholder EXISTS when the frame lands: the mid-turn
#: ``hydrateVerbatimNow`` pass resolves it IN PLACE.
#: * ``MIDTURN_FENCE_AFTER_MARKER`` — the fence rides the final answer
#: (the realistic shape: the model quotes after the call), so the
#: mid-turn panel is appended and the ``done`` pass re-homes it onto
#: the fence and drops the twin (task 03's ``dropPlacedTwin``).
#: * ``MIDTURN_TWO_RANGES_MARKER`` — TWO ``show_text`` calls (the two
#: ranges the question names), no fences: two panels, in call order.
#: Stateless per the house rule (the tool-result count is the state).
#:
#: Deliberately NOT gated on ``_SHOW_TEXT_CLAUSE_MARKER``: the kill-switch
#: arm boots a ``BOR_ANSWER_TEXT=0`` app and asks the SAME question, where
#: the attempt must reach the server-side ``UNKNOWN_TOOL`` refusal (the
#: tool is not offered) so the wire can be pinned byte-identical to the
#: pre-phase baseline. The trigger phrase appears in no other suite's
#: question (verified by the suite's own grep) and no other trigger is a
#: substring of it.
MIDTURN_TRIGGER = "walk through the stored text as you go"
#: The module KB's source + path (the locator the panel captions).
MIDTURN_SOURCE = "midturnlab"
MIDTURN_PATH = "runbooks/valve-panel.md"
#: The ``show_text`` ``path`` argument: the COMBINED ``source/path`` identity
#: (``app/rag/agent.py`` resolves it through ``_resolve_path`` — a bare
#: ``path`` names a document that does not exist, the phase-72 refusal). The
#: call identity stays DERIVED, never a label (phase 191 LOCKED D2).
MIDTURN_COMBINED = f"{MIDTURN_SOURCE}/{MIDTURN_PATH}"
#: The range the single-call arms request.
MIDTURN_START, MIDTURN_END = 12, 20
#: The second call's range in the two-range arm.
MIDTURN_SECOND_START, MIDTURN_SECOND_END = 1, 10
#: Arm markers (question text only — never rendered by the app).
MIDTURN_FENCE_BEFORE_MARKER = "fence it before the call"
MIDTURN_FENCE_AFTER_MARKER = "fence it in the answer"
MIDTURN_TWO_RANGES_MARKER = "then the next block"
#: The round-1 content delta — on screen BEFORE the panel, so the suite can
#: assert the panel lands BELOW the text the model had already written.
MIDTURN_INTRO = "Let me pull the stored block up here — MIDTURN_INTRO_4412"
#: The final answer's witness token: it exists ONLY in the deltas that
#: stream after the frame, so "no sentinel in the DOM" == "the turn is live".
MIDTURN_SENTINEL = "MIDTURN_SENTINEL_7731"
#: Bytes the model re-types into a fence body — they must NEVER paint.
MIDTURN_FENCE_BODY_MARKER = "MIDTURN-FENCE-BODY-NEVER-ON-SCREEN"
#: The silence between the ``verbatim_block`` frame and the answer's first
#: delta (seconds). 1.5 s is comfortably longer than any Playwright query
#: round trip and comfortably shorter than every turn guard.
MIDTURN_GAP_S = 1.5
#: The unfenced / fence-first answer (byte-exact: the suite compares the
#: concatenated ``delta`` payloads against it on BOTH apps).
MIDTURN_PLAIN_ANSWER = (
"The panel above is the file itself, not my retyping of it.\n\n"
f"{MIDTURN_SENTINEL} — the answer kept streaming after the panel landed.\n\n"
"(Deterministic mock answer for E2E.)"
)
#: The fence-after-call answer (the dedupe arm).
MIDTURN_FENCED_ANSWER = (
"The block I pulled is quoted below.\n\n"
f"```bor {MIDTURN_COMBINED}#{MIDTURN_START}-{MIDTURN_END}\n"
f"{MIDTURN_FENCE_BODY_MARKER} — the model re-typed these bytes\n"
"```\n\n"
f"{MIDTURN_SENTINEL} — the answer kept streaming after the panel landed.\n\n"
"(Deterministic mock answer for E2E.)"
)
def _midturn_fence() -> str:
"""One CLOSED ```bor fence for the arm's primary range (the model's
own bytes inside — the placeholder drops them), padded by blank lines:
the client concatenates a mid-round ``delta`` and the next round's answer
with NO separator, so a fence that ends without its own trailing newline
has its closing ``` glued to the following word and stops being a fence
(a REAL model writes the blank line — the mock has to be as well-formed
as the thing it stands in for)."""
return (
f"```bor {MIDTURN_COMBINED}#{MIDTURN_START}-{MIDTURN_END}\n"
f"{MIDTURN_FENCE_BODY_MARKER} — the model re-typed these bytes\n"
"```\n\n"
)
def _midturn_ranges(text: str) -> list[tuple[int, int]]:
"""Every ``N-M`` line range the question names, in order (the two-range
arm carries two). Empty → the primary range."""
found = [
(int(start), int(end))
for start, end in re.findall(r"\b(\d+)\s*-\s*(\d+)\b", text)
]
return found or [(MIDTURN_START, MIDTURN_END)]
def _midturn_flow(body: dict) -> tuple[str, ...] | None:
"""The deterministic phase-192 mid-turn step (see MIDTURN_TRIGGER).
``("call", path, start, end, mid_round_delta)`` — the round's content
delta (possibly empty) followed by the scripted ``show_text`` call (the
line numbers are STRINGS, the ``_show_text_request`` convention);
``("answer", text)`` — the final answer, streamed after
``MIDTURN_GAP_S`` of silence.
"""
text = _user(body)
lower = text.lower()
if MIDTURN_TRIGGER not in lower:
return None
ranges = _midturn_ranges(text)
results = _tool_results(body)
if not results:
delta = MIDTURN_INTRO
if MIDTURN_FENCE_BEFORE_MARKER in lower:
delta = f"{MIDTURN_INTRO}\n\n{_midturn_fence()}"
start, end = ranges[0]
return ("call", MIDTURN_COMBINED, str(start), str(end), delta)
if (
MIDTURN_TWO_RANGES_MARKER in lower
and len(results) == 1
and len(ranges) > 1
):
start, end = ranges[1]
return ("call", MIDTURN_COMBINED, str(start), str(end), "")
return (
"answer",
MIDTURN_FENCED_ANSWER
if MIDTURN_FENCE_AFTER_MARKER in lower
else MIDTURN_PLAIN_ANSWER,
)
# ---------------------------------------------------------------------------
# Phase 67 (LLM retry, TODO.md L3): deterministic failure injection
# ---------------------------------------------------------------------------
@@ -5710,6 +5877,34 @@ def chat_completions(body: dict[str, Any]) -> Any:
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 192 (task 04): the MID-TURN show_text turn — the round-1
# content delta (+ optionally the fence itself) then the call, the
# final answer only after MIDTURN_GAP_S of silence, so the panel
# the verbatim_block frame paints is on screen BEFORE any answer
# token. Checked BEFORE the phase-182 show_text flow (disjoint
# trigger phrases — the ordering convention); that flow's own arms
# stay byte-identical.
midturn = _midturn_flow(body)
if midturn is not None:
if midturn[0] == "call":
stream = _midround_tool_stream(
"",
midturn[4],
"show_text",
_show_text_call_arguments(midturn[1], midturn[2], midturn[3]),
"call_midturn_show",
)
else: # "answer" — the final answer, after the observation gap
stream = _sse_stream(
_apply_max_tokens(midturn[1], body.get("max_tokens")),
0.0,
pre_content_delay=MIDTURN_GAP_S,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
turn_progress = _turn_progress_flow(body)
if turn_progress is not None:
stream = _turn_progress_stream(turn_progress)
File diff suppressed because it is too large Load Diff
+27 -7
View File
@@ -28,8 +28,9 @@ Postgres so the fixtures' exact text is the assertion baseline):
badge (locked A5). Wire pins: exactly FIVE tools offered (the three base
+ ``show_image`` + ``show_text``), exactly one executed call, and — from
an in-page fetch tap over the real ``/api/chat`` stream — the turn's SSE
``type`` set stays inside the pre-phase seven with ``answer_text``
riding INSIDE ``done`` (no eighth event type).
``type`` set stays inside the pre-phase seven PLUS the one additive type
phase 192 introduced (``verbatim_block``, the mid-turn copy of a shown
block), with ``answer_text`` still riding INSIDE ``done``.
- *append path* — the same turn with ``describe it without a block`` (no
fence in the answer): the shown block renders as a panel APPENDED to the
bubble body, below the answer text and ABOVE the source clusters (the
@@ -246,7 +247,7 @@ DOC_A_LINES = DOC_A_CONTENT.split("\n")
DOC_B_LINES = DOC_B_CONTENT.split("\n")
#: The SSE ``type`` values the pre-phase stream already carried
#: (``app/schemas.py``'s chat events): the phase added NO eighth type.
#: (``app/schemas.py``'s chat events): phase 182 added NO eighth type.
PRE_PHASE_EVENT_TYPES = {
"delta",
"thinking",
@@ -257,6 +258,15 @@ PRE_PHASE_EVENT_TYPES = {
"retry",
}
#: The ONE type phase 192 (task 01, A1) adds: the mid-turn
#: ``verbatim_block`` frame, sent by the turn whose ``show_text`` call just
#: succeeded (one frame per shown block, before the next model round). App A
#: calls ``show_text``, so its vocabulary is the seven PLUS this type and
#: NOTHING else; a turn with no successful show never carries it, and the
#: kill-switch stream's byte-identity is pinned on the wire in
#: ``tests/integration/test_chat_api.py::test_kill_switch_stream_is_byte_identical``.
PHASE_192_EVENT_TYPES = {"verbatim_block"}
# ---------------------------------------------------------------------------
# Module apps: verbatim ON (App A) and the kill switch OFF (App B).
@@ -847,13 +857,23 @@ def test_show_text_turn_paints_the_files_bytes(
"Text shown to the user" in r for r in _tool_results(captured[1])
), "the show_text confirming result never reached the model"
# --- the frame vocabulary (no eighth SSE type) ----------------------
# --- the frame vocabulary (the seven + phase 192's ONE type) --------
frames = _sse_frames(page)
types: set[str] = {str(f.get("type")) for f in frames}
assert types <= PRE_PHASE_EVENT_TYPES, (
order = [str(f.get("type")) for f in frames]
types: set[str] = set(order)
allowed = PRE_PHASE_EVENT_TYPES | PHASE_192_EVENT_TYPES
assert types <= allowed, (
f"the stream invented an event type: {sorted(types)}"
)
assert len(types) <= 7
assert len(types) <= 8
# Phase 192 (task 01): the shown block also rides its own mid-turn frame,
# ahead of the answer's deltas (the panel is on screen while the model is
# still talking) — ADDITIVE, the done frame below still carries it.
block_frames = [f for f in frames if f.get("type") == "verbatim_block"]
assert [b["locator"] for b in block_frames] == [seeded["a_locator"]]
i_block = order.index("verbatim_block")
assert "delta" not in order[:i_block], order
assert order.index("tool") < i_block, order
assert "answer_text" not in types, "answer_text leaked out as its own event"
done = [f for f in frames if f.get("type") == "done"]
assert len(done) == 1
+9 -9
View File
@@ -47,14 +47,14 @@ from app.config import Settings
from app.db import SessionLocal
from app.models import Document, FolderSummary, GitSource
from app.rag import agent
from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL, AgentHolder, run_agent
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
from app.rag.agent import (
AGENT_TOOLS,
SHOW_TEXT_TOOL,
AgentHolder,
AgentPiece,
run_agent,
)
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
if TYPE_CHECKING:
from app.rag.scaffolding import ScaffoldingFilter
@@ -286,8 +286,8 @@ async def _consume(
llm: LLMClient,
db_factory: Callable[[], Session],
holder: AgentHolder,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
) -> list[AgentPiece]:
out: list[AgentPiece] = []
async for piece in run_agent(
llm,
db_factory, # SEC-14-04: session factory (short-lived sessions)
+4 -4
View File
@@ -28,8 +28,8 @@ from sqlalchemy.orm import Session
from app.config import Settings
from app.db import SessionLocal
from app.models import Document, GitSource
from app.rag.agent import AgentHolder, _budget_note, run_agent
from app.rag.llm import LLMClient, RetryPiece, StreamPiece, ToolCallPiece, ToolResultPiece
from app.rag.agent import AgentHolder, AgentPiece, _budget_note, run_agent
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
if TYPE_CHECKING:
from app.rag.scaffolding import ScaffoldingFilter
@@ -128,9 +128,9 @@ def _run_call(
async def _consume(
llm: LLMClient, db: Session, holder: AgentHolder
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""SEC-14-04: uses a short-lived session per tool call."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
out: list[AgentPiece] = []
async for piece in run_agent(
llm,
lambda: SessionLocal(), # SEC-14-04: session factory (short-lived sessions)
+254 -1
View File
@@ -41,7 +41,7 @@ from app.rag.importer import import_sources
from app.rag.llm import EmbeddingError, LLMError, StreamPiece, ToolCallPiece, ToolResultPiece
from app.rag.prompts import SHOW_TEXT_CLAUSE, build_high_prompt
from app.rag.retriever import TRUNCATION_MARKER
from app.schemas import ChatDoneEvent, SourceRef
from app.schemas import ChatDoneEvent, SourceRef, VerbatimBlockRef
from tests.conftest import ADMIN_PASSWORD
#: The fixture documents' fixed creation date (phase 106, D5): the
@@ -3276,6 +3276,9 @@ def test_grounded_turn_carries_show_text_clause_and_tool_per_the_flag(
#: A fixture doc with a stable, countable body (16 lines as imported).
_K8S = "docs/homelab/kubernetes.md"
#: The second countable fixture doc (phase 192: the two-call turn needs two
#: different canonical locators — the ``shown_texts`` dedupe is per locator).
_BACKUPS = "docs/homelab/backups.md"
def _verbatim_settings(**overrides: Any) -> Settings:
@@ -3299,6 +3302,20 @@ def _doc_lines(db, combined: str) -> list[str]:
return content.splitlines()
def _doc_title(db, combined: str) -> str:
"""The indexed row's ``title`` — the same authority the resolver puts on
the block, never a string retyped here (the phase-182 ``_doc_lines``
convention)."""
source, _, path = combined.partition("/")
title = db.scalar(
select(Document.title).where(
Document.source == source, Document.path == path
)
)
assert title, combined
return str(title)
def test_show_text_turn_done_frame_carries_the_server_block(
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
) -> None:
@@ -3608,3 +3625,239 @@ def test_resolution_failure_degrades_to_no_blocks(
assert any("verbatim block resolution failed" in m for m in lines)
turn = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert turn and turn[-1].endswith("verbatim_blocks=0")
# ---------------------------------------------------------------------------
# Phase 192 (task 01, A1) — the additive mid-turn ``verbatim_block`` frame.
#
# A successful ``show_text`` used to reach the client ONLY on the ``done``
# frame's ``answer_text`` list (phase 182 task 05), so every panel popped in
# after the model had finished talking about the bytes. The agent loop now
# also yields the block the moment the call succeeds, and the endpoint turns
# it into one additive SSE frame — after the round's ``tool`` /
# ``tool_result`` frames, before the next model round. The ``done`` frame,
# the saved record and the ``verbatim_blocks=N`` log field keep the phase-182
# contract byte for byte; a ``BOR_ANSWER_TEXT=0`` turn streams NOTHING new.
# ---------------------------------------------------------------------------
def test_verbatim_block_frame_lands_midturn_before_done(
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
) -> None:
"""A scripted grounded turn whose model calls
``show_text(docs/homelab/kubernetes.md, offset=3, limit=3)`` → the stream
carries exactly ONE ``type="verbatim_block"`` frame with the EXACT
:class:`VerbatimBlockRef` field set (plus ``type``), positioned after the
round's ``tool`` frame and before the answer's ``delta`` frames and the
``done`` frame. Its bytes are the DB's, its extent is the server's, and
the ``done`` frame still carries the SAME block in ``answer_text`` (the
restore/shared-page contract is untouched) — counted ONCE by the per-turn
``verbatim_blocks=N`` line, which never counts mid-turn frames."""
scripted = FakeRagLLM(
tool_script=[
[
ToolCallPiece(
id="call_1",
name="show_text",
arguments={"path": _K8S, "offset": 3, "limit": 3},
)
]
]
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
types = [f["type"] for f in frames]
assert types.count("verbatim_block") == 1
i_tool, i_block, i_done = types.index("tool"), types.index("verbatim_block"), len(frames) - 1
assert types[i_done] == "done"
# The frame sits inside its own round: after the ``tool`` frame, before
# the next round's answer deltas and long before ``done``.
assert i_tool < i_block < types.index("delta") < i_done
lines = _doc_lines(db, _K8S)
frame = frames[i_block]
assert frame == {
"type": "verbatim_block",
"source": "docs",
"path": "homelab/kubernetes.md",
"title": "Kubernetes Homelab Cluster",
"start_line": 3,
"end_line": 5,
"total_lines": len(lines),
"text": "\n".join(lines[2:5]),
"truncated": False,
"locator": f"{_K8S}#3-5",
}
# The key set is the done-frame item's set plus ``type`` — the client's
# ``isVerbatimBlock`` guard accepts the event as-is.
assert set(frame) == set(VerbatimBlockRef.model_fields) | {"type"}
# The done frame is UNCHANGED from phase 182: the same block, in the same
# shape, at the same place (the mid-turn frame is additive, never a
# replacement — the client's idempotence keeps it from painting twice).
done = frames[i_done]
assert done["answer_text"] == [{k: v for k, v in frame.items() if k != "type"}]
turn = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert turn and turn[-1].endswith("verbatim_blocks=1")
def test_kill_switch_stream_is_byte_identical(
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The wire pin for the kill switch: with ``BOR_ANSWER_TEXT=0`` the tool
is never offered, so a scripted ``show_text`` attempt lands on the
phase-182 refusal path and the captured frame sequence is EXACTLY the
pre-phase baseline — one ``tool`` frame, the answer's ``delta`` frames,
the pre-phase ``done`` frame — with NO ``verbatim_block`` frame anywhere
and no ``answer_text`` key on any frame."""
answer = "The flag is off, so nothing is displayed here."
monkeypatch.setattr(
chat_api, "get_settings", lambda: _verbatim_settings(answer_text=False)
)
scripted = FakeRagLLM(
answer=answer,
tool_script=[
[
ToolCallPiece(
id="call_1",
name="show_text",
arguments={"path": _K8S, "offset": 3, "limit": 3},
)
]
],
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
# The documented pre-phase baseline, spelled out: the refused call's
# ``tool`` frame (the exact ChatToolEvent bytes — no range keys, they are
# ``read``-only), then one ``delta`` frame per 12-char slice of the
# canned answer, then ``done``. Nothing else, in that order.
expected = [{"type": "tool", "name": "show_text", "argument": _K8S}] + [
{"type": "delta", "text": answer[i : i + 12]} for i in range(0, len(answer), 12)
]
assert frames[:-1] == expected
assert frames[-1]["type"] == "done"
assert set(frames[-1]) == {"type", "deflected", "sources", "related", "suggestions"}
assert "verbatim_block" not in json.dumps(frames)
assert "answer_text" not in json.dumps(frames)
# The refusal the model saw is the phase-182 one (the tool does not exist
# while the flag is off), and no block was recorded anywhere.
tool_message = scripted.seen_messages[1][3]
assert tool_message["role"] == "tool"
assert tool_message["content"].startswith("Unknown tool.")
def test_done_frame_and_log_line_unchanged_by_midturn_frames(
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
) -> None:
"""Phase 192 (task 03, A4) — the frame the mid-turn pass adds is a COPY of
a done-frame item, never a participant in the done-frame contract.
Two agent rounds, two successful ``show_text`` calls, no fence in the
answer: the stream carries TWO ``verbatim_block`` frames (each in its own
round, before ``done``) and the ``done`` frame still carries exactly the
phase-182 resolution — both blocks in CALL order with the server's
extents, byte-identical to what the resolver reads out of the row. That
list is the saved record's ``answer_text`` and what the reload / shared
page hydrates from, so the dedupe the client performs lands on data the
server never changed. The per-turn line therefore reads
``verbatim_blocks=2`` — the DONE-frame count — and never 4: the mid-turn
frames are counted NOWHERE."""
scripted = FakeRagLLM(
tool_script=[
[
ToolCallPiece(
id="call_1",
name="show_text",
arguments={"path": _K8S, "offset": 3, "limit": 3},
)
],
[
ToolCallPiece(
id="call_2",
name="show_text",
arguments={"path": _BACKUPS, "offset": 1, "limit": 2},
)
],
]
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
types = [f["type"] for f in frames]
done = frames[-1]
assert types[-1] == "done"
mid = [f for f in frames if f["type"] == "verbatim_block"]
assert len(mid) == 2 and types.count("verbatim_block") == 2
# Each frame rides its own round: after that round's ``tool`` frame and
# long before ``done`` (the phase-192 emission order, twice).
tool_positions = [i for i, t in enumerate(types) if t == "tool"]
assert len(tool_positions) == 2
for round_index, frame in enumerate(mid):
assert tool_positions[round_index] < frames.index(frame) < len(frames) - 1, (
"the frame lands inside its OWN round: after its tool line, before done"
)
# The done frame is the phase-182 resolution, spelled out from the row —
# the same two blocks, in call order, with the server's extent/locator.
k8s_lines = _doc_lines(db, _K8S)
backup_lines = _doc_lines(db, _BACKUPS)
assert done["answer_text"] == [
{
"source": "docs",
"path": "homelab/kubernetes.md",
"title": _doc_title(db, _K8S),
"start_line": 3,
"end_line": 5,
"total_lines": len(k8s_lines),
"text": "\n".join(k8s_lines[2:5]),
"truncated": False,
"locator": f"{_K8S}#3-5",
},
{
"source": "docs",
"path": "homelab/backups.md",
"title": _doc_title(db, _BACKUPS),
"start_line": 1,
"end_line": 2,
"total_lines": len(backup_lines),
"text": "\n".join(backup_lines[0:2]),
"truncated": False,
"locator": f"{_BACKUPS}#1-2",
},
]
# Each mid-turn frame is EXACTLY its done-frame twin minus ``type``: the
# client hydrates both from one shape, and the mid-turn copy resolved at
# the call moment cannot disagree with the done-time resolution.
assert done["answer_text"] == [
{k: v for k, v in frame.items() if k != "type"} for frame in mid
]
# The done frame's key set is the phase-182 set plus the additive
# ``answer_text`` — no new key, none renamed, none dropped.
assert set(done) == {
"type",
"deflected",
"sources",
"related",
"suggestions",
"answer_text",
}
# The ONE log line counts the blocks SENT on ``done`` (2), never the sum
# of the mid-turn frames and the done items (4) — AGENTS.md rule 10.
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert len(lines) == 1, lines
assert lines[-1].endswith("verbatim_blocks=2"), lines[-1]
assert "verbatim_blocks=4" not in json.dumps(lines)
@@ -71,16 +71,11 @@ from app.rag.agent import (
REREAD_PARTIAL,
SHOW_TEXT_TOOL,
AgentHolder,
AgentPiece,
run_agent,
)
from app.rag.importer import import_sources
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
)
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece, ToolResultPiece
from app.rag.pdf import extract_pdf_text
from app.rag.retriever import TRUNCATION_MARKER
from tests.fakes import FakeEmbedder
@@ -278,11 +273,11 @@ async def _consume(
llm: HandbookAgentLLM,
holder: AgentHolder,
settings: Settings,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one scripted ``run_agent`` turn (the owner's question,
no seed docs). SEC-14-04: a short-lived session per tool call —
the same pattern as production and the phase-70 integration suite."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
out: list[AgentPiece] = []
async for piece in run_agent(
cast("LLMClient", llm),
lambda: SessionLocal(),
@@ -296,7 +291,7 @@ async def _consume(
return out
def _answer(pieces: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]) -> str:
def _answer(pieces: list[AgentPiece]) -> str:
"""The turn's final answer text (the content pieces)."""
return "".join(
piece.text for piece in pieces if isinstance(piece, StreamPiece) and piece.kind == "content"
@@ -79,15 +79,10 @@ from app.rag import agent
from app.rag.agent import (
READ_TRUNCATION_NOTICE,
AgentHolder,
AgentPiece,
run_agent,
)
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
)
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece, ToolResultPiece
from app.rag.retriever import TRUNCATION_MARKER
if TYPE_CHECKING:
@@ -291,11 +286,11 @@ async def _consume(
llm: TwoRangeAgentLLM,
holder: AgentHolder,
settings: Settings,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one scripted ``run_agent`` turn. SEC-14-04: a
short-lived session per tool call — the same pattern as
production and the phase-70/131 integration suites."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
out: list[AgentPiece] = []
async for piece in run_agent(
cast("LLMClient", llm),
lambda: SessionLocal(),
@@ -309,7 +304,7 @@ async def _consume(
return out
def _answer(pieces: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]) -> str:
def _answer(pieces: list[AgentPiece]) -> str:
"""The turn's final answer text (the content pieces)."""
return "".join(
piece.text for piece in pieces if isinstance(piece, StreamPiece) and piece.kind == "content"
+252 -11
View File
@@ -51,11 +51,19 @@ retries (a dead-then-recovered round restarts before its first piece
with a ``RetryPiece``; a mid-stream drop stays terminal — locked A2;
the forced final no-tools call retries too; ``llm_retries=0`` is one
plain attempt; retries are invisible to the round cap; consumer abandon
mid-retry-sleep leaks nothing).
mid-retry-sleep leaks nothing), and the phase-192 (task 01) mid-turn
``VerbatimBlockPiece``: exactly ONE piece per SUCCESSFUL ``show_text`` —
after the round's ``tool`` / ``tool_result`` pieces and before the next
model round, the piece's ``block`` being the holder's own
``shown_texts`` entry — and ZERO pieces on every refusal path (missing
``path``, the ``answer_text=False`` posture, no document, an ``is_image``
row, the spent ``show_text_max_blocks`` budget, the already-shown dedupe)
or on the ``BOR_ANSWER_TEXT=0`` turn where the tool is not even offered.
"""
from __future__ import annotations
import asyncio
import dataclasses
import json
import logging
import uuid
@@ -82,7 +90,9 @@ from app.rag.agent import (
SHOWN_TEXT_TRUNCATION_NOTICE,
UNKNOWN_TOOL,
AgentHolder,
AgentPiece,
MalformedReplyError,
VerbatimBlockPiece,
agent_tools,
run_agent,
)
@@ -96,6 +106,7 @@ from app.rag.llm import (
)
from app.rag.prompts import TOOLS_SECTION, _base, build_deflect_prompt, build_high_prompt
from app.rag.retriever import TRUNCATION_MARKER
from app.rag.verbatim import VerbatimBlock
if TYPE_CHECKING:
from app.rag.scaffolding import ScaffoldingFilter
@@ -201,12 +212,14 @@ async def _run(
seed_docs: list[Document] | None = None,
history: Sequence[dict[str, Any]] = (),
db_factory: Callable[[], Session] | None = None,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one ``run_agent`` turn; *history* (phase 74) is the
client's prior turns spliced between system and user (default
``()`` — the pre-phase-74 two-message request). Phase 95: the loop
may also yield a ``ToolResultPiece`` (a truncated ``read``)."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
may also yield a ``ToolResultPiece`` (a truncated ``read``), and
phase 192 a ``VerbatimBlockPiece`` (a successful ``show_text``) —
both members of the one :data:`app.rag.agent.AgentPiece` union."""
out: list[AgentPiece] = []
factory = db_factory or (lambda: _mock_session())
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5143,8 +5156,8 @@ def test_round_failure_after_first_piece_is_terminal(monkeypatch: pytest.MonkeyP
)
sleeps = _record_sleeps(monkeypatch)
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async def drain() -> list[AgentPiece]:
out: list[AgentPiece] = []
with pytest.raises(LLMError, match="mid-stream drop"):
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5205,8 +5218,8 @@ def test_zero_retries_is_one_plain_attempt(monkeypatch: pytest.MonkeyPatch) -> N
llm = FailingLLM([([], LLMError("connection refused"))])
sleeps = _record_sleeps(monkeypatch)
async def drain() -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async def drain() -> list[AgentPiece]:
out: list[AgentPiece] = []
with pytest.raises(LLMError, match="connection refused"):
async for piece in run_agent(
cast("LLMClient", llm),
@@ -5255,9 +5268,7 @@ def test_abandon_mid_retry_sleep_leaks_nothing(monkeypatch: pytest.MonkeyPatch)
holder=holder,
)
async def consumer() -> list[
StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece
]:
async def consumer() -> list[AgentPiece]:
return [p async for p in gen]
task = asyncio.ensure_future(consumer())
@@ -6149,3 +6160,233 @@ def test_show_text_scripted_turn_returns_the_slice_to_the_model(
assert len(holder.shown_texts) == 1
assert holder.shown_texts[0].locator == f"{_SHOW_IDENTITY}#3-5"
assert holder.tool_calls == 1
# ---------------------------------------------------------------------------
# Phase 192 (task 01, A1) — the mid-turn ``VerbatimBlockPiece``
#
# The block a successful ``show_text`` appended to ``holder.shown_texts``
# used to reach the client only on the ``done`` frame (phase 182 task 05),
# so every verbatim panel popped in AFTER the model finished talking about
# the bytes. The loop now also yields ONE ``VerbatimBlockPiece`` the moment
# the block exists — after the round's ``tool`` / ``tool_result`` pieces and
# before the next model round — and the API layer turns it into the additive
# ``verbatim_block`` SSE frame (pinned in ``tests/integration/test_chat_api.py``
# and on the client in task 02). Success-only: every refusal path returns
# before the ``shown_texts.append``, so it yields NOTHING.
# ---------------------------------------------------------------------------
def _patch_two_docs(
monkeypatch: pytest.MonkeyPatch, *docs: Document
) -> None:
"""Resolve several fixture rows by ``(source, path)`` (the multi-document
variant of :func:`_patch_show_doc` — a truncated ``read`` doc AND a
showable doc in one turn)."""
def _find(_db: Session, source: str, path: str) -> Document | None:
for doc in docs:
if (source, path) == (doc.source, doc.path):
return doc
return None
monkeypatch.setattr(agent, "find_document", _find)
_DEFAULT_SHOW_DOC = object() # "the resolvable fixture row" sentinel
def _show_turn(
monkeypatch: pytest.MonkeyPatch,
arguments: dict[str, Any],
settings: Settings | None = None,
doc: Document | None | object = _DEFAULT_SHOW_DOC,
) -> tuple[AgentHolder, ScriptedLLM, list[AgentPiece]]:
"""One scripted turn whose model calls ``show_text`` with *arguments*
(the default resolves the fixture row; pass ``None`` for the
no-document refusal, or an ``is_image`` row for the A6 refusal)."""
_patch_show_doc(
monkeypatch, _show_doc() if doc is _DEFAULT_SHOW_DOC else cast("Document | None", doc)
)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="show_text", arguments=arguments)],
[StreamPiece("content", "ans")],
)
return holder, llm, asyncio.run(_run(llm, holder, settings or _settings()))
def test_verbatim_block_piece_is_one_frozen_block() -> None:
"""The piece's shape (the :class:`app.rag.llm.ToolResultPiece` pattern):
a FROZEN dataclass with exactly one field, ``block``, holding the
server's :class:`app.rag.verbatim.VerbatimBlock` — an immutable wire
value, never a re-derivation."""
block = VerbatimBlock(
source=_SHOW_SRC,
path=_SHOW_PATH,
title="Gufo quadlet",
start_line=2,
end_line=3,
total_lines=5,
text="L2-two\nL3-three",
)
piece = VerbatimBlockPiece(block=block)
assert [f.name for f in dataclasses.fields(piece)] == ["block"]
assert piece.block is block
with pytest.raises(dataclasses.FrozenInstanceError):
piece.block = None # pyright: ignore[reportAttributeAccessIssue]
def test_show_text_success_yields_one_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The success pin: a scripted ``show_text`` yields EXACTLY ONE
``VerbatimBlockPiece``, AFTER the round's ``ToolCallPiece`` (the ``tool``
frame) and BEFORE the next model round's first piece (the answer's
``StreamPiece``). Its ``block`` IS the ``holder.shown_texts`` entry — the
same object, so the mid-turn frame and the done frame can never
disagree."""
_patch_show_doc(monkeypatch, _show_doc())
holder = AgentHolder()
llm = ScriptedLLM(
[
ToolCallPiece(
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2}
)
],
[StreamPiece("content", "here"), StreamPiece("content", " it is")],
)
out = asyncio.run(_run(llm, holder, _settings()))
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
assert len(pieces) == 1
piece = pieces[0]
# Order: the tool call, the block, THEN the next round's answer pieces.
assert isinstance(out[0], ToolCallPiece) and out[0].name == "show_text"
assert out[1] is piece
assert all(isinstance(p, StreamPiece) for p in out[2:])
# The block is the holder's own entry (identity, not a copy).
assert piece.block is holder.shown_texts[0]
assert piece.block.locator == f"{_SHOW_IDENTITY}#2-5"
assert piece.block.text == "L2-two\nL3-three\nL4-four\nL5-five"
# The done-frame bookkeeping is untouched by the piece (task 05's path).
assert holder.tool_calls == 1
assert len(holder.shown_texts) == 1
assert len(llm.requests) == 2 # the tool round + the answer round
def test_verbatim_block_piece_lands_after_a_truncation_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The emission-order contract with BOTH derived pieces in one turn: a
truncated ``read`` (round 1 → ``ToolResultPiece``) then a
``show_text`` (round 2 → ``VerbatimBlockPiece``) stream
``tool, tool_result, tool, verbatim_block`` and only THEN the answer —
every derived piece stays inside its own round."""
big = _doc("S", "big.md", "Big", "\n".join(f"line {i}" for i in range(1, 30)))
_patch_two_docs(monkeypatch, big, _show_doc())
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments={"path": "S/big.md"})],
[ToolCallPiece(id="c2", name="show_text", arguments={"path": _SHOW_IDENTITY})],
[StreamPiece("content", "ans")],
)
out = asyncio.run(_run(llm, holder, _settings(read_max_chars=20)))
kinds = [
(
"tool:" + p.name
if isinstance(p, ToolCallPiece)
else "trunc"
if isinstance(p, ToolResultPiece)
else "block"
if isinstance(p, VerbatimBlockPiece)
else "retry"
if isinstance(p, RetryPiece)
else p.kind
)
for p in out
]
assert kinds == [
"tool:read",
"trunc",
"tool:show_text",
"block",
"content",
]
assert len([p for p in out if isinstance(p, VerbatimBlockPiece)]) == 1
@pytest.mark.parametrize(
"case",
["missing_args", "flag_off", "no_document", "image_doc", "block_cap"],
)
def test_show_text_refusals_yield_no_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch, case: str
) -> None:
"""Every refusal path yields ZERO pieces — the single success gate is
the ``shown_texts.append`` (the mid-turn frame is success-only by
construction, and the refusal itself is byte-identical to phase 182):
the missing-``path`` refusal, the ``answer_text=False`` UNKNOWN_TOOL
posture, the no-document teaching, the ``is_image`` refusal (LOCKED A6)
and the spent ``show_text_max_blocks`` budget."""
if case == "missing_args":
arguments: dict[str, Any] = {}
settings, doc = None, _DEFAULT_SHOW_DOC
elif case == "flag_off":
arguments = {"path": _SHOW_IDENTITY}
settings, doc = _settings(answer_text=False), _DEFAULT_SHOW_DOC
elif case == "no_document":
arguments = {"path": "nosuch/file.md"}
settings, doc = None, None
elif case == "image_doc":
arguments = {"path": _SHOW_IDENTITY}
settings, doc = None, _show_doc(is_image=True)
else:
arguments = {"path": _SHOW_IDENTITY}
settings, doc = _settings(show_text_max_blocks=0), _DEFAULT_SHOW_DOC
holder, _llm, out = _show_turn(monkeypatch, arguments, settings, doc)
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
_assert_refusal(holder) # counts in nothing, displays nothing
def test_show_text_already_shown_dedupe_yields_no_second_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The canonical-locator dedupe is a REFUSAL (D7), so the second,
identical show adds no holder entry and yields no second piece —
exactly ONE ``VerbatimBlockPiece`` across both rounds (the client can
therefore never be told to paint the same block twice)."""
_patch_show_doc(monkeypatch, _show_doc())
holder = AgentHolder()
call = ToolCallPiece(
id="c1", name="show_text", arguments={"path": _SHOW_IDENTITY, "offset": 2, "limit": 2}
)
llm = ScriptedLLM(
[call],
[ToolCallPiece(id="c2", name="show_text", arguments=call.arguments)],
[StreamPiece("content", "ans")],
)
out = asyncio.run(_run(llm, holder, _settings()))
pieces = [p for p in out if isinstance(p, VerbatimBlockPiece)]
assert len(pieces) == 1
assert pieces[0].block.locator == f"{_SHOW_IDENTITY}#2-3"
assert len(holder.shown_texts) == 1 # the dedupe refused the second show
def test_kill_switch_turn_yields_no_verbatim_block_piece(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""``BOR_ANSWER_TEXT=0`` (the phase-182 D8 posture): the tool is NOT
offered — so no call can succeed — and a scripted call anyway yields
zero pieces. The turn's piece sequence is byte-identical to pre-phase
(the wire pin lives in ``tests/integration/test_chat_api.py``)."""
holder, llm, out = _show_turn(
monkeypatch, {"path": _SHOW_IDENTITY}, _settings(answer_text=False)
)
offered = [t["function"]["name"] for t in (llm.requests[0][1] or [])]
assert "show_text" not in offered
assert [p for p in out if isinstance(p, VerbatimBlockPiece)] == []
_assert_refusal(holder)
+4 -10
View File
@@ -19,14 +19,8 @@ from sqlalchemy.orm import Session
from app.config import Settings
from app.models import Document
from app.rag import agent
from app.rag.agent import AgentHolder, run_agent
from app.rag.llm import (
LLMClient,
RetryPiece,
StreamPiece,
ToolCallPiece,
ToolResultPiece,
)
from app.rag.agent import AgentHolder, AgentPiece, run_agent
from app.rag.llm import LLMClient, StreamPiece, ToolCallPiece
#: Fixture document creation date (phase 106, D5).
_FIXTURE_CREATED_AT = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
@@ -131,9 +125,9 @@ async def _consume(
holder: AgentHolder,
settings: Settings,
seed_docs: list[Document] | None = None,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
) -> list[AgentPiece]:
"""Consume one ``run_agent`` turn."""
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
out: list[AgentPiece] = []
async for piece in run_agent(
cast("LLMClient", llm),
db_factory,
+34 -2
View File
@@ -1471,6 +1471,30 @@ PINNED_RUN_TURN = """function runTurn(text, { reask = false, image = null } = {}
}
}
}
} else if (ev.type === "verbatim_block") {
// Phase 192 (task 02, A2): the show_text panel lands at the moment
// the call succeeds — the fence it names hydrates IN PLACE; an
// unfenced block lands at the answer position now, not at done.
// The done pass re-runs the same functions and can never double this
// panel: hydrateVerbatim is idempotent (only .verbatim-pending is
// touched), it DROPS this appended twin when the answer fences the
// same locator later (dropPlacedTwin, phase 192 A3), and
// appendUnlocatedVerbatim skips an already-placed locator. NO
// client-side answer_text gate: the server's gate is the only one
// (a BOR_ANSWER_TEXT=0 turn sends no frame, so this branch never
// fires — a second, client-side gate would only drift).
// `wrap` always exists here (the round's `tool` frame created the
// live bubble) but the optional chain keeps a frame that arrives
// without one a silent no-op, like the `tool_result` branch.
const n = hydrateVerbatimNow(
wrap?.querySelector(".bubble"),
ev,
VERBATIM_LINK_OPTS
);
// The append is the ZERO-match path only: a fence already on screen
// keeps its panel IN PLACE (the model's order), and only a block no
// placeholder names lands at the answer position.
if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);
} else if (ev.type === "error") {
// Phase 182 (task 07): an `error` frame is terminal — the `done`
// hydrate will never run, so a placeholder that streamed in stays at
@@ -1701,14 +1725,22 @@ def test_run_turn_is_pinned_at_phase_159_state() -> None:
Copy buttons BEFORE ``appendSources``; the ``answer_text`` meta key rides
``rememberBrainTurn`` with ``|| undefined``; and the terminal paths — the
``error`` frame, the 300 s guard, its own abort surface, and the Stop
branch — settle any leftover placeholder to the unresolved state):
branch — settle any leftover placeholder to the unresolved state)
and phase 192 (task 02)'s sanctioned addition (the ONE new SSE branch:
``ev.type === "verbatim_block"`` settles the pending fence the shown
block names IN PLACE through ``hydrateVerbatimNow`` and appends the block
through ``appendUnlocatedVerbatim`` ONLY when that pass matched nothing —
the mid-turn landing, so a panel appears the moment the ``show_text``
call succeeds instead of at ``done``; no client-side gate, the server's
``BOR_ANSWER_TEXT`` gate is the only one):
the full source must match the pin — the ``reask`` path, the turn-local
resets, the
SSE contract, the save points, and the sanctioned additions (the
task-01 ``lastUserWrap`` assignment + the idle-settle mark
pass, the task-03 aborted-settle mark pair, the phase-160
done-settle pair, the phase-161 live-container additions, the
D5 refinement, and the phase-182 verbatim settle/hydrate pair) included."""
D5 refinement, the phase-182 verbatim settle/hydrate pair and the
phase-192 mid-turn branch) included."""
assert _fn_source(_js(), "runTurn") == PINNED_RUN_TURN
+846
View File
@@ -27,6 +27,17 @@ Why these particular pins (the phase's drift risks, in order):
call the SAME globals, in the SAME order (hydrate → append the unlocated →
settle the code blocks, all inside the ``answer_image`` band and BEFORE the
source clusters), so one answer cannot render three different ways.
* **The mid-turn pass settles ONLY what it matches** (phase 192, task 02):
the additive ``verbatim_block`` frame carries ONE block, and
``hydrateVerbatimNow`` may replace only the pending placeholders that block
matches — every other placeholder STAYS ``.verbatim-pending`` for the
``done`` / failure path, because burning a fence for a block that has not
arrived yet is how a resolved panel turns into "I could not verify this".
* **One canonical locator, one panel** (phase 192, A3): a mid-turn block that
no fence named yet is appended at the answer position, and when the answer
fences it later the hydration drops that appended twin (``dropPlacedTwin``,
bounded by the ``.msg`` wrap) — the scan is PER LOCATOR, so two honest
ranges of one file stay two panels.
* **No panel is ever left lying.** Every terminal path — the ``error`` frame,
the 300 s guard, the Stop action, the failed-turn funnel — settles the
leftover placeholders; nothing reads "Retrieving the exact text…" after the
@@ -43,6 +54,7 @@ import re
import shutil
import subprocess
from pathlib import Path
from typing import Any
import pytest
@@ -80,6 +92,7 @@ COPY_ARIA = "Copy ${shown} lines from ${identity}"
PUBLIC_GLOBALS = (
"buildVerbatimPanel",
"hydrateVerbatim",
"hydrateVerbatimNow", # phase 192 (task 02): the MID-TURN pass
"appendUnlocatedVerbatim",
"settleVerbatimPlaceholders",
"enhanceAnswerCodeBlocks",
@@ -112,6 +125,28 @@ def _body(js: str, name: str) -> str:
return _fn(js, name)
def _doc(js: str, name: str) -> str:
"""The ``/* … */`` doc comment that sits directly above
``function <name>`` — the house docstring-pinning convention for the
clients: the prose IS the contract record, so a refactor that "simplifies"
the rule away has to delete the sentence in the same change."""
start = js.find(f"\nfunction {name}(")
assert start != -1, f"function {name}() must exist"
open_ = js.rfind("/*", 0, start)
close = js.find("*/", open_)
assert open_ != -1 and close != -1 and close < start, (
f"{name}() must carry the doc comment above it"
)
return js[open_ : close]
def _flow(text: str) -> str:
"""Comment prose as one line — the pins quote the SENTENCE, not its
wrapping (the `` * `` line leaders come out with the wrap)."""
lines = [re.sub(r"^\s*\*+\s?", " ", line) for line in text.splitlines()]
return " ".join(" ".join(lines).split())
# ---------------------------------------------------------------------------
# the client ships, and ships in the right shape
# ---------------------------------------------------------------------------
@@ -346,6 +381,134 @@ def test_hydration_is_idempotent_and_never_throws_on_garbage() -> None:
assert "Array.isArray(blocks)" in guard, "a missing frame key is not an error"
def test_midturn_hydration_settles_only_the_placeholders_one_block_matches() -> None:
"""Phase 192 (task 02, A2) — the heart of the mid-turn pass. The additive
``verbatim_block`` frame carries ONE block, and ``hydrateVerbatimNow``:
* walks the SAME pending set as the done pass
(``.verbatim-pending[data-bor-locator]``, the attribute ``markdown.js``
normalized — no second grammar);
* matches through the ONE rule (``matchVerbatimBlock`` with a one-block
list) so the mid-turn pass and the ``done`` pass can never disagree
about which fence this block owns;
* replaces ONLY the matched nodes (``buildVerbatimPanel``) — the unmatched
stay ``.verbatim-pending``, NEVER the honest unresolved state, which is
the done/failure path's job (``settleVerbatimPlaceholders``);
* returns the number settled (``0`` is the caller's signal to append);
* is total without a ``try`` — a missing root, a root that is not an
element, or a frame that is not a block all return ``0``."""
js = _text(VERBATIM_JS)
fn = _fn(js, "hydrateVerbatimNow")
assert '.verbatim-pending[data-bor-locator]"' in fn, (
"the mid-turn walk is the done path's pending set"
)
assert 'node.getAttribute("data-bor-locator")' in fn, (
"the key is the attribute markdown.js wrote — never re-derived"
)
assert "matchVerbatimBlock(" in fn, "the ONE match rule, not a mid-turn variant"
assert "const list = [block]" in fn, "the one-block frame rides the done list shape"
assert "replaceWith(buildVerbatimPanel(" in fn, "only a MATCH builds a panel"
assert "buildVerbatimUnresolved" not in fn, (
"an unmatched fence stays pending mid-turn — settling is the terminal path's job"
)
assert "if (!matched) return" in fn, "the unmatched node is left in place"
assert "settled += 1" in fn and "return settled" in fn, (
"the count is the caller's append/no-append signal"
)
assert 'typeof root.querySelectorAll !== "function"' in fn, "a missing root is 0"
assert "isVerbatimBlock(block)" in fn, "a malformed frame settles NOTHING"
assert "try {" not in fn, "the guards make it total — no exception path"
assert "innerHTML" not in fn
def test_hydration_drops_the_midturn_twin_and_never_an_unresolved_one() -> None:
"""Phase 192 (A3): a block that arrived mid-turn before its fence was
appended at the answer position; when its fence hydrates, that twin is
removed so ONE locator keeps ONE panel and the panel sits where the model
fenced it. The shape of the scan is the contract:
* bounded by the message wrap (the appended panel lives in ``.msg-body``,
beside the bubble — a bubble-local walk would miss it);
* ``.verbatim-pending`` and ``.verbatim-unresolved`` are NOT placements
(the ``appendUnlocatedVerbatim`` exclusion, shared);
* the key is the ONE ``verbatimPlacedKey`` derivation (the same one the
append-side scan reads — two derivations is how the dedupe drifts);
* it compares the canonical LOCATOR (never an identity — two ranges of one
file are two panels) and never removes the placeholder being hydrated."""
js = _text(VERBATIM_JS)
twin = _fn(js, "dropPlacedTwin")
assert 'host.querySelectorAll(".verbatim[data-bor-locator]")' in twin
assert 'el.classList.contains("verbatim-pending")' in twin
assert 'el.classList.contains("verbatim-unresolved")' in twin
assert 'el.getAttribute("data-bor-locator") !== placedKey' in twin, (
"PER LOCATOR — an identity scan would collapse two honest ranges"
)
assert "verbatimSplitLocator" not in twin and "verbatimIdentity(" not in twin, (
"no identity fallback in the twin scan"
)
assert "if (el === keep) return" in twin, "the hydrating placeholder is never the twin"
assert "el.remove()" in twin
hydrate = _fn(js, "hydrateVerbatim")
assert 'root.closest(".msg")' in hydrate, "the scan is bounded by the message wrap"
assert "if (block) dropPlacedTwin(host, block, node);" in hydrate, (
"only a MATCHED placeholder drops a twin"
)
# The ONE key derivation, read by both dedupe sides.
assert js.count("function verbatimPlacedKey(") == 1
assert "verbatimPlacedKey(block)" in _fn(js, "appendUnlocatedVerbatim")
#: The sentences the phase-192 (A3) prose must keep carrying. They are the
#: only place a reader of ``verbatim.js`` learns the dedupe's two rules, and
#: the fastest way to lose a rule is a refactor that drops its documentation
#: first and its code second.
TWIN_DOC_SENTENCES = (
# PER LOCATOR, never per identity — the two-ranges-stay clause.
"PER LOCATOR, never per identity: two different ranges of one file are"
" two honest panels",
# The in-text panel wins over the mid-turn appended twin.
"The IN-TEXT panel wins",
)
def test_the_twin_dedupe_docstring_keeps_the_per_locator_clause() -> None:
"""Phase 192 (task 03, A3): the prose is part of the contract. The
``dropPlacedTwin`` docstring names the per-LOCATOR rule (and the
two-honest-ranges clause that follows from it) and why the in-text panel
wins; ``hydrateVerbatim``'s docstring says the matched placeholder drops
that twin ("one locator, one panel, and it sits where the model fenced
it"); and the module header states the same rule for whoever reads the
file top-down."""
js = _text(VERBATIM_JS)
twin = _flow(_doc(js, "dropPlacedTwin"))
for sentence in TWIN_DOC_SENTENCES:
assert sentence in twin, f"the docstring lost: {sentence}"
assert "appendUnlocatedVerbatim" in twin, (
"the placement exclusion is documented as SHARED with the append side"
)
hydrate = _flow(_doc(js, "hydrateVerbatim"))
assert "dropPlacedTwin" in hydrate
assert "one locator, one panel, and it sits where the model fenced it" in hydrate
header = _flow(js[: js.find("*/")])
assert "ONE CANONICAL LOCATOR, ONE PANEL (phase 192)" in header
assert "two honest ranges of one file stay two panels" in header
def test_the_twin_scan_bounds_to_the_msg_wrap_or_the_root() -> None:
"""The scan's host is the message wrap (the appended twin lives in
``.msg-body``, beside the bubble), and when a root has no ``.msg``
ancestor the ROOT bounds it — a missing wrap narrows the scan, it never
switches the dedupe off."""
hydrate = _fn(_text(VERBATIM_JS), "hydrateVerbatim")
assert 'root.closest(".msg")' in hydrate
assert 'root.closest(".msg") : null) || root' in hydrate, (
"no .msg ancestor → the root bounds the scan"
)
assert "document.querySelectorAll" not in hydrate, (
"never a page-wide scan — one message's twin, one message's panels"
)
def test_unlocated_blocks_land_in_the_answer_image_band() -> None:
"""The blocks no placeholder matched (a ``show_text`` call the prose never
named) append to ``.msg-body`` — the same band as the answer image, below
@@ -458,6 +621,50 @@ def test_the_done_handler_hydrates_before_the_source_clusters() -> None:
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatim(") :][:200]
def _midturn_branch(js: str) -> str:
start = js.index('ev.type === "verbatim_block"')
end = js.index('ev.type === "error"', start)
return js[start:end]
def test_the_midturn_frame_handler_hydrates_in_place_and_appends_only_on_zero() -> None:
"""Phase 192 (task 02, A2): the live turn's ``verbatim_block`` branch.
The shape IS the behaviour: hydrate the pending fence FIRST, and reach
for ``appendUnlocatedVerbatim`` ONLY on the zero-match path — a fence that
is already on screen keeps its panel where the model put it, and only a
block no placeholder names lands at the answer position (the
``answer_image`` band). The branch sits in the same ``readSSE`` chain, after
``tool_result`` (the server's emission order) and before ``error``.
And there is NO client-side kill switch: the server's
``BOR_ANSWER_TEXT`` gate is the only gate, so a disabled turn simply sends
no frame (a second, client-side settings check is a drift risk, exactly
the phase's reasoning for keeping the gate server-side)."""
js = _text(APP_JS)
branch = _midturn_branch(js)
assert "hydrateVerbatimNow(" in branch, "the mid-turn pass settles the fence"
assert "appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS)" in branch, (
"the unmatched block lands at the answer position, ONE opts object"
)
assert re.search(r"if \(\w+ === 0\)\s*appendUnlocatedVerbatim\(", branch), (
"the append is the ZERO-match path only — never alongside an in-place panel"
)
order = [branch.index("hydrateVerbatimNow("), branch.index("appendUnlocatedVerbatim(")]
assert order == sorted(order), "hydrate in place FIRST, append only if nothing matched"
assert "VERBATIM_LINK_OPTS" in branch[branch.index("hydrateVerbatimNow(") :][:200], (
"the mid-turn panel navigates exactly like the done panel (chip parity)"
)
# No second gate: nothing in the branch reads settings / a flag / the
# done frame's key — the arrival of the frame is the whole condition.
code = _strip_comments(branch)
for gate in ("answer_text", "answerText", "settings", "await ", "fetch("):
assert gate not in code, f"{gate} looks like a client-side gate (server-only)"
# The frame type is the ONE new thing the live turn knows (the server
# sends it between `tool_result` and the next round).
assert js.index('ev.type === "tool_result"') < js.index('ev.type === "verbatim_block"')
def test_the_answer_renderers_opt_in_and_nothing_else_does() -> None:
"""The per-delta write and the ``done``-time re-render both pass
``{ verbatim: true }`` (the third site is the restore renderer, pinned in
@@ -865,3 +1072,642 @@ def test_malformed_blocks_are_dropped_and_never_throw(tmp_dir: Path) -> None:
assert out["identity"] == {"ok": True, "value": "homelab/runbooks/a.md"}
assert out["unresolved_copy"] == {"ok": True, "value": UNRESOLVED_COPY}
assert out["badge_title"] == {"ok": True, "value": BADGE_TITLE}
# ---------------------------------------------------------------------------
# node: the MID-TURN pass (phase 192, task 02), run against the real file on
# a minimal DOM — the marker pins above prove the SHAPE, these prove the
# behaviour the E2E (task 04) will show in a browser: matched fences settle in
# place, everything else stays pending, a bad frame settles nothing.
# ---------------------------------------------------------------------------
#: The minimal DOM + the fixtures (``pendingBubble`` / ``shownBlock`` /
#: ``snapshot``). ``querySelectorAll`` understands the ONE compound selector
#: the client walks (``.a.b[attr]``) and throws on anything else, so a
#: selector drift in the client fails loudly instead of matching nothing.
_DOM_LIB = """
class El {
constructor(tag) {
this.tag = tag;
this.children = [];
this.parent = null;
this.attrs = {};
this.dataset = {};
this.classes = new Set();
this._own = "";
}
get className() { return [...this.classes].join(" "); }
set className(v) { this.classes = new Set(String(v).split(/\\s+/).filter(Boolean)); }
get classList() {
return { contains: (c) => this.classes.has(c), add: (c) => this.classes.add(c) };
}
setAttribute(k, v) { this.attrs[k] = String(v); }
addEventListener(ev, fn) { (this.listeners = this.listeners || {})[ev] = fn; }
getAttribute(k) {
if (Object.prototype.hasOwnProperty.call(this.attrs, k)) return this.attrs[k];
if (k.startsWith("data-")) {
const camel = k.slice(5).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
if (Object.prototype.hasOwnProperty.call(this.dataset, camel)) {
return String(this.dataset[camel]);
}
}
return null;
}
appendChild(c) { c.parent = this; this.children.push(c); return c; }
append(...cs) { for (const c of cs) this.appendChild(c); }
insertBefore(box, ref) {
const i = this.children.indexOf(ref);
this.children.splice(i < 0 ? this.children.length : i, 0, box);
box.parent = this;
return box;
}
replaceWith(node) {
if (!this.parent) throw new Error("replaceWith on a detached node");
const i = this.parent.children.indexOf(this);
this.parent.children[i] = node;
node.parent = this.parent;
this.parent = null;
}
remove() {
if (!this.parent) return;
const i = this.parent.children.indexOf(this);
if (i >= 0) this.parent.children.splice(i, 1);
this.parent = null;
}
closest(sel) {
let cur = this;
while (cur) {
if (matches(cur, sel)) return cur;
cur = cur.parent;
}
return null;
}
querySelectorAll(sel) { return descendants(this).filter((el) => matches(el, sel)); }
querySelector(sel) { return descendants(this).find((el) => matches(el, sel)) || null; }
get textContent() {
if (!this.children.length) return this._own;
return this.children.map((c) => c.textContent).join("");
}
set textContent(v) { this.children = []; this._own = String(v); }
}
function descendants(el, out) {
const list = out || [];
for (const c of el.children) { list.push(c); descendants(c, list); }
return list;
}
function matches(el, sel) {
const parts = sel.match(/\\.[A-Za-z0-9_-]+|\\[[^[\\]]+\\]/g) || [];
if (!parts.length) throw new Error("unsupported selector: " + sel);
return parts.every((p) =>
p.startsWith(".")
? el.classes.has(p.slice(1))
: el.getAttribute(p.slice(1, -1)) !== null
);
}
globalThis.document = {
createElement: (t) => new El(t),
createTextNode: (t) => {
const node = new El("#text");
node._own = String(t);
return node;
},
querySelectorAll: () => [],
body: new El("body"),
};
function pendingBubble(locators) {
const bubble = new El("div");
bubble.classes.add("bubble");
for (const loc of locators) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = loc;
bubble.appendChild(node);
}
return bubble;
}
function shownBlock(over) {
return Object.assign({
source: "homelab",
path: "runbooks/a.md",
title: "Runbook A",
start_line: 12,
end_line: 40,
total_lines: 210,
text: "line twelve\\nline thirteen",
truncated: false,
locator: "homelab/runbooks/a.md#12-40",
}, over || {});
}
function pendingWrap(locators) {
const msg = new El("div");
msg.className = "msg brain";
const body = new El("div");
body.className = "msg-body";
const bubble = new El("div");
bubble.className = "bubble";
msg.appendChild(body);
body.appendChild(bubble);
for (const loc of locators) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = loc;
bubble.appendChild(node);
}
return msg;
}
function findClass(root, cls) {
return descendants(root).find((el) => el.classes.has(cls)) || null;
}
function inBubble(el) {
let cur = el.parent;
while (cur) {
if (cur.classes.has("bubble")) return true;
cur = cur.parent;
}
return false;
}
/* The RESOLVED panels in document order — a pending or unresolved node is
* NOT a placement (appendUnlocatedVerbatim's own convention), which is what
* makes this the "one locator, one panel" reading. */
function placed(msg) {
return descendants(msg)
.filter((el) => el.classes.has("verbatim"))
.filter((el) => !el.classes.has("verbatim-pending"))
.filter((el) => !el.classes.has("verbatim-unresolved"))
.map((el) => ({
locator: el.getAttribute("data-bor-locator"),
where: inBubble(el) ? "bubble" : "body",
}));
}
/* The fence the model writes in a LATER round (the mid-turn frame arrived
* before it): a fresh pending node inside the bubble. */
function addFence(msg, locator) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = locator;
msg.querySelector(".bubble").appendChild(node);
return node;
}
/* The same later-round fence inside a BARE bubble (no `.msg` wrap) — the arm
* that proves the ROOT bounds the twin scan when there is no message wrap. */
function bareFence(bubble, locator) {
const node = new El("div");
node.className = "verbatim verbatim-pending";
node.dataset.borLocator = locator;
bubble.appendChild(node);
return node;
}
function snapshot(root) {
return descendants(root)
.filter((el) => el.classes.has("verbatim"))
.map((el) => {
const caption = findClass(el, "verbatim-caption");
const pre = findClass(el, "verbatim-text");
const link = findClass(el, "verbatim-open");
return {
pending: el.classes.has("verbatim-pending"),
unresolved: el.classes.has("verbatim-unresolved"),
locator: el.getAttribute("data-bor-locator"),
caption: caption ? caption.textContent : null,
text: pre ? pre.textContent : null,
href: link ? link.href : null,
};
});
}
module.exports = {
El, pendingBubble, pendingWrap, shownBlock, snapshot, placed, addFence,
bareFence, findClass, descendants,
};
"""
_DOM_HARNESS = """
const fs = require("fs");
const src = fs.readFileSync(process.argv[2], "utf8");
const dom = require(process.argv[3]); // installs globalThis.document
const cases = JSON.parse(fs.readFileSync(process.argv[4], "utf8"));
const panel = new Function(
src +
"\\nreturn { hydrateVerbatimNow, hydrateVerbatim, appendUnlocatedVerbatim," +
" buildVerbatimPanel, isVerbatimBlock };"
)();
const scope = Object.assign({}, panel, dom);
const out = {};
for (const [name, body] of Object.entries(cases)) {
try {
const fn = new Function("api", "with (api) { return (" + body + "); }");
out[name] = { ok: true, value: fn(scope) };
} catch (err) {
out[name] = { ok: false, error: String(err && err.message ? err.message : err) };
}
}
process.stdout.write(JSON.stringify(out));
"""
#: The block a mid-turn ``verbatim_block`` frame carries in these cases.
SHOWN_BLOCK_LOCATOR = "homelab/runbooks/a.md#12-40"
#: The OTHER fence on screen mid-answer — the one this frame must NOT touch.
OTHER_PENDING_LOCATOR = "homelab/runbooks/b.md#1-5"
#: The caption the panel must carry in every arm: the SERVER's extent (the
#: phase-182 contract), never the range the model typed into its fence.
CAPTION_SERVER_EXTENT = "homelab/runbooks/a.md \u00b7 lines 12\u201340 of 210"
def _node_dom(cases: dict[str, str], tmp_dir: Path) -> dict[str, Any]:
"""Run *cases* (JS expressions with the panel API + the DOM fixtures in
scope) under node against the real ``verbatim.js`` on the minimal DOM.
``Any`` on purpose: the DOM cases return nested objects/arrays, and the
assertions below walk them (``value["nodes"][0]["caption"]``)."""
lib = tmp_dir / "bor_dom.cjs"
lib.write_text(_DOM_LIB, encoding="utf-8")
script = tmp_dir / "bor_dom_harness.cjs"
script.write_text(_DOM_HARNESS, encoding="utf-8")
case_file = tmp_dir / "dom_cases.json"
case_file.write_text(json.dumps(dict(cases)), encoding="utf-8")
proc = subprocess.run(
["node", str(script), str(VERBATIM_JS), str(lib), str(case_file)],
capture_output=True,
text=True,
timeout=60,
check=False,
)
assert proc.returncode == 0, proc.stderr
return json.loads(proc.stdout)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_pass_settles_the_matched_fence_and_leaves_the_rest(tmp_dir: Path) -> None:
"""The behaviour of ``hydrateVerbatimNow`` against the real file: the fence
whose canonical locator the frame carries settles IN PLACE with the
server's caption and bytes; the next fence in the same answer is NOT
touched (no panel, no unresolved sentence — it stays a promise the ``done``
frame still has to keep)."""
out = _node_dom(
{
"exact": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"wrong_range_uses_the_servers_extent": """(() => {
const bubble = pendingBubble(["homelab/runbooks/a.md#99-120"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {});
return { n, nodes: snapshot(bubble) };
})()""",
"two_fences_one_locator": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, pending: snapshot(bubble).filter((x) => x.pending).length }};
}})()""",
"opts_thread_through": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{
documentHref: (s, p) => "HOOK/" + s + "/" + p,
}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
},
tmp_dir,
)
exact = out["exact"]
assert exact["ok"] is True, exact
assert exact["value"]["n"] == 1, "exactly the matching placeholder settles"
settled, untouched = exact["value"]["nodes"]
assert settled == {
"pending": False,
"unresolved": False,
"locator": SHOWN_BLOCK_LOCATOR,
"caption": CAPTION_SERVER_EXTENT,
"text": "line twelve\nline thirteen",
"href": "/document.html?source=homelab&path=runbooks%2Fa.md",
}, "the panel is the done path's panel, built from the SERVER's bytes"
assert untouched == {
"pending": True,
"unresolved": False,
"locator": OTHER_PENDING_LOCATOR,
"caption": None,
"text": None,
"href": None,
}, "a fence this block does not name STAYS pending"
# The model's range was wrong: the identity fallback still applies, and
# the caption / data attribute carry the SERVER's extent (mid-turn and
# done can never disagree — the phase-182 contract).
fallback = out["wrong_range_uses_the_servers_extent"]
assert fallback["ok"] is True, fallback
assert fallback["value"]["n"] == 1
assert fallback["value"]["nodes"][0]["locator"] == SHOWN_BLOCK_LOCATOR
assert fallback["value"]["nodes"][0]["caption"] == CAPTION_SERVER_EXTENT
dupes = out["two_fences_one_locator"]
assert dupes["ok"] is True, dupes
assert dupes["value"]["n"] == 2 and dupes["value"]["pending"] == 0
hooked = out["opts_thread_through"]
assert hooked["ok"] is True, hooked
assert hooked["value"]["n"] == 1
assert hooked["value"]["nodes"][0]["href"] == "HOOK/homelab/runbooks/a.md", (
"opts (the chat's modal hook) reach the mid-turn panel"
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_pass_is_total_on_garbage_and_no_match(tmp_dir: Path) -> None:
"""Nothing a mid-turn frame can carry settles a fence the block does not
name \u2014 and nothing throws: no match, a malformed frame (the ``isVerbatimBlock``
guard), a null root, a root that is not an element, no root at all. The
pending node stays ``.verbatim-pending`` in every arm (NEVER the honest
unresolved sentence \u2014 that is the terminal path's verdict, not this one)."""
out = _node_dom(
{
"other_file": f"""(() => {{
const bubble = pendingBubble(["{OTHER_PENDING_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, shownBlock(), {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"malformed_no_source": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, {{ path: "runbooks/a.md", text: "x" }}, {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"malformed_not_an_object": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}"]);
const n = hydrateVerbatimNow(bubble, "verbatim", {{}});
return {{ n, nodes: snapshot(bubble) }};
}})()""",
"null_root": "hydrateVerbatimNow(null, shownBlock(), {})",
"plain_object_root": "hydrateVerbatimNow({}, shownBlock(), {})",
"undefined_root": "hydrateVerbatimNow(undefined, shownBlock(), undefined)",
},
tmp_dir,
)
for name in ("other_file", "malformed_no_source", "malformed_not_an_object"):
case = out[name]
assert case["ok"] is True, (name, case)
assert case["value"]["n"] == 0, name
node = case["value"]["nodes"][0]
assert node["pending"] is True and node["unresolved"] is False, (
f"{name}: the fence stays a promise, never a burned placeholder"
)
assert node["text"] is None, f"{name}: no panel, so no bytes on screen"
for name in ("null_root", "plain_object_root", "undefined_root"):
assert out[name] == {"ok": True, "value": 0}, name
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_midturn_panel_survives_the_done_pass(tmp_dir: Path) -> None:
"""The parity that keeps the phase additive: the mid-turn pass runs, then
the ``done`` frame arrives with the SAME block. ``hydrateVerbatim`` only
touches ``.verbatim-pending``, so the panel the mid-turn pass built is the
SAME node afterwards (its ``Copied`` state and scroll position survive),
and the unrelated fence settles to the honest sentence."""
out = _node_dom(
{
"then_done": f"""(() => {{
const bubble = pendingBubble(["{SHOWN_BLOCK_LOCATOR}", "{OTHER_PENDING_LOCATOR}"]);
const n1 = hydrateVerbatimNow(bubble, shownBlock(), {{}});
const midturnPanel = findClass(bubble, "verbatim");
const n2 = hydrateVerbatim(bubble, [shownBlock()], {{}});
const nodes = snapshot(bubble);
return {{
n1,
n2,
sameNode: findClass(bubble, "verbatim") === midturnPanel,
nodes,
}};
}})()""",
},
tmp_dir,
)
case = out["then_done"]
assert case["ok"] is True, case
value = case["value"]
assert value["n1"] == 1 and value["n2"] == 1
assert value["sameNode"] is True, (
"the done pass re-runs over the mid-turn panel without rebuilding it"
)
settled, unresolved = value["nodes"]
assert settled["pending"] is False and settled["locator"] == SHOWN_BLOCK_LOCATOR
assert unresolved["pending"] is False and unresolved["unresolved"] is True, (
"the fence no frame ever resolved lands on the honest state at done"
)
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_one_locator_one_panel_across_the_midturn_and_done_passes(tmp_dir: Path) -> None:
"""The duplicate the mid-turn landing would otherwise leave behind (the
phase's A3): a ``show_text`` block arrives BEFORE the answer fences it, so
the mid-turn pass appends it at the answer position; the fence lands in a
later round and hydrates at ``done``. Exactly ONE panel remains and it sits
IN THE BUBBLE, where the model put the fence.
The two arms that must NOT collapse: two different ranges of one file are
two honest panels (the scan is per canonical locator), and a block the
answer never fences keeps its single appended panel."""
out = _node_dom(
{
"fence_arrives_later": """(() => {
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
const mid = hydrateVerbatimNow(bubble, shownBlock(), {});
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
addFence(msg, "homelab/runbooks/a.md#99-120");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { mid, appended, settled, panels: placed(msg) };
})()""",
"two_ranges_one_file": """(() => {
const b1 = shownBlock();
const b2 = shownBlock({ start_line: 10, end_line: 14,
locator: "homelab/runbooks/a.md#10-14" });
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
appendUnlocatedVerbatim(msg, [b1], {});
addFence(msg, "homelab/runbooks/a.md#10-14");
addFence(msg, "homelab/runbooks/a.md#12-40");
hydrateVerbatim(bubble, [b1, b2], {});
return { panels: placed(msg) };
})()""",
"never_fenced_keeps_its_panel": """(() => {
const msg = pendingWrap([]);
const bubble = msg.querySelector(".bubble");
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
const again = appendUnlocatedVerbatim(msg, [shownBlock()], {});
return { appended, settled, again, panels: placed(msg) };
})()""",
"repeated_fence_shows_the_bytes_once": """(() => {
const msg = pendingWrap(["homelab/runbooks/a.md#12-40",
"homelab/runbooks/a.md#12-40"]);
const bubble = msg.querySelector(".bubble");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { settled, panels: placed(msg) };
})()""",
"unresolved_twin_is_not_a_placement": """(() => {
const msg = pendingWrap(["homelab/runbooks/a.md#1-2"]);
const bubble = msg.querySelector(".bubble");
hydrateVerbatim(bubble, [], {}); // the honest state, same locator
const appended = appendUnlocatedVerbatim(msg, [shownBlock()], {});
return { appended, panels: placed(msg) };
})()""",
},
tmp_dir,
)
fenced = out["fence_arrives_later"]
assert fenced["ok"] is True, fenced
assert fenced["value"] == {
"mid": 0,
"appended": 1,
"settled": 1,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
}, "the fence wins: one panel, in the answer, the mid-turn twin gone"
ranges = out["two_ranges_one_file"]
assert ranges["ok"] is True, ranges
assert ranges["value"]["panels"] == [
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
], "two ranges of one file are TWO panels — the scan is per locator"
unfenced = out["never_fenced_keeps_its_panel"]
assert unfenced["ok"] is True, unfenced
assert unfenced["value"] == {
"appended": 1,
"settled": 0,
"again": 0,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}],
}, "a block no fence names keeps exactly its one answer-position panel"
repeated = out["repeated_fence_shows_the_bytes_once"]
assert repeated["ok"] is True, repeated
assert repeated["value"]["settled"] == 2
assert repeated["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
], "the same fence twice shows those bytes once"
unresolved = out["unresolved_twin_is_not_a_placement"]
assert unresolved["ok"] is True, unresolved
assert unresolved["value"]["appended"] == 1, (
"an unresolved node is not a placement, so the block still lands"
)
assert unresolved["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
]
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_the_twin_scan_bounds_to_the_message_it_hydrates(tmp_dir: Path) -> None:
"""The dedupe's BOUND, run against the real file (task 03): the scan reads
the message the hydration belongs to.
* a bare bubble with no ``.msg`` ancestor still dedupes — the ``|| root``
fallback means "no wrap" narrows the scan to the root, it never turns
the dedupe off (two panels for one locator is the bug either way);
* a second range of the same file keeps its own panel under that fallback
too (per locator, never per identity — the fallback is not a wider scan);
* a twin that belongs to ANOTHER message survives: hydrating one answer
never reaches into the next one's panels."""
out = _node_dom(
{
"bare_bubble_dedupes_too": """(() => {
const bubble = pendingBubble([]);
const appended = appendUnlocatedVerbatim(bubble, [shownBlock()], {});
bareFence(bubble, "homelab/runbooks/a.md#12-40");
const settled = hydrateVerbatim(bubble, [shownBlock()], {});
return { appended, settled, panels: placed(bubble) };
})()""",
"bare_bubble_two_ranges_stay_two": """(() => {
const other = shownBlock({ start_line: 10, end_line: 14,
locator: "homelab/runbooks/a.md#10-14" });
const bubble = pendingBubble([]);
appendUnlocatedVerbatim(bubble, [shownBlock()], {});
bareFence(bubble, "homelab/runbooks/a.md#10-14");
const settled = hydrateVerbatim(bubble, [shownBlock(), other], {});
return { settled, panels: placed(bubble) };
})()""",
"another_message_is_out_of_bounds": """(() => {
const here = pendingWrap(["homelab/runbooks/a.md#12-40"]);
const neighbour = pendingWrap([]);
appendUnlocatedVerbatim(neighbour, [shownBlock()], {});
const settled = hydrateVerbatim(
here.querySelector(".bubble"), [shownBlock()], {});
return { settled, here: placed(here), neighbour: placed(neighbour) };
})()""",
},
tmp_dir,
)
bare = out["bare_bubble_dedupes_too"]
assert bare["ok"] is True, bare
assert bare["value"] == {
"appended": 1,
"settled": 1,
"panels": [{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}],
}, "no .msg wrap still means ONE panel for one locator"
ranges = out["bare_bubble_two_ranges_stay_two"]
assert ranges["ok"] is True, ranges
assert ranges["value"]["settled"] == 1
assert ranges["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
{"locator": "homelab/runbooks/a.md#10-14", "where": "bubble"},
], "the fallback bounds the scan — it does not widen it past the locator"
neighbour = out["another_message_is_out_of_bounds"]
assert neighbour["ok"] is True, neighbour
assert neighbour["value"]["settled"] == 1
assert neighbour["value"]["here"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"}
], "this answer hydrates to one panel"
assert neighbour["value"]["neighbour"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "body"}
], "the next answer's panel is not this scan's to remove"
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_a_stored_record_restores_one_panel_per_locator(tmp_dir: Path) -> None:
"""The restore/shared parity of the dedupe (task 03): both call sites run
the SAME two passes in the SAME order — hydrate the record's fences, then
append the blocks the answer never located. So a stored answer that repeats
one locator twice and additionally carries an unlocated block comes back as
exactly TWO panels: the locator once, IN the text, and the unlocated block
once, in the answer band. One reload, one render — the duplicate the
mid-turn frame could leave behind never reaches the record's reader."""
out = _node_dom(
{
"restore": f"""(() => {{
const msg = pendingWrap(["{SHOWN_BLOCK_LOCATOR}", "{SHOWN_BLOCK_LOCATOR}"]);
const bubble = msg.querySelector(".bubble");
const record = [
shownBlock(),
shownBlock({{
source: "homelab", path: "runbooks/c.md", title: "Runbook C",
start_line: 1, end_line: 3, total_lines: 9,
text: "c one\\nc two", locator: "homelab/runbooks/c.md#1-3",
}}),
];
const settled = hydrateVerbatim(bubble, record, {{}});
const appended = appendUnlocatedVerbatim(msg, record, {{}});
return {{ settled, appended, panels: placed(msg) }};
}})()""",
},
tmp_dir,
)
case = out["restore"]
assert case["ok"] is True, case
assert case["value"]["settled"] == 2, "every fence settles — nothing is left pending"
assert case["value"]["appended"] == 1, (
"only the block the answer never located lands in the answer band"
)
assert case["value"]["panels"] == [
{"locator": SHOWN_BLOCK_LOCATOR, "where": "bubble"},
{"locator": "homelab/runbooks/c.md#1-3", "where": "body"},
], "one locator, one panel — the same answer on every surface"
+114
View File
@@ -1401,3 +1401,117 @@ def test_saved_chat_total_budget_counts_verbatim_block_text() -> None:
assert "8,000,000-character budget" in exc.value.errors()[0]["msg"]
# 14 × 560 000 = 7 840 000 fits (plus the small text, still inside).
assert len(SavedChatCreate.model_validate({"messages": heavy[:-1]}).messages) == 14
# ---------------------------------------------------------------------------
# Phase 192 (task 01, A1): ChatVerbatimBlockEvent — the mid-turn twin of the
# done frame's ``answer_text`` item.
# ---------------------------------------------------------------------------
def test_chat_verbatim_block_event_shape() -> None:
"""The mid-turn frame's wire shape: ``type`` FIRST (the sibling events'
convention) then the exact :class:`VerbatimBlockRef` field set, the
``verbatim_block`` default, ``truncated`` defaulting False (the block was
not capped), and no ``model_config`` — the phase-95
:class:`ChatToolResultEvent` convention mirrored: an SSE frame is
server-built, so unknown keys are ignored rather than forbidden."""
from app.schemas import ChatVerbatimBlockEvent
ev = ChatVerbatimBlockEvent(**_verbatim_block())
dumped = ev.model_dump()
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
assert dumped == {"type": "verbatim_block", **_verbatim_block()}
minimal = ChatVerbatimBlockEvent(
source="Homelab",
path="a.md",
title="a.md",
start_line=1,
end_line=1,
total_lines=1,
text="one line",
locator="Homelab/a.md",
).model_dump()
assert minimal["truncated"] is False
assert minimal["type"] == "verbatim_block"
# The sibling convention mirrored EXACTLY: an SSE event model declares no
# ``extra="forbid"`` (the frame is server-built, so an unknown key is
# ignored rather than rejected — like ``ChatToolResultEvent``).
from app.schemas import ChatToolResultEvent
assert ChatVerbatimBlockEvent.model_config.get("extra") is None
assert ChatToolResultEvent.model_config.get("extra") is None
def test_chat_verbatim_block_event_field_set_matches_the_done_item() -> None:
"""The drift pin: the mid-turn frame carries EXACTLY the done frame's
item fields (plus ``type``) with the SAME caps — the one client
(``isVerbatimBlock`` + ``buildVerbatimPanel``) hydrates both from one
shape, so a field added to :class:`VerbatimBlockRef` without its twin
here fails this test."""
from app.schemas import ChatVerbatimBlockEvent
assert set(ChatVerbatimBlockEvent.model_fields) == set(
VerbatimBlockRef.model_fields
) | {"type"}
for name, ref_field in VerbatimBlockRef.model_fields.items():
mine = ChatVerbatimBlockEvent.model_fields[name]
assert mine.annotation == ref_field.annotation, name
assert mine.is_required() == ref_field.is_required(), name
assert mine.metadata == ref_field.metadata, name # the caps, verbatim
# The caps fire (the same limits, enforced on this model too).
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(text="x" * 600_001))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(source="s" * 121))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(path="p" * 1001))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(title="t" * 501))
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(locator="l" * 2101))
for field in ("start_line", "end_line", "total_lines"):
with pytest.raises(ValidationError):
ChatVerbatimBlockEvent(**_verbatim_block(**{field: -1}))
# ``isVerbatimBlock`` (frontend/assets/verbatim.js) needs exactly these
# three: a non-empty ``source`` string, a non-empty ``path`` string and a
# ``text`` string — so the mid-turn event is accepted as-is.
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
assert isinstance(dumped["source"], str) and dumped["source"]
assert isinstance(dumped["path"], str) and dumped["path"]
assert isinstance(dumped["text"], str)
def test_chat_verbatim_block_event_dump_keys_are_the_client_contract() -> None:
"""Phase 192 (task 03, A4): the key-set pin the ONE panel client leans on.
``frontend/assets/verbatim.js`` has no schema of its own: the
``isVerbatimBlock`` guard is a shape check and ``buildVerbatimPanel``
reads the fields by name — so the mid-turn frame is only renderable while its
``model_dump()`` key set is EXACTLY :class:`VerbatimBlockRef`'s key set
plus the ``type`` discriminator. A field added to one model and not the
other, a rename, or an ``exclude=`` on the dump all fail here (the client
would silently lose the caption, the extent, or the whole block).
"""
from app.schemas import ChatVerbatimBlockEvent
dumped = ChatVerbatimBlockEvent(**_verbatim_block()).model_dump()
assert set(dumped) == set(VerbatimBlockRef.model_fields) | {"type"}
# ``type`` FIRST and the ref's fields in declaration order — the sibling
# SSE events' convention, unchanged by the dump.
assert list(dumped) == ["type", *VerbatimBlockRef.model_fields]
# The three keys ``isVerbatimBlock`` cannot live without, plus the keys
# ``buildVerbatimPanel`` prints (the caption's extent and its open link).
for key in ("source", "path", "text", "title", "start_line", "end_line",
"total_lines", "truncated", "locator"):
assert key in dumped, f"the client reads {key} off both frames"
# The done item round-trips the SAME keys: the mid-turn frame is a copy of
# a done item, so the restore/shared hydration needs no second reader.
assert (
VerbatimBlockRef.model_validate(
{k: v for k, v in dumped.items() if k != "type"}
).model_dump()
== {k: v for k, v in dumped.items() if k != "type"}
)