phase: 95_read_truncation_cap

All gates are green. Final verification is complete — no fixes were needed; the phase is fully implemented and passing. **Phase 95 final verification report** - Verified all 3 tasks already implemented (task files in `complete/`): read cap + notice + `ToolResultPiece` (task 01), `tool_result` SSE + live/saved/shared marker (task 02), E2E suite (task 03). - Code checks passed: `BOR_READ_MAX_CHARS` (default 128 000, validator), pinned `READ_TRUNCATION_NOTICE` + `TRUNCATION_MARKER`, byte-identical at/under-cap read, holder `read_truncations` (no `tool_calls` impact), piece order (after `tool`, before next round), `ChatToolResultEvent`, `ToolCall` back-compat fields, `app.js` live/restore handlers, `shared.js`, `.truncated-note` CSS, `.env.example` entry, mock-LLM echo script. - **Tests:** `uv run pytest` → 1966 passed; `uv run pytest --cov=app --cov-report=term-missing` → all green, TOTAL **99%** (>90% gate). - **E2E:** `uv run pytest tests/e2e/test_read_truncation_cap.py -v --no-cov` → **3 passed** (frame order + live marker + LLM notice via echo; save→shared fidelity; short-read control). - **Regression (isolated):** `test_agent_document_tools` 4 ✓, `test_chat_history` 5 ✓, `test_share_chat` 4 ✓, `test_big_read_progress` 4 ✓, `test_stop_generation` 3 ✓. - **Lint/types:** `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings. **Completion criteria:** ① over-cap read → first-cap-chars + marker + pinned notice — ✓ (unit-pinned: at-cap/cap+1/notice tests); ② user marker live/saved/shared — ✓ (E2E + frontend tests); ③ at/under cap byte-identical, no frame — ✓ (unit + control E2E); ④ top-2 `<documents>` retrieval untouched — ✓ (`app/rag/retriever.py` unmodified vs HEAD); ⑤ suite green, >90% coverage, ruff+pyright clean — ✓; ⑥ no completed-phase behavior change — ✓ (all gates green; commit left to harness per pass rules). - No defects found; no changes made this pass. Next pending phase: none in `todo/` (96 is the next free number).
2026-09-11 03:42:51 -04:00
parent d4943b4822
commit bcaef800c5
36 changed files with 2836 additions and 43 deletions
@@ -41,7 +41,15 @@ def _source_ref() -> dict:


 def _tool_call() -> dict:
-    return {"name": "read", "argument": "Homelab/kubernetes.md"}
+    # Phase 95 (task 02): the truncation record rides the same entry —
+    # the CURRENT full shape (additive fields, defaults for a plain read).
+    return {
+        "name": "read",
+        "argument": "Homelab/kubernetes.md",
+        "truncated": False,
+        "chars_shown": None,
+        "chars_total": None,
+    }


 def _user_message(text: str = "How did I install k3s?") -> ChatMessage:
@@ -341,8 +349,22 @@ def test_realistic_bor_chat_v1_payload_round_trips() -> None:
            "suggestions": None,
            "thinking": "The kubernetes doc covers the cluster layout…",
            "tools": [
-                {"name": "read", "argument": "Homelab/kubernetes.md"},
-                {"name": "ls", "argument": None},
+                # Phase 95 (task 02): the current full shape — one
+                # truncated read (the marker record) + a plain ls.
+                {
+                    "name": "read",
+                    "argument": "Homelab/kubernetes.md",
+                    "truncated": True,
+                    "chars_shown": 128_000,
+                    "chars_total": 204_000,
+                },
+                {
+                    "name": "ls",
+                    "argument": None,
+                    "truncated": False,
+                    "chars_shown": None,
+                    "chars_total": None,
+                },
            ],
            "stopped": None,
        },
@@ -393,3 +415,143 @@ def test_realistic_payload_round_trips_through_update_model() -> None:
    }
    payload = SavedChatUpdate.model_validate({"messages": [msg]})
    assert payload.model_dump()["messages"] == [msg]
+
+
+# ---------------------------------------------------------------------------
+# Phase 95 (task 02): ToolCall truncation fields + ChatToolResultEvent
+# ---------------------------------------------------------------------------
+
+
+def test_tool_call_round_trips_with_truncation_fields() -> None:
+    """The CURRENT full shape (phase 95 task 02): a truncated read's
+    record (``truncated: True`` + the two non-negative counts) validates
+    and round-trips ``model_dump()`` unchanged — the save payload carries
+    it with zero other change (the UI re-renders the marker from it)."""
+    raw = {
+        "name": "read",
+        "argument": "Homelab/big.md",
+        "truncated": True,
+        "chars_shown": 128_000,
+        "chars_total": 204_000,
+    }
+    call = ToolCall.model_validate(raw)
+    assert call.truncated is True
+    assert call.chars_shown == 128_000
+    assert call.chars_total == 204_000
+    assert call.model_dump() == raw
+
+
+def test_tool_call_old_shape_validates_with_defaults() -> None:
+    """Backward-compat (the phase-50 rule): a saved chat written BEFORE
+    phase 95 — tool records without the truncation fields — validates
+    UNCHANGED: ``truncated`` defaults to False (the marker is absent),
+    the counts to None. No migration (``ChatMessage.tools`` is JSON);
+    only the dump gains the additive keys with their defaults."""
+    old_shape = {"name": "read", "argument": "Homelab/kubernetes.md"}
+    call = ToolCall.model_validate(old_shape)
+    assert call.truncated is False
+    assert call.chars_shown is None
+    assert call.chars_total is None
+    dumped = call.model_dump()
+    assert dumped["name"] == "read" and dumped["argument"] == "Homelab/kubernetes.md"
+    assert dumped["truncated"] is False
+    assert dumped["chars_shown"] is None and dumped["chars_total"] is None
+
+
+def test_tool_call_old_shape_message_still_round_trips_as_record() -> None:
+    """The record-level backward-compat: a pre-phase-95 brain message
+    (old-shape ``tools``) validates inside ``ChatMessage`` and dumps back
+    as a VALID record of the same shape (the frontend renders it without
+    the marker — ``truncated`` falsy)."""
+    old_message = {
+        "who": "brain",
+        "text": "You've got this!",
+        "sources": None,
+        "deflected": False,
+        "suggestions": None,
+        "thinking": None,
+        "tools": [
+            {"name": "read", "argument": "Homelab/kubernetes.md"},
+            {"name": "read_document", "argument": "Homelab/legacy.md"},
+        ],
+        "stopped": None,
+    }
+    msg = ChatMessage.model_validate(old_message)
+    assert all(t.truncated is False for t in (msg.tools or []))
+    # Re-validating the dump is a no-op (lossless record round-trip).
+    ChatMessage.model_validate(msg.model_dump())
+
+
+def test_tool_call_counts_reject_negative() -> None:
+    """Phase 83 bounds philosophy: the counts are non-negative
+    (``ge=0``) — a negative count is not a real record."""
+    with pytest.raises(ValidationError):
+        ToolCall.model_validate(
+            {
+                "name": "read",
+                "argument": "x",
+                "truncated": True,
+                "chars_shown": -1,
+                "chars_total": 5,
+            }
+        )
+    with pytest.raises(ValidationError):
+        ToolCall.model_validate(
+            {
+                "name": "read",
+                "argument": "x",
+                "truncated": True,
+                "chars_shown": 5,
+                "chars_total": -1,
+            }
+        )
+
+
+def test_chat_tool_result_event_shape() -> None:
+    """The A15 extension's wire shape (phase 95 task 02): the seventh,
+    OPTIONAL SSE event type — ``{type, name, argument, truncated,
+    chars_shown, chars_total}`` — with the pinned field order, the
+    ``tool_result`` default, ``truncated`` defaulting True (the emission
+    trigger), and the non-negative counts."""
+    from app.schemas import ChatToolResultEvent
+
+    ev = ChatToolResultEvent(
+        name="read",
+        argument="docs/big.md",
+        truncated=True,
+        chars_shown=128_000,
+        chars_total=204_000,
+    )
+    dumped = ev.model_dump()
+    assert list(dumped) == [
+        "type",
+        "name",
+        "argument",
+        "truncated",
+        "chars_shown",
+        "chars_total",
+    ]
+    assert dumped == {
+        "type": "tool_result",
+        "name": "read",
+        "argument": "docs/big.md",
+        "truncated": True,
+        "chars_shown": 128_000,
+        "chars_total": 204_000,
+    }
+    # The emission trigger defaults: the pump always builds the frame from
+    # a piece, so a frame that ever exists carries the truncation truth.
+    minimal = ChatToolResultEvent(
+        name="read", argument=None, chars_shown=0, chars_total=0
+    ).model_dump()
+    assert minimal["truncated"] is True
+
+
+def test_chat_tool_result_event_counts_reject_negative() -> None:
+    """The frame's counts are non-negative (``ge=0``), like the record's."""
+    from app.schemas import ChatToolResultEvent
+
+    with pytest.raises(ValidationError):
+        ChatToolResultEvent(name="read", argument="x", chars_shown=-1, chars_total=5)
+    with pytest.raises(ValidationError):
+        ChatToolResultEvent(name="read", argument="x", chars_shown=5, chars_total=-1)