phase: 182_verbatim_show_text
Build and Push Containers / build-and-push-app (push) Successful in 1m46s
Build and Push Containers / build-and-push-db (push) Successful in 12s

**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**

- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
2026-10-15 08:33:12 -04:00
parent 814a400e6d
commit b34b8ca672
104 changed files with 10852 additions and 203 deletions
+154 -6
View File
@@ -23,7 +23,7 @@ from app.config import Settings
from app.main import app as fastapi_app
from app.models import Document, KbOverview, QueryLog
from app.rag import agent as rag_agent
from app.rag.agent import AGENT_TOOLS
from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL
from app.rag.llm import StreamPiece, ToolCallPiece
from app.rag.prompts import SUGGEST_INTRO, build_deflect_prompt
from app.rag.retriever import RetrievedChunk, weak_hit_titles
@@ -1135,10 +1135,13 @@ def test_endpoint_grounded_turn_runs_agent_loop_with_tools(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 37: a grounded endpoint turn runs the agent loop — the
single no-tool-call request carries ``AGENT_TOOLS`` (the default
round cap keeps the tools offered), no ``tool`` frames stream, and
the ``done`` event is the plain retrieval shape (the tool-free
answer is byte-identical)."""
single no-tool-call request carries the offered tool list (the
default round cap keeps the tools offered), no ``tool`` frames
stream, and the ``done`` event is the plain retrieval shape (the
tool-free answer is byte-identical). Phase 182 (task 03) pin
update, deliberate: the offer is the three base tools plus
``show_text`` (``answer_text`` defaults ON), the base three staying
element-for-element ``AGENT_TOOLS`."""
_session, llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([_chunk(doc, 0.90)]))
@@ -1149,7 +1152,12 @@ def test_endpoint_grounded_turn_runs_agent_loop_with_tools(
assert frames[-1]["deflected"] is False
assert not any(f["type"] == "tool" for f in frames)
assert len(llm.seen) == 1
assert llm.seen_tools == [AGENT_TOOLS] # one request, tools offered
# Phase 182 (task 03): ``answer_text`` defaults ON — the offer is
# the base three (byte-identical) plus ``show_text``.
assert llm.seen_tools == [[*AGENT_TOOLS, SHOW_TEXT_TOOL]] # one request
offered = llm.seen_tools[0]
assert offered is not None
assert offered[:3] == AGENT_TOOLS # the base offer untouched
# The system prompt is the HIGH prompt with the <tools> instructions
# (phase 70: the harness-aligned ls/read/grep copy — new names in,
# old phase-37/68 names out).
@@ -1600,6 +1608,9 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
"total_ms=",
"retries=",
"scaffold_stripped=",
# phase 182 (task 05): the PLAN §9 additive tail — the blocks
# actually SENT this turn, last on the line.
"verbatim_blocks=",
)
idx = -1
for field in order:
@@ -1669,3 +1680,140 @@ def test_endpoint_log_line_records_suggested_after_summary_hits(
line = _last_log_line()
assert "thinking=- thinking_chars=" in line
assert "thinking=high" not in line
# ---------------------------------------------------------------------------
# Phase 182 (task 05): the per-turn log line's ``verbatim_blocks=N`` tail
# and the deflected turn's resolve-noting posture (D9). The count is the
# blocks ACTUALLY SENT (after the canonical-locator dedupe and the
# ``show_text_max_blocks`` cut), so the resolver is faked here — its own
# behaviour is pinned in ``tests/unit/test_verbatim_resolver.py`` and the
# real end-to-end fusion in ``tests/integration/test_chat_api.py``.
# ---------------------------------------------------------------------------
def _block(n: int) -> Any:
"""One fake resolved block (the resolver's return type — the counting
and the frame wiring are what these pins are about)."""
from app.rag.verbatim import VerbatimBlock
return VerbatimBlock(
source="Homelab",
path=f"quadlet/gufo.container{n}",
title=f"gufo{n}",
start_line=1,
end_line=4,
total_lines=12,
text=f"[Unit{n}\nExec=gufo",
truncated=False,
locator=f"Homelab/quadlet/gufo.container{n}#1-4",
)
def test_log_line_ends_with_verbatim_blocks_zero_on_a_plain_turn(
client: TestClient,
gate_env: tuple[_FakeSession, _CannedLLM],
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""A plain grounded turn (no tool blocks, no fence in the answer)
ends ``… scaffold_stripped=0 verbatim_blocks=0`` — the PLAN §9 field
is UNIFORM across turn shapes (the phase-67 ``retries=N`` pattern),
and the ``done`` frame carries no ``answer_text`` key (A9)."""
_session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([_chunk(doc, 0.30)]))
with caplog.at_level("INFO", logger="app.chat"):
frames = _ask(client, "How is my Kubernetes cluster set up?")
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines
assert lines[-1].endswith("scaffold_stripped=0 verbatim_blocks=0")
done = [f for f in frames if f.get("type") == "done"]
assert done and "answer_text" not in done[0]
def test_log_line_counts_the_verbatim_blocks_sent(
client: TestClient,
gate_env: tuple[_FakeSession, _CannedLLM],
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Two resolved blocks → ``verbatim_blocks=2`` AND both ride the
``done`` frame's ``answer_text`` in the resolver's order (the frame
and the line count the SAME list — never the raw tool-call count)."""
_session, _llm = gate_env
doc = _doc("Kubernetes Homelab Cluster", "TALOS_DOC_SENT")
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([_chunk(doc, 0.30)]))
monkeypatch.setattr(
chat_api, "resolve_answer_blocks", lambda *_a: [_block(1), _block(2)]
)
with caplog.at_level("INFO", logger="app.chat"):
frames = _ask(client, "How is my Kubernetes cluster set up?")
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and lines[-1].endswith("verbatim_blocks=2")
done = [f for f in frames if f.get("type") == "done"]
assert done
assert done[0]["answer_text"] == [
{
"source": "Homelab",
"path": "quadlet/gufo.container1",
"title": "gufo1",
"start_line": 1,
"end_line": 4,
"total_lines": 12,
"text": "[Unit1\nExec=gufo",
"truncated": False,
"locator": "Homelab/quadlet/gufo.container1#1-4",
},
{
"source": "Homelab",
"path": "quadlet/gufo.container2",
"title": "gufo2",
"start_line": 1,
"end_line": 4,
"total_lines": 12,
"text": "[Unit2\nExec=gufo",
"truncated": False,
"locator": "Homelab/quadlet/gufo.container2#1-4",
},
]
def test_deflected_turn_resolves_nothing_and_logs_zero(
client: TestClient,
gate_env: tuple[_FakeSession, _CannedLLM],
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""D9: a DEFLECTED turn resolves NOTHING even when its own answer
text carries a `` ```bor `` fence — a hallucinated locator must not
leak KB text through the turn that just disclaimed it. The resolver
is never called, the line reads ``verbatim_blocks=0`` and the frame
carries no ``answer_text`` key."""
_session, llm = gate_env
llm.answer = (
"I haven't done anything like that.\n"
"```bor Homelab/quadlet/gufo.container 1-4\n"
"Exec=/usr/bin/lie\n"
"```"
)
doc = _doc("Sourdough Starter", "BREAD_DOC_SENT")
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([_chunk(doc, 0.10)]))
calls: list[Any] = []
monkeypatch.setattr(
chat_api,
"resolve_answer_blocks",
lambda *_a: calls.append(1) or [], # pyright: ignore[reportFunctionMemberAccess]
)
with caplog.at_level("INFO", logger="app.chat"):
frames = _ask(client, "How do I bake sourdough bread?")
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and lines[-1].endswith("verbatim_blocks=0")
assert calls == [] # the resolver never ran on a deflected turn
done = [f for f in frames if f.get("type") == "done"]
assert done and done[0]["deflected"] is True
assert "answer_text" not in done[0]