phase: 182_verbatim_show_text
Build and Push Containers / build-and-push-app (push) Successful in 1m46s
Build and Push Containers / build-and-push-db (push) Successful in 12s

**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**

- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
2026-10-15 08:33:12 -04:00
parent 814a400e6d
commit b34b8ca672
104 changed files with 10852 additions and 203 deletions
+128 -3
View File
@@ -9,7 +9,7 @@ The pins (00_phase, Testing & Quality):
* the auth matrix — anonymous AND a signed-in token USER get 403 on ALL
four routes (the router-wide ``require_admin`` closes the surface on
every method), the admin 200s on all four;
* the shape — ``items`` length == ``len(Settings.model_fields)`` (78),
* the shape — ``items`` length == ``len(Settings.model_fields)`` (one per field),
groups in ``GROUPS`` order (the stable order the E2E pins against),
``totals.count`` / ``overridden`` correct, ``overlay_enabled`` true;
on an empty table a field whose ``BOR_*`` var the test env sets
@@ -152,7 +152,7 @@ def test_auth_matrix_all_four_routes(client: TestClient, db: Session) -> None:
def test_catalogue_shape_and_sources_on_empty_table(admin_client) -> None:
"""The whole catalogue, grouped and stable: 78 items (exactly
"""The whole catalogue, grouped and stable: one item per field (exactly
``Settings.model_fields``), groups in ``GROUPS`` order (each group a
contiguous block, the block order = ``GROUPS``), ``totals`` correct,
``overlay_enabled`` true. On an empty table: the env-var-set field
@@ -418,7 +418,7 @@ def test_kill_switch_disables_the_mechanism_not_the_panel(
def test_catalogue_stays_under_the_byte_budget(admin_client) -> None:
"""The whole catalogue stays under the phase-167 INFO bar (64 KB —
``BOR_LOG_LARGE_RESPONSE_BYTES``), so ``GET /api/admin/settings``
never floods the log (78 items ≈ 20 KB; the audit script re-verifies
never floods the log (the whole catalogue ≈ 20 KB; the audit script re-verifies
the 256 KB budget in task 06)."""
r = admin_client.get(API)
assert r.status_code == 200
@@ -708,6 +708,131 @@ def test_llm_summary_thinking_level_row_contract(
admin_client.delete(f"{API}/llm_summary_thinking_level") # no row survives
def test_verbatim_display_rows_contract(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 182 (task 01, LOCKED A8): the three verbatim-display rows are
present in the ``Agent tools`` group with the right labels and types
(``bool`` + two ``int``s, defaults ``true`` / ``12000`` / ``5``) and
behave through the generic overlay machinery:
(a) the GET rows carry group/label/type/env_var, are neither secret nor
restart-required (the chat request path reads them per turn — live);
(b) a PUT stores the RAW string in ``app_settings`` and the value
resolves live WITHOUT a restart;
(c) ``0`` is storable on both budgets — it is INERT, not an error
(``app_settings`` holds ``"0"``, the effective value is ``0``);
(d) a negative value is refused by the validate-before-store gate (A5)
with a 422 naming the field and NOTHING stored — the same rule that
refuses startup; a ``false`` for the master switch is storable
(the kill switch);
(e) a DELETE resets each to the env default;
(f) the non-admin 403 gate applies.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code defaults, no row in the table.
for var in (
"BOR_ANSWER_TEXT",
"BOR_SHOW_TEXT_MAX_CHARS",
"BOR_SHOW_TEXT_MAX_BLOCKS",
):
monkeypatch.delenv(var, raising=False)
invalidate_settings()
# (a) the rows.
r = admin_client.get(API)
assert r.status_code == 200, r.text
expect = {
"answer_text": ("Answer text", "bool", "true", "BOR_ANSWER_TEXT"),
"show_text_max_chars": (
"Show text max chars",
"int",
"12000",
"BOR_SHOW_TEXT_MAX_CHARS",
),
"show_text_max_blocks": (
"Show text max blocks",
"int",
"5",
"BOR_SHOW_TEXT_MAX_BLOCKS",
),
}
for key, (label, typ, default, envname) in expect.items():
item = _item(r.json(), key)
assert item["group"] == "Agent tools"
assert item["label"] == label
assert item["type"] == typ
assert item["env_var"] == envname
assert item["secret"] is False
assert item["masked"] is False
assert item["restart_required"] is False
assert item["overridable"] is True
assert item["value"] == default
assert item["default"] == default
assert item["source"] == "default" # env var unset, no row
assert item["has_override"] is False
assert item["db_value"] is None
# (b) PUT stores the raw string and resolves live, no restart.
for key, sent, resolved in (
("answer_text", "false", False),
("show_text_max_chars", "2000", 2000),
("show_text_max_blocks", "2", 2),
):
r = admin_client.put(f"{API}/{key}", json={"value": sent})
assert r.status_code == 200, r.text
assert r.json()["source"] == "db"
assert r.json()["has_override"] is True
assert r.json()["db_value"] == sent
assert db.execute(
text("SELECT value FROM app_settings WHERE key = :k"), {"k": key}
).scalar() == sent # stored raw, exactly as sent
assert getattr(get_settings(), key) == resolved # live, no restart
# (c) ``0`` is storable on both budgets (inert, not an error).
for key in ("show_text_max_chars", "show_text_max_blocks"):
r = admin_client.put(f"{API}/{key}", json={"value": "0"})
assert r.status_code == 200, r.text
assert r.json()["value"] == "0"
assert db.execute(
text("SELECT value FROM app_settings WHERE key = :k"), {"k": key}
).scalar() == "0"
assert getattr(get_settings(), key) == 0
# (d) a negative value is refused before anything is stored (A5) —
# the 422 names the field, the previous row is untouched.
for key in ("show_text_max_chars", "show_text_max_blocks"):
r = admin_client.put(f"{API}/{key}", json={"value": "-1"})
assert r.status_code == 422, r.text
assert key in r.json()["detail"]
assert db.execute(
text("SELECT value FROM app_settings WHERE key = :k"), {"k": key}
).scalar() == "0" # unchanged
assert getattr(get_settings(), key) == 0 # the live view is undisturbed
# (e) DELETE resets each to the env default (the vars are unset here,
# so the code default wins again).
for key, default in (
("answer_text", True),
("show_text_max_chars", 12_000),
("show_text_max_blocks", 5),
):
r = admin_client.delete(f"{API}/{key}")
assert r.status_code == 200, r.text
assert r.json()["has_override"] is False
assert r.json()["source"] == "default"
assert getattr(get_settings(), key) == default
assert db.execute(
text("SELECT count(*) FROM app_settings WHERE key = :k"), {"k": key}
).scalar() == 0
assert db.execute(text("SELECT count(*) FROM app_settings")).scalar() == 0
# (f) the non-admin 403 gate (asserted once for this family — the full
# four-route matrix is ``test_auth_matrix_all_four_routes``).
anon = TestClient(fastapi_app)
r = anon.put(f"{API}/answer_text", json={"value": "false"})
assert r.status_code == 403 and r.json() == {"detail": "admin only"}
def test_docs_token_is_write_only_masked_end_to_end(
admin_client: TestClient, db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
+70 -1
View File
@@ -13,7 +13,10 @@ the ``scripts/``-consumer surface (``import_extension_set``), and the
phase-172 thinking level (``llm_chat_thinking_level`` — the value the
LLM layer reads per request via ``get_settings()``; the off switch is
live-settable, A2) and its phase-178 summary twin (task 02:
``llm_summary_thinking_level``, default ``""`` = off, A2).
``llm_summary_thinking_level``, default ``""`` = off, A2), and the
phase-182 verbatim-display family (task 01: ``answer_text`` — the master
switch — plus the two ``show_text_max_*`` budgets, whose inert ``0``
values and reset-to-default resolve live with no restart).
The rows are inserted directly through the ``db`` session (the admin
API is task 04); the save itself is ``invalidate_settings()`` —
@@ -249,6 +252,72 @@ def test_llm_summary_thinking_level_live_apply_no_restart(
assert get_settings().llm_summary_thinking_level == ""
# ---------------------------------------------------------------------------
# the verbatim-display switch + budgets: save/reset resolve the live values
# (phase 182, task 01)
# ---------------------------------------------------------------------------
def test_verbatim_display_settings_live_apply_no_restart(
db: Session, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 182 (task 01, LOCKED A8): the verbatim-display master switch
and its two budgets follow the overlay live — a save (the PUT's write,
minus the API: ``_save`` + ``invalidate_settings()`` is exactly what
the PUT/DELETE call after the write) resolves
``get_settings().answer_text`` / ``.show_text_max_chars`` /
``.show_text_max_blocks`` WITHOUT a restart (the chat request path
reads them per turn via ``get_settings()``), the ``0`` values resolve
live as INERT (not an error, not coerced to the default), the bool
accepts the env-native ``false``/``0`` spellings, and a reset (the
DELETE) returns the env default.
"""
# A deterministic base (machine-independent, the kill-switch test's
# idiom): the code defaults, no row in the table.
for var in (
"BOR_ANSWER_TEXT",
"BOR_SHOW_TEXT_MAX_CHARS",
"BOR_SHOW_TEXT_MAX_BLOCKS",
):
monkeypatch.delenv(var, raising=False)
invalidate_settings()
assert get_settings().answer_text is True
assert get_settings().show_text_max_chars == 12_000
assert get_settings().show_text_max_blocks == 5
# Save: every value resolves through the overlay, no restart.
_save(db, "answer_text", "false")
_save(db, "show_text_max_chars", "2500")
_save(db, "show_text_max_blocks", "1")
invalidate_settings()
assert get_settings().answer_text is False
assert get_settings().show_text_max_chars == 2500
assert get_settings().show_text_max_blocks == 1
# ``0`` is the inert state (the field is not treated as "unset"), and
# the bool's other env-native spelling resolves to True.
_save(db, "answer_text", "1")
_save(db, "show_text_max_chars", "0")
_save(db, "show_text_max_blocks", "0")
invalidate_settings()
assert get_settings().answer_text is True
assert get_settings().show_text_max_chars == 0
assert get_settings().show_text_max_blocks == 0
# The reset returns the env defaults (no process restart in between).
db.execute(
text(
"DELETE FROM app_settings WHERE key IN "
"('answer_text', 'show_text_max_chars', 'show_text_max_blocks')"
)
)
db.commit()
invalidate_settings()
assert get_settings().answer_text is True
assert get_settings().show_text_max_chars == 12_000
assert get_settings().show_text_max_blocks == 5
# ---------------------------------------------------------------------------
# the chat concurrency semaphore follows the saved cap
# ---------------------------------------------------------------------------
+7 -7
View File
@@ -47,7 +47,7 @@ from app.config import Settings
from app.db import SessionLocal
from app.models import Document, FolderSummary, GitSource
from app.rag import agent
from app.rag.agent import AGENT_TOOLS, AgentHolder, run_agent
from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL, AgentHolder, run_agent
from app.rag.llm import (
LLMClient,
RetryPiece,
@@ -319,7 +319,7 @@ def test_ls_top_level_lists_registered_sources_through_run_agent(
holder, llm = _run_call(db, "ls", {})
assert llm.requests[0][1] == AGENT_TOOLS
assert llm.requests[0][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
assert llm.requests[1][0][3]["content"].startswith(
"2 sources:\n"
"\n"
@@ -482,7 +482,7 @@ def test_ls_path_like_scope_unknown_source_gets_no_source_refusal(kb, src, db) -
agent.NO_SOURCE_NOT_A_DIRECTORY.format(scope="app")
)
assert holder.tool_calls == 0 and holder.read_docs == []
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
assert llm.requests[1][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL] # rejected → tools stay offered
def test_ls_unknown_folder_gets_not_a_folder_with_parents_subfolders(
@@ -505,7 +505,7 @@ def test_ls_unknown_folder_gets_not_a_folder_with_parents_subfolders(
"backups/ containers/ networking/"
)
assert holder.tool_calls == 0 and holder.read_docs == []
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
assert llm.requests[1][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL] # rejected → tools stay offered
# ---------- read (the canonical combined source/path form) ----------
@@ -595,7 +595,7 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) ->
"No document at 'deep/nested/doc.md' — "
"did you mean 'Alpha/deep/nested/doc.md'?"
)
assert llm.requests[1][1] == AGENT_TOOLS
assert llm.requests[1][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
# Round 2: the corrected combined identity succeeds — the full
# content (plus the phase-106 D5 date line), the holder records the
# row, and it counts.
@@ -604,7 +604,7 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) ->
"date: 2024-06-15\n"
"FULL-TEXT"
)
assert llm.requests[2][1] == AGENT_TOOLS
assert llm.requests[2][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
# SEC-14-04: short-lived session loads fresh copies
assert [d.id for d in holder.read_docs] == [created.id]
assert holder.tool_calls == 1 # only the corrected read executed
@@ -670,7 +670,7 @@ def test_grep_whole_kb_through_run_agent(kb, db) -> None:
holder, llm = _run_call(db, "grep", {"pattern": "needle"})
# Offered: the first request carries AGENT_TOOLS (the 3-tool list).
assert llm.requests[0][1] == AGENT_TOOLS
assert llm.requests[0][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
# Executed against the real DB: catalog order, grep-style lines.
assert llm.requests[1][0][3]["content"].startswith(
"Alpha/a/one.md:2: needle in one\n"
+416 -10
View File
@@ -36,10 +36,10 @@ from app.config import Settings, get_settings
from app.main import app as fastapi_app
from app.models import Chunk, Document, GitSource, QueryLog
from app.rag import agent
from app.rag.agent import AGENT_TOOLS, READ_TRUNCATION_NOTICE
from app.rag.agent import AGENT_TOOLS, READ_TRUNCATION_NOTICE, SHOW_TEXT_TOOL
from app.rag.importer import import_sources
from app.rag.llm import EmbeddingError, LLMError, StreamPiece, ToolCallPiece, ToolResultPiece
from app.rag.prompts import build_high_prompt
from app.rag.prompts import SHOW_TEXT_CLAUSE, build_high_prompt
from app.rag.retriever import TRUNCATION_MARKER
from app.schemas import ChatDoneEvent, SourceRef
from tests.conftest import ADMIN_PASSWORD
@@ -1291,9 +1291,12 @@ def test_grounded_turn_streams_tool_frames_and_cites_read_doc(
# (not spent budgets) bounds the loop, and the model answered while
# still being offered the tools (2 rounds < default cap 10).
assert len(scripted.seen_messages) == 3
assert scripted.seen_tools[0] == AGENT_TOOLS
assert scripted.seen_tools[1] == AGENT_TOOLS
assert scripted.seen_tools[2] == AGENT_TOOLS
# Phase 182 (task 03) pin update, deliberate: ``answer_text``
# defaults ON, so the grounded offer is the base three
# (element-for-element ``AGENT_TOOLS``) plus ``show_text``.
assert scripted.seen_tools[0] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
assert scripted.seen_tools[1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
assert scripted.seen_tools[2] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
# The query_log row carries the same combined source list.
(row,) = db.scalars(select(QueryLog)).all()
@@ -2887,7 +2890,10 @@ def test_show_image_turn_done_frame_carries_answer_image(
# The cited tier includes the same doc (the scripted read).
assert ("docs", "pic.png") in [(s["source"], s["path"]) for s in done["sources"]]
# The toggle was honoured on the wire: the first (agent) request
# offered the 4th tool.
# offered the 4th tool. Phase 182 (task 03) pin update, deliberate:
# with ``answer_text`` ON (the shipped default) ``show_text`` rides
# LAST too — the ``answer_images`` gate itself is unchanged (it
# still adds exactly ``show_image``, 3rd-position-offer order kept).
offered = scripted.seen_tools[0]
assert offered is not None
assert [t["function"]["name"] for t in offered] == [
@@ -2895,6 +2901,7 @@ def test_show_image_turn_done_frame_carries_answer_image(
"read",
"grep",
"show_image",
"show_text",
]
@@ -2903,8 +2910,9 @@ def test_toggle_off_same_flow_offers_three_tools_and_no_answer_image_key(
) -> None:
"""The SAME scripted flow with ``answer_images=False`` (the
default) → ``show_image`` is NOT in the offered tools (the mock's
recorded request carries exactly the three base tools) and the
``done`` frame has NO ``answer_image`` key — the feature's
recorded request carries the three base tools — plus ``show_text``,
phase 182's default-ON display tool, pin updated here deliberately)
and the ``done`` frame has NO ``answer_image`` key — the feature's
server-side gate is the OFFER (a model that was never offered the
tool never calls it, so no such call occurs in the flow)."""
_seed_image_doc(db) # resolvable, but the tool is never offered
@@ -2928,10 +2936,16 @@ def test_toggle_off_same_flow_offers_three_tools_and_no_answer_image_key(
done = frames[-1]
assert done["deflected"] is False
assert "answer_image" not in done # ABSENT, not null
# The offer: exactly the three base tools — no ``show_image``.
# The offer: the three base tools — no ``show_image`` (phase 182:
# ``show_text`` rides last, the default being ON).
offered = scripted.seen_tools[0]
assert offered is not None
assert [t["function"]["name"] for t in offered] == ["ls", "read", "grep"]
assert [t["function"]["name"] for t in offered] == [
"ls",
"read",
"grep",
"show_text",
]
def test_zero_tool_grounded_turn_done_frame_has_no_answer_image_key(
@@ -3202,3 +3216,395 @@ def test_grounded_turn_suggested_plain_image_doc_block_is_byte_identical(
"</document>"
)
assert "Referenced in" not in system
# ---------- phase 182 (task 04): the show_text clause reaches the model ----------
def _gate_settings(**overrides: Any) -> Settings:
"""``Settings`` for the endpoint with the live honesty gate plus
*overrides* (the thinking-kill-switch pattern above) — so only the
flag under test changes from the calibrated default."""
live = get_settings()
return Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
relevance_threshold=live.relevance_threshold,
**overrides,
)
def test_grounded_turn_carries_show_text_clause_and_tool_per_the_flag(
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 182 (task 04, LOCKED A1/A8): on a GROUNDED turn the model
request carries the ``SHOW_TEXT_CLAUSE`` inside ``<tools>`` exactly
when ``BOR_ANSWER_TEXT`` is on — and the ``show_text`` tool itself is
offered on the same condition, so the model is never taught a tool it
was not given nor handed one it was not taught (``answer_images``
stays off here, so the offer is the three base tools alone off, the
three base tools + ``show_text`` on)."""
for flag in (True, False):
monkeypatch.setattr(
chat_api, "get_settings", lambda f=flag: _gate_settings(answer_text=f)
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
first_request = len(seeded_kb.seen_tools)
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["deflected"] is False, f"flag={flag}"
system = seeded_kb.seen_messages[first_request][0]["content"]
assert (SHOW_TEXT_CLAUSE in system) is flag, f"flag={flag}"
if flag:
assert system.index("<tools>") < system.index(SHOW_TEXT_CLAUSE)
assert system.index(SHOW_TEXT_CLAUSE) < system.index("</tools>")
offered = [t["function"]["name"] for t in (seeded_kb.seen_tools[first_request] or [])]
assert ("show_text" in offered) is flag, f"flag={flag}"
assert len(offered) == (4 if flag else 3), f"flag={flag}: {offered}"
# ---------------------------------------------------------------------------
# Phase 182 (task 05): the done frame's ADDITIVE ``answer_text`` array —
# the SERVER-resolved verbatim blocks (the ``show_text`` tool's and the
# ones the answer's own ```bor locator fences name), the byte-identity of
# every frame WITHOUT them, and the per-turn line's ``verbatim_blocks=N``
# tail. The resolver itself is pinned in tests/unit/test_verbatim_resolver.py;
# here the wire is the subject.
# ---------------------------------------------------------------------------
#: A fixture doc with a stable, countable body (16 lines as imported).
_K8S = "docs/homelab/kubernetes.md"
def _verbatim_settings(**overrides: Any) -> Settings:
"""The endpoint's settings view with the live gate plus *overrides*
(the ``_gate_settings`` pattern above) — only the flag under test
changes from the calibrated default."""
return _gate_settings(**overrides)
def _doc_lines(db, combined: str) -> list[str]:
"""The indexed row's content split the way the resolver splits it
(``splitlines()``, 1-based) — the authority the frame's text is
compared against, never a copy typed into this file."""
source, _, path = combined.partition("/")
content = db.scalar(
select(Document.content).where(
Document.source == source, Document.path == path
)
)
assert content is not None, combined
return content.splitlines()
def test_show_text_turn_done_frame_carries_the_server_block(
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
) -> None:
"""A scripted grounded turn whose model calls
``show_text(docs/homelab/kubernetes.md, offset=3, limit=100)`` → the
``done`` frame carries ONE ``answer_text`` entry whose ``text`` is
byte-identical to ``documents.content`` lines 3–16 (the overshooting
``limit`` CLAMPED, LOCKED A2), whose extent + ``total_lines`` are the
SERVER's, and whose ``locator`` is the canonical
``source/path#3-16`` (D7)."""
scripted = FakeRagLLM(
tool_script=[
[
ToolCallPiece(
id="call_1",
name="show_text",
arguments={"path": _K8S, "offset": 3, "limit": 100},
)
]
]
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level(logging.DEBUG, logger="app.response_budget"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
lines = _doc_lines(db, _K8S)
# Phase 167's byte-budget line covers the done-bearing turn (the
# stream total, ``stream=1``) — the audit surface that shows the
# blocks' growth, pinned WELL inside the 256 KB budget here.
budget = [
r.getMessage()
for r in caplog.records
if " path=/api/chat " in r.getMessage()
]
assert len(budget) == 1, budget
size = int(re.search(r"bytes=(\d+)", budget[0]).group(1)) # pyright: ignore[reportOptionalMemberAccess]
assert 0 < size < 256 * 1024, budget[0]
assert done["deflected"] is False
assert done["answer_text"] == [
{
"source": "docs",
"path": "homelab/kubernetes.md",
"title": "Kubernetes Homelab Cluster",
"start_line": 3,
"end_line": len(lines),
"total_lines": len(lines),
"text": "\n".join(lines[2:]),
"truncated": False,
"locator": f"{_K8S}#3-{len(lines)}",
}
]
def test_answer_fence_resolves_the_same_block_without_a_tool_call(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""D1/D3: the ANSWER alone carries a ```bor locator fence (no tool
call at all) and the block still resolves — resolution is a plain
``(source, path)`` lookup, not a replay of a tool call. The frame's
text is the DB's bytes: the model's (deliberately corrupt) fence body
appears NOWHERE on the wire."""
scripted = FakeRagLLM(
answer=(
"Here it is:\n"
"```bor docs/homelab/kubernetes.md 3-5\n"
"Exec=/MODEL_TYPED_LIES/never-render.bin\n"
"```\n"
"That is the whole range."
)
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
lines = _doc_lines(db, _K8S)
done = frames[-1]
assert done["answer_text"] == [
{
"source": "docs",
"path": "homelab/kubernetes.md",
"title": "Kubernetes Homelab Cluster",
"start_line": 3,
"end_line": 5,
"total_lines": len(lines),
"text": "\n".join(lines[2:5]),
"truncated": False,
"locator": f"{_K8S}#3-5",
}
]
# The model's fence body never reaches the wire as a RESOLVED block
# (the delta frames still carry the answer text — task 06/07 drop the
# body at render time; here the pin is that the frame's bytes are the
# DB's, not the model's).
assert done["answer_text"][0]["text"] == "\n".join(lines[2:5])
assert "MODEL_TYPED_LIES" not in json.dumps(done["answer_text"])
# The prose frames still stream the model's own answer text (the
# render-time body drop is task 06/07's job, pinned there) — the
# frame is what carries the truth.
answer = "".join(
f["text"] for f in frames if f.get("type") == "delta"
)
assert "MODEL_TYPED_LIES" in answer
def test_fence_plus_identical_tool_call_produce_exactly_one_block(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""D2 dedupe: the tool shows the range AND the answer locates the
same canonical locator → exactly ONE entry (the fence wins, the block
is not painted twice)."""
scripted = FakeRagLLM(
answer=(
"The block:\n"
"```bor docs/homelab/kubernetes.md#3-5\n"
"```"
),
tool_script=[
[
ToolCallPiece(
id="call_1",
name="show_text",
arguments={"path": _K8S, "offset": 3, "limit": 3},
)
]
],
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert len(done["answer_text"]) == 1
assert done["answer_text"][0]["locator"] == f"{_K8S}#3-5"
def test_three_blocks_with_max_blocks_two_send_two_in_answer_order(
client,
db,
seeded_kb: FakeRagLLM,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""D8: three fences in the answer, ``BOR_SHOW_TEXT_MAX_BLOCKS=2`` →
exactly TWO entries, in ANSWER order (the cut is the resolver's, the
frame keeps its order, and the log line counts what was SENT)."""
monkeypatch.setattr(
chat_api, "get_settings", lambda: _verbatim_settings(show_text_max_blocks=2)
)
scripted = FakeRagLLM(
answer=(
"A:\n```bor docs/homelab/kubernetes.md 1-2\n```\n"
"B:\n```bor docs/homelab/backups.md 1-2\n```\n"
"C:\n```bor docs/deployments/new-service.md 1-2\n```"
)
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert [b["locator"] for b in done["answer_text"]] == [
"docs/homelab/kubernetes.md#1-2",
"docs/homelab/backups.md#1-2",
]
lines = [
r.getMessage() for r in caplog.records if "question=" in r.getMessage()
]
assert lines and lines[-1].endswith("verbatim_blocks=2")
def test_turn_without_blocks_serializes_without_the_answer_text_key(
client, db, seeded_kb: FakeRagLLM, caplog: pytest.LogCaptureFixture
) -> None:
"""A plain grounded turn (no tool call, no fence): the ``done`` frame
has NO ``answer_text`` key — checked on the RAW wire text — and its
key set is exactly the pre-phase one (the A9 byte-identity pin), with
``verbatim_blocks=0`` on the per-turn line."""
scripted = FakeRagLLM()
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert "answer_text" not in done
assert set(done) == {
"type",
"deflected",
"sources",
"related",
"suggestions",
}
lines = [
r.getMessage() for r in caplog.records if "question=" in r.getMessage()
]
assert lines and lines[-1].endswith("verbatim_blocks=0")
def test_answer_text_off_keeps_the_pre_phase_frame_and_no_tool(
client, db, seeded_kb: FakeRagLLM, monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""The master switch OFF (LOCKED A8/A9) with an answer that DOES
carry a fence: the frame is byte-for-byte the pre-phase shape (no
key), the line reads ``verbatim_blocks=0``, and the model was never
offered ``show_text`` — every byte-identity contract is exactly
pre-phase."""
monkeypatch.setattr(
chat_api, "get_settings", lambda: _verbatim_settings(answer_text=False)
)
scripted = FakeRagLLM(
answer=(
"Here it is:\n"
"```bor docs/homelab/kubernetes.md 3-5\n"
"Exec=/MODEL_TYPED_LIES/never-render.bin\n"
"```"
)
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is False
assert set(done) == {"type", "deflected", "sources", "related", "suggestions"}
offered = [t["function"]["name"] for t in (scripted.seen_tools[0] or [])]
assert "show_text" not in offered
lines = [
r.getMessage() for r in caplog.records if "question=" in r.getMessage()
]
assert lines and lines[-1].endswith("verbatim_blocks=0")
def test_deflected_frame_has_no_answer_text_even_with_a_fence(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""D9: a deflected answer whose mocked text contains a fence still
resolves NOTHING — the frame has no ``answer_text`` key (the
disclaimer never leaks KB bytes) and no tools were offered."""
scripted = FakeRagLLM(
answer=(
"I haven't done anything like that.\n"
"```bor docs/homelab/kubernetes.md 3-5\n"
"```\n"
"Maybe try a saved runbook?"
)
)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, OFF_TOPIC)
finally:
fastapi_app.dependency_overrides.clear()
done = frames[-1]
assert done["deflected"] is True
assert "answer_text" not in done
def test_resolution_failure_degrades_to_no_blocks(
client,
db,
seeded_kb: FakeRagLLM,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""The answer already went out — a DB failure inside the done-time
resolution must not break the stream: the turn settles with the plain
pre-phase ``done`` frame (no key), the line reads
``verbatim_blocks=0``, and the failure is named on the log (the
query_log-write degradation precedent, never a silent swallow)."""
def _boom(*_a: Any, **_kw: Any) -> list[Any]:
raise RuntimeError("the KB went away mid-answer")
monkeypatch.setattr(chat_api, "resolve_answer_blocks", _boom)
scripted = FakeRagLLM(answer="Here:\n```bor docs/homelab/kubernetes.md 1-2\n```")
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
with caplog.at_level("INFO", logger="app.chat"):
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["type"] == "done"
assert "answer_text" not in frames[-1]
lines = [
r.getMessage()
for r in caplog.records
if "question=" in r.getMessage() or "verbatim" in r.getMessage()
]
assert any("verbatim block resolution failed" in m for m in lines)
turn = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert turn and turn[-1].endswith("verbatim_blocks=0")
+114
View File
@@ -2302,3 +2302,117 @@ def test_record_without_segments_round_trips_key_absent(
assert r2.status_code == 201, r2.text
got2 = admin_client.get(f"/api/chats/{r2.json()['id']}")
assert got2.json()["messages"][0]["segments"] == []
# ---------------------------------------------------------------------------
# Phase 182 (task 05): the answer's VERBATIM display blocks on the BRAIN
# record — the persisted twin of the done frame's ``answer_text`` (the
# show_text tool's blocks and the ones a ```bor locator fence resolved).
# Same phase-113 lesson as ``related`` / ``answer_image``: the key must be
# ACCEPTED (``extra="forbid"`` otherwise 422s the done-time auto-save and
# the quiet failure leaves the turn unsaved), round-trip byte-for-byte so
# the restore path and the anonymous shared page re-render the panels from
# the record's own bytes (LOCKED A4 — no new route), and stay ABSENT for
# every record without blocks (A9, the phase-50 round-trip contract).
# ---------------------------------------------------------------------------
#: One resolved block — the done frame's ``answer_text`` item shape
#: (the SERVER's text + its 1-based inclusive extent + the canonical
#: locator the client matches its placeholder against).
VERBATIM_BLOCK: dict[str, Any] = {
"source": "Homelab",
"path": "quadlet/gufo.container",
"title": "gufo.container",
"start_line": 3,
"end_line": 9,
"total_lines": 21,
"text": "[Unit]\nDescription=gufo\n[Service]\nExecStart=/usr/bin/gufo",
"truncated": False,
"locator": "Homelab/quadlet/gufo.container#3-9",
}
def test_create_round_trips_the_brain_answer_text_blocks(
admin_client: TestClient, db: Session
) -> None:
"""A brain record carrying ``answer_text`` → 201 (NOT the silent 422)
and the blocks survive byte-for-byte in the stored JSONB, the create
response and the admin detail — the panels restore from the record."""
brain = {**FULL_BRAIN, "answer_text": [VERBATIM_BLOCK]}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, f"expected 201, got {r.status_code}: {r.text}"
body = r.json()
assert body["messages"][1]["answer_text"] == [VERBATIM_BLOCK]
assert "answer_text" not in body["messages"][0]
raw = db.execute(
text("SELECT messages FROM saved_chats WHERE id = :id"),
{"id": body["id"]},
).scalar_one()
assert raw[1]["answer_text"] == [VERBATIM_BLOCK]
got = admin_client.get(f"/api/chats/{body['id']}")
assert got.status_code == 200
assert got.json()["messages"][1]["answer_text"] == [VERBATIM_BLOCK]
def test_create_without_answer_text_round_trips_unchanged(
admin_client: TestClient,
) -> None:
"""A record WITHOUT the key (every pre-phase-182 chat, every turn
with no blocks) round-trips UNCHANGED — no ``answer_text`` key
anywhere, ABSENT rather than ``null``/``[]`` on every surface (A9)."""
brain = {**FULL_BRAIN}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, r.text
body = r.json()
assert body["messages"] == _expect([_user(FIRST_QUESTION), brain])
assert "answer_text" not in body["messages"][0]
assert "answer_text" not in body["messages"][1]
got = admin_client.get(f"/api/chats/{body['id']}")
assert got.status_code == 200
assert "answer_text" not in got.json()["messages"][1]
def test_empty_answer_text_list_is_dropped_not_stored(
admin_client: TestClient,
) -> None:
"""An empty ``answer_text`` (the client saving a turn whose resolver
returned nothing) serializes AWAY — never ``null``, never ``[]`` —
so the stored record is byte-identical to the no-key shape (A9)."""
brain = {**FULL_BRAIN, "answer_text": []}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, r.text
assert "answer_text" not in r.json()["messages"][1]
def test_shared_serve_includes_the_brain_answer_text_blocks(
admin_client: TestClient,
) -> None:
"""The anonymous shared page renders the SAME panels from the share's
own record (A4 — no new bytes route): the public snapshot's brain
record carries the blocks verbatim, the user record does not."""
brain = {**FULL_BRAIN, "answer_text": [VERBATIM_BLOCK]}
r = admin_client.post(
"/api/chats",
json={"messages": [_user(FIRST_QUESTION), brain]},
)
assert r.status_code == 201, r.text
share_url = _share(admin_client, r.json()["id"])["share_url"]
anon = TestClient(fastapi_app) # fresh jar: truly anonymous
got = anon.get(f"/api{share_url}")
assert got.status_code == 200
messages = got.json()["messages"]
assert messages[1]["answer_text"] == [VERBATIM_BLOCK]
assert "answer_text" not in messages[0]
@@ -145,6 +145,10 @@ def test_no_row_high_prompt_byte_identical_to_pre_phase(
notes=[],
kb_overview=None,
folder_lines=_suggested_folder_lines(db),
# Phase 182 (task 04): the grounded build carries the show_text
# clause exactly when BOR_ANSWER_TEXT is on (the default) — read
# the live flag so this pin holds with the switch either way.
show_text=get_settings().answer_text,
)
assert system["content"] == expected
assert "<knowledge_base>" not in system["content"]
@@ -208,6 +212,8 @@ def test_row_high_prompt_carries_kb_section_exactly(
notes=[],
kb_overview=OVERVIEW,
folder_lines=_suggested_folder_lines(db),
# Phase 182 (task 04): same as above — the flag is the live one.
show_text=get_settings().answer_text,
)
assert system["content"] == expected
@@ -69,6 +69,7 @@ from app.rag.agent import (
AGENT_TOOLS,
READ_TRUNCATION_NOTICE,
REREAD_PARTIAL,
SHOW_TEXT_TOOL,
AgentHolder,
run_agent,
)
@@ -361,8 +362,11 @@ def test_grep_then_range_read_answers_the_find_traps_question(
assert len(llm.requests) == 3
assert [call.name for call in llm.calls] == ["grep", "read"]
# The tools stay offered on every request (phase 45 — no budgets).
assert llm.requests[0][1] == AGENT_TOOLS
assert llm.requests[2][1] == AGENT_TOOLS
# Phase 182 (task 03) pin update, deliberate: ``answer_text``
# defaults ON, so the grounded offer is the base three
# (element-for-element ``AGENT_TOOLS``) plus ``show_text``.
assert llm.requests[0][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
assert llm.requests[2][1] == [*AGENT_TOOLS, SHOW_TEXT_TOOL]
# Round 1: the lowercase pattern (grep is case-insensitive).
assert llm.calls[0].arguments == {"pattern": PATTERN}