Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 182_verbatim_show_text
Build and Push Containers / build-and-push-app (push) Successful in 1m46s
Details
Build and Push Containers / build-and-push-db (push) Successful in 12s
Details

Browse Source 
**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**

- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
ducoterra
2026-10-15 08:33:12 -04:00
parent 814a400e6d
commit b34b8ca672
104 changed files with 10852 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/rag/agent.py
+292 -13
View File
@@ -211,6 +211,29 @@ task 04):
— its ``"No matches for …"`` line is a (counted) result. A rejected
call still consumes a *round* in the loop, so a pathological stream
that keeps emitting rejected calls is bounded by the cap (point 4).
Display tools (phase 124 ``show_image``, phase 182 ``show_text`` —
both composed by :func:`agent_tools`, each behind its own flag): they
display content FOR THE USER. ``show_image`` never adds context; the
5th tool ``show_text`` (phase 182, task 03) returns the resolved
slice back to the model ONLY as the truth of what was displayed
(LOCKED A3 — with a do-NOT-retype instruction: the model describes
the block, the user copies the server's bytes), and its block rides
``holder.shown_texts``, never ``read_docs``. The LOCKED core contract
A1–A3/A6: **the model names a location, the server shows the bytes**
— the handler resolves the slice through
:func:`app.rag.verbatim.block_for` (same ``(source, path)`` identity,
same 1-based ``splitlines()`` indexing as ``read``/``grep``, same
char-budget-cut-at-a-line-boundary), the DISPLAYED EXTENT is the
server's (an overshooting ``limit`` clamps into the result's ``lines:``
header, an ``offset`` past the end is the :data:`OFFSET_PAST_END`
teaching naming the true line count, the char cap shrinks the extent
— the same strings ``read`` uses, one copy each), images belong to
``show_image`` (a show of an ``is_image`` row is the
:data:`SHOW_TEXT_NOT_A_TEXT_DOC` refusal, the exact mirror of
:data:`NOT_AN_IMAGE_DOC` — never a silent conversion), and the inline
``` ```bor ``` locator-fence form of the same contract (phase 182 D2)
is resolved at the API layer (task 05), NOT here.
4. Every call the model emits is appended back to the message history as
the assistant tool-call message + the tool result (refusals included),
consumes one round, and the model is called again. Every tool result
@@ -317,6 +340,7 @@ from app.rag.llm import (
from app.rag.retriever import TRUNCATION_MARKER, _fold
from app.rag.scaffolding import ScaffoldingFilter
from app.rag.source_removal import resolve_source_name
from app.rag.verbatim import VerbatimBlock, block_for
logger = logging.getLogger("app.agent")
@@ -535,20 +559,104 @@ SHOW_IMAGE_TOOL: dict[str, Any] = {
}
def agent_tools(settings: Settings) -> list[dict[str, Any]]:
"""The grounded turn's tool offer list (phase 124, task 02).
#: The 5th agent tool (phase 182, task 03; LOCKED A1–A3/A6) — offered
#: ONLY when ``settings.answer_text`` (``BOR_ANSWER_TEXT``, default ON):
#: :func:`agent_tools` composes the offer list (the three base tools stay
#: :data:`AGENT_TOOLS`, byte-identical). The description IS the behavior
#: contract (the phase-124 discipline): the model NAMES a location, the
#: SERVER shows the bytes verbatim — the user copies what is displayed,
#: so the model must NOT retype it. Same OpenAI function shape as the
#: other tools: one required string ``path`` (the combined
#: ``source/path`` identity — the bare-path-will-not-resolve teaching
#: mirrored from ``read``) plus the OPTIONAL ``offset``/``limit`` line
#: range (the ``read`` wording reused, so every line-number surface in
#: the app teaches one grammar).
SHOW_TEXT_TOOL: dict[str, Any] = {
"type": "function",
"function": {
"name": "show_text",
"description": (
"Display part of a document to the user EXACTLY as stored "
"— the user sees the server's bytes and can copy them. "
"Call it when the user wants to see, copy or run a file (a "
"quadlet, a config, a script): they copy from what is "
"displayed, so do NOT retype the content yourself after "
"calling it — describe it instead. `path` is the combined "
"`source/path` string exactly as shown in `ls`. `offset` "
"(1-based first line) and `limit` (line count) are "
"optional — omit them to display the whole document; use "
"them when `grep`/`read` told you which lines matter. The "
"displayed line numbers come from the server, not from "
"your arguments: an overshooting `limit` is clamped and an "
"`offset` past the end is refused with the true line "
"count. Only text documents — for an image use "
"`show_image`. Call one tool at a time — wait for this "
"result before your next call."
),
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": (
"The document to display, as the combined "
"`source/path` string exactly as shown in the "
"`ls` output (e.g. "
"'homelab/active/container_caddy/caddy.md'). "
"A bare document path (without the source "
"name) will not resolve. Only text documents "
"— for an image use `show_image`."
),
},
"offset": {
"type": "integer",
"description": (
"The first line to display, 1-based. Omit it "
"to display from the beginning of the "
"document. Use it to display a specific part "
"of a large document — grep reports the line "
"numbers of what you found."
),
},
"limit": {
"type": "integer",
"description": (
"The maximum number of lines to display, "
"starting at offset. Omit it to display as "
"many lines as fit under the display cap. Use "
"it to show the part the user asked about — "
"you are showing a part, not the document."
),
},
},
"required": ["path"],
},
},
}
The three base tools (:data:`AGENT_TOOLS`, byte-identical) plus
def agent_tools(settings: Settings) -> list[dict[str, Any]]:
"""The grounded turn's tool offer list (phase 124, task 02; the 5th
tool, phase 182, task 03).
The three base tools (:data:`AGENT_TOOLS`, byte-identical) plus the
TWO display tools, each gated by its own flag:
:data:`SHOW_IMAGE_TOOL` when ``settings.answer_images``
(``BOR_ANSWER_IMAGES``) is true — otherwise exactly the three base
tools (the pre-phase-124 offer, byte-identical: no 4th tool in the
wire ``tools`` array). A FRESH list on every call — callers never
mutate the result or the constants (``run_agent`` passes it straight
to the LLM request).
(``BOR_ANSWER_IMAGES``) and :data:`SHOW_TEXT_TOOL` when
``settings.answer_text`` (``BOR_ANSWER_TEXT``) — the offer order is
``ls``, ``read``, ``grep``, ``show_image``, ``show_text``. With both
flags off: exactly the three base tools (the pre-phase-124 offer,
byte-identical: no display tool in the wire ``tools`` array); with
only ``answer_text`` off it is the pre-phase-182 offer
byte-identical. A FRESH list on every call — callers never mutate
the result or the constants (``run_agent`` passes it straight to the
LLM request).
"""
tools: list[dict[str, Any]] = list(AGENT_TOOLS)
if settings.answer_images:
tools.append(SHOW_IMAGE_TOOL)
if settings.answer_text:
tools.append(SHOW_TEXT_TOOL)
return tools
#: Tool refusal texts (phase 37): rejected calls count in nothing
@@ -587,6 +695,56 @@ ALREADY_SHOWING = (
"Answer from what you have."
)
#: The ``show_text`` refusals (phase 182, task 03): the same contract as
#: every other refusal — counts in nothing, consumes a round, changes no
#: context. ``MISSING_SHOW_TEXT_ARGS`` mirrors ``MISSING_SHOW_ARGS``.
#: ``SHOW_TEXT_NOT_A_TEXT_DOC``: one ``{arg}`` field, the argument echoed
#: as passed — the row is a standalone IMAGE; image display belongs to
#: ``show_image`` (LOCKED A6 — the exact mirror of
#: :data:`NOT_AN_IMAGE_DOC`, never a silent conversion). The range
#: problems REUSE :data:`BAD_RANGE_ARGS` and :data:`OFFSET_PAST_END`
#: verbatim — the same teaching, the same echoed values, one copy of
#: each string in the codebase.
MISSING_SHOW_TEXT_ARGS = "show_text requires a string argument 'path'."
SHOW_TEXT_NOT_A_TEXT_DOC = (
"'{arg}' is a standalone image document — use show_image to display "
"images."
)
#: One ``{max}`` field (``settings.show_text_max_blocks``): the
#: per-answer block budget, enforced SERVER-side BEFORE the resolve work
#: (the model is taught the budget in the description; the handler is
#: the actual gate). ``0`` therefore disables the tool's effect while
#: keeping it offered (the house kill-switch pattern).
SHOW_TEXT_BLOCK_CAP = (
"{max} text blocks are already displayed this answer — describe them "
"instead of showing more."
)
#: One ``{locator}`` field — the canonical ``source/path[#first-last]``
#: the model just asked for: the dedupe mirror of
#: :data:`ALREADY_SHOWING`, but per BLOCK under the block budget (the
#: budget allows SEVERAL different blocks; the same block twice teaches).
SHOW_TEXT_ALREADY_SHOWN = (
"{locator} is already on screen this answer — say so instead of "
"showing it again."
)
#: The ``show_text`` truncation notice (phase 182, task 03) — appended as
#: the result's LAST line when the per-block char budget
#: (``settings.show_text_max_chars``) cut the displayed slice (the block
#: text already ends with the shared :data:`TRUNCATION_MARKER` on its own
#: line, and the ``lines:`` extent already shrank to the last line
#: actually displayed). In the :data:`READ_TRUNCATION_NOTICE` spirit it
#: names the truth the model must not paper over — the rest is NOT
#: displayed — and the way the user gets it (open the document; NO
#: ``offset``/``limit`` re-show loop is implied, only the displayed
#: slice is what the user copies). Fields: ``{source}``/``{path}``, the
#: RESOLVED identity, plain (no ``!r``).
SHOWN_TEXT_TRUNCATION_NOTICE = (
"TRUNCATED — the rest of {source}/{path} is NOT displayed. The user "
"can open the document to read it in full; describe the displayed "
"lines instead of showing it again."
)
#: The read-truncation notice (phase 95, task 01; REWRITTEN phase 131,
#: task 01) — appended, after the shared :data:`TRUNCATION_MARKER`, to
#: a ``read`` result whose document is LONGER than
@@ -1642,6 +1800,17 @@ class AgentHolder:
nothing). Read by ``app.api.chat`` after the stream for the done
frame's ``answer_image`` field (task 04) — the ``read_docs``
precedent (an API-layer read-after-stream signal).
``shown_texts`` (phase 182, task 03): the verbatim blocks
``show_text`` displayed, in CALL order, capped by
``settings.show_text_max_blocks`` (a call over the budget is refused
with :data:`SHOW_TEXT_BLOCK_CAP` and a duplicate of a shown locator
with :data:`SHOW_TEXT_ALREADY_SHOWN` — both append nothing). Each
entry is the RESOLVED
:class:`app.rag.verbatim.VerbatimBlock` — the server's bytes plus
the server's extent (the canonical ``locator`` is the dedupe key and
the client's matching key, D7). Read by ``app.api.chat`` after the
stream for the done frame's ``answer_text`` array (task 05) — the
``shown_images``/``read_docs`` read-after-stream precedent.
``read_coverage`` (phase 141, task 01): one ``(source, path,
first_line, last_line)`` record per covered span — INCLUSIVE line
numbers over the document's ``content.splitlines()``, in read
@@ -1676,6 +1845,7 @@ class AgentHolder:
read_truncations: list[tuple[str, int, int]] = field(default_factory=list)
shown_images: list[Document] = field(default_factory=list)
read_coverage: list[tuple[str, str, int, int]] = field(default_factory=list)
shown_texts: list[VerbatimBlock] = field(default_factory=list)
def _execute_tool(
@@ -2065,6 +2235,113 @@ def _execute_tool(
f"{IMAGE_DOC_MARKER}\n"
f"{doc.content}"
)
if call.name == "show_text":
# Phase 182 (task 03; LOCKED A1–A3/A6): the 5th agent tool — the
# model NAMES a location, the SERVER shows the bytes verbatim.
# Offered only when ``settings.answer_text`` (``agent_tools``);
# the handler enforces the contract server-side. Every refusal
# below appends nothing, increments nothing and consumes a
# round, like every other refusal.
raw_path = call.arguments.get("path")
arg = raw_path.strip() if isinstance(raw_path, str) else ""
if not arg:
return MISSING_SHOW_TEXT_ARGS
if not settings.answer_text:
# Defensive — the tool is NOT OFFERED while the flag is off,
# so from the model's side this is a tool that does not
# exist: the same refusal an un-offered name gets (the
# tools-off posture — "Unknown tool."), counts in nothing.
return UNKNOWN_TOOL
doc, _source, _path = _resolve_path(db, arg)
if doc is None:
# The same phase-72 "did you mean …?" teaching as the read
# branch (a refusal — not counted, no context, nothing
# displayed).
return _no_document_refusal(db, arg)
if doc.is_image:
# LOCKED A6: an image row is refused, NEVER silently
# converted — image display is ``show_image``'s job (the
# mirror of the show_image branch's NOT_AN_IMAGE_DOC).
return SHOW_TEXT_NOT_A_TEXT_DOC.format(arg=arg)
# The OPTIONAL line range — the SAME validator, the SAME echoed
# BAD_RANGE_ARGS teaching as ``read`` (one copy of each string);
# ``offset`` without ``limit`` displays to the end, ``limit``
# without ``offset`` counts from line 1 — ``slice_lines`` applies
# both one-sided semantics with ``read``'s exact indexing.
ok, range_error, offset, limit = _read_range_args(
call.arguments.get("offset"), call.arguments.get("limit")
)
if not ok:
return range_error
# The per-answer block budget — checked BEFORE the resolve work
# (and, with a 0 char budget, before ``cap_slice`` could raise):
# either zeroed kill switch refuses like a spent budget, so
# ``0`` disables the tool's EFFECT while it stays offered (D8).
block_budget = (
settings.show_text_max_blocks
if settings.show_text_max_chars > 0
else 0
)
if len(holder.shown_texts) >= block_budget:
return SHOW_TEXT_BLOCK_CAP.format(max=block_budget)
block = block_for(
db,
arg,
offset=offset,
limit=limit,
cap=settings.show_text_max_chars,
doc=doc,
)
if block is None:
# An empty document or an ``offset`` past the last line —
# the SAME OFFSET_PAST_END teaching as ``read``, the true
# line count named (the same ``len(content.splitlines())``
# expression ``read`` uses). Counts in nothing; a corrected
# retry still proceeds.
return OFFSET_PAST_END.format(
source=doc.source,
path=doc.path,
total_lines=len(doc.content.splitlines()),
offset=offset if offset is not None else 1,
)
if any(shown.locator == block.locator for shown in holder.shown_texts):
# The canonical-locator dedupe (D7): the identical block is
# already on screen — say so instead of painting it twice.
return SHOW_TEXT_ALREADY_SHOWN.format(locator=block.locator)
# Success: the block rides the holder for the done frame
# (task 05) and the call counts (executed-only, the house rule).
holder.shown_texts.append(block)
holder.tool_calls += 1
# The result contract (D5 — the ``read``/``show_image`` header
# contract the E2E mock's ``_READ_RESULT_PREFIX`` parses): a NEW
# first line (absent from ``read`` — the model must know the
# show registered AND must not retype), then the exact document
# headers ``read`` established (date line, phase-145 origin
# lines), then the SERVER's resolved extent — clamped ``limit``
# and cap-shrunk end included, the caption never lies — and the
# verbatim slice itself (A21: LLM calls are free, and returning
# the text means the model's commentary can never disagree with
# the panel).
origin = origin_for_source(db, doc.source)
result = (
"Text shown to the user — these lines are displayed "
"verbatim; describe them, do NOT retype them.\n"
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{_origin_lines(origin)}"
f"lines: {block.start_line}-{block.end_line} of "
f"{block.total_lines}\n"
f"{block.text}"
)
if block.truncated:
# The char cap cut the slice: the block already ends with
# the shared marker on its own line — append the pinned
# truth (the rest is NOT displayed; the user opens the
# document; no re-show loop implied).
result += "\n" + SHOWN_TEXT_TRUNCATION_NOTICE.format(
source=doc.source, path=doc.path
)
return result
if call.name == "grep":
raw_pattern = call.arguments.get("pattern")
pattern = raw_pattern.strip() if isinstance(raw_pattern, str) else ""
@@ -2229,11 +2506,13 @@ async def run_agent(
# Phase 45: no per-tool budgets — the tools stay offered for the
# whole turn, bounded by the round cap. ``0`` is the no-tools kill
# switch: exactly one request with ``tools=None`` (the pre-phase-37
# path). Phase 124 (task 02): the offer list is :func:`agent_tools`
# — the three base tools (byte-identical) plus ``show_image`` only
# when ``settings.answer_images`` (``BOR_ANSWER_IMAGES``) is on;
# off (the default) the offer is element-for-element the
# pre-phase-124 list.
# path). Phase 124 (task 02) / 182 (task 03): the offer list is
# :func:`agent_tools` — the three base tools (byte-identical) plus
# ``show_image`` only when ``settings.answer_images``
# (``BOR_ANSWER_IMAGES``) and ``show_text`` only when
# ``settings.answer_text`` (``BOR_ANSWER_TEXT``, default ON); both
# off: the offer is element-for-element the pre-phase-124 list,
# ``answer_text`` alone off: the pre-phase-182 list.
max_rounds = settings.agent_max_rounds
tools: list[dict[str, Any]] | None = (
agent_tools(settings) if max_rounds > 0 else None