phase: 182_verbatim_show_text
Build and Push Containers / build-and-push-app (push) Successful in 1m46s
Build and Push Containers / build-and-push-db (push) Successful in 12s

**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**

- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
2026-10-15 08:33:12 -04:00
parent 814a400e6d
commit b34b8ca672
104 changed files with 10852 additions and 203 deletions
+531
View File
@@ -0,0 +1,531 @@
/* Brain of Reese — the phase-182 VERBATIM panel client (task 07).
*
* A ` ```bor <source/path>#12-40 ` locator fence is a CLAIM about a file,
* never its contents: `markdown.js` (task 06) drops the model's body and
* paints a `.verbatim.verbatim-pending` placeholder keyed by the canonical
* locator, and the SERVER (`app.rag.verbatim.resolve_answer_blocks`,
* task 05) puts the real bytes on the `done` frame as `answer_text`. This
* file is the other half of that handshake: it RESOLVES the placeholders
* against the frame's blocks (`hydrateVerbatim`), places the blocks the
* answer never located (`appendUnlocatedVerbatim`), settles leftovers when
* a turn ends without a `done` (`settleVerbatimPlaceholders`), gives every
* settled answer code block a Copy button (`enhanceAnswerCodeBlocks`) and
* owns the ONE clipboard path (`copyTextWithFallback`).
*
* WHY IT IS A CLASSIC SCRIPT, NOT A MODULE (the `markdown.js` pattern):
* `index.html` and `shared.html` load it as a plain `<script>` right after
* `markdown.js` and before their page modules, so the top-level function
* declarations below are globals both module pages can call — the ONE
* implementation shared by the live turn, the reload restore and the
* anonymous shared page (parity is the phase's hard requirement). The
* Containerfile minifies it WITHOUT `--bundle` for the same reason.
*
* NO LOCATOR GRAMMAR LIVES HERE (the phase's main drift risk): the canonical
* string is produced in exactly ONE client place,
* `markdown.js::borLocator` (the mirror of Python
* `app.rag.verbatim.canonical_locator`). Hydration reads the already
* normalized `data-bor-locator` attribute and compares it to the server's
* `locator` with a plain string `===`. Nothing here re-parses a range,
* canonicalizes an en dash, or normalizes a path.
*
* THE CONTRACT WITH THE SERVER (LOCKED A2/A4/A7):
* • `block.text` is painted with `textContent` and copied with
* `navigator.clipboard` VERBATIM — no trim, no re-wrap, no join, no
* model byte ever substituted in, and the line numbers never enter the
* clipboard (they live in the `user-select: none` gutter, a separate
* node the copy path never reads);
* • the caption's numbers are the SERVER's resolved extent, so a locator
* whose range the model got wrong shows what the server actually read
* (A2 made visible);
* • a locator the server could not resolve says so — the unresolved node
* carries NO badge, NO body and NONE of the model's bytes.
*
* The house frontend rules hold throughout: every node is built with
* `createElement` + `textContent` (no HTML string anywhere — the data here
* is model-written text and indexed file bytes), every handler is attached
* with `addEventListener` (the CSP is `default-src 'self'`: no inline
* handlers, no `onclick=`/`oncopy=` attribute), and the states are carried
* by TEXT and STRUCTURE, never by colour alone (WCAG 1.4.1).
*/
/* The one honest sentence for a locator nothing resolved (the pinned copy —
* the whole unresolved state is this sentence plus the document link, so a
* screen reader, a monochrome theme and a screenshot all agree on it). */
function verbatimUnresolvedCopy() {
return "I could not verify this against the knowledge base";
}
/* The badge's claim, spelled out for the pointer tooltip — the WORD is the
* signal (locked A5), the tooltip only explains it. */
function verbatimBadgeTitle() {
return "These bytes are copied from the knowledge base, not written by the model.";
}
/* ---------- the small guards (a malformed frame never throws) ----------
* `answer_text` crosses the wire, so every field is treated as untrusted
* shape: a block that is not a plain object with the three fields the panel
* cannot live without is DROPPED, exactly like the restore path drops a
* malformed timeline entry. */
function isVerbatimBlock(block) {
return !!(
block &&
typeof block === "object" &&
typeof block.source === "string" &&
block.source &&
typeof block.path === "string" &&
block.path &&
typeof block.text === "string"
);
}
function verbatimBlocks(blocks) {
return Array.isArray(blocks) ? blocks.filter(isVerbatimBlock) : [];
}
/* A positive line count, or null (a missing/garbage count never prints). */
function verbatimCount(value) {
return Number.isInteger(value) && value > 0 ? value : null;
}
/* The one `source/path` identity the whole app uses (the read/grep
* coordinate). NO normalization here — see the header: markdown.js owns the
* grammar, this only joins the two halves the server already resolved. */
function verbatimIdentity(source, path) {
return `${source}/${path}`;
}
/* Split a canonical locator into its identity half and its range suffix.
* The split is on the `#` the server PUT THERE (canonical_locator's own
* separator) — never a re-parse of the range. */
function verbatimSplitLocator(locator) {
const text = typeof locator === "string" ? locator : "";
const hash = text.indexOf("#");
return hash === -1
? { identity: text, hasRange: false }
: { identity: text.slice(0, hash), hasRange: true };
}
/* The plain viewer href (the shared page's whole affordance — that page has
* no modal, and the documents API is admin-gated so a guest lands on the
* sign-in page by design). app.js OVERRIDES this through
* `opts.documentHref` with `documentUrl(…, "/")`, the chip's own
* back-to-the-chat URL (phase 153). A rooted path built from
* encodeURIComponent — never a `javascript:`/`data:` target. */
function verbatimDocumentHref(source, path) {
return (
"/document.html?source=" +
encodeURIComponent(source) +
"&path=" +
encodeURIComponent(path)
);
}
/* ---------- the panel's text pieces ---------- */
/* `homelab/a.md · lines 12–40 of 210`, or `homelab/a.md · all 210 lines`
* when the block IS the whole document (the caption is the SERVER's extent:
* `start_line`/`end_line` describe exactly the lines `text` contains after
* any character cap, so a capped block reads as a range, never as "all"). */
function verbatimCaptionText(block) {
const identity = verbatimIdentity(block.source, block.path);
const start = verbatimCount(block.start_line);
const end = verbatimCount(block.end_line);
const total = verbatimCount(block.total_lines);
if (start === null || end === null || end < start) return identity;
if (!block.truncated && start === 1 && total !== null && end === total) {
return `${identity} · all ${total} lines`;
}
return total === null
? `${identity} · lines ${start}–${end}`
: `${identity} · lines ${start}–${end} of ${total}`;
}
/* `Verbatim text from homelab/a.md, lines 12–40` (the range clause only when
* the panel shows a range — a whole-document panel is named by its file). */
function verbatimPanelLabel(block) {
const identity = verbatimIdentity(block.source, block.path);
const start = verbatimCount(block.start_line);
const end = verbatimCount(block.end_line);
if (start === null || end === null || end < start) {
return `Verbatim text from ${identity}`;
}
if (start === end) return `Verbatim text from ${identity}, line ${start}`;
return `Verbatim text from ${identity}, lines ${start}–${end}`;
}
/* The lines the gutter numbers: one entry per LINE OF THE SERVER TEXT, each
* `{ shown, number }`. A truncated block's last line is the server's
* `[…truncated…]` marker — a real line of the panel, but NOT a line of the
* file, so it gets no number (an empty cell keeps the rows aligned; the
* phase's rule is that the gutter never claims a coordinate the file does
* not have). */
function verbatimGutter(block) {
const lines = String(block.text).split("\n");
const start = verbatimCount(block.start_line);
const end = verbatimCount(block.end_line);
const numbered = start !== null && end !== null && end >= start ? end - start + 1 : 0;
return lines.map((_, i) => ({
number: i < numbered ? String(start + i) : "\u00a0",
}));
}
/* ---------- the controls ---------- */
/* The select-on-focus of the clipboard fallback field (an `<a>` has no
* `.select()`, a range does the job). Best-effort: a selection failure only
* means the reader copies by hand — the field still shows the whole text.
* (app.js's `copyShareLinkWithFallback` used to own this; task 07 moved the
* pair into verbatim.js so the code blocks and the share link share ONE
* degradation.) */
function selectAllInFallbackField(el) {
try {
const range = document.createRange();
range.selectNodeContents(el);
const sel = window.getSelection();
sel.removeAllRanges();
sel.addRange(range);
} catch {
/* selection is best-effort — the field still carries the full text */
}
}
/* The phase-55 degradation, generalized (LOCKED: the owner-locked share
* strings and behaviour are unchanged — app.js delegates here).
* `navigator.clipboard` is unavailable on a non-secure (http) homelab origin
* and rejects outside a user gesture, so the failure path renders ONE
* transient link field inside `hostEl` that selects its text on focus
* (click or Tab, then Ctrl/Cmd+C). Returns true when the clipboard took it.
* NEVER reads the DOM — the caller hands over the exact string, which is
* what makes "the clipboard content equals the server text" true. */
async function copyTextWithFallback(text, hostEl, title) {
const host = hostEl || document.body;
document.querySelectorAll(".share-link-fallback").forEach((el) => el.remove());
try {
await navigator.clipboard.writeText(text);
return true;
} catch {
const field = document.createElement("a");
field.className = "share-link-fallback";
// An <a> needs an href to focus; a code block has none, so it takes the
// tabindex instead and stays a select-to-copy field either way.
if (/^(?:https?:|mailto:|\/)/i.test(text)) field.href = text;
else field.tabIndex = 0;
field.textContent = text; // the text is data — textContent, never innerHTML
field.title = title || "Click, then copy (Ctrl/Cmd+C)";
field.addEventListener("focus", () => selectAllInFallbackField(field));
host.appendChild(field); // beside the control that failed, never elsewhere
field.focus({ preventScroll: true });
return false;
}
}
/* One pill: `Copy` → `Copied` (the TEXT carries the state; `.copied` only
* supports it — WCAG 1.4.1). The label rides a `<span>` because the clipped
* label is the CSS ellipsis target (the phase-156 rule) while the control's
* `aria-label` names the full path. Sticky by design: no timer, so a reader
* who tabs away and back finds the state the click left. */
function buildCopyButton(className, text, ariaLabel, hostEl) {
const btn = document.createElement("button");
btn.type = "button";
btn.className = className;
btn.setAttribute("aria-label", ariaLabel);
const label = document.createElement("span");
label.textContent = "Copy";
btn.appendChild(label);
btn.addEventListener("click", () => {
copyTextWithFallback(text, hostEl || btn.parentElement, ariaLabel).then((ok) => {
if (!ok) return; // the field is the feedback (it is focused and selected)
btn.classList.add("copied");
label.textContent = "Copied";
});
});
return btn;
}
/* The panel's link to the document. `opts.onOpen(source, path, linkEl)` is
* the chat page's modal hook (phase 26/153 — left-click opens the viewer IN
* the page, the href stays the `/document.html` escape hatch for
* middle-click and copy-link); with no hook the href simply navigates,
* which is the shared page's whole affordance. */
function buildVerbatimOpenLink(source, path, opts, text) {
const link = document.createElement("a");
link.className = "verbatim-open";
const identity = verbatimIdentity(source, path);
const href =
opts && typeof opts.documentHref === "function"
? opts.documentHref(source, path)
: verbatimDocumentHref(source, path);
link.href = href;
link.textContent = text || identity;
link.title = identity; // the full path as the native tooltip (chip pattern)
if (opts && typeof opts.onOpen === "function") {
link.addEventListener("click", (event) => {
event.preventDefault(); // no new tab — the modal takes over
event.stopPropagation();
opts.onOpen(source, path, link);
});
}
return link;
}
/* ---------- the panel ---------- */
/* buildVerbatimPanel(block, opts) → the `.verbatim` node.
*
* .verbatim[role=group][aria-label][data-bor-locator]
* .verbatim-head sticky (PLAN §7.1)
* .verbatim-caption source/path · lines 12–40 of 210 (clipped)
* .verbatim-badge VERBATIM (a WORD)
* .verbatim-open → the document viewer (optional)
* .verbatim-copy Copy → Copied (aria-label
* names path)
* .verbatim-body the bounded scroll panel
* ol.verbatim-lines the SERVER's line numbers (user-select: none)
* pre.verbatim-text block.text, byte for byte
* .verbatim-more the truncation line + the link (optional)
*
* `opts = { documentHref, onOpen }` — both optional (see the two builders
* above). Every value reaches the DOM through `textContent`/`title`/a
* built `href`: the caption carries indexed text and the aria-labels carry
* model-written locators, so there is no HTML string to escape. */
function buildVerbatimPanel(block, opts) {
const identity = verbatimIdentity(block.source, block.path);
const root = document.createElement("div");
root.className = "verbatim";
root.setAttribute("role", "group");
root.setAttribute("aria-label", verbatimPanelLabel(block));
// The SERVER's canonical locator is the identity of the placed block (the
// pending placeholder's own attribute is replaced with it — that is how
// `appendUnlocatedVerbatim` knows this block already has a home).
root.dataset.borLocator =
typeof block.locator === "string" && block.locator
? block.locator
: identity;
const head = document.createElement("div");
head.className = "verbatim-head";
const caption = document.createElement("span");
caption.className = "verbatim-caption";
caption.textContent = verbatimCaptionText(block);
caption.title = verbatimCaptionText(block); // the clipped text stays reachable
const badge = document.createElement("span");
badge.className = "verbatim-badge";
badge.textContent = "VERBATIM";
badge.title = verbatimBadgeTitle();
head.append(caption, badge);
head.appendChild(buildVerbatimOpenLink(block.source, block.path, opts));
const gutter = verbatimGutter(block);
// The clipboard's line count (the aria-label's N): the SERVER's extent,
// not the marker line the cap appended.
const from = verbatimCount(block.start_line);
const to = verbatimCount(block.end_line);
const shown = from !== null && to !== null && to >= from ? to - from + 1 : gutter.length;
head.appendChild(
buildCopyButton(
"verbatim-copy",
block.text,
`Copy ${shown} lines from ${identity}`,
root
)
);
root.appendChild(head);
const body = document.createElement("div");
body.className = "verbatim-body";
const numbers = document.createElement("ol");
numbers.className = "verbatim-lines";
numbers.setAttribute("aria-hidden", "true"); // the range rides the panel's label
for (const row of gutter) {
const li = document.createElement("li");
li.textContent = row.number;
numbers.appendChild(li);
}
const pre = document.createElement("pre");
pre.className = "verbatim-text";
pre.textContent = block.text; // THE POINT OF THE PHASE: the server's bytes
body.append(numbers, pre);
root.appendChild(body);
if (block.truncated) {
// One muted line naming where the rest lives, plus the same viewer link
// — the panel never pretends the slice is the file.
const more = document.createElement("div");
more.className = "verbatim-more";
const note = document.createElement("span");
note.textContent = "Cut short by the size limit — the rest is in the document. ";
more.appendChild(note);
more.appendChild(
buildVerbatimOpenLink(block.source, block.path, opts, `open ${identity}`)
);
root.appendChild(more);
}
return root;
}
/* The honest state for a locator the server did not resolve (and for a
* placeholder no terminal frame ever settled): the sentence, plus the
* document link ONLY when the identity actually parsed into a
* `source/path`. NO badge, NO body, NONE of the model's bytes — the fence's
* own text is already gone (task 06 dropped it), so there is nothing here
* to leak even by accident. */
function buildVerbatimUnresolved(locator, opts) {
const root = document.createElement("div");
root.className = "verbatim verbatim-unresolved";
root.setAttribute("role", "note");
const key = typeof locator === "string" ? locator : "";
if (key) root.dataset.borLocator = key;
root.dataset.verbatimState = (opts && opts.kind) || "unverified";
const state = document.createElement("span");
state.className = "verbatim-state";
state.textContent = verbatimUnresolvedCopy();
root.appendChild(state);
const { identity } = verbatimSplitLocator(key);
const slash = identity.indexOf("/");
if (slash > 0 && slash < identity.length - 1) {
root.appendChild(document.createTextNode(" — open "));
root.appendChild(
buildVerbatimOpenLink(identity.slice(0, slash), identity.slice(slash + 1), opts)
);
}
return root;
}
/* ---------- hydration ---------- */
/* The ONE match rule (LOCKED A2 made visible):
* 1. the canonical locator, compared as PLAIN STRINGS against the
* server's `locator` — the attribute markdown.js already normalized;
* 2. failing that, exactly ONE block sharing the same `source/path`
* (the model's RANGE was wrong; the panel shows the server's extent);
* 3. failing that, null → the unresolved state. An ambiguous identity
* (two blocks, same file, different ranges) never guesses. */
function matchVerbatimBlock(blocks, key) {
for (const block of blocks) {
if (block.locator === key) return block;
}
const identity = verbatimSplitLocator(key).identity;
const samePath = blocks.filter(
(block) => verbatimIdentity(block.source, block.path) === identity
);
return samePath.length === 1 ? samePath[0] : null;
}
/* hydrateVerbatim(root, blocks, opts) — replace every PENDING placeholder
* under `root` with its panel, or with the unresolved state.
*
* Only `.verbatim-pending` is touched, so the pass is idempotent: a panel
* this function built (or one restored from storage) is skipped on a re-run,
* which keeps a `Copied` state and a scrolled body alive. Never throws —
* a missing/malformed `blocks` settles every placeholder to the honest
* state instead. Returns the number of placeholders settled. */
function hydrateVerbatim(root, blocks, opts) {
if (!root || typeof root.querySelectorAll !== "function") return 0;
const list = verbatimBlocks(blocks);
let settled = 0;
root.querySelectorAll(".verbatim-pending[data-bor-locator]").forEach((node) => {
const key = node.getAttribute("data-bor-locator") || "";
const block = matchVerbatimBlock(list, key);
node.replaceWith(
block ? buildVerbatimPanel(block, opts) : buildVerbatimUnresolved(key, opts)
);
settled += 1;
});
return settled;
}
/* appendUnlocatedVerbatim(wrap, blocks, opts) — the blocks NO placeholder
* matched (a `show_text` call whose answer never located the file), appended
* in the frame's order at the `answer_image` position: below the answer
* text, above the source clusters (the phase-124/130 DOM order — callers run
* this BEFORE `appendSources`).
*
* State-free idempotence: a block is already placed when the wrap holds a
* resolved panel carrying its canonical `locator`, so calling this twice
* (or after a hydration that matched) appends nothing. */
function appendUnlocatedVerbatim(wrap, blocks, opts) {
if (!wrap || typeof wrap.querySelectorAll !== "function") return 0;
const list = verbatimBlocks(blocks);
if (!list.length) return 0;
const host = wrap.querySelector(".msg-body") || wrap;
const placed = new Set();
wrap.querySelectorAll(".verbatim[data-bor-locator]").forEach((el) => {
// An unresolved placeholder is NOT a placement (it names a locator the
// server never returned, and the block belongs on screen anyway).
if (el.classList.contains("verbatim-unresolved")) return;
placed.add(el.getAttribute("data-bor-locator"));
});
let appended = 0;
for (const block of list) {
const key =
typeof block.locator === "string" && block.locator
? block.locator
: verbatimIdentity(block.source, block.path);
if (placed.has(key)) continue;
host.appendChild(buildVerbatimPanel(block, opts));
placed.add(key);
appended += 1;
}
return appended;
}
/* settleVerbatimPlaceholders(root, kind) — the terminal-frame sweep the
* failure paths run (the `error` frame, the 300 s guard, the Stop action,
* the failed-turn funnel): a panel NEVER sits at "Retrieving the exact
* text…" once a turn is over. `kind` is recorded on the node
* (`data-verbatim-state`) for the next reader; every kind renders the same
* honest sentence, because "I could not verify this" is the truth for a
* turn that ended before `done` too. */
function settleVerbatimPlaceholders(root, kind) {
if (!root || typeof root.querySelectorAll !== "function") return 0;
let settled = 0;
root.querySelectorAll(".verbatim-pending").forEach((node) => {
node.replaceWith(
buildVerbatimUnresolved(node.getAttribute("data-bor-locator") || "", { kind })
);
settled += 1;
});
return settled;
}
/* ---------- the ordinary answer code block ---------- */
/* enhanceAnswerCodeBlocks(root) — every `<pre>` inside an answer bubble that
* is NOT inside a `.verbatim` gets the `.code-block` wrapper and a Copy
* button, ONCE the turn settled (the callers run it on the settled answer,
* never mid-stream). NO badge, NO caption: a `VERBATIM` claim belongs only
* to a server-resolved block (LOCKED A5) — plainly the model's, or verbatim
* and badged, never a hybrid.
*
* Idempotent on re-render: a `<pre>` already inside a `.code-block` is left
* alone (the live path re-runs the renderer per delta; the restore path runs
* this once). Returns the number of blocks wrapped. */
function enhanceAnswerCodeBlocks(root) {
if (!root || typeof root.querySelectorAll !== "function") return 0;
const bubbles = [];
if (root.classList && root.classList.contains("bubble")) bubbles.push(root);
root.querySelectorAll(".bubble").forEach((el) => bubbles.push(el));
let wrapped = 0;
for (const bubble of bubbles) {
bubble.querySelectorAll("pre").forEach((pre) => {
if (pre.closest(".verbatim")) return; // the panel owns its own Copy
const parent = pre.parentElement;
if (!parent || parent.classList.contains("code-block")) return;
const box = document.createElement("div");
box.className = "code-block";
parent.insertBefore(box, pre);
box.appendChild(pre);
box.appendChild(
buildCopyButton(
"code-block-copy",
pre.textContent || "",
"Copy this code block",
box
)
);
wrapped += 1;
});
}
return wrapped;
}