phase: 182_verbatim_show_text
**Phase 182 — final verification pass (all 8 tasks already complete; this was verification + docs + one defect fix).**
- Verified: `app/rag/verbatim.py` (locator grammar, `splitlines()` slice, line-boundary cap, `resolve_answer_blocks`), `SHOW_TEXT_TOOL` + `shown_texts`, `SHOW_TEXT_CLAUSE`, `done.answer_text` (9 fields, key dropped when empty) + `ChatMessageRecord.answer_text` (≤50, 600 000-char cap counted in the 8 M budget), `verbatim.js` + 4 `{verbatim:true}` renderers, `Containerfile` minify step, `.env.example` 3 vars.
- `uv run python -c "…answer_text, show_text_max_chars, show_text_max_blocks"` → `True 12000 5`; each negative refuses startup naming its field; `0` accepted as inert ✅
- `bash .agents/validate.sh` → **validation OK**: 4320 unit+integration tests green, `TOTAL 6849 17 → 99%` (gate app/ >90% OK), `ruff check .` "All checks passed!", `pyright` 0 errors (re-run after my edits, 08:28).
- E2E, each in isolation `--no-cov`: `test_verbatim_show_text.py` 13 passed (panel bytes == `documents.content`, server caption, clipboard == server string, `/document.html?source=…&path=…`, honest unresolved state, reload+share, `show_image` regression, flag-off app) ✅; green: `test_answer_images` `test_harness_aligned_tools` `test_agent_document_tools` `test_markdown_tables` `test_markdown_inline_links_images` `test_svg_document_preview` `test_smoke` `test_document_viewer` `test_markdown_blockquotes` `test_chat_persistence` `test_thinking_timeline` `test_chat_rag` `test_settings_mobile_value_width`.
- **Defect found & fixed (pre-existing, not phase 182):** `tests/e2e/test_admin_settings_panel.py::test_admin_sees_groups_and_source_badges` failed — its whole-page `assert "reese" not in page.content()` tripped on the shell's onboarding chips, which follow the newest saved chats in the one shared E2E DB (a leftover `reeselink.json` question from a tool suite). Scoped the strict sweep to `.settings-shell` and kept a whole-page `reese:reese` credential needle → 12 passed with the leftovers still present.
- Docs: README fixed 2 inconsistencies — Quick Start omitted the **required** `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` (the app fail-loud refuses boot) and the broken anchor `#chat--` → `#chat-`; full re-read of README (1816 lines) + AGENTS confirmed the phase-182 feature/settings/log/UI-contract text matches the code, and every `.env.example` var + every `Settings` field is documented.
- Notable: kept `resolve_answer_blocks` synchronous on the SSE terminal path (bounded indexed SELECTs, matching the existing `query_log` precedent and the phase's documented fence-lookup bound); no live-infra changes made.
- Next pending phase: none — `todo/` holds only 182 (its 8 task files are in `complete/`; the harness commits + moves `00_phase.md`).
This commit is contained in:
+138
-6
@@ -102,6 +102,43 @@
|
||||
* out of scope. Everything without a link/image/`####`+ construct renders
|
||||
* byte-identically to
|
||||
* the pre-phase renderer (pinned by the phase-180 unit suite).
|
||||
*
|
||||
* Phase 182 (task 06) adds the OPT-IN verbatim pass and the second,
|
||||
* optional argument: `renderMarkdown(md, opts)`, where the ONLY truthy path
|
||||
* is `opts && opts.verbatim === true`. When on, a ` ```bor ` LOCATOR fence
|
||||
* (``` ```bor homelab/runbooks/nginx.md#12-40 ````) is replaced — info
|
||||
* line, body and closing fence — with a `\u0000VERBATIMn\u0000` placeholder
|
||||
* that the final restore step emits as
|
||||
* `<div class="verbatim verbatim-pending" data-bor-locator="…">`.
|
||||
* WHY THE MODEL'S BODY IS DISCARDED (the phase's core contract, locked A7):
|
||||
* the bytes inside a `bor` fence are the MODEL's re-typed copy of a
|
||||
* knowledge-base file, and a re-typed copy is exactly the bug this phase
|
||||
* kills — it can silently differ from the indexed bytes. So the renderer
|
||||
* keeps only the LOCATION (normalized by the local `borLocator`, the client
|
||||
* mirror of Python `app.rag.verbatim.canonical_locator`) and paints nothing
|
||||
* but the pending caption; the server
|
||||
* (`app.rag.verbatim.resolve_answer_blocks`) supplies the real bytes and
|
||||
* `verbatim.js` (task 07) hydrates the placeholder by matching
|
||||
* `data-bor-locator`. An UNPARSEABLE locator is put back byte-for-byte and
|
||||
* falls through to the ordinary fence pass — no placeholder, no badge, the
|
||||
* model's own body — and a fence left OPEN at the very end of the input
|
||||
* (the info line typed, the body not yet streamed) shows that same pending
|
||||
* state instead of raw backticks. The server's `fences_in` is STRICTER
|
||||
* there (it ignores an unclosed fence: the server only ever sees a finished
|
||||
* answer, the client sees a partial one); a half-typed BODY therefore still
|
||||
* reads as ordinary text until its closing fence lands, exactly like every
|
||||
* other incomplete fence does today. PASS ORDER with the option on:
|
||||
* verbatim → fences → tables → escape+inline → blockquotes → paragraphs →
|
||||
* restore (a ` ``` ` inside a `bor` body ends the fence at the FIRST closer
|
||||
* — the documented limitation the server shares, so both sides agree). The
|
||||
* placeholder is the one block element the paragraph pass lets through
|
||||
* unwrapped. BYTE-IDENTICAL WHEN OFF: with no second argument — every
|
||||
* existing call site, the document viewer (`document.js`), the thinking
|
||||
* blocks and the user bubbles — `opts` is undefined, the pass never runs,
|
||||
* no `\u0000VERBATIM` placeholder exists anywhere in the text, the restore
|
||||
* replace is a no-op, and a document that legitimately contains a ` ```bor `
|
||||
* fence still renders its own text (the phase-44/138/180 fixtures stay
|
||||
* byte-identical, pinned by their own suites).
|
||||
*/
|
||||
|
||||
function escapeHtml(s) {
|
||||
@@ -156,10 +193,83 @@ function mdAnchorOpen(safe) {
|
||||
);
|
||||
}
|
||||
|
||||
function renderMarkdown(md) {
|
||||
// Phase 182 (task 06): the CLIENT MIRROR of Python
|
||||
// `app.rag.verbatim.canonical_locator` — the tolerant locator grammar of
|
||||
// phase 182 D7, re-implemented here because the renderer runs in the
|
||||
// browser and cannot import Python. PARITY IS LOAD-BEARING: the canonical
|
||||
// string this returns becomes the `data-bor-locator` key that `verbatim.js`
|
||||
// (task 07) matches against the server's `done.answer_text[].locator`, so a
|
||||
// divergence means a resolved block never hydrates. The parity table in
|
||||
// tests/unit/test_frontend_verbatim_markdown.py pins both sides; `fences_in`
|
||||
// on the server pins the fence shape.
|
||||
//
|
||||
// Grammar: a combined `source/path` (the `read`/`grep` identity contract,
|
||||
// split at the FIRST `/`) plus an OPTIONAL separator (` `, `:`, `#`, `#L`)
|
||||
// and line range — `12-40`, `12\u201340` (en dash), `L12-L40`, or a bare `12`
|
||||
// (one line). Returns `source/path` for a whole document and
|
||||
// `source/path#12-40` for a range (canonical DIGITS only, never the en dash,
|
||||
// never the `L`), or `""` (NOT a locator) for an empty/whitespace input, a
|
||||
// value with no `/` in the identity, a non-numeric or non-positive bound,
|
||||
// `start > end`, trailing junk after the range, a dangling separator
|
||||
// (`a.md#`), or a leading `bor` token the caller left in.
|
||||
function borLocator(raw) {
|
||||
const text = String(raw).trim();
|
||||
if (!text) return "";
|
||||
// A caller that pasted the fence's language token in has already lost the
|
||||
// identity — refuse rather than guess which token was the path.
|
||||
if (text.split(" ")[0] === "bor") return "";
|
||||
// Leftmost match of: one-or-more separators + the range, ANCHORED to the
|
||||
// end of the string (so a range-shaped tail followed by more text is
|
||||
// junk, not a range). Everything before it is the identity.
|
||||
const match = /[ \t:#]+(L?\d+(?:[ \t]*[-\u2013][ \t]*L?\d+)?)$/i.exec(text);
|
||||
const identity = match ? text.slice(0, match.index) : text;
|
||||
const rangePart = match ? match[1] : null;
|
||||
// One whitespace-free `source/path` token, never ending in a dangling
|
||||
// separator (`a.md#` with no range is a typo, not a whole document).
|
||||
if (!identity || /\s/.test(identity)) return "";
|
||||
if (/[#:\-]$/.test(identity)) return "";
|
||||
if (!identity.includes("/")) return "";
|
||||
if (rangePart === null) return identity;
|
||||
const bounds = rangePart.split(/[\u2013-]/).map((b) => b.trim().replace(/^[Ll]+/, ""));
|
||||
if (bounds.some((b) => !/^\d+$/.test(b))) return "";
|
||||
const start = parseInt(bounds[0], 10);
|
||||
const end = bounds.length === 2 ? parseInt(bounds[1], 10) : start;
|
||||
if (start < 1 || end < 1 || start > end) return "";
|
||||
return `${identity}#${start}-${end}`;
|
||||
}
|
||||
|
||||
function renderMarkdown(md, opts) {
|
||||
// 0. Protect the phase-182 ` ```bor ` LOCATOR fences — BEFORE the fence
|
||||
// pass, so a locator fence never reaches the ordinary code renderer.
|
||||
// OPT-IN ONLY (the byte-identity contract in the header): `opts` is
|
||||
// undefined at every existing call site and `opts.verbatim === true` is
|
||||
// the single truthy path. The whole match is DROPPED — info line, body
|
||||
// and closing fence — because the model's copy of the file is exactly
|
||||
// what must never be painted (locked A7); only the normalized locator is
|
||||
// kept. The closing alternative (`(?![\s\S])`) matches a fence left OPEN
|
||||
// at the very end of the input, so as soon as the model has typed the
|
||||
// info line the answer shows the pending state instead of raw backticks
|
||||
// (the deliberate asymmetry with the server's stricter `fences_in`: the
|
||||
// server only ever sees a finished answer, the client sees a partial
|
||||
// one). A bare ` ```bor ` with no path does NOT match — it is an ordinary
|
||||
// code block and must survive to the fence pass untouched — and an
|
||||
// unparseable locator is returned byte-for-byte, degrading to the
|
||||
// `<pre><code>` that pass produces today with the model's own body.
|
||||
const verbatim = [];
|
||||
let text = md;
|
||||
if (opts && opts.verbatim === true) {
|
||||
text = text.replace(/^```bor[ \t]+\S[^\n]*\n(?:[\s\S]*?^```[ \t]*$|(?![\s\S]))/gm, (match) => {
|
||||
const infoLine = match.slice(0, match.indexOf("\n")).replace(/^```bor[ \t]+/, "").trim();
|
||||
const canonical = borLocator(infoLine);
|
||||
if (canonical === "") return match;
|
||||
verbatim.push(canonical);
|
||||
return `\u0000VERBATIM${verbatim.length - 1}\u0000`;
|
||||
});
|
||||
}
|
||||
|
||||
// 1. Protect fenced code blocks.
|
||||
const codeBlocks = [];
|
||||
let text = md.replace(/```(\w*)\n([\s\S]*?)```/g, (_m, _lang, code) => {
|
||||
text = text.replace(/```(\w*)\n([\s\S]*?)```/g, (_m, _lang, code) => {
|
||||
codeBlocks.push(`<pre><code>${escapeHtml(code.replace(/\n$/, ""))}</code></pre>`);
|
||||
return `\u0000CODE${codeBlocks.length - 1}\u0000`;
|
||||
});
|
||||
@@ -356,19 +466,41 @@ function renderMarkdown(md) {
|
||||
}
|
||||
text = quoted.join("\n");
|
||||
|
||||
// 3. Paragraphs (double newline separated).
|
||||
// 3. Paragraphs (double newline separated). The first guard test is the
|
||||
// untouched pre-existing one; the second is the phase-182 placeholder,
|
||||
// which restores to a `<div>` (a BLOCK element) and must not be wrapped
|
||||
// in a `<p>` the way a code placeholder is. It can only ever match when
|
||||
// `opts.verbatim` was on, so every other caller keeps byte-identical
|
||||
// paragraph behavior.
|
||||
text = text
|
||||
.split(/\n{2,}/)
|
||||
.map((block) => {
|
||||
const b = block.trim();
|
||||
if (!b) return "";
|
||||
if (/^<(h\d|ul|ol|pre|li|blockquote)/.test(b)) return b;
|
||||
if (/^<(h\d|ul|ol|pre|li|blockquote)/.test(b) || /^\u0000VERBATIM\d+\u0000$/.test(b)) return b;
|
||||
return `<p>${b.replace(/\n/g, "<br>")}</p>`;
|
||||
})
|
||||
.join("");
|
||||
|
||||
// 4. Restore protected spans: tables (already final HTML) and code.
|
||||
// 4. Restore protected spans: tables (already final HTML), code, and the
|
||||
// phase-182 verbatim placeholders. A placeholder restores as the PENDING
|
||||
// panel and NEVER as the model's bytes (locked A7): the caption carries
|
||||
// the `source/path` identity, the state line is a polite live region
|
||||
// (`role="status"` announces the pending text once and the task-07
|
||||
// hydration swap as a change), and every interpolated value goes through
|
||||
// escapeHtml — the locator is model-written text landing in an ATTRIBUTE,
|
||||
// so this is the escape-first boundary with no exception.
|
||||
return text
|
||||
.replace(/\u0000TABLE(\d+)\u0000/g, (_m, i) => tables[Number(i)])
|
||||
.replace(/\u0000CODE(\d+)\u0000/g, (_m, i) => codeBlocks[Number(i)]);
|
||||
.replace(/\u0000CODE(\d+)\u0000/g, (_m, i) => codeBlocks[Number(i)])
|
||||
.replace(/\u0000VERBATIM(\d+)\u0000/g, (_m, i) => {
|
||||
const canonical = verbatim[Number(i)];
|
||||
const identity = canonical.split("#")[0];
|
||||
return (
|
||||
`<div class="verbatim verbatim-pending" data-bor-locator="${escapeHtml(canonical)}">` +
|
||||
`<span class="verbatim-caption">${escapeHtml(identity)}</span>` +
|
||||
`<span class="verbatim-state" role="status">Retrieving the exact text…</span>` +
|
||||
"</div>"
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user